Robot Framework review: tutorial, pros and cons

A Robot Framework file is a set of sections: Settings, Variables, Test Cases or Tasks, and Keywords. Each row in a test starts with a keyword name followed by its arguments, with cells separated by two or more spaces or by pipes. Keywords come from two places. User keywords are written in Robot syntax from other keywords, and live in the same file or in resource files shared across suites. Library keywords are implemented in code. Running robot on a directory turns the folder tree into a suite tree, and setups and teardowns at suite and test level wrap the steps, which is how browsers are opened once per suite and pages once per test.

A library is usually a Python module or class, and each public function or method becomes a keyword: fill_text becomes Fill Text. Arguments arrive as strings and are converted according to the function’s type hints, and 7.5 adds support for Python’s type statement aliases. Standard libraries such as BuiltIn, Collections, OperatingSystem, Process and String ship with the framework, while Browser, SeleniumLibrary, RequestsLibrary and AppiumLibrary are separate projects installed from PyPI. Browser library runs Playwright in a Node.js process behind its Python keywords. A remote library interface lets keywords be served from other languages, but Python is the natural home.

Every run writes output.xml, or JSON if you name the output file with a .json extension, plus log.html and report.html generated from it. The log is a tree of every suite, test and keyword with arguments, timings, messages and embedded screenshots; the report summarises results by suite and tag. Rebot, the companion tool, regenerates reports from output files and merges several of them, which is how Pabot combines parallel runs and how a rerun of failed tests replaces the original results. The exit code is the number of failed tests, capped at 250, so CI fails without any extra wiring.

Variables can be set in files, from the command line with --variable, from environment variables with %{NAME} syntax, or inside tests with the VAR syntax added in 7.0. Since 7.3 a variable can declare a type, such as ${count: int}, and be converted on creation. Since 7.4 the Secret type hides a value in data and logs, even at trace level, and a secret can only be created from another secret or an environment variable, never a literal. Command-line values override file values, which is how one suite targets several environments without edits.

The structure that keeps Robot suites maintainable is three levels. Tests read as business steps and contain no selectors. Business keywords in resource files, such as Sign In As or Create Order For, combine page interactions. Library keywords from Browser or SeleniumLibrary sit at the bottom and are only called from resource files. Keep one resource file per area of the product, prefer role and data-testid selectors, and assert inside keywords with Browser library’s assertion operators, which retry. A redesign then changes a resource file, not every test, and a reviewer can read the test file alone.

Pabot runs suites in parallel processes, by default as many as the larger of two and the CPU count, and merges the outputs into one log and report. It splits by suite unless you pass --testlevelsplit, which spreads individual tests but adds overhead, since each process starts its own browser. Make suites independent first: each should create its own data and sign in itself. PabotLib provides locks and shared value sets for the few resources that cannot be duplicated. For flaky infrastructure, rerun only the failed tests and merge them back rather than retrying everything; note that --rerunfailed stops with an error when nothing failed, so run it only after a failing run.

The pipeline installs Python and Node.js, installs pinned packages, and lets rfbrowser init download Chromium with its system dependencies. Pass secrets as environment variables, which the Secret-typed variable in the resource file reads, so they never appear in the log. Robot’s exit code is the number of failed tests, so the job fails without extra configuration. Upload the whole results folder even when the run fails, because log.html with its embedded screenshots is what someone will open first. If installing Node.js is a problem on your runners, robotframework-browser[bb] bundles it and replaces rfbrowser init with rfbrowser install.

For new web suites, Browser library is the stronger default. It runs on Playwright, so actions auto-wait, strict mode fails a selector that matches more than one element, assertion operators on Get keywords retry, and pages and contexts opened in a test close automatically at the end of it. Its defaults are a 10 second timeout, 1 second of assertion retries and a screenshot on failure. SeleniumLibrary 6.9 runs on Selenium 4 with Selenium Manager handling drivers, has a 5 second default timeout, an implicit wait of 0 and Capture Page Screenshot on failure. Choose it when you need real Safari, an existing Selenium Grid or cloud provider, or when a large SeleniumLibrary suite already exists.

Robot Framework 7.5 is a feature release with few incompatible changes for normal users, but it deprecates several things worth fixing now. The built-in Testdoc tool and the Telnet library are deprecated in favour of external projects. Tag patterns written like XORY must separate operators, as X OR Y, and the & operator gives way to AND. Regular expression matching in Collections will require a full match, and the list_ and time_ argument names become list and time in Robot Framework 8.0. If you are coming from 6.x, the [Return] setting has warned since 7.0 and singular section headers such as Test Case are deprecated. Run the suite once, read the deprecation warnings in the log and fix them before 8.0 removes them.