Karate review: API testing tutorial, pricing and setup

A Karate test is a .feature file of Scenarios, each a list of steps such as url, path, request, method and match. Karate parses the file itself rather than mapping steps to Java glue code as Cucumber does, so there is no step definition layer to write. Variables, JSON and XML are first-class: a step can declare a JSON literal, embed expressions with #(), call another feature, or run JavaScript. The interpreter, HTTP client and reports all run on the JVM, so Java is needed at run time even though tests contain none, and Java classes can be called when the DSL runs out.

Karate 2.0.0, released on 26 March 2026, replaced the engine. GraalJS gave way to karate-js, an embedded JavaScript engine designed to be thread-safe, and parallel execution moved to Java 21 virtual threads. The HTTP client moved to Apache HttpClient 5.6, the mock server was rebuilt on Netty, Gatling integration was reimplemented in Java without Scala, and the HTML reports were redesigned. Backward compatibility shims keep the v1 com.intuit.karate Runner, Results and MockServer classes working, so the migration guide says most projects only need a dependency and Java version change. The exceptions are in parallel tags, reporting and logging.

Assertions centre on match. A single match compares a whole JSON or XML document against an expected shape, with markers such as #string, #uuid, #regex, #notnull, #present and #ignore standing in for values that vary, and a double hash making a field optional. match each applies a shape to every array element, and contains and within check subsets and ranges. A failed match in version 2 reports every mismatched path with actual and expected values. This is why Karate tests are short: one match replaces a dozen field assertions.

Before each scenario, Karate runs karate-config.js, a JavaScript function that returns variables available to every feature. It reads karate.env, set with -Dkarate.env in Maven or -e on the CLI, to choose base URLs and credentials per environment, and can call configure for timeouts, headers, logging or authentication. Version 2 adds a declarative configure auth for Basic, Bearer, OAuth2 with automatic token refresh, and NTLM. Because the config runs per scenario, expensive setup such as fetching a token belongs in karate.callSingle or callonce, not in plain config code.

Karate’s own recommendation is to keep feature files next to the Java runner under src/test/java, grouped by domain, such as orders and accounts, rather than in a separate resources tree. Put karate-config.js at the classpath root and keep environment differences there: base URLs and non-secret settings per karate.env, secrets from environment variables or system properties. Put shared flows, such as obtaining a token, in their own features and call them with callonce so they run once per feature rather than once per scenario. Tag scenarios with @smoke, @regression or @ignore from the start, because tags are how the runner and CI select subsets.

For CI, run everything through a single JUnit test that calls the parallel runner rather than through one runner class per feature. Runner.path selects features, tags filters them, and parallel sets the thread count and must be the last call. Version 2 produces only HTML reports by default, so turn on JUnit XML explicitly if your CI publishes test results. Scenarios must be independent: they run in any order and on different threads, so a scenario that relies on data created by another will fail intermittently. Where scenarios genuinely share a resource, serialise just those with @lock=name rather than slowing the whole suite.

A Karate suite in a Maven project runs with a normal build: set up Java 21, run the suite test, and upload target/karate-reports so the HTML report, with every request and response, is one click from a failed job. Pass the environment with -Dkarate.env and secrets as environment variables read by karate.sysenv. Karate’s own CI guide also recommends scanning published reports for leaked tokens, since the HTML report records headers and bodies; configure logging masks for Authorization headers before reports leave the build.

Work through the migration guide on a branch. Swap karate-junit5 for karate-junit6, add junit-jupiter explicitly because v2 no longer bundles it, move the build to Java 21 and Surefire 3.2.5 or later, and run the suite. Then search for the changes that do not fail loudly. @parallel=false is ignored without a warning, so replace it with @lock. karate-summary.json and the per-feature JSON files are gone; scrapers should read the karate-events.jsonl stream. Logging options such as logPrettyRequest are now no-ops under configure logging. karate-robot is not part of v2, and image comparison moved to the karate-image extension. Compare a v1 and v2 report for the same commit before switching CI.

Karate suites degrade in a recognisable way: scenarios grow long JavaScript blocks, called features take many arguments, and nobody can tell what a failure means. Keep JavaScript in small named functions, defined once and read in, or move real logic into a Java helper called through Java.type. Prefer one match against a full expected shape over many single-field checks, and store expected payloads as JSON files when they are long. Add a Gherkin comment above important match steps: in version 2 that comment becomes the failure label in the report, which makes CI failures readable to people who did not write the test.