Pairwise testing tool

Pairwise testing, also called all-pairs testing, is a combinatorial technique that covers every possible pair of values across a set of parameters without testing every full combination. With five parameters of a few values each, exhaustive testing needs hundreds of tests. Covering every pair needs a small fraction of that, because each test covers many pairs at once: a test with five parameters covers ten pairs.

It suits configuration and compatibility testing, where the parameters are independent choices: browser and operating system, device and network, user role and feature flag, payment method and currency.

Research by the US National Institute of Standards and Technology on failures in real systems found that most were triggered by a single parameter value or by the interaction of two, and that the share needing three or more parameters fell quickly. Covering every pair therefore catches a large share of interaction defects for a small share of the cost of exhaustive testing.

It is not a guarantee. A defect that needs three specific values together can slip through a pairwise suite. When a combination is known to be risky, add it as an explicit test alongside the generated suite, and consider three-way coverage for the most critical parameters.

Real parameters are rarely fully independent. Safari does not run on Windows or Android. A guest checkout may not offer a saved wallet. A suite that ignores this wastes tests on impossible configurations, or reports failures that are not defects. Constraints tell the generator which pairs must never appear together.

Occasionally constraints make a pair impossible to place in any complete test, even though the pair itself is allowed. The tool reports these pairs rather than silently dropping them, because they usually point to a constraint that is stricter than intended.

The generator is greedy, in the style of the AETG algorithm. Each new test starts from the value that appears in the most pairs not yet covered, then fills the other parameters with whichever values cover the most new pairs, compares many candidate tests and keeps the best. Several runs are compared and the smallest suite wins.

The result is not guaranteed to be the theoretical minimum, which is a hard problem in general, but it is close: it is never smaller than the largest number of allowed pairs between any two parameters, and the tool shows that minimum next to the suite size so you can judge for yourself. The generator runs in your browser and handles models of up to 50,000 value pairs, which covers almost every real configuration matrix. For larger models, a command-line tool such as Microsoft PICT is the better choice.