82 TestNG interview questions with answers

What they are assessing

Whether you understand it as a framework rather than a tool.

Model answer

TestNG is a Java testing framework inspired by JUnit but designed for a wider range of testing, not just unit testing. It provides annotations for structuring tests, flexible configuration through an XML suite file, grouping, dependencies between tests, data driven testing through data providers, parallel execution and a listener model for extension. In automation work its value over plain JUnit is mainly the suite XML, grouping, dependency handling and built in parallelism, which are what you need once a suite grows past a few dozen tests.

Likely follow-up

What does TestNG give you that JUnit 5 does not?

What they are assessing

Comparative knowledge, asked in almost every TestNG interview.

Model answer

TestNG has richer configuration annotations covering suite, test, class, method and group levels, where JUnit 4 had only class and method. TestNG supports dependencies between test methods natively, groups, and parameterisation from XML as well as from a data provider. Parallel execution is built into the suite XML rather than requiring extra configuration. TestNG does not require methods to be static in the same way older JUnit did, and it handles suite level setup more cleanly. JUnit 5 has closed much of the gap with extensions, nested tests and tags.

What they are assessing

Practical execution knowledge.

Model answer

Several ways. From an IDE by right clicking a class or method, which is what you use while writing. From a testng.xml suite file, which is the normal way to organise a real suite and is what you point CI at. From Maven through the Surefire plugin, which picks up either the XML or the annotated classes. From Gradle through the useTestNG configuration. And from the command line with java org.testng.TestNG testng.xml. In CI it is almost always Maven or Gradle invoking the suite XML.

What they are assessing

A detail that causes flaky suites when assumed wrongly.

Model answer

TestNG does not guarantee alphabetical or declaration order by default; the order is effectively unpredictable unless you impose one. You impose it with priority, with dependsOnMethods, or by setting preserve-order in the suite XML which applies to classes and tests rather than methods within a class. The important point for an interview is that relying on incidental ordering is a design problem: well written tests should be independent, and needing a specific order usually indicates shared state that will break under parallel execution.

Trap to avoid

Saying tests run in the order they are written. They do not, and building a suite on that assumption produces failures that appear only on another machine.

What they are assessing

Precision about a commonly misused attribute.

Model answer

priority orders methods within the same class, lowest value first, with the default being zero. Methods with equal priority run in unpredictable order relative to each other. Its limits matter: it does not order across classes, it does not create a dependency so a failed high priority test does not stop a lower one running, and negative values are allowed which surprises people. If you need one test to skip because another failed, that is dependsOnMethods, not priority.

Likely follow-up

Two methods both have priority 1. Which runs first?

What they are assessing

Vocabulary that matters when reading a suite XML.

Model answer

In the XML hierarchy, a suite is the top level element and can contain several test tags. A test tag is a logical grouping that can contain classes, packages or methods, and it is the level at which parameters and some parallel settings apply. A class is a Java class containing methods annotated with @Test. The confusing part is that test in TestNG means the XML element, not an individual test method, which is why @BeforeTest runs once per XML test tag rather than before each method.

What they are assessing

Knowledge of conditional skipping.

Model answer

Throw a SkipException from within the test or a configuration method, which marks it skipped rather than failed. That is the right mechanism when the decision depends on runtime state: an environment that does not support the feature, a missing dependency, or a feature flag being off. The alternatives are enabled equals false on the annotation, which is a permanent compile time decision, and an IAnnotationTransformer listener, which can disable tests programmatically before the run starts based on external configuration.

What they are assessing

Basic recall.

Model answer

@Test marks a test method. The configuration annotations are @BeforeSuite and @AfterSuite, @BeforeTest and @AfterTest, @BeforeGroups and @AfterGroups, @BeforeClass and @AfterClass, and @BeforeMethod and @AfterMethod. Then @DataProvider for data driven tests, @Factory for creating test instances at runtime, @Parameters for injecting values from the suite XML, and @Listeners for attaching listeners to a class. There is also @Ignore at class level in newer versions.

What they are assessing

The single most asked TestNG question.

Model answer

Before annotations run outermost first: @BeforeSuite, then @BeforeTest, then @BeforeClass, then @BeforeMethod, then the @Test method, then @AfterMethod, @AfterClass, @AfterTest, @AfterSuite. @BeforeGroups and @AfterGroups fit around the group they name. The pattern to remember is that the before methods go from broadest to narrowest and the after methods reverse it. @BeforeMethod and @AfterMethod run around every test method, while @BeforeClass runs once per class and @BeforeSuite once for the entire suite.

Likely follow-up

With two test classes in one XML test tag, how many times does @BeforeTest run?

What they are assessing

The distinction people get wrong most often.

Model answer

@BeforeMethod runs before every single test method, so with ten test methods it executes ten times. @BeforeTest runs once before all the classes inside a given test tag in the suite XML, regardless of how many methods those classes contain. So browser launch per test method goes in @BeforeMethod, while one-off configuration for a logical group of classes goes in @BeforeTest. The naming is genuinely misleading and interviewers ask precisely because of that.

Trap to avoid

Assuming @BeforeTest runs before each test method. It runs once per XML test tag, and misplacing driver setup there is a classic cause of shared driver bugs.

What they are assessing

Knowledge of configuration failure behaviour.

Model answer

On a configuration method, alwaysRun equals true forces it to execute even if a preceding configuration method failed or the test it belongs to was skipped. It is normally set on @AfterMethod and @AfterClass so cleanup still happens when setup failed, which otherwise leaves browsers or connections open. On a @Test method, alwaysRun makes it run even when the methods it depends on failed, which mostly defeats the point of the dependency and is rarely what you want.

What they are assessing

Repeat execution knowledge.

Model answer

invocationCount runs the same test method a set number of times, which is useful for flakiness investigation or simple load. threadPoolSize, used alongside it, runs those invocations across that many threads concurrently, so invocationCount ten with threadPoolSize three runs ten executions across three threads. There is also successPercentage, which lets a method be reported as passed if at least that percentage of invocations succeeded, which is occasionally useful and frequently a way of hiding a genuine intermittent defect.

Likely follow-up

Would you use successPercentage on a UI test? Why not?

What they are assessing

Negative testing mechanics.

Model answer

With the expectedExceptions attribute on @Test, giving the exception class, and optionally expectedExceptionsMessageRegExp to assert on the message. The test passes only if that exception is thrown. The weakness is that it passes regardless of where in the method the exception occurred, so a setup line throwing the same type produces a false pass. For anything beyond trivial cases I prefer an explicit try and catch with an Assert.fail after the call, or assertThrows style helpers, because it pins the assertion to the specific invocation.

What they are assessing

Understanding of hang protection.

Model answer

timeOut sets a maximum duration in milliseconds for a test or configuration method; exceeding it fails the method with a ThreadTimeoutException. It is genuinely useful as a safety net in CI, because a hung test blocks the build far longer than a failed one, and the usual causes are a lost browser session or a network call with no client timeout. It can be set per method or globally in the suite XML. It is a backstop rather than a substitute for proper waits and client side timeouts.

What they are assessing

A distinction that separates experienced TestNG users.

Model answer

A data provider supplies different data to the same test method, so the method runs several times on one instance of the class. A factory creates multiple instances of the test class itself, each configured differently, and every test method in each instance then runs. So data provider varies the input to a method; factory varies the object under test. The practical use of a factory is running an entire test class against several configurations, such as the same suite against three user roles or three browsers, without duplicating the class.

Likely follow-up

Can a factory method itself use a data provider?

What they are assessing

Whether you know the less used annotations honestly.

Model answer

@BeforeGroups runs once before the first method belonging to any of the named groups executes, and @AfterGroups after the last one. The realistic use is expensive setup shared by a group of tests but not needed by the rest of the suite: seeding a specific data set for the payments group, or standing up a mock service only the integration group needs. In practice it is used rarely, because most teams achieve the same thing with @BeforeClass and class level organisation, and it is worth saying so rather than inventing a use case.

What they are assessing

The practical setup decision.

Model answer

For an independent, parallel-safe suite, in @BeforeMethod with the driver held in a ThreadLocal, and quit in @AfterMethod with alwaysRun set. That gives every test a clean browser and works under parallel execution. Initialising in @BeforeClass is faster because the browser is reused, but tests then share cookies, local storage and session, which couples them and causes order dependent failures. @BeforeSuite is worse still under parallelism because a single static driver is shared across threads.

Trap to avoid

Putting the driver in a plain static field. It works serially and breaks the moment anyone enables parallel execution.

What they are assessing

Understanding of failure propagation.

Model answer

The test methods it precedes are marked skipped rather than failed, and TestNG continues with the rest of the run. That distinction matters when reading a report: a large number of skips usually points at one configuration failure rather than many broken tests, so you investigate the configuration first. The corresponding @AfterMethod does not run unless alwaysRun is set, which is why cleanup is commonly missed and browsers accumulate. The configfailurepolicy setting controls whether subsequent configuration methods are skipped or still attempted.

Likely follow-up

Your report shows 200 skipped and 1 failed. Where do you look?

What they are assessing

Framework design rather than annotation recall.

Model answer

A base class holding the configuration methods, which test classes extend, is the common approach and works well for driver lifecycle. Inheritance has limits though, particularly when different classes need different setup, so an alternative is listeners implementing IInvokedMethodListener or ITestListener that apply setup centrally without touching the class hierarchy. A third option is composition: a helper or fixture object created in @BeforeMethod. I would usually combine a thin base class for lifecycle with listeners for cross cutting concerns such as screenshots and logging.

What they are assessing

A subtlety that causes confusing behaviour.

Model answer

Configuration methods in a superclass run as well as those in the subclass, and the before methods run parent first while the after methods run child first, which mirrors constructor and destructor semantics. If both parent and child define @BeforeMethod, both execute. Overriding the parent method rather than defining a new one means only the child version runs, which is usually the intent when you want to replace rather than add. Not knowing this produces duplicated setup that appears to work but runs twice.

What they are assessing

Core assertion knowledge.

Model answer

A hard assertion, from the Assert class, throws immediately on failure so the rest of the test method does not execute. A soft assertion, using SoftAssert, records the failure and continues, reporting all failures together when assertAll is called. Soft assertions are useful when you want to check several independent properties of one page or response in a single test. The critical detail is that assertAll must be called, otherwise the failures are collected and never reported and the test passes.

Trap to avoid

Forgetting assertAll. The test passes while assertions failed, which is worse than no test at all because it creates false confidence.

What they are assessing

Judgement rather than mechanics.

Model answer

When a failure invalidates everything after it. If the login assertion fails softly, every subsequent step runs against a page that is not there and produces a cascade of meaningless failures that obscure the real one. So the rule I apply is hard assertions for anything that is a precondition for the rest of the test, soft assertions only for independent checks at the same point in the flow, such as validating several fields on a form that has already loaded correctly.

What they are assessing

Whether you think about failure diagnostics.

Model answer

It discards the information you need when it fails. assertTrue(actual.equals(expected)) reports only that false was not true, whereas assertEquals(actual, expected) reports both values in the failure message. The difference matters at three in the morning when a CI failure is all you have. So I use the most specific assertion available, and where assertTrue is genuinely the right shape I always supply the message parameter explaining what was expected.

Likely follow-up

What is the argument order for assertEquals in TestNG, and why does it matter?

What they are assessing

Maintainability thinking.

Model answer

By making the assertion carry context rather than relying on the stack trace. Every assertion gets a message naming what was being checked and, where useful, the identifier of the data involved, since a data driven test failing on iteration forty is unhelpful without knowing which row. Beyond that, an ITestListener that attaches a screenshot, the page URL and relevant logs on failure gives the diagnostic context automatically for every test rather than depending on each author. Assertion libraries such as AssertJ also produce considerably better messages than the built in ones.

What they are assessing

Data driven testing basics.

Model answer

A method annotated with @DataProvider returning Object[][] or an Iterator of Object[], where each inner array is one set of parameters for one invocation of the test. The test method references it with @Test(dataProvider = "name"), and its parameters must match the array contents in number and type. TestNG then runs the test once per row, reporting each as a separate result. Returning an Iterator rather than an array is preferable for large data sets because rows are produced lazily rather than all held in memory.

What they are assessing

Practical organisation.

Model answer

Mark the provider method static in the other class and reference it with both attributes: @Test(dataProvider = "name", dataProviderClass = DataProviders.class). Keeping providers in a separate class is worth doing once several tests share the same data, because otherwise the data logic gets duplicated or the test class becomes half data plumbing. The requirement for it to be static is the detail people forget, and the resulting error message is not especially clear.

What they are assessing

Real data driven implementation.

Model answer

The provider method reads the file and returns the rows. For CSV, a reader or a library such as OpenCSV; for Excel, Apache POI; for JSON, Jackson or Gson mapping to objects. I would return a two dimensional array of a domain object rather than loose strings where the data has structure, because a test signature taking one well named object is far more readable than one taking eight strings. Failing to close the resource is the usual defect, and files should be read once rather than per row.

Likely follow-up

Where would you keep the data file so it works both locally and in CI?

What they are assessing

Lesser known but genuinely useful capability.

Model answer

Yes. A data provider method can take a java.lang.reflect.Method parameter, which TestNG injects with the test method it is about to supply, so the provider can return different data depending on the method name. It can also take an ITestContext parameter to access suite level parameters and the current test context. That is how one provider serves several tests with different data sets, and how a provider reads a parameter such as environment from the suite XML to decide which data to return.

What they are assessing

Knowledge of a specific parallelism mode.

Model answer

Set parallel equals true on the @DataProvider annotation, which runs the iterations concurrently, with the thread count controlled by the data provider thread count setting in the suite XML or on the command line. It is effective for API tests where each row is independent. For UI tests it requires the driver to be genuinely thread confined, typically through ThreadLocal, and any shared test data must be distinct per row or you get collisions. Enabling it on a suite with shared state is a reliable way to produce intermittent failures.

Trap to avoid

Enabling parallel data providers on a suite with a single shared WebDriver. It will fail in ways that look like application defects.

What they are assessing

Reporting quality at scale.

Model answer

By ensuring the report names the data, not just the method. Passing a domain object with a meaningful toString, or implementing ITest so the test name is set per iteration, both work; without one of them the report shows forty results with identical names and you cannot tell which row failed. I would also include the identifying data in every assertion message. This matters more than it sounds: a data driven suite whose failures cannot be attributed to a row wastes significant triage time.

What they are assessing

Basic suite organisation.

Model answer

Groups are labels applied to test methods or classes with the groups attribute, allowing you to include or exclude sets of tests at run time from the suite XML or the command line. They are how you build one suite that serves several purposes: a smoke group running in five minutes on every commit, a regression group nightly, and groups by feature area or by risk. A method can belong to several groups, which is what makes them more flexible than organising purely by class.

What they are assessing

Practical XML knowledge.

Model answer

Inside a test tag, a groups element containing a run element with include and exclude children naming the groups. Exclusion wins over inclusion, so a method in both an included and an excluded group does not run, which is the behaviour to remember. Groups also support regular expressions in the name, so you can include everything matching a pattern. The same can be driven from the command line with the groups and excludegroups options, which is how CI usually selects what to run without editing files.

Likely follow-up

A method is in groups smoke and flaky. You include smoke and exclude flaky. Does it run?

What they are assessing

Awareness of a rarely used feature.

Model answer

A define element in the XML creates a meta group containing other groups, which can then be included by one name. It is worth using when you have many fine grained groups and want a stable name for a combination, such as a release group composed of smoke, payments and auth. In practice most teams manage with a modest number of flat groups, and heavy use of meta groups tends to make it hard to work out what a given run will actually execute, which is worth saying rather than presenting it as best practice.

What they are assessing

Judgement about quarantine.

Model answer

Tag the test into a quarantine group excluded from the blocking pipeline, so it stops failing the build, but run that group on a schedule and report on it so it stays visible. The important part is the accompanying discipline: a ticket per quarantined test with an owner and a date, and a hard limit on how many tests may be in quarantine at once. Without that, quarantine becomes a permanent home and the suite quietly loses coverage while appearing green, which is worse than the original flakiness.

What they are assessing

Dependency basics.

Model answer

It declares that a test method should run only after the named methods, and only if they passed. If a dependency fails, the dependent test is skipped rather than failed, which keeps the report honest: the dependent test was never actually exercised. This differs from priority, which only orders and does not skip. The usual legitimate use is a genuine precondition, such as a login test that other tests require, rather than using it to sequence tests that should be independent.

What they are assessing

Scale of dependency declaration.

Model answer

dependsOnMethods names specific methods, which is precise but brittle because renaming a method breaks it silently at runtime rather than at compile time. dependsOnGroups depends on every method in the named groups, which is more maintainable when the dependency is conceptual: this test needs the setup group to have run. dependsOnGroups also works across classes naturally, whereas method dependencies across classes require the fully qualified name and are fragile.

Likely follow-up

What happens if you create a circular dependency between two methods?

What they are assessing

Whether you have opinions formed by maintaining a suite.

Model answer

Sparingly. They are legitimate for real preconditions, and they keep reports readable by skipping rather than cascading failures. But a suite with extensive dependencies cannot be parallelised effectively, cannot be run partially, and fails in long chains where one root cause produces fifty skips. My preference is independent tests that create their own preconditions, using API calls for setup rather than driving the UI through a dependency chain, and reserving dependencies for cases where the setup is genuinely expensive and shared.

What they are assessing

Core parallelism knowledge.

Model answer

Set on the suite or test tag: methods runs individual test methods in separate threads, classes runs each class in its own thread with methods inside it serial, tests runs each XML test tag in its own thread, and instances runs instances produced by a factory in parallel. thread-count controls the pool size. The choice matters: methods gives the most parallelism but requires complete independence, while classes is the safest starting point because methods within a class often share class level state.

Likely follow-up

Which mode would you pick for a Selenium suite, and why?

What they are assessing

The most common practical parallelism problem.

Model answer

Hold the driver in a ThreadLocal so each thread has its own instance, with a getter returning the current thread's driver, initialised in @BeforeMethod and removed in @AfterMethod. Removing matters as well as quitting, because thread pools reuse threads and a stale ThreadLocal leaks. Everything the driver touches must follow the same rule: page objects should be created per test rather than held statically, and any shared utility holding state becomes a race condition.

Trap to avoid

A static WebDriver field. It is the single most common cause of a suite that passes serially and fails randomly in parallel.

What they are assessing

Experience of actually doing it.

Model answer

Shared state, in roughly this order of frequency: static drivers or page objects, static utility classes holding per test data, test data collisions where several tests use the same account and interfere, and order dependencies that were invisible when everything ran sequentially. Then environmental limits: too many browsers for the machine, connection pool exhaustion on the application, and rate limiting. Then reporting, where listeners writing to a shared file without synchronisation produce corrupted output. I would expect to fix data isolation first, because it causes the most confusing failures.

What they are assessing

Empirical rather than arbitrary configuration.

Model answer

By measuring rather than guessing. Start low, increase, and watch both total duration and failure rate, because past a point failures rise from resource contention while duration stops improving. The constraints are usually the load generator machine for browser based tests, roughly one to two browsers per available core as a starting point, and the application under test, which may rate limit or simply be a shared environment other people are using. The right number is the one that gives the shortest reliable run, not the highest.

What they are assessing

Whether you have organised a suite properly.

Model answer

It is the suite definition file: which classes, packages or methods to run, grouped into one or more test tags. It configures parameters, group inclusion and exclusion, parallel mode and thread count, listeners, and suite level settings such as verbosity and preserve-order. Its value is separating what runs from how the tests are written, so CI can run smoke or regression from the same codebase without any code change. It is also what lets you define several logical runs, such as one test tag per browser.

What they are assessing

Parameterisation via XML.

Model answer

Define a parameter element with a name and value at suite or test level, then annotate the receiving method with @Parameters listing the names, with the method taking matching arguments. Parameters can be injected into configuration methods as well as tests, which is how a browser name reaches the driver setup in @BeforeMethod. Marking the argument with @Optional provides a default when the parameter is absent, which prevents the run failing outright when someone uses a different XML file.

Likely follow-up

How would you override an XML parameter from the command line in CI?

What they are assessing

Ordering control at the XML level.

Model answer

preserve-order, true by default, makes TestNG run classes and methods listed in a test tag in the order they appear in the XML rather than reordering them. It applies to the listed elements, not to method ordering within a class, which is still governed by priority and dependencies. It matters when you have deliberately sequenced classes, for example a data setup class before the tests that use it, and it is ignored under parallel execution, which is a common source of confusion.

What they are assessing

Framework organisation at scale.

Model answer

One approach is a test tag per browser within a suite, each passing a browser parameter, with parallel set to tests so they run concurrently. Environment is better handled as a system property or an XML parameter overridden at run time rather than a separate file per environment, because duplicated XML files drift. A parent suite using suite-files to include child suites helps when several teams own different areas. The principle is one source of truth for the test list, with variation supplied as parameters.

What they are assessing

Knowledge of the extension model.

Model answer

ITestListener for test start, success, failure and skip events, which is where screenshot on failure and custom logging usually live. ISuiteListener for suite start and finish. IInvokedMethodListener for before and after every method including configuration methods. IAnnotationTransformer for modifying annotations at runtime, which is how you apply a retry analyser globally. IReporter for generating custom reports after the run. IMethodInterceptor for reordering or filtering the method list before execution.

What they are assessing

The three mechanisms and their scope.

Model answer

Three ways. The @Listeners annotation on a test class, which is simple but applies only where it is declared and gets copy pasted. A listeners element in testng.xml, which applies to the whole suite and is usually the right choice. Or the ServiceLoader mechanism, placing the implementation class name in a META-INF services file, which makes the listener apply automatically wherever the jar is on the classpath, which is how shared framework listeners are distributed across projects.

What they are assessing

A very common real requirement, and its risks.

Model answer

Implement IRetryAnalyzer with a counter and a maximum, returning true from retry while under the limit. Attach it either per test with the retryAnalyzer attribute, or globally with an IAnnotationTransformer that sets it on every method, which is the maintainable option. The caveat matters as much as the mechanism: retries hide flakiness rather than fixing it, and a suite that passes on the third attempt is not passing. I would cap at one retry, report retried tests separately so the flake rate stays visible, and treat a rising retry count as a defect.

Trap to avoid

Presenting retries as the solution to flakiness. Interviewers are listening for whether you know it is a mitigation that must be measured.

What they are assessing

Practical diagnostics implementation.

Model answer

In an ITestListener implementation, override onTestFailure, retrieve the driver instance for the current test, take the screenshot and attach it to the report. Getting the driver is the awkward part under parallelism: the listener needs access to the correct thread's instance, which is why a ThreadLocal driver holder exposed through a static getter is the usual pattern, or retrieving it from the test instance via ITestResult.getInstance and a cast to a base class. Saving with a name including the test and timestamp keeps artefacts navigable.

Likely follow-up

How do you get the driver in the listener when tests run in parallel?

What they are assessing

Baseline knowledge.

Model answer

By default it writes to a test-output folder: index.html as the main report, emailable-report.html as a single file summary suitable for sending, and testng-results.xml which is the machine readable result set that CI tools and other reporters consume. There is also a testng-failed.xml generated after a run containing only the failed tests, which can be executed directly to rerun just those, which is useful during triage.

What they are assessing

Awareness of the reporting ecosystem.

Model answer

Allure is the most common choice, giving step level detail, attachments, history and trend across runs, integrated through an adapter and an annotation set. ExtentReports is the other frequent answer, simpler to set up and producing a good single file report. Both are driven from listeners so no test code changes. Beyond the tool, the things that make a report useful are attaching screenshots and logs on failure, naming data driven iterations meaningfully, and keeping history so flakiness is visible as a trend rather than a single run.

What they are assessing

Whether you optimise for the person triaging.

Model answer

By attaching everything needed at the moment of failure: screenshot, page source or response body, the browser console log, the network HAR if available, the test data used, and the application logs for that window. All of it from a listener so it applies uniformly. Then keep the artefacts with the build for long enough to investigate. The test of this is whether someone can diagnose a failure from the build page alone, and most suites fail that test because the only artefact is a stack trace.

What they are assessing

Build integration basics.

Model answer

Through the Surefire plugin, configured in the POM either to point at a suiteXmlFiles entry naming your testng.xml, or to pick up classes by naming convention. Surefire runs during the test phase. The common additions are passing system properties through argLine or systemPropertyVariables, setting skipTests for builds that should not run them, and configuring the failIfNoTests behaviour. For integration or UI suites that should not run on every build, the Failsafe plugin bound to the verify phase is usually a better fit than Surefire.

Likely follow-up

Why might you use Failsafe rather than Surefire for a UI suite?

What they are assessing

Whether suites are portable.

Model answer

As system properties or environment variables read at runtime, rather than hard coded or committed in a properties file per environment. In Maven that means passing them with -D and reading with System.getProperty, with a sensible default so a local run works without arguments. XML parameters are the alternative for values that belong to the suite definition. Credentials come from the CI secret store, never from the repository, and should not be printed to the log by any listener.

What they are assessing

Distributed execution knowledge.

Model answer

The driver setup in @BeforeMethod creates a RemoteWebDriver pointing at the grid or cloud endpoint, with capabilities built from parameters supplied by the suite XML or system properties, so browser and platform vary without code changes. Parallel mode plus thread count controls how many sessions run concurrently, and that number must respect the grid's capacity or sessions queue and time out. For cloud providers, the session name and build name should be set from the TestNG method and suite names so their dashboard is navigable.

What they are assessing

Pipeline design.

Model answer

By group, driven from the command line so one codebase serves every stage. A smoke group on every commit, running in a few minutes against a deployed environment. A broader regression group on merge to the main branch or nightly, parallelised. Longer running or environment specific suites on a schedule. Each stage fails the build on failure except the quarantine group, which reports but does not block. The key property is that selecting what runs requires no code change, only a different group argument.

What they are assessing

The classic suite level bug.

Model answer

Shared state, almost always. Static fields carrying data between tests, a browser session reused with cookies or local storage from a previous test, test data modified by one test and depended on by another, and database records left behind. Then ordering assumptions, where a test relied on something an earlier test happened to create. I would isolate by running pairs to find the interacting tests, then make each test create and clean its own preconditions rather than adding ordering to paper over it.

What they are assessing

Report interpretation.

Model answer

Usually a single configuration failure rather than many broken tests: a @BeforeClass or @BeforeMethod threw, so everything it precedes was skipped. The other common cause is a failed dependency, where one method with dependsOnMethods failing cascades skips through everything downstream. So I would find the one failure and the skips will normally resolve with it. The related finding is that a suite reporting mostly skips has effectively not run, which is worth flagging rather than reporting as a partial pass.

Likely follow-up

How do you make sure cleanup still runs when setup failed?

What they are assessing

Practical debugging of a stuck build.

Model answer

Most often a browser session or a network call with no timeout, so the thread waits forever. Adding timeOut on the tests turns the hang into a failure with a stack trace, which is the first thing I would do because it makes the problem diagnosable. Beyond that, a thread dump from the CI agent shows where threads are parked, and browser processes left running from earlier failures accumulate until the machine stops. A missing @AfterMethod quit with alwaysRun is the usual root cause of the accumulation.

What they are assessing

Systematic isolation.

Model answer

Bisect. Run the failing test alone to confirm it passes, then run it with half the suite before it, then a quarter, until you have the smallest set that reproduces. TestNG makes this easier because you can list specific methods in an XML file. Once the pair is identified, look for what the first test leaves behind: static state, database rows, browser session, a feature flag toggled. Randomising execution order deliberately is a good ongoing practice, because it surfaces these dependencies early rather than when someone enables parallelism.

What they are assessing

Environment reasoning.

Model answer

Timing, since CI machines are usually slower and more contended, so implicit assumptions about speed break. Screen resolution and headless mode differences affecting element visibility and click interception. Timezone and locale differences between the agent and the developer machine. Data state, where local runs use data left from previous manual testing. Network differences including proxies and restricted egress. And parallelism, if CI runs the suite parallel and local runs are serial. I would start by reproducing locally in headless mode with the CI timezone set.

What they are assessing

Whether you treat suite speed as a deliverable.

Model answer

Measure per test first, since the distribution is usually skewed and a handful of tests account for a large share. Then attack in order: move coverage down the pyramid so journeys that could be API tests are not UI tests, remove redundant tests covering the same path, replace UI setup with API calls for preconditions, enable parallelism once isolation allows it, and remove fixed sleeps, which are almost always present and almost always the cheapest win. Reporting the distribution to the team usually prompts the redundancy conversation on its own.

What they are assessing

Simple recall with a follow-on about practice.

Model answer

enabled equals false on a @Test stops the method being run, and it can also be applied at class level to disable every test in the class. It is a compile time decision, unlike SkipException which is runtime. The practical caution is that disabled tests accumulate silently: a suite with thirty disabled tests looks healthy while coverage has quietly gone. I would require a ticket reference in a comment next to any disabled test and review them periodically.

What they are assessing

Attention to reporting quality.

Model answer

It sets a human readable description that appears in reports alongside the method name. It is worth using because method names are constrained by Java naming and often end up either terse or unreadably long, whereas a description can state the business behaviour being verified. In a report shared with people who do not read the code, the difference between testCheckout2 and a sentence describing what was verified is the difference between a useful report and a list of identifiers.

What they are assessing

Fine grained selection.

Model answer

Inside a class element, add a methods element with include or exclude entries naming the methods. It is useful for temporarily narrowing a run during debugging, and it is how testng-failed.xml is structured when TestNG generates it after a failed run. For ongoing organisation groups are better, because method lists in XML go stale as soon as anyone renames a method and fail silently by simply running nothing.

Trap to avoid

Using method includes as the main organising mechanism. A renamed method silently drops out of the run with no error.

What they are assessing

Knowledge of runtime annotation modification.

Model answer

It lets you modify test annotations programmatically before execution. The most common use is applying a retry analyser to every test without annotating each one. Others include disabling tests based on external configuration such as a feature flag or an environment, adding groups dynamically, or setting a global timeout. It runs before the suite starts, so decisions must be based on information available at that point. It is the cleanest way to apply cross cutting policy without touching hundreds of test classes.

What they are assessing

Awareness of execution list manipulation.

Model answer

It receives the list of methods TestNG intends to run and returns a modified list, so you can filter, reorder or prioritise before execution. Realistic uses are running previously failed tests first to get fast feedback, ordering by historical duration to improve parallel packing, or filtering by an external source such as a test management system deciding today's scope. It is powerful and worth using sparingly, because a run whose contents are decided by code is harder to reason about than one decided by the XML.

What they are assessing

Knowledge of the runtime context object.

Model answer

It represents the context of a test tag at runtime and can be injected into configuration and test methods. It exposes the suite and test names, the XML parameters, the start and end times, and the passed, failed and skipped result sets. Practical uses are reading parameters without the @Parameters annotation, sharing state between methods within a test tag through setAttribute and getAttribute, and building custom summaries in a listener. Using it as a general shared state mechanism is tempting and usually leads to coupling.

What they are assessing

Alternatives and their trade-offs.

Model answer

You can catch the assertion error and record it yourself, or accumulate messages in a list and assert at the end, which is effectively hand rolled soft assertion. Assertion libraries such as AssertJ provide SoftAssertions with better messages and a fluent API. The reason to reach for one of these rather than TestNG SoftAssert is usually message quality, particularly for collections and objects, where the built in assertions produce output that is technically correct and hard to read.

What they are assessing

Design thinking about data.

Model answer

By keeping the provider a reader rather than a repository. The data belongs in a versioned file or is generated from a defined rule, and the provider loads it, so there is one place to change a value. Where data must exist in the system under test, the better pattern is creating it through an API in setup and returning identifiers, rather than hard coding identifiers that assume a particular database state. Hard coded identifiers are the reason suites work on one environment and fail on another.

What they are assessing

Detail knowledge.

Model answer

Both. Putting groups on the @Test annotation at class level applies it to every test method in that class, and individual methods can add further groups of their own. That is convenient for organising by feature where the class boundary matches the feature. The thing to be careful about is that the class level annotation does not apply to methods that carry their own @Test annotation with a groups attribute unless you repeat it, which produces tests that quietly fall outside every group and therefore never run.

What they are assessing

Alternative designs.

Model answer

Create the data through an API in a configuration method rather than through another test, so the precondition is setup rather than a test dependency. If the setup is genuinely expensive and shared, do it once in @BeforeClass or a suite level fixture and have each test derive its own isolated slice from it, for example its own account within a shared tenant. That keeps tests independent and parallel safe while paying the expensive cost once, which a dependency chain does not achieve.

What they are assessing

Traceability in practice.

Model answer

Usually with a listener that reads the result on test finish and posts it to the tool's API, mapped by an identifier carried on the test, either in the description, a custom annotation or the method name. The mapping is the part that decays: identifiers get copied when tests are duplicated, and deleted tests leave orphaned cases. I would keep the mapping in one place and validate it as part of the build, otherwise the integration reports confidently against cases that no longer exist.

What they are assessing

Measurement rather than anecdote.

Model answer

Persist results across runs rather than looking at one build. The metric that matters is the proportion of runs in which a given test failed while the code did not change, and the count of tests that required a retry. Allure retains history natively; otherwise push the results XML into a database or a dashboard. Once it is a visible number per test, the conversation changes from arguing about whether a test is flaky to ranking which ones to fix first, which is the actual point of measuring it.

What they are assessing

Leadership on a common situation.

Model answer

Measure first: the pass rate, the flake rate per test and the proportion of failures that turned out to be test defects rather than product defects. That number is usually the reason for the distrust and nobody has quantified it. Then quarantine the worst offenders so the pipeline goes green and stays meaningful, with a ticket and owner each. Then fix them in priority order and reintroduce. Green has to mean something before anyone will look at red, so restoring the signal comes before adding coverage.

Likely follow-up

How do you stop the quarantine list growing indefinitely?

What they are assessing

Test architecture judgement.

Model answer

By pushing each check to the lowest layer that can answer it honestly. Business rules belong in unit tests where feedback is in seconds. Contract and data handling belong at API level, which in a Java stack often means TestNG driving REST Assured rather than a browser. Only genuine end to end journeys belong in the UI suite, because those are slowest and most brittle. A TestNG suite that has grown to eight hundred UI tests is usually carrying several hundred that should have been API tests, and reducing it is a faster win than parallelising it.

What they are assessing

Understanding of layering.

Model answer

They operate at different levels and are frequently used together. TestNG is the execution framework: it runs methods, manages lifecycle, parallelism and reporting. Cucumber is a specification layer that maps plain language scenarios to step definitions, and those step definitions still need a runner underneath, which is often TestNG through the AbstractTestNGCucumberTests class. So the question of TestNG or Cucumber is usually a false choice; the real question is whether the plain language layer earns its maintenance cost.

What they are assessing

Knowledge of a useful suite setting.

Model answer

A suite level attribute controlling what happens after a configuration method fails. The default is skip, meaning subsequent configuration methods of that type are skipped along with the dependent tests. Setting it to continue makes TestNG attempt them anyway. continue is occasionally useful when one class's setup failing should not prevent other classes being attempted, but it can also produce a cascade of confusing failures from tests running without their preconditions, so I would default to skip and be deliberate about changing it.

What they are assessing

Precision about a less used mode.

Model answer

classes puts each test class in its own thread. instances applies when a @Factory has produced several instances of the same class, and runs those instances in parallel rather than the classes. So if a factory creates one instance per user role, instances runs the roles concurrently while keeping each instance's methods serial. Choosing classes in that situation would not parallelise them, because they are all the same class. It is a narrow distinction and only matters once factories are in use.

What they are assessing

Practical triage knowledge.

Model answer

After a run, TestNG writes this file into the output folder containing only the tests that failed, plus their configuration methods. Running it re-executes just those, which is far faster than a full suite when you are verifying a fix or checking whether a failure is intermittent. The caveat is that it does not reproduce the original suite conditions, so a test that failed because of interference from an earlier test will often pass in isolation, which is itself diagnostic information.

What they are assessing

A frustrating and common situation.

Model answer

Work through the filters in order. Is the class actually listed in the XML, or matched by the package or naming convention Surefire expects. Is the method annotated with @Test. Is enabled false. Is it in a group that is excluded, or not in any included group when includes are specified. Is it in a method include list that no longer matches after a rename. And is the class in the compiled output at all, since a source folder not configured as a test root produces exactly this silent absence.

What they are assessing

A considered opinion rather than loyalty.

Model answer

It depends on the stack and the team. For a Java project where the team already knows it, yes: the suite XML, grouping and parallelism are mature and it integrates with everything. For a new project with no Java constraint, I would weigh JUnit 5, which has closed most of the historical gaps and has a more modern extension model, and if the application is JavaScript I would question using a Java framework at all rather than Playwright. Choosing TestNG because it is what we have always used is not a reason I would offer in an interview.