TestNG Tutorial (2027): Write and Run Your First Selenium Test, Step by Step

TestNG is a Java testing framework, and it is the runner behind a large share of Selenium suites. If you are starting out in automation testing, it is a good first framework: it gives you setup and teardown methods, assertions, groups and reports without any extra libraries.

This tutorial takes you from an empty folder to a working Selenium test run by TestNG. You will create a Maven project, write three tests against a public practice login page, run them from the command line, read a real failure and find the reports. Every file and every output below comes from our own run on 28 October 2026, on Selenium 4.51.0, TestNG 7.12.0, JDK 21 and Maven 3.9.16, in Chrome 155 and Firefox 159.

When your first test works, our TestNG framework guide covers groups, data providers, dependencies and parallel runs.

What has changed since this tutorial was first written

The first version of this tutorial, from 2016, used Selenium 2.53.0 with Firefox 45. You downloaded a jar by hand, added it to the Eclipse build path and kept it matched to one browser version. That setup does not work with any current browser.

Two changes make the modern version shorter. Maven downloads Selenium and TestNG from two lines in a file. And since Selenium 4.6, a tool called Selenium Manager ships with Selenium. Its documentation says it "automatically discovers, downloads, and caches the drivers required by Selenium". You no longer download ChromeDriver or GeckoDriver, and you no longer set a driver path.

Step 1: Create the Maven project

Make a folder called first-testng-test. Inside it, create the folders src/test/java/io/qable/firsttest. Maven expects test code under src/test/java, and the rest of the path is your package name.

In the project folder, create a file called pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0">
  <modelVersion>4.0.0</modelVersion>
  <groupId>io.qable</groupId>
  <artifactId>first-testng-test</artifactId>
  <version>1.0</version>

  <properties>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <surefire.version>3.5.6</surefire.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>4.51.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>7.12.0</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${surefire.version}</version>
      </plugin>
    </plugins>
  </build>
</project>

Three parts matter. The two dependencies bring in Selenium and TestNG, both with test scope because only your tests use them. The Surefire plugin is what runs tests when you type mvn test, and it detects TestNG automatically. The release property compiles for Java 17, so any JDK from 17 upwards will build it.

We pinned Surefire to 3.5.6 for a reason covered later. For this project, the newest release, 3.6.1, ran the same three tests with the same results.

Step 2: Write your first TestNG class

Create LoginTest.java in src/test/java/io/qable/firsttest. It tests the login page of the-internet, a practice site published by Sauce Labs for exactly this purpose. Its username and password are printed on the page, so you can run this test yourself.

package io.qable.firsttest;

import org.openqa.selenium.By;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class LoginTest {

    private WebDriver driver;

    @BeforeMethod
    public void openBrowser() {
        // Selenium Manager finds or downloads the right driver: no driver path, no jar matching.
        String browser = System.getProperty("browser", "chrome");
        if (browser.equals("firefox")) {
            FirefoxOptions options = new FirefoxOptions();
            options.addArguments("-headless");
            driver = new FirefoxDriver(options);
        } else {
            ChromeOptions options = new ChromeOptions();
            options.addArguments("--headless=new");
            driver = new ChromeDriver(options);
        }
        Capabilities caps = ((RemoteWebDriver) driver).getCapabilities();
        System.out.println("Browser: " + caps.getBrowserName() + " " + caps.getBrowserVersion());
        driver.get("https://the-internet.herokuapp.com/login");
    }

    @Test
    public void validLoginOpensTheSecureArea() {
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("SuperSecretPassword!");
        driver.findElement(By.cssSelector("button[type='submit']")).click();

        Assert.assertTrue(driver.getCurrentUrl().endsWith("/secure"), "should land on /secure");
        Assert.assertTrue(driver.findElement(By.id("flash")).getText().contains("You logged into a secure area!"));
    }

    @Test
    public void wrongPasswordShowsAnError() {
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("wrong-password");
        driver.findElement(By.cssSelector("button[type='submit']")).click();

        Assert.assertTrue(driver.getCurrentUrl().endsWith("/login"), "should stay on /login");
        Assert.assertTrue(driver.findElement(By.id("flash")).getText().contains("Your password is invalid!"));
    }

    @Test
    public void flashMessageMatchesExactly() {
        // Written to show a real failure: the visible message carries more than the words.
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("SuperSecretPassword!");
        driver.findElement(By.cssSelector("button[type='submit']")).click();

        Assert.assertEquals(driver.findElement(By.id("flash")).getText(), "You logged into a secure area!");
    }

    @AfterMethod
    public void closeBrowser() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Here is what each part does:

• @BeforeMethod: runs before every test. It starts a new headless browser, prints the browser name and version, and opens the login page.
• @Test: marks a test. TestNG finds these methods itself, so you never call them.
• @AfterMethod: runs after every test, even a failed one. It closes the browser.
• Assert: TestNG's assertion class. If an assertion fails, the test fails and the report shows the expected and the actual value.

The browser is chosen with a system property, so the same code runs in Chrome or Firefox. Headless mode runs the browser without a window, which is what a CI server needs. Remove the headless line if you want to watch the test run.

Two details in that setup catch beginners out. First, the headless switch is spelled differently for each browser: Chrome takes --headless=new, while Firefox takes -headless, with a single dash, as in our code. Second, System.getProperty("browser", "chrome") reads a value you pass on the Maven command line, and falls back to Chrome when you pass nothing. Surefire hands that value to the separate Java process that runs your tests, which is why our Firefox runs printed Browser: firefox 159.0 without any change to the code. The Browser: line itself is worth keeping in your own tests. When a result differs between two machines, the first question is which browser and version ran, and the log already answers it.

Two tests should pass. The third is written to fail, so you can see what a real failure looks like before you meet one in your own work.

Choosing the right setup annotation

@BeforeMethod is one of four setup annotations. They differ only in how often they run, and the TestNG documentation defines each one. Each has a matching @After annotation that runs at the same level when the work is done.

Start with @BeforeMethod for the browser. Sharing one browser across a class looks faster, but one test's leftover state, such as a logged-in session, can then decide whether the next test passes.

Step 3: Run the test with Maven

Open a terminal in the project folder and run:

mvn test

The first run takes longer, because Maven downloads Selenium and TestNG and Selenium Manager downloads the browser driver. This is the test section of our Chrome run, with the Java stack trace and Maven's blank lines removed and nothing else changed:

[INFO] Running io.qable.firsttest.LoginTest
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Browser: chrome 155.0.8059.85
Browser: chrome 155.0.8059.85
Browser: chrome 155.0.8059.85
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.24 s <<< FAILURE! -- in io.qable.firsttest.LoginTest
[ERROR] io.qable.firsttest.LoginTest.flashMessageMatchesExactly -- Time elapsed: 0.673 s <<< FAILURE!
java.lang.AssertionError: 
expected [You logged into a secure area!] but found [You logged into a secure area!
×]
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURE

The three SLF4J warnings are harmless. They mean Selenium found no logging library, so its own log messages are switched off.

To run the same tests in Firefox, pass the browser property:

mvn test -Dbrowser=firefox

Firefox gave identical results: 3 tests run, 1 failure, 20.9 seconds in total. On our very first Firefox attempt, though, the browser refused to start, and every test was skipped with a SessionNotCreatedException. We changed nothing and ran it again. It started, and it started on every run after that. If a session will not start, re-run once before you change your code.

Run one test or one class

While you write a test, you rarely want the whole suite. Surefire's test property picks a class, or a single method after a hash sign:

mvn test -Dtest=LoginTest#validLoginOpensTheSecureArea

Our run of that command opened one browser and ran one test:

[INFO] Running io.qable.firsttest.LoginTest
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Browser: chrome 155.0.8059.85
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.266 s -- in io.qable.firsttest.LoginTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

Use -Dtest=LoginTest for the whole class. The same property works in a CI job, which makes it the quickest way to re-run a test that failed overnight.

Step 4: Read the failure

TestNG reports a failed assertion with both values. This is what our failing test printed:

java.lang.AssertionError: 
expected [You logged into a secure area!] but found [You logged into a secure area!
×]

The two values look the same, but the found value has a second line. The flash message on the page contains a close link, shown as a multiplication sign, and getText() returns all the visible text inside the element, the link included. An exact comparison therefore fails, even though the login worked.

This is the most useful lesson in a first test. The page was right and the assertion was wrong. You have two good fixes: check for the words with contains(), as the first test does, or read the text from a narrower element.

Fix the failing test

The login page gives the message and the close link no separate elements, so here the fix is to compare only the first line of the text:

    @Test
    public void flashMessageFirstLineMatchesExactly() {
        driver.get("https://the-internet.herokuapp.com/login");
        driver.findElement(By.id("username")).sendKeys("tomsmith");
        driver.findElement(By.id("password")).sendKeys("SuperSecretPassword!");
        driver.findElement(By.cssSelector("button[type='submit']")).click();

        // The close link adds a second line; compare only the message line.
        String message = driver.findElement(By.id("flash")).getText().lines().findFirst().orElse("");
        Assert.assertEquals(message, "You logged into a secure area!");
    }

This version passed in both of our runs. It still checks the exact wording, which a contains() check does not, so a typo in the message would still fail the test.

Whichever fix you choose, prove the test can still fail. Put a deliberate typo in the expected text, run it once, see it go red, then remove the typo. A test that cannot fail protects nothing.

Step 5: Find the test reports

When Maven runs TestNG, the reports go to target/surefire-reports. Open emailable-report.html in a browser for a one-page summary, or index.html for the full report.

testng-failed.xml is the file to remember. It is a ready-made suite that re-runs only the tests that failed.

Re-run only the tests that failed

This is the testng-failed.xml TestNG wrote after our Chrome run:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Failed suite [Surefire suite]" verbose="0">
  <test thread-count="5" name="Surefire test(failed)" verbose="0">
    <classes>
      <class name="io.qable.firsttest.LoginTest">
        <methods>
          <include name="flashMessageMatchesExactly"/>
          <include name="closeBrowser"/>
          <include name="openBrowser"/>
        </methods>
      </class> <!-- io.qable.firsttest.LoginTest -->
    </classes>
  </test> <!-- Surefire test(failed) -->
</suite> <!-- Failed suite [Surefire suite] -->

It lists the failed test and the setup and teardown methods that test needs, so the browser still opens and closes around it. To use it, copy it out of target, which Maven clears on the next build, and pass it to Surefire:

mvn test -Dsurefire.suiteXmlFiles=testng-failed.xml

Our run of that command ran one test instead of three:

[INFO] Running TestSuite
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Browser: chrome 155.0.8059.85
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.716 s <<< FAILURE! -- in TestSuite
[ERROR] io.qable.firsttest.LoginTest.flashMessageMatchesExactly -- Time elapsed: 0.832 s <<< FAILURE!
java.lang.AssertionError: 
expected [You logged into a secure area!] but found [You logged into a secure area!
×]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO] BUILD FAILURE

The test failed again, because we had not fixed it yet. That is the point: once you have a fix, this file lets you confirm it in seconds, without waiting for the whole suite. Run the full suite once more before you merge, because a fix for one test can break another.

Once you have more than one TestNG test case, you can also write your own testng.xml to decide which tests run together.

Running the same test in Eclipse

You can run everything above without an IDE. If you prefer Eclipse, install the TestNG plugin from the Eclipse Marketplace first. The screenshots below come from the first version of this tutorial. The dialogs are the same in current Eclipse, but the versions and package names in them are from 2016.

The simplest route is to import the Maven project, with File, then Import, then Existing Maven Projects. Your pom.xml brings in Selenium and TestNG, so you can skip the first two steps below.

Create a Java project in Eclipse

Without Maven, you start from File, then New, then Java Project, and give the project a name.

Add the TestNG library to the build path

Right-click the project and choose Build Path, then Add Libraries, then TestNG. This adds TestNG only. You would still need to add Selenium yourself, which is why Maven is the better route.

Add a TestNG class

Right-click your package and choose New, then Other, then TestNG class.

Choose @BeforeMethod and @AfterMethod

In the dialog, name the class and tick the @BeforeMethod and @AfterMethod boxes. Eclipse generates an empty class with those methods, which you can fill in with the code from step 2.

Run as TestNG Test

Right-click the class or the project and choose Run As, then TestNG Test. Results appear in the TestNG view at the bottom of the window.

Open the report in test-output

A run from Eclipse writes its report to a test-output folder, which the TestNG documentation gives as the default report directory. Right-click the HTML report and open it with the web browser.

In IntelliJ IDEA, add TestNG as a Maven dependency and the IDE runs TestNG tests from the green arrow next to each class and method.

Mistakes to avoid in your first TestNG test

The first version of this tutorial created the driver as a field: WebDriver driver = new FirefoxDriver();. It then closed that driver in @AfterMethod. That works for exactly one test. We copied the pattern into a class with two tests and ran it:

[ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 4.982 s <<< FAILURE! -- in io.qable.firsttest.FieldInitTest
[ERROR] io.qable.firsttest.FieldInitTest.open -- Time elapsed: 0.319 s <<< FAILURE!
org.openqa.selenium.NoSuchSessionException: 
Session ID is null. Using WebDriver after calling quit()?

Surefire counts the setup and teardown methods too, so it reports 4 run and 2 skipped. TestNG's own report shows the two tests: 1 passed and 1 skipped.

The first test passed and closed the browser. TestNG reuses the same class instance for every test, so @BeforeMethod for the second test called a driver that had already quit. TestNG then skipped the second test. Create the driver inside @BeforeMethod, as step 2 does, and every test gets a fresh browser.

The earlier version also paused with Thread.sleep(1000). A fixed pause slows every run and still fails when the page is slower than the pause. Our login tests need no pauses: Selenium waits for each page to finish loading after get() and after the login click, so each element is already there when findElement looks for it.

Waiting for content that loads later

Content that appears after the page has loaded is different. The practice site's dynamic loading page shows "Hello World!" a few seconds after you press Start. We wrote two tests against it: one that looks straight away, and one that uses WebDriverWait.

    @Test
    public void waitForContentThatLoadsLater() {
        driver.get("https://the-internet.herokuapp.com/dynamic_loading/2");
        driver.findElement(By.cssSelector("#start button")).click();

        long start = System.nanoTime();
        WebElement finish = new WebDriverWait(driver, Duration.ofSeconds(10))
                .until(ExpectedConditions.visibilityOfElementLocated(By.id("finish")));
        System.out.println("Waited " + (System.nanoTime() - start) / 1_000_000 + " ms");

        Assert.assertEquals(finish.getText(), "Hello World!");
    }

    @Test
    public void lookingTooSoonFails() {
        driver.get("https://the-internet.herokuapp.com/dynamic_loading/2");
        driver.findElement(By.cssSelector("#start button")).click();

        // No wait: the element does not exist yet, so this throws NoSuchElementException.
        driver.findElement(By.id("finish"));
    }

The test without a wait failed with NoSuchElementException in both of our runs. The test with a wait passed. This is the line it printed, then the other test's failure, from our first run:

Waited 5176 ms
[ERROR] io.qable.firsttest.NextStepsTest.lookingTooSoonFails -- Time elapsed: 3.362 s <<< FAILURE!
org.openqa.selenium.NoSuchElementException: 
no such element: Unable to locate element: {"method":"css selector","selector":"#finish"}

The wait returned as soon as the text appeared, about five seconds after the click. Thread.sleep(1000) would have failed here, and Thread.sleep(10000) would have wasted five seconds on every run. Set the wait's limit to the longest delay you will accept, and let it return early.

A note on Maven Surefire versions

Maven Surefire 3.6 changed how it runs TestNG. On our runs, 3.6.1 handled this tutorial's project without a problem, because the project has no suite file. Once you add a testng.xml and list it in the Surefire configuration, 3.6.1 stops the build with an error. Our TestNG framework guide shows the error and the fix.

That is why the pom.xml above pins 3.5.6. It costs nothing now and saves you a confusing failure later. When you outgrow this pin, you can pass a different version on the command line, as we did: mvn test -Dsurefire.version=3.6.1.

Where this test sits among test levels

The ISTQB Certified Tester Foundation Level syllabus, version 4.0.1, separates tests by level. It describes component testing, "also known as unit testing", as testing "components in isolation", and notes that it "often requires specific support, such as test harnesses or unit test frameworks". System testing, by contrast, "focuses on the overall behavior and capabilities of an entire system or product, often including functional testing of end-to-end tasks".

The login test in this tutorial is a system-level test: it drives a real browser against a deployed site. TestNG's own site says it is "designed to cover all categories of tests: unit, functional, end-to-end, integration", so one framework runs both kinds. Keep the levels in separate classes or groups. Fast component tests can then run on every commit, and browser tests can run where the extra time is worth it.

Where to go next

With one class working, the next steps are the features TestNG is known for:

• Groups: tag tests as smoke or regression and run one group at a time.
• Data providers: run one test with many sets of input data.
• Parallel runs: run tests on several threads to cut suite time.
• testng.xml: decide which classes and groups run together.

Our TestNG framework guide covers each of these with tested code. Keep the habits from this tutorial as the suite grows: one browser per test, assertions that check what matters, and versions pinned in pom.xml.

If you want help building a TestNG and Selenium suite that stays reliable as it grows, talk to QAble's automation testing services team.