DS DevShelfHub Projects · AI tools
Cheatsheets / TestNG
Cheatsheet · Dev tooling

TestNG Cheatsheet: Annotations, Suites and DataProviders Reference

By DevShelfHub

Annotations, suites, groups, data providers, dependencies, listeners, parallel — the Java testing framework as a one-page reference.

105 items 8 min Suites Groups DataProviders

Start hereQuick start · 6 you’ll reach for daily

Test method@Test public void testX()
Data-driven@Test(dataProvider="rows")
Group filter@Test(groups={"smoke"})
Setup hook@BeforeClass void setUp()
AssertAssert.assertEquals(actual, exp)
Suitetestng.xml · parallel="methods"

Target versions · paceVersions

Targets: testng ≥ 7.10 java ≥ 11 maven-surefire ≥ 3.2

TestNG 7 dropped Java 8 long ago — pin to Java 11+ everywhere. @BeforeMethod / @AfterMethod / @DataProvider + group filters from testng.xml are the defining feature vs JUnit — lean on them. Listeners auto-discover via META-INF/services/org.testng.ITestNGListener. Soft assertions live in org.testng.asserts.SoftAssert. This sheet pins to TestNG 7.10+.

Maven · Gradle · runSetup

bash


  org.testng
  testng
  7.10.2
  test




  maven-surefire-plugin
  3.2.5
  
    
      src/test/resources/testng.xml
    
  


# Gradle — build.gradle.kts
# testImplementation("org.testng:testng:7.10.2")
# tasks.test { useTestNG { suiteXmlFiles += file("testng.xml") } }

# Run
mvn test                                    # all tests in the suite
mvn test -Dgroups="smoke,api"               # group filter
mvn test -Dtest=LoginTest#testValid         # single method
mvn test -DsuiteXmlFile=integration.xml     # alternate suite
gradle test --tests com.acme.LoginTest

Where things liveCommon imports

import org.testng.annotations.*Test, Before*, After*, DataProvider, Parameters, Listeners.
import org.testng.AssertStatic asserts. assertEquals(actual, expected) — argument order!
import org.testng.asserts.SoftAssertCollect failures, fail at end (.assertAll()).
import org.testng.ITestListener, IRetryAnalyzer, IAnnotationTransformerListener interfaces.
import org.testng.SkipExceptionSkip from inside a running test (e.g. precondition failed).
import org.testng.ITestContext, ITestResultRuntime introspection inside hooks / listeners.
import org.assertj.core.api.Assertions / org.hamcrest.MatcherAssertPrefer AssertJ or Hamcrest for fluent assertions on objects.

@Test · lifecycle hooksAnnotations & lifecycle

Test methods

@Test public void testFoo()A test method. Public, void, no args (or use data provider).
@Test(description="...", priority=1)Description appears in reports. Lower priority runs first.
@Test(groups={"smoke","api"})Tag for selective runs.
@Test(enabled=false)Skip permanently — better than commenting out.
@Test(timeOut=5000)Per-test timeout in ms.
@Test(invocationCount=10, threadPoolSize=4)Repeat in parallel — load / flake hunting.
@Test(expectedExceptions=IllegalStateException.class, expectedExceptionsMessageRegExp=".*closed.*")Assert thrown exception.
@Test(dependsOnMethods="login")Order + skip cascade if predecessor fails.
@Test(dependsOnGroups="auth.*")Depend on a regex group.
@Test(retryAnalyzer=RetryAnalyzer.class)Retry transient failures via IRetryAnalyzer.

Lifecycle hooks

@BeforeSuite / @AfterSuiteOnce per testng.xml suite.
@BeforeTest / @AfterTestPer <test> tag in the suite XML.
@BeforeGroups("smoke") / @AfterGroupsAround all methods of a group.
@BeforeClass / @AfterClassOnce per class instance.
@BeforeMethod / @AfterMethodBefore / after each @Test method.
alwaysRun=trueRun the hook even when dependsOn short-circuits the chain.
@FactoryProgrammatically instantiate test classes with parameters.

Hard, soft, AssertJAssertions

Assert.assertEquals(actual, expected, message)Note the argument order is actual, expected — opposite of JUnit.
Assert.assertTrue / assertFalse / assertNull / assertNotNullBasic conditions.
Assert.assertEqualsNoOrder(actualArr, expectedArr)Array compare ignoring order.
Assert.assertThrows(Err.class, () -> ...)Lambda-style exception check.
Assert.fail("explain")Force failure with a message.
SoftAssert s = new SoftAssert(); s.assertEquals(...); s.assertAll();Collect multiple failures; only blow up at the end.
throw new SkipException("precondition failed")Skip dynamically from inside the test.
Assertions.assertThat(list).containsExactly("a","b")AssertJ — fluent, readable. Preferred for complex objects.
assertThat(map, hasEntry("k", "v"))Hamcrest matchers.

Parameterized testsData providers

@DataProvider(name="rows") public Object[][] rows()Static table. Inner arrays match the test signature.
@Test(dataProvider="rows") public void it(String s, int n)Wire the provider in. One run per row.
@DataProvider(parallel=true)Run rows in parallel — per-method threads.
dataProviderClass = TestData.classReuse a provider from another class.
return Iterator<Object[]>Lazy / large datasets — rows generated on demand.
indices = {0, 2, 4}Pick subset of rows (7.6+).
first arg: ITestContext / MethodTestNG injects context when these are the first parameter of a provider.
@Parameters({"baseUrl","apiKey"})Inject values from <parameter> tags in testng.xml.
javascript
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;

public class BillingTest {

    // Static, in-class data table
    @DataProvider(name = "plans")
    public Object[][] plans() {
        return new Object[][] {
            { "free", 0 },
            { "pro", 10 },
            { "max", 50 },
        };
    }

    // Use it — TestNG matches arity / types
    @Test(dataProvider = "plans", groups = {"smoke"})
    public void feeForPlan(String plan, int expected) {
        assertEquals(Billing.feeFor(plan), expected);
    }

    // Parallel data-provider — N parametrized runs in parallel
    @DataProvider(name = "skus", parallel = true)
    public Object[][] skus() {
        return new Object[][] { {"a"}, {"b"}, {"c"}, {"d"} };
    }

    // Cross-class: dataProviderClass points to a static method elsewhere
    @Test(dataProvider = "users", dataProviderClass = TestData.class)
    public void roleIsAssigned(User u, Role r) { /* ... */ }

    // Stream-friendly: return Iterator for big or lazy datasets
    @DataProvider(name = "csv")
    public java.util.Iterator rows() throws Exception {
        return CsvLoader.load("users.csv").iterator();
    }
}

Group expressions · dependsOnGroups & dependencies

@Test(groups={"smoke","api"})Attach multiple tags.
@Test(groups={"smoke"}, dependsOnGroups={"login.*"})Run after every login.* group passes.
dependsOnMethods={"setUpUser"}Direct method dependency.
alwaysRun=trueHonor failed dependencies — useful for cleanup.
@BeforeGroups("smoke")Per-group fixture — runs once before any test in the group.
testng.xml <groups><run><include name="smoke"/>Filter at the suite level.
-Dgroups="smoke,!flaky"Maven surefire CLI filter.
priority = 1Lower-priority methods first within the same group. Not a global order.

Declarative test runsSuite XML

<suite name="..." parallel="methods" thread-count="4">Parallel modes: none, methods, classes, instances, tests.
<test name="...">A group of classes / packages sharing config.
<packages><package name="com.acme.tests"/>Discover by package.
<classes><class name="..."/>Discover by class.
<groups><run><include name="..."/><exclude name="..."/>Filter by group.
<parameter name="..." value="..."/>Wires into @Parameters-annotated methods.
<listeners><listener class-name="..."/>Programmatic alternative to @Listeners.
preserve-order="false"Allow TestNG to reorder within a <test>.
configfailurepolicy="continue"Keep running tests when a config method fails.
data-provider-thread-count="4"Parallel size for parallel=true data providers.
markdown






  
  
    
    
  

  
  

  
    
      
        
        
        
      
    
    
      
    
  

  
    
      
      
        
                    
        
      
    
  

Hook into the runnerListeners & hooks

implements ITestListenerPer-test events: onTestStart, onTestSuccess, onTestFailure, onTestSkipped.
implements ISuiteListenerSuite-level events.
implements IRetryAnalyzerPer-test retry logic — return true while you want a retry.
implements IAnnotationTransformerMutate annotation values at runtime (e.g. attach a retry analyzer to every @Test).
implements IInvokedMethodListenerWrap @Before* / @After* too, not just tests.
implements IExecutionListenerWhole-execution start / end — spin up a Grid / Docker once per run.
@Listeners({MyListener.class, RetryListener.class})Attach via annotation. testng.xml + SPI also work.
META-INF/services/org.testng.ITestNGListenerAuto-discovery file — one FQCN per line.
Reporter.log("...", true)Attach a log line to the HTML report.

Threads · CI · retriesParallel & CI

parallel="methods" thread-count="4"Threads share state — make tests stateless or use ThreadLocal.
parallel="classes"One thread per class. Safer when methods share fields.
parallel="instances"One thread per @Factory instance.
parallel="tests"One thread per <test> tag — biggest unit of isolation.
@Test(threadPoolSize=N, invocationCount=M)Run one method M times across N threads.
timeOut="60000" (suite or test)Cap individual or suite duration in ms.
surefire-report.xml / testng-results.xmlCI-readable outputs — pair with Jenkins / GitHub annotations.
org.testng.reporters.XMLReporter / EmailableReporter2Built-in reporters. Allure / ReportPortal plug in via listeners.
mvn -Dthreadcount=8 -Dgroups=smokeOverride at the surefire level on CI.

UI · mobile · GridTestNG + Selenium

@BeforeClass void setUp() { driver = new ChromeDriver(...); }One browser per class. parallel="classes" keeps state safe.
ThreadLocal<WebDriver> driver = new ThreadLocal<>()When using parallel="methods" with one driver per thread.
@AfterClass(alwaysRun=true) void tearDown() { driver.quit(); }Always close — orphan browsers OOM the runner.
@Parameters({"browser"}) before driver new-upPick browser per <test> tag from testng.xml.
@Listeners(RetryListener.class) + IRetryAnalyzerRetry flaky UI tests once. Track flake rate — don’t hide it.
ScreenshotOnFailureListener (custom)onTestFailure((TakesScreenshot) driver).getScreenshotAs(...) → attach to report.
RemoteWebDriver + Grid hub URLRun UI tests across a Selenium Grid.
Allure @Step / @AttachmentStep-by-step report + screenshots / traces.

Login flow · data-drivenEnd-to-end · LoginTest

Class-level Chrome lifecycle, data-provider for credentials, group + dependency to wire a downstream test.

javascript
package com.acme.tests;

import org.testng.annotations.*;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import static org.testng.Assert.*;

public class LoginTest {

    private WebDriver driver;

    @BeforeClass(alwaysRun = true)
    public void setUpBrowser() {
        ChromeOptions opts = new ChromeOptions();
        opts.addArguments("--headless=new", "--window-size=1280,800");
        driver = new ChromeDriver(opts);
    }

    @AfterClass(alwaysRun = true)
    public void tearDown() {
        if (driver != null) driver.quit();           // always clean up
    }

    @DataProvider(name = "users")
    public Object[][] users() {
        return new Object[][] {
            { "ana@x.com", "secret", true  },
            { "bob@x.com", "wrong",  false },
        };
    }

    @Test(groups = {"smoke"}, dataProvider = "users",
          description = "Login with valid + invalid creds")
    public void canLogIn(String email, String pw, boolean ok) {
        driver.get("https://example.com/login");
        new LoginPage(driver).fill(email, pw).submit();
        assertEquals(driver.getCurrentUrl().contains("/dashboard"), ok);
    }

    @Test(dependsOnMethods = "canLogIn", groups = {"regression"})
    public void canViewDashboard() {
        assertTrue(driver.getPageSource().contains("Welcome"));
    }
}

Best practiceGood to know

Remember the (actual, expected) order in Assert.assertEquals. TestNG’s order is the opposite of JUnit. Swapping them flips the diff in the report — the test still passes / fails correctly, but the message is misleading. Migrating teams burn an afternoon on this.
Use alwaysRun=true on cleanup hooks. Without it, @AfterClass is skipped when a @BeforeClass or dependent test fails — leaving DB rows, browsers, or processes around.
Filter groups in testng.xml, not by commenting out @Test. Mark each test with groups={"smoke","regression"} once, then build suites for each CI lane. Same code, different filter — no risk of forgetting a commented-out test.

Common trapsWatch out for

parallel="methods" shares one class instance across threads. Mutable instance fields race. Use ThreadLocal for the driver / state, or switch to parallel="classes" / "instances" if your tests assume isolation.
priority is a sort, not a dependency. Priority orders tests within the same instance but doesn’t guarantee execution — failures don’t cascade. Use dependsOnMethods / dependsOnGroups for real ordering.
SoftAssert without .assertAll() silently passes. Collected failures only fire when assertAll() is called. Forget it and you get a green test that should be red. Put assertAll() at the end — or call it from an @AfterMethod on a shared SoftAssert.

Go deeperSee also

TestNG FAQ

What is TestNG used for?

TestNG is a Java testing framework inspired by JUnit and NUnit but with additional features for enterprise testing. It supports test suites, parallel execution, data-driven tests via @DataProvider, dependency ordering, grouping, and flexible configuration through testng.xml. It is widely used for API, integration, and Selenium UI test automation.

What is the difference between TestNG and JUnit?

TestNG and JUnit 5 are both mature Java testing frameworks. TestNG has built-in suite XML configuration, native parallel test support, and @DataProvider for parameterized tests without needing extensions. JUnit 5 has a more modular architecture with @ParameterizedTest and extensions. JUnit 5 is more common for unit tests; TestNG is popular in Selenium/API automation stacks.

How does @DataProvider work in TestNG?

@DataProvider annotates a method that returns an Object[][] (or Iterator<Object[]>) of input sets. Reference it from a test method with @Test(dataProvider='myProvider'). TestNG runs the test once per row, passing each array as arguments. Add parallel=true to the annotation to run data sets concurrently.

What is testng.xml used for?

testng.xml is the suite configuration file. It defines which test classes or packages to run, organizes them into suites and tests, sets thread counts for parallel execution, includes or excludes groups, and passes parameters to tests. Running mvn test or gradle test picks it up via the surefire/testng plugin configuration.

How does parallel test execution work in TestNG?

Set parallel='methods', 'classes', 'tests', or 'instances' in the <suite> tag of testng.xml, along with thread-count='N'. Methods parallelism runs individual @Test methods concurrently; tests parallelism runs each <test> block in its own thread. Ensure test methods are thread-safe — use ThreadLocal for WebDriver or database connections to avoid sharing state.

Is TestNG free and open source?

Yes. TestNG is Apache-2.0 licensed and free to use commercially. Add it as a test-scope dependency via Maven Central (org.testng:testng). No subscription is required.