Order of execution of tests in TestNG
TestingTestngOrder of-ExecutionTesting Problem Overview
How to customize the order of execution of tests in TestNG?
For example:
public class Test1 {
@Test
public void test1() {
System.out.println("test1");
}
@Test
public void test2() {
System.out.println("test2");
}
@Test
public void test3() {
System.out.println("test3");
}
}
In the above suite, the order of execution of tests is arbitrary. For one execution the output may be:
test1
test3
test2
How do I execute the tests in the order in which they've been written?
Testing Solutions
Solution 1 - Testing
This will work.
@Test(priority=1)
public void Test1() {
}
@Test(priority=2)
public void Test2() {
}
@Test(priority=3)
public void Test3() {
}
priority
encourages execution order but does not guarantee the previous priority level has completed. test3
could start before test2
completes. If a guarantee is needed, then declare a dependency.
Unlike the solutions which declare dependencies, tests which use priority
will execute even if one test fails. This problem with dependencies can be worked around with @Test(...alwaysRun = true...)
according to documentation.
Solution 2 - Testing
In TestNG, you use dependsOnMethods and/or dependsOnGroups:
@Test(groups = "a")
public void f1() {}
@Test(groups = "a")
public void f2() {}
@Test(dependsOnGroups = "a")
public void g() {}
In this case, g() will only run after f1() and f2() have completed and succeeded.
You will find a lot of examples in the documentation: http://testng.org/doc/documentation-main.html#test-groups
Solution 3 - Testing
To address specific scenario in question:
@Test
public void Test1() {
}
@Test (dependsOnMethods={"Test1"})
public void Test2() {
}
@Test (dependsOnMethods={"Test2"})
public void Test3() {
}
Solution 4 - Testing
@Test(dependsOnMethods="someBloodyMethod")
Solution 5 - Testing
If you don't want to use the @Test(priority = )
option in TestNG, you can make use of the javaassist library and TestNG's IMethodInterceptor
to prioritize the tests according to the order by which the test methods are defined in the test class. This is based on the solution provided here.
Add this listener to your test class:
package cs.jacob.listeners;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
import javassist.NotFoundException;
import org.testng.IMethodInstance;
import org.testng.IMethodInterceptor;
import org.testng.ITestContext;
public class PriorityInterceptor implements IMethodInterceptor {
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
Comparator<IMethodInstance> comparator = new Comparator<IMethodInstance>() {
private int getLineNo(IMethodInstance mi) {
int result = 0;
String methodName = mi.getMethod().getConstructorOrMethod().getMethod().getName();
String className = mi.getMethod().getConstructorOrMethod().getDeclaringClass().getCanonicalName();
ClassPool pool = ClassPool.getDefault();
try {
CtClass cc = pool.get(className);
CtMethod ctMethod = cc.getDeclaredMethod(methodName);
result = ctMethod.getMethodInfo().getLineNumber(0);
} catch (NotFoundException e) {
e.printStackTrace();
}
return result;
}
public int compare(IMethodInstance m1, IMethodInstance m2) {
return getLineNo(m1) - getLineNo(m2);
}
};
IMethodInstance[] array = methods.toArray(new IMethodInstance[methods.size()]);
Arrays.sort(array, comparator);
return Arrays.asList(array);
}
}
This basically finds out the line numbers of the methods and sorts them by ascending order of their line number, i.e. the order by which they are defined in the class.
Solution 6 - Testing
>Use this:
public class TestNG
{
@BeforeTest
public void setUp()
{
/*--Initialize broowsers--*/
}
@Test(priority=0)
public void Login()
{
}
@Test(priority=2)
public void Logout()
{
}
@AfterTest
public void tearDown()
{
//--Close driver--//
}
}
Usually TestNG provides number of annotations, We can use @BeforeSuite, @BeforeTest, @BeforeClass
for initializing browsers/setup.
We can assign priority if you have written number of test cases in your script and want to execute as per assigned priority then use:
@Test(priority=0)
starting from 0,1,2,3....
Meanwhile we can group number of test cases and execute it by grouping.
for that we will use @Test(Groups='Regression')
At the end like closing the browsers we can use @AfterTest, @AfterSuite, @AfterClass
annotations.
Solution 7 - Testing
If I understand your question correctly in that you want to run tests in a specified order, TestNG IMethodInterceptor can be used. Take a look at http://beust.com/weblog2/archives/000479.html on how to leverage them.
If you want run some preinitialization, take a look at IHookable http://testng.org/javadoc/org/testng/IHookable.html and associated thread http://groups.google.com/group/testng-users/browse_thread/thread/42596505990e8484/3923db2f127a9a9c?lnk=gst&q=IHookable#3923db2f127a9a9c
Solution 8 - Testing
Piggy backing off of user1927494's answer, In case you want to run a single test before all others, you can do this:
@Test()
public void testOrderDoesntMatter_1() {
}
@Test(priority=-1)
public void testToRunFirst() {
}
@Test()
public void testOrderDoesntMatter_2() {
}
Solution 9 - Testing
By Specifying test methods to be executed in testNg.xml we can execute the test cases in desired order
<suite>
<test name="selenium1">
<classes>
<class name="com.test.SeleniumTest" >
<methods><include name="methodB"></include>
<include name="methodA"></include>
</methods>
</class>
</classes>
</test>
</suite>
Solution 10 - Testing
By using priority paramenter for @Test we can control the order of test execution.
Solution 11 - Testing
The ordering of methods in the class file is unpredictable, so you need to either use dependencies or include your methods explicitly in XML.
By default, TestNG will run your tests in the order they are found in the XML file. If you want the classes and methods listed in this file to be run in an unpredictible order, set the preserve-order attribute to false
Solution 12 - Testing
I've faced the same issue, the possible reason is due to parallel execution of testng and the solution is to add Priority option or simply update preserve-order="true" in your testng.xml.
<test name="Firefox Test" preserve-order="true">
Solution 13 - Testing
An answer with an important explanation:
There are two parameters of "TestNG" who are supposed to determine the order of execution the tests:
@Test(dependsOnGroups= "someGroup")
And:
@Test(dependsOnMethods= "someMethod")
In both cases these functions will depend on the method or group,
But the differences:
In this case:
@Test(dependsOnGroups= "someGroup")
The method will be dependent on the whole group, so it is not necessarily that immediately after the execution of the dependent function, this method will also be executed, but it may occur later in the run and even after other tests run.
It is important to note that in the case and there is more than one use within the same set of tests in this parameter, this is a safe recipe for problems because the dependent methods of the entire set of tests will run first and only then the methods that depend on them.
However, in this case:
@Test(dependsOnMethods= "someMethod")
Even if this parameter is used more than once within the same set of tests, the dependent method will still be executed after the dependent method is executed immediately.
Hope it's clear and helps.
Solution 14 - Testing
There are ways of executing tests in a given order. Normally though, tests have to be repeatable and independent to guarantee it is testing only the desired functionality and is not dependent on side effects of code outside of what is being tested.
So, to answer your question, you'll need to provide more information such as WHY it is important to run tests in a specific order.
Solution 15 - Testing
In case you happen to use additional stuff like dependsOnMethods
, you may want to define the entire @Test flow in your testng.xml file. AFAIK, the order defined in your suite XML file (testng.xml) will override all other ordering strategies.
Solution 16 - Testing
use: preserve-order="true" enabled="true" that would run test cases in the manner in which you have written.
<suite name="Sanity" verbose="1" parallel="" thread-count="">
<test name="Automation" preserve-order="true" enabled="true">
<listeners>
<listener class-name="com.yourtest.testNgListner.RetryListener" />
</listeners>
<parameter name="BrowserName" value="chrome" />
<classes>
<class name="com.yourtest.Suites.InitilizeClass" />
<class name="com.yourtest.Suites.SurveyTestCases" />
<methods>
<include name="valid_Login" />
<include name="verifyManageSurveyPage" />
<include name="verifySurveyDesignerPage" />
<include name="cloneAndDeleteSurvey" />
<include name="createAndDelete_Responses" />
<include name="previewSurvey" />
<include name="verifySurveyLink" />
<include name="verifySurveyResponses" />
<include name="verifySurveyReports" />
</methods>
</classes>
</test>
</suite>
Solution 17 - Testing
Tests like unit tests? What for? Tests HAVE to be independant, otherwise.... you can not run a test individually. If they are independent, why even interfere? Plus - what is an "order" if you run them in multiple threads on multiple cores?