TestNG Reporting Customization

27 Comments / TestNG Tutorial

We have introduced how to create TestNG default report Html and XML files in the article TestNG Report Example. But you may find it is not enough for you. Because the original reports generated by TestNG have some inconvenience, it is so basic. TestNG provide an IReporter interface that you can implement to create a test context listener. Then you can customize the report in its generateReport() method. This example will show you how to customize the TestNG report step by step.

Table of Contents

1. Default TestNG Generated Reports.

We will customize the default report generated in the article TestNG Report Example. Below are the testng.xml and test java files used in that article.

main-suite.xml
suite1.xml
suite2.xml

TestCaseClass1.java
TestCaseClass2.java
TestCaseClass3.java

TestCaseClass4.java
TestCaseClass5.java
TestCaseClass6.java

The default TestNG emailable report contains the test suites summary area and suite test class summary area, it has the below disadvantages.

The Time column is not user-friendly.

Do not provide test start time and end time.

We will customize it to the below report. It has below improvements.

Add Browser, Start Time, and End Time columns for test suite summary report.

Add Start Time, Parameter, Reporter Message, and Exception Message columns for test method summary report.
Convert Execution Time value to a more human-readable format.

2. Customize TestNG Report Steps.

customize-emailable-report-template.html : This is the template Html for customized reports. It contains Html layout and three customize TestNG report element place holder.

  <body>
    <table>
      <tr><center><b>$TestNG_Custom_Report_Title$</b></center></tr>
      <thead>
        <tr>
          <th>Test Name</th>
          <th># Total Method</th>
          <th># Passed</th>
          <th># Skipped</th>
          <th># Failed</th>
          <th>Browser</th>
          <th>Start Time</th>
          <th>End Time</th>
          <th>Execute Time (mm:hh:ss:ms)</th>
          <th>Included Groups</th>
          <th>Excluded Groups</th>
        </tr>
       </thead> 
       $Test_Case_Summary$
    </table>
    <table id="summary">
      <thead>
        <tr>
          <th>Class</th>
          <th>Method</th>
          <th>Start Time</th>
          <th>Execution Time (mm:hh:ss:ms)</th>
          <th>Parameter</th>
          <th>Reporter Message</th>
          <th>Exception Message</th>
        </tr>
      </thead> 
      $Test_Case_Detail$
    </table>
  </body>

main-suite.xml : Add test listener in this TestNG suite XML file. We will introduce the test listener later.

<listeners>
	<listener class-name="com.dev2qa.example.testng.report.custom.CustomTestNGReporter" />
  </listeners>

CustomTestNGReporter.java : This is the java class that implements the IReporter interface. It just needs to override the IReporter interface generateReport() method. You can see below code comments for more explanation. Besides you add the TestNG listener class in XML settings files, you can also add it in the java class use the below source code.

// Create an instance of the TestNG listener.
private CustomTestNGReporter listener = new CustomTestNGReporter();

// Create an instance of the TestNG.
TestNG testng = new TestNG();

// Add the TestNG listener to the TestNG object.
testng.addListener(listener);

Package : com.dev2qa.example.testng.report.custom

public class CustomTestNGReporter implements IReporter {
	
	//This is the customize emailabel report template file path.
	private static final String emailableReportTemplateFile = "C:/WorkSpace/dev2qa.com/TestNGExampleProject/src/com/dev2qa/example/testng/report/custom/customize-emailable-report-template.html";
	
	@Override
	public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
		
		try
		{
			// Get content data in TestNG report template file.
			String customReportTemplateStr = this.readEmailabelReportTemplate();
			
			// Create custom report title.
			String customReportTitle = this.getCustomReportTitle("Custom TestNG Report");
			
			// Create test suite summary data.
			String customSuiteSummary = this.getTestSuiteSummary(suites);
			
			// Create test methods summary data.
			String customTestMethodSummary = this.getTestMehodSummary(suites);
			
			// Replace report title place holder with custom title.
			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$TestNG_Custom_Report_Title\\$", customReportTitle);
			
			// Replace test suite place holder with custom test suite summary.
			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$Test_Case_Summary\\$", customSuiteSummary);
			
			// Replace test methods place holder with custom test method summary.
			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$Test_Case_Detail\\$", customTestMethodSummary);
			
			// Write replaced test report content to custom-emailable-report.html.
			File targetFile = new File(outputDirectory + "/custom-emailable-report.html");
			FileWriter fw = new FileWriter(targetFile);
			fw.write(customReportTemplateStr);
			fw.flush();
			fw.close();
			
		}catch(Exception ex)
		{
			ex.printStackTrace();
		}
	}
	
	/* Read template content. */
	private String readEmailabelReportTemplate()
	{
		StringBuffer retBuf = new StringBuffer();
		
		try {
		
			File file = new File(this.emailableReportTemplateFile);
			FileReader fr = new FileReader(file);
			BufferedReader br = new BufferedReader(fr);
			
			String line = br.readLine();
			while(line!=null)
			{
				retBuf.append(line);
				line = br.readLine();
			}
			
		} catch (FileNotFoundException ex) {
			ex.printStackTrace();
		}finally
		{
			return retBuf.toString();
		}
	}
	
	/* Build custom report title. */
	private String getCustomReportTitle(String title)
	{
		StringBuffer retBuf = new StringBuffer();
		retBuf.append(title + " " + this.getDateInStringFormat(new Date()));
		return retBuf.toString();
	}
	
	/* Build test suite summary data. */
	private String getTestSuiteSummary(List<ISuite> suites)
	{
		StringBuffer retBuf = new StringBuffer();
		
		try
		{
			int totalTestCount = 0;
			int totalTestPassed = 0;
			int totalTestFailed = 0;
			int totalTestSkipped = 0;
			
			for(ISuite tempSuite: suites)
			{
				retBuf.append("<tr><td colspan=11><center><b>" + tempSuite.getName() + "</b></center></td></tr>");
				
				Map<String, ISuiteResult> testResults = tempSuite.getResults();
				
				for (ISuiteResult result : testResults.values()) {
					
					retBuf.append("<tr>");
					
					ITestContext testObj = result.getTestContext();
					
					totalTestPassed = testObj.getPassedTests().getAllMethods().size();
					totalTestSkipped = testObj.getSkippedTests().getAllMethods().size();
					totalTestFailed = testObj.getFailedTests().getAllMethods().size();
					
					totalTestCount = totalTestPassed + totalTestSkipped + totalTestFailed;
					
					/* Test name. */
					retBuf.append("<td>");
					retBuf.append(testObj.getName());
					retBuf.append("</td>");
					
					/* Total method count. */
					retBuf.append("<td>");
					retBuf.append(totalTestCount);
					retBuf.append("</td&gt;");
					
					/* Passed method count. */
					retBuf.append("<td bgcolor=green>");
					retBuf.append(totalTestPassed);
					retBuf.append("</td>");
					
					/* Skipped method count. */
					retBuf.append("<td bgcolor=yellow>");
					retBuf.append(totalTestSkipped);
					retBuf.append("</td>");
					
					/* Failed method count. */
					retBuf.append("<td bgcolor=red>");
					retBuf.append(totalTestFailed);
					retBuf.append("</td>");
					
					/* Get browser type. */
					String browserType = tempSuite.getParameter("browserType");
					if(browserType==null || browserType.trim().length()==0)
					{
						browserType = "Chrome";
					}
					
					/* Append browser type. */
					retBuf.append("<td>");
					retBuf.append(browserType);
					retBuf.append("</td>");
					
					/* Start Date*/
					Date startDate = testObj.getStartDate();
					retBuf.append("<td>");
					retBuf.append(this.getDateInStringFormat(startDate));
					retBuf.append("</td>");
					
					/* End Date*/
					Date endDate = testObj.getEndDate();
					retBuf.append("<td>");
					retBuf.append(this.getDateInStringFormat(endDate));
					retBuf.append("</td>");
					
					/* Execute Time */
					long deltaTime = endDate.getTime() - startDate.getTime();
					String deltaTimeStr = this.convertDeltaTimeToString(deltaTime);
					retBuf.append("<td>");
					retBuf.append(deltaTimeStr);
					retBuf.append("</td>");
					
					/* Include groups. */
					retBuf.append("<td>");
					retBuf.append(this.stringArrayToString(testObj.getIncludedGroups()));
					retBuf.append("</td>");
					
					/* Exclude groups. */
					retBuf.append("<td>");
					retBuf.append(this.stringArrayToString(testObj.getExcludedGroups()));
					retBuf.append("</td>");
					
					retBuf.append("</tr>");
				}
			}
		}catch(Exception ex)
		{
			ex.printStackTrace();
		}finally
		{
			return retBuf.toString();
		}
	}

	/* Get date string format value. */
	private String getDateInStringFormat(Date date)
	{
		StringBuffer retBuf = new StringBuffer();
		if(date==null)
		{
			date = new Date();
		}
		DateFormat df = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
		retBuf.append(df.format(date));
		return retBuf.toString();
	}
	
	/* Convert long type deltaTime to format hh:mm:ss:mi. */
	private String convertDeltaTimeToString(long deltaTime)
	{
		StringBuffer retBuf = new StringBuffer();
		
		long milli = deltaTime;
		
		long seconds = deltaTime / 1000;
		
		long minutes = seconds / 60;
		
		long hours = minutes / 60;
		
		retBuf.append(hours + ":" + minutes + ":" + seconds + ":" + milli);
		
		return retBuf.toString();
	}
	
	/* Get test method summary info. */
	private String getTestMehodSummary(List<ISuite> suites)
	{
		StringBuffer retBuf = new StringBuffer();
		
		try
		{
			for(ISuite tempSuite: suites)
			{
				retBuf.append("<tr><td colspan=7><center><b>" + tempSuite.getName() + "</b></center></td></tr>");
				
				Map<String, ISuiteResult> testResults = tempSuite.getResults();
				
				for (ISuiteResult result : testResults.values()) {
					
					ITestContext testObj = result.getTestContext();

					String testName = testObj.getName();
					
					/* Get failed test method related data. */
					IResultMap testFailedResult = testObj.getFailedTests();
					String failedTestMethodInfo = this.getTestMethodReport(testName, testFailedResult, false, false);
					retBuf.append(failedTestMethodInfo);
					
					/* Get skipped test method related data. */
					IResultMap testSkippedResult = testObj.getSkippedTests();
					String skippedTestMethodInfo = this.getTestMethodReport(testName, testSkippedResult, false, true);
					retBuf.append(skippedTestMethodInfo);
					
					/* Get passed test method related data. */
					IResultMap testPassedResult = testObj.getPassedTests();
					String passedTestMethodInfo = this.getTestMethodReport(testName, testPassedResult, true, false);
					retBuf.append(passedTestMethodInfo);
				}
			}
		}catch(Exception ex)
		{
			ex.printStackTrace();
		}finally
		{
			return retBuf.toString();
		}
	}
	
	/* Get failed, passed or skipped test methods report. */
	private String getTestMethodReport(String testName, IResultMap testResultMap, boolean passedReault, boolean skippedResult)
	{
		StringBuffer retStrBuf = new StringBuffer();
		
		String resultTitle = testName;
		
		String color = "green";
		
		if(skippedResult)
		{
			resultTitle += " - Skipped ";
			color = "yellow";
		}else
		{
			if(!passedReault)
			{
				resultTitle += " - Failed ";
				color = "red";
			}else
			{
				resultTitle += " - Passed ";
				color = "green";
			}
		}
		
		retStrBuf.append("<tr bgcolor=" + color + "><td colspan=7><center><b>" + resultTitle + "</b></center></td></tr>");
			
		Set<ITestResult> testResultSet = testResultMap.getAllResults();
			
		for(ITestResult testResult : testResultSet)
		{
			String testClassName = "";
			String testMethodName = "";
			String startDateStr = "";
			String executeTimeStr = "";
			String paramStr = "";
			String reporterMessage = "";
			String exceptionMessage = "";
			
			//Get testClassName
			testClassName = testResult.getTestClass().getName();
				
			//Get testMethodName
			testMethodName = testResult.getMethod().getMethodName();
				
			//Get startDateStr
			long startTimeMillis = testResult.getStartMillis();
			startDateStr = this.getDateInStringFormat(new Date(startTimeMillis));
				
			//Get Execute time.
			long deltaMillis = testResult.getEndMillis() - testResult.getStartMillis();
			executeTimeStr = this.convertDeltaTimeToString(deltaMillis);
				
			//Get parameter list.
			Object paramObjArr[] = testResult.getParameters();
			for(Object paramObj : paramObjArr)
			{
				paramStr += (String)paramObj;
				paramStr += " ";
			}
				
			//Get reporter message list.
			List<String> repoterMessageList = Reporter.getOutput(testResult);
			for(String tmpMsg : repoterMessageList)				
			{
				reporterMessage += tmpMsg;
				reporterMessage += " ";
			}
				
			//Get exception message.
			Throwable exception = testResult.getThrowable();
			if(exception!=null)
			{
				StringWriter sw = new StringWriter();
				PrintWriter pw = new PrintWriter(sw);
				exception.printStackTrace(pw);
				
				exceptionMessage = sw.toString();
			}
			
			retStrBuf.append("<tr bgcolor=" + color + ">");
			
			/* Add test class name. */
			retStrBuf.append("<td>");
			retStrBuf.append(testClassName);
			retStrBuf.append("</td>");
			
			/* Add test method name. */
			retStrBuf.append("<td>");
			retStrBuf.append(testMethodName);
			retStrBuf.append("</td>");
			
			/* Add start time. */
			retStrBuf.append("<td>");
			retStrBuf.append(startDateStr);
			retStrBuf.append("</td>");
			
			/* Add execution time. */
			retStrBuf.append("<td>");
			retStrBuf.append(executeTimeStr);
			retStrBuf.append("</td>");
			
			/* Add parameter. */
			retStrBuf.append("<td>");
			retStrBuf.append(paramStr);
			retStrBuf.append("</td>");
			
			/* Add reporter message. */
			retStrBuf.append("<td>");
			retStrBuf.append(reporterMessage);
			retStrBuf.append("</td>");
			
			/* Add exception message. */
			retStrBuf.append("<td>");
			retStrBuf.append(exceptionMessage);
			retStrBuf.append("</td>");
			
			retStrBuf.append("</tr>");

		}
		
		return retStrBuf.toString();
	}
	
	/* Convert a string array elements to a string. */
	private String stringArrayToString(String strArr[])
	{
		StringBuffer retStrBuf = new StringBuffer();
		if(strArr!=null)
		{
			for(String str : strArr)
			{
				retStrBuf.append(str);
				retStrBuf.append(" ");
			}
		}
		return retStrBuf.toString();
	}

}

Right-click main-suite.xml, click the Run As —> TestNG Suite menu item in the popup menu list to run it.
After execution, you can see the custom-emailable-report.html in the left project panel under the test-output folder. Open it in your favorite web browser, you can see the customized TestNG reports.

3. Question & Answer.

3.1 How to include test method execution result in the TestNG customize report.

I am looking for a method to include the test method executed result in the TestNG Email-able report. In my test case, the test method will return a UUID, an error code number, and an error message if the test method throws errors. And I want to include the test method returned UUID, error code number, and error message in the customized TestNG report with separate columns. I think this need to use the TestNG listener to implement it, but I do not know how to implement it. Please give me some help, thanks a lot.

3.2 How to customize the TestNG report summary section.

The TestNG emailable report is very helpful, and I also learned from this article that I can customize the TestNG emailable report with a Listener class. But what I need is to add a more detailed information table in the report summary section to show the more detailed test method execution result in that area. Is there any method to implement this?

27 thoughts on “TestNG Reporting Customization”

Aayush
January 4, 2024 at 7:06 am

If the code is not able to generate the failed and skipped cases, you can do it by changing 1 line of code, here it is

1. search for “public void generateReport” method in the CustomTestNGReporter class
2. now search for this line of code
customReportTemplateStr = customReportTemplateStr.replaceAll(“\\$Test_Case_Detail\\$”, customTestMethodSummary);

and replace it with this:

customReportTemplateStr = customReportTemplateStr.replaceAll(“\\$Test_Case_Detail\\$”, Matcher.quoteReplacement(customTestMethodSummary));

Now your code will start generating the skipped and failed cases in the report

Reply
shubham
January 6, 2021 at 1:01 pm

Did you find the solution. if ys then please share

Reply
Cruze
September 29, 2020 at 12:35 pm

Hi ,

When I run the XML suite using maven build pom.xml file with TestNG plugin and when there are test failures and maven build gets failed . The customized report is not getting generated. Please let me know if i should add any parameters in pom.xml file. Thanks in advance.

Reply
Sapan Sharma
February 16, 2020 at 5:32 am

Hi,
When I am getting custom emailable report on my target folder but nothing is written into it. My code is same as above mentioned. Please help me .

Reply
Sisira
September 4, 2019 at 11:35 am

Thanks for sharing,
This implemenation working fine with my project except failure details not passing,
please support on following enhancements?
1. How to get report for multiple browser executions
2. How to get screen captures on failures + why failure details not passing

Reply
nesli
July 30, 2019 at 12:39 pm

i am getting lot of errors, it doesn’t work 🙁

Reply
swati
May 27, 2019 at 5:42 am

Hi,

I am getting the same error. Can anyone help me with this

Reply
S P
April 24, 2019 at 8:05 pm

Need help….

How is the suite-parameter named “browserType” set? I don’t see it being set anywhere in JAVA or XML files

Reply
S Patel
April 24, 2019 at 7:59 pm

Thanks for the nice example.

I have one question, though. How is the suite’s parameter “browserType” passed to this listener. I checked the java code, and the TestNG XML files – and did not find a mention of the parameter.

Reply
nishadhi
February 6, 2019 at 9:32 am

When I implement the custom-emaiable-report.html now in my test output directory I am getting both the default emaialble-report.html and custom-emailable-report.html. But I want to have only the custom report in the output directory. Is there a possible way to programmatically disable the creation of default emailable-report.html

Reply
sreedevi
January 23, 2019 at 4:43 am

i am getting java.lang.IllegalArgumentException: Illegal group reference at below lines :

// Replace report title place holder with custom title.
customReportTemplateStr = customReportTemplateStr.replaceAll(“\\$TestNG_Custom_Report_Title\\$”, customReportTitle);

// Replace test suite place holder with custom test suite summary.
customReportTemplateStr = customReportTemplateStr.replaceAll(“\\$Test_Case_Summary\\$”, customSuiteSummary);

// Replace test methods place holder with custom test method summary.
customReportTemplateStr = customReportTemplateStr.replaceAll(“\\$Test_Case_Detail\\$”, customTestMethodSummary);

please help.

Reply
1. Ajay
  April 22, 2019 at 7:07 am
  
  Did you find any solution for this? I’m also getting same error
  
  Reply
2. Arno
  May 27, 2019 at 9:40 am
  
  If you change “customReportTitle” to “Matcher.quoteReplacement(customReportTitle)” it will work. Same thing with the other two lines.
  
  For me it was also failing here, because my exception messages included $ character. By using Matcher.quoteReplacement these $ characters will not be treated as special characters.
  
  Reply
3. Balazs Gyore
  January 22, 2020 at 5:07 pm
  
  I did change line 55. From replaceAll. to replace, also change the regexp and it’s working now.
  
  line 55 looks like that now:
  
  customReportTemplateStr = customReportTemplateStr.replace(“$Test_Case_Detail$”, customTestMethodSummary);
  
  Reply
  1. venkat
    February 4, 2020 at 12:07 pm
    
    Yes, super bro now working fine for exceptions also.
    
    Reply
Rakesh kumar Kandanuru
November 21, 2018 at 2:55 pm

Tests and results in the report are not published in executed order. In TestNG XML i have mentioned preserve-order=”true”, still the same.
Is there a way i can publish test results(test methods names) in the order of execution.

Reply
vik
August 7, 2018 at 7:43 pm

Hi – is there a way to get the log messages in different rows instead of all getting congested in the same cell one after the other ?

and all the headers like execution time etc. are not very clearly visible. Can’t we add borders to those headers ?

Reply
Murali V
February 22, 2018 at 6:09 am

Hi,

Thank you so much for code.It works fine for all test cases passed.

Some of the points identified while working on above script:

+ I am unable to get the customized template report when test case failed.
+ If any exception raises at script level, which doesn’t show in customized report.
+ It doesn’t show total count of methods in the particular project/product

Could you please help us in above points.

Thanks in Advance.

Best Regards,
Murali V.

Reply
1. viki
  August 7, 2018 at 7:56 pm
  
  I am also not getting failed test cases, its showing everything passed,
  
  Reply
Murali V
February 16, 2018 at 9:51 am

@Senthil, You can import statements below:

I am getting Customized report successfully for every run.Hope it helps for you.

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.apache.batik.bridge.UserAgent;
import org.openqa.selenium.Capabilities;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.BrowserType;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.IReporter;
import org.testng.IResultMap;
import org.testng.ISuite;
import org.testng.ISuiteResult;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.xml.XmlSuite;

Reply
Senthilkumar S Selvaraj
February 14, 2018 at 5:59 am

please share the import statements

i am getting lot of errors

Reply
1. Senthilkumar S Selvaraj
  February 14, 2018 at 6:06 am
  
  import java.awt.List;
  import java.io.BufferedReader;
  import java.io.File;
  import java.io.FileNotFoundException;
  import java.io.FileReader;
  import java.io.FileWriter;
  import java.io.PrintWriter;
  import java.io.StringWriter;
  import java.text.DateFormat;
  import java.text.SimpleDateFormat;
  import java.util.Date;
  import java.util.Map;
  import java.util.Set;
  
  import org.testng.IReporter;
  import org.testng.IResultMap;
  import org.testng.ISuite;
  import org.testng.ISuiteResult;
  import org.testng.ITestContext;
  import org.testng.ITestResult;
  import org.testng.Reporter;
  
  what imports are missing here?
  
  Reply
  1. Senthilkumar S Selvaraj
    February 14, 2018 at 7:24 am
    
    finally i fixed all the errors
    
    running the testng suite , not generating the customer report
    
    Please help
    
    Reply
Ami
January 19, 2018 at 11:53 am

unable to automatically create custom emailable report html file , can you help ? I have followed every steps right

Reply
1. happyzhaosong
  January 20, 2018 at 4:47 pm
  
  What error have you encountered? Can you show the error messages?
  
  Reply
2. shubham
  January 6, 2021 at 1:00 pm
  
  Did you get solution for this? because i am getting same problem
  
  Reply
Murali V
January 3, 2018 at 6:50 am

Nice Article…Very Helpful..:)

Reply