Add execution reports in Azure DevOps pipeline jobs

This article explains how to display simple latest execution reports in the Azure DevOps pipeline for quick analysis, as shown in the following image. Detail reports and historical execution trends are available in Perfecto on the CI Dashboard.

The task mentioned here is independent of the framework. You can use it with any language framework. This example uses the Java-based Quantum framework to run the test and display the results in the pipeline.

Prerequisites

Before you start, make sure that:

  • You have integrated Perfecto Smart Reporting in the test framework.

  • The test framework accepts external variables for BuildName and BuildNumber while creating reportiumClient, and the execution results are available in the cloud's CI Dashboard view.

Step-by-step instructions

  1. Add the following Python file into your framework and upload it to the repository: PerfectoReportsHTMLCreation.py

  2. Under Pipelines > Tasks, create a new pipeline and add a Maven task, providing the pom.xml file path and goal.

    Copy

    Goal

    clean install -Dreportium-job-name=$(Build.DefinitionName) -Dreportium-job-number=$(Build.BuildId)

    The ReportiumClient in the framework will receive the given JobName and JobNumber and create the CI Dashboard view under Test Analysis in Perfecto. Post execution, we will retrieve the same results and display them in the pipeline.

  3. Under Pipelines > Tasks, add the Use Python 3.7 task and then add the latest or latest -1 Python 3 version.

  4. Under Pipelines > Variables, create a variable for the cloud name and the cloud security token.

  5. Under Pipelines > Tasks, add a Command Line Script task. In the Script field, enter the following code:

    Copy 
    echo "**************Installing Python Pre requisite Libraries************"

    pip install requests

    echo "**************Execute Python File***************"

    #Remove if HTML file available 
    rm PerfectoReport.html

    #Run python script
    python PerfectoReportsHTMLCreation.py $(build.cloudName) $(Build.DefinitionName) $(Build.BuildId) $(build.cloudSecurityToken)

    echo $(Pipeline.Workspace)/s

    ls -la $(Pipeline.Workspace)/s

    The python file will retrieve the results of the execution that was executed in the Maven task using the Smart Reporting Public API.

  6. Add a Publish Pipeline Artifact task.

    The Python script executed from the Command Line Script task will create the PerfectoReport.html file. Publish this artifact into the $(Pipeline.Workspace) location. We will use this path to get the HTML file to publish in the pipeline.

  7. Install the Html viewer extension in your Azure DevOps. This will allow you to add the new Publish HTML Report task.

  8. Add a Publish HTML Report task. In the HTML file Directory, add the following path:

    Copy 
    $(Pipeline.Workspace)/s/PerfectoReport.html

  9. Click Save & Queue to save and execute the pipeline. The script will execute the tests, and the Python task will generate the PerfectoReport.html and publish the results.