Xray test management for Atlassian Jira

Learn how to integrate Xray, a JIRA plugin for test management, with Perfecto. The instructions refer to the TestNG framework, but you can use the same APIs for any other framework.

Integration sample files are available on GitHub: https://github.com/PerfectoMobileSA/XraySample

For additional resources, see:

Important: This document includes references to third-party products, Atlassian Jira and Xray. The user interface and usage of third-party products are subject to change without notice. For the latest published information about Atlassian Jira, see https://support.atlassian.com/jira-software-cloud/resources/. For the latest published information about Xray, see https://docs.getxray.app/display/XRAY/Getting+started.

Prerequisites

Integrating Xray with Perfecto requires the following:

  • Jira
  • TestNG
  • Java
  • Maven
    The integration relies on the following Maven dependencies:

    Copy
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.10</version>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.0.0</version>
    </dependency>

1 | Set up Xray and configure Jira issues

In this step, you install Xray, add the relevant issue types to your Jira project, and generate the keys and tokens required to connect Perfecto with Xray and Jira.

To set up Xray and configure Jira issues:

  1. Install Xray from the Atlassian Marketplace.
  2. To add Xray issue types to your Jira project, configure the project settings as follows:
    1. Under Xray Settings, on the Summary tab, click the link to edit issue types for the Xray project.  

      The following Xray issue types are available: Test, Precondition, Test Set, Test Plan, Test Execution, and Sub Test Execution.

    2. On the Issue type scheme tab, drag the required Xray issue type from the Available Issue Types pane to the Issue Type for Current Scheme pane and click Save

      The Issue Type for Current Scheme pane includes all issue types already configured for your project.

      When the Xray issue types are configured, they appear in the list of issue types to select from when you create a new issue.

  3. Create the Xray API key and Jira token. We need them later for the integration with Perfecto.

    Important: To be able to create the key and token, you need to be a Jira administrator.
    • To create the API Key: 
      1. On the menu on the left, click Xray > API Keys.
      2. On the API Keys tab, click Create API Key.

      3. In the Create API Key dialog box, select the user for which you want to generate the key and click Generate.
    • To create the Jira token: 
      1. Click the user icon at the bottom left; then click the gear icon and select Account settings.

      2. Under Atlassian account, on the Security tab, click Create and manage API tokens.

      3. Under API tokens, click Create API token. Jira creates the token and lists it on the page.

  4. In JIRA, create an issue of type Test for each test or scenario in your automation project. Make sure there is one-to-one mapping between Jira Test issues and tests in the automation project.
  5. Create the test execution Jira issues. These issues define the execution batches for the different test phases, such as smoke, sanity, and regression testing. This step includes adding the relevant test cases to the appropriate test execution issue. In the example below, the test execution issue (called PERFECTO-4) includes two test issues (PERFECTO-1 and PERFECTO-2). The integration with Perfecto shows PASSED (or FAILED) in the Status column.

2 | Configure the automation project

In this step, you configure parameters in your automation project and map the test cases with the respective Xray issues in Jira.

To configure the automation project:

  1.  Configure the relevant parameters as shown in the following code snippet. The goal here is to include the execution results of smoke, sanity, and regression testing in the Xray issues in Jira.

    See GitHub for non-Quantum Java TestNG and Quantum and Gherkin TestNG sample files.

    Copy
    <parameter name="globalJiraSwitchKey" value="true"></parameter>
    <parameter name="createNewJiraTasks" value="true" />
    <parameter name="jiraHostURL" value="https://jiraperfecto.atlassian.net" />
    <parameter name="jiraUser" value="johndoe@gmail.com" />
    <parameter name="jiraPassword" value="<your_password>" />
    <parameter name="projectKey" value="PERFECTO"></parameter>
    <parameter name="xrayClientID" value="your_clientID"></parameter>
    <parameter name="xrayClientSecret" value="Your_clientSecret"></parameter>
    <parameter name="xrayHost" value="https://xray.cloud.xpand-it.com"></parameter>
    <parameter name="testExecutionKey" value="PERFECTO-4"></parameter>

    where:

    • globalJiraSwitchKey enables or disables the Jira/XRay integration.

    • createNewJiraTasks enables or disables the creation of a new Jira task for each automation failure.

    • jiraHostURL is the URL of your Jira host.

    • jiraUser is the email address of the user used to log in to Jira.

    • jiraPassword is the Jira API token that we created earlier.

    • projectKey is the key of the Jira project. You can identify the key by looking at your issue identifiers. For example, if all your issues are named PERFECTO-xx, the project key is PERFECTO.

    • xrayClientID and xrayClientSecret are the values provided when we created the Xray API token.

    • xrayHost is the URL of your Xray host (see Xray REST API for information on the base URL). For privately hosted clouds, this URL is different from the above sample. The administrator should be able to provide these details.

    • testExecutionKey is the identifier for the relevant test execution issue in Jira. This key is different for smoke, sanity, and regression test executions.

  2. To map the automation scripts with the actual Jira test cases, tag each test case with the Jira issue number. The listener file (see next step) uses this code to find out which test cases to update. For example:

    • In a non-Quantum Java test case, add the following: 

      Copy
      @Test(groups = { "@PERFECTO-1" })
    • In a Quantum Java test case (Cucumber), add the following: 

      Copy
      @WebSearch @PERFECTO-1
  3. Add the following files to the project: 

    • The listener file: JiraListener.java

      Add this file in the appropriate package. In the GitHub examples, the packages are:

      <listener class-name="com.quantum.listeners.QuantumReportiumListener" />
      <listener class-name="com.quantum.listeners.JiraListener" />

      or

      <listener class-name="com.listeners.JiraListener" />
    • The .json resource files:

      jiraNewTask.json
      xrayTestExUpdate.json
      xrayTestFail.json
      xrayTestPass.json
  4. In the xrayTestExUpdate.json file, edit the following to include the name of your Jira user:

    Copy
    "user" : "UPDATE_YOUR_JIRA_USERNAME_HERE"
  5. (Optional) In the fileJiraListener.java file, update the value of the following string variable with the path of your .json resource files:

    Copy
    public static final String RESOURCESLOC = "resources/";
    public static final String RESOURCESLOC = "resources/";