CI Dashboard view

When using a Continuous Integration (CI) tool, for example Jenkins, each test set is assigned a job name and, usually, a job number. The CI Dashboard displays an overview of the history of the tests listed by their job name.

Watch this short video for a demonstration of how to navigate the CI Dashboard view.


Jobs tab

The Jobs tab shows the following information:

  • The Job Name and the details of the selected test run.
  • A statistical overview of the selected test run results: 
    • Number of tests included
    • Number of tests that reported "passed"
    • Number of tests that reported "failed"
    • Number of tests that did not report a final status or indeterminate result
  • A set of bars that (graphically) represent the history of the job's test runs, where each bar represents the distribution of the test results for the particular run
  • A line graph that represents the duration history of the job's test runs, where each node on the line represents the duration of the particular run.
Note:

The job number listed in the Job/Branch Details column is the latest job number assigned by the CI tool (for example Jenkins). Each bar in the Result History column represents the results for a different job number. If the CI tool resets the job numbers assigned (for example as a result of a re-installation or general reset), the Test Analysis view displays a new bar (and node) for equal job numbers if the start-time of the two jobs is more than 24 hours apart.

Clicking information in the Last Build Summary column or a bar in the Result History column opens the report in the Report Library. Clicking an icon in the right-most column opens the Insights view () or HeatMap () for the job/branch.

History information

The Result History bars and Duration History nodes are coordinated. Each bar represents the same job as the corresponding node. Moving the pointer over a bar or node displays a tooltip with the following statistic information:

  • Job Number
  • Date and time that the test run was executed
  • Duration of the test run (when moving the pointer over the duration node)
  • A statistical overview of the run (when moving the pointer over the history bar)
  • Heatmap icon to drill down to the Heatmap for this build
  • Insights icon to drill down to Insights for this build

The statistical overview shows the actual test results for the test run.

Branches By Job tab

Each CI Job (see above) can also be divided into different branches that may, for example, define a different flavor of test for the features under test. If the tester declared a test run to be associated with a particular branch (by using the withBranch method or with the reportium-job-branch command-line parameter), then the job and its branches are displayed in the Branches by Job tab.

The tab displays a separate line for each job that includes:

  • The total number of branches in the job
  • Summary of the status of the most recent branch run

Clicking the expansion button on the left or the job name opens a list of all the branches' history overview (similar to the job history overview described above), showing one line for each branch of the job.