Identify the cause of failure

When a test fails, you need to be able to quickly identify the cause of failure. For failed tests, Perfecto's single test report (STR) includes execution insights to help you understand why a failure may have occurred. These insights:

  • Reveal why a specific failure reason was assigned as a result of visual analysis: Perfecto's root cause analysis (RCA) feature uses artificial intelligence (AI) to identify the underlying error behind test failures. In cases where visual analysis is used to assign a failure reason, Perfecto finds the last screenshot taken before the error occurred and analyzes it.

    In the Execution insights pane, the Failure reason identified card summarizes these findings. Selecting the card displays the last screenshot taken before the error occurred in the visual artifacts area.

  • Allow for viewing side-by-side screenshots of the failed execution step and the same step in the last successful execution: The last successful execution is always the last successful test run before the test run captured in the current report. Comparing these screenshots can help you get a better idea of why the failure occurred.

    In the Execution insights pane, the Screen comparison card identifies the last successful run, including a link to the report. Selecting the card displays the current and last successful screenshot in the visual artifacts area.

    Restriction: To retrieve the screenshot of the same step in the last successful execution, the current and last successful tests need to be exactly the same, meaning they need to be based on the same script and capabilities and have the same steps up to the failed step. Tests that include conditions, loops, or other dynamic elements may introduce additional steps during execution that make it impossible for Perfecto to retrieve the screenshot of the last successful run.

    In addition, Perfecto does not provide side-by-side screenshots when:

    • No reference test is available

    • The failed test does not include any failed commands

    • The last successful execution didn't include a screenshot or a video

    • The last successful execution occurred more than 45 days ago

If a report includes execution insights, the Execution insights pane is open by default, with the first insight selected.

To view why a specific failure reason was assigned:

  1. In the Execution insights pane on the right, select the Failure reason identified insight.

  2. Review the content in the pane and the screenshot displayed in the visual artifacts area.

To compare screenshots:

  1. In the Execution insights pane on the right, select the Screen comparison available insight.

  2. Analyze the side-by-side screenshots displayed in the visual artifact area. Identifying even small differences might lead to a better understanding of what could have caused the failure.

  3. (Optional) View the screenshots in full-screen mode by clicking the full-screen mode button .

  4. (Optional) Open the last successful report by clicking the open report button . The report opens in a new browser tab, automatically displaying the corresponding command that matches the screenshot.

To re-open the Execution insights pane:

  • In the STR, in the visual artifacts area, click the execution insights button .