Manage failure reasons

Restriction: The Failure Reasons tab is only available to admin users.

The Failure Reasons tab displays a list of failure reasons on the left and steps on how to add custom failure reasons on the right. You recognize system-supplied failure reasons by the Perfecto logo preceding them.

Blocked failure reasons do not appear in the list. They are not configurable.

To access the Failure reasons view:

Log in to Perfecto at https://<cloud-name>.app.perfectomobile.com.
On the UI landing page, click ACCOUNT SETUP.
In the header, click Failure reasons.

On this page:

Failure reasons and categories

Perfecto supplies the categories and failure reasons listed in the following table.

Category	Description	Failure reason
Application issue	Failures related to issues with the application. These are typically caused by bugs in the tested application.	App crashed App failure App management App version Assertion failed
Scripting	Failures that are most likely related to errors in the script, such as incorrect commands, parameters, or values. Custom failure reasons that existed in the system when Insights was first added are also mapped to this category.	Element not found Invalid XPath syntax Multiple elements found Popup handling Invalid expression Index out of bounds Json error Class not found Illegal argument Unsupported command Element not interactable Invalid session
Test data	Failures related to incorrect test data used to drive the tests, typically caused by outdated test data, insufficient volume of test data, or test data not meeting requirements, such as uniqueness.
Test environment	Failures related to the setup of the test environment, typically caused by incorrectly configured back-end services or unavailable third-party dependencies.	Backend error Network error
Text execution	Failures that cause a test to fail but that are not related to the application, scripting issues, or the environment. Possible examples are disabled notifications, disabled location settings, or login failures.	Device settings Authentication issue

Each name is associated with an ID and a color. In the Report Library and Insights, the color helps identify the different failure reasons for the test reports.

You can change the color of system-supplied failure reasons or define additional failure reasons. Because of Perfecto's intricate mapping logic, it is not possible to change the name of pre-defined failure reasons.

Each failure reason is mapped to one of the existing categories. The categories are designed to help you work more efficiently. Perfecto expertise automatically classifies test failures by similar root causes that can be resolved using a similar approach. This kind of information jump starts your root cause analysis. For more information on categories, see the documentation on the Failures by category widget in Insights.

In automation scripts, you can group failures under the same system-supplied or custom failure reason by supplying the failure reason ID in the TestResultFactory's createFailure()method. See also Use failure reasons in test scripts.

To edit the color of a system-supplied failure reason:

Move the pointer over the respective table row.
On the right, click the edit button .
Select a different color.
To accept your change, click the check mark on the right.

The changes will not affect any tests already categorized.

Blocked failure reasons

Following is a list of blocked failure categories and reasons. They provide insights into why a test's execution could not be completed.

You cannot view or configure blocked failure reasons in this view.

Category	Description	Blocked failure reason
Lab	Failures related to physical issues on mobile or web devices, such as Wi-Fi problems or devices that are in error or disconnected.	Device in error Device not connected Close app failure Execution error Install app failure Internal BT problem Launch app failure Perfecto Connect error
Orchestration	Failures related to issues with organizing tests, such as failing to reserve a device or devices being already in use when a test is run.	All licenses in use App not found Capability not supported Device in use Device not available Device not found Device not reserved Device reserved No licenses found Session inactivity User not authorized

Custom failure reasons

You can add up to 15 custom failure reasons. Perfecto automatically assigns an ID to each new custom failure reason. You can edit an existing custom failure reason at any time, but the changes will not affect any reports already categorized.

To add a custom failure reason:

At the bottom of the failure reasons list, click Add another failure reason. You may need to scroll down to see this option.
In the Failure reason column, in the text box, enter the name string for the new failure reason. The string cannot exceed 25 characters.
In the Color column, select a color for the failure reason.
In the Category column, select a category from the list.
To finalize the addition, click the check mark.

To edit a custom failure reason:

Hover over the respective table row.
On the right, click the edit button .
Change the name, color, or category as needed.
To accept your changes, click the check mark on the right.

To delete a custom failure reason:

Hover over the respective table row.
On the right, click the delete button .
When prompted for confirmation, click YES.

Copy a failure reason ID

Perfecto assigns each custom failure reason a unique ID, as displayed in the ID column. You can copy-paste this ID into your test scripts or into a text file used by the test scripts to indicate what caused a test to fail.

To copy a failure reason ID:

Hover over the ID to display the copy button .
Click the copy button to copy the ID to the clipboard.
Paste the ID into the destination text (or code) file.

Use failure reasons in test scripts

The Smart Reporting SDK and the External Test Information SDK support supplying the failure reason for a test script or report using the TestResultFactory's createFailure() method. This applies to both system-supplied and custom failure reasons.

The method supports the following parameters:

A failure message that is added to the test report
The actual error notification (throwable entity)
A failure reason ID string
The string must match one of the defined failure reason IDs as shown in the UI. If the provided string does not match any of the defined failure reasons, it will be ignored.

The following example shows how to create the failure TestResult with a failure reason (one of the basic failure reasons supplied by Perfecto):

Copy

TestResult testResult = TestResultFactory.createFailure("Test stop failure", e, "bEC3xAdMTz");  
//Paste the failure reason ID after copying from the Failure reasons tab
reportiumClient.testStop(testResult);

See the full example test code here.