AI Visual Comparison command

Utilizes AI to compare two device screens like a human tester would. Use this command to detect visual differences in before-and-after test result screenshots, and fail the test if the differences are in certain categories.

An AI Visual Comparison command compares two device screens and detects and describes visual differences in test result screenshots. The test fails if the differences are in certain categories. You define a baseline screenshot and future tests are compared against it. This test step detects, for example, missing, added, or moved UI elements, and changes in values, color, or text style.

For more usage best practices, see Best practices for working with AI visual comparisons.

Name

Type

Default

Description

Device ID*

Device

 

The device for this command. By default, this is the device under test (DUT).

Baseline ID*

String

 

Screenshots of a previous, named run are used as the baseline for the comparison with the checkpoints created during later runs. Choose a name for the baseline indicating the screen being compared.

Failure Criteria*

List

device

value

style

missing

addition

moved

error

uncategorized

pixelDifference

Select at least one or multiple values to determine which visual differences in the checkpoint fail the test when compared to the baseline. All criteria are analyzed, but only the ones you list in the parameter fail the test. For best pratcies, see see Compare screens with AI vision.

  • Device - Signal strength, battery level, time of the day

  • Style - Text style, font, text size, color, alignment, borders, shading

  • Value - Text content or numeric values

  • Missing - UI element is absent

  • Moved - UI element is present but in a different position

  • Addition - A new UI element has appeared

  • Error - Error message, UI elements cut off or overlapping

  • Uncategorized - Differences detected but they are not in any of the categories.

  • Pixel difference - No meaningful differences detected, just common anti-aliasing, image compression, or image scaling artifacts

Comment

String

 

Adds a comment for reference.

On-fail Result*

Condition

Ignore | Break | Continue | Abort | Catch

The behavior when the user action fails. Default is to abort.

* Mandatory