Visual relations

Visual relations is useful when searching visually for a non-unique object on a dynamic device screen. Relating the non-unique object to another object will define it precisely and ensure a stable search.

In the following image, the FREE button is not unique on the screen. A default search cannot be used because it will return the first occurrence of the FREE object. We solve this conundrum using Visual Relations.

Visual relations is used to find the "Free" needle (object or text) in relation to the "Paypal" needle as a two-step process. The first needle image object ("PayPal") is aligned to the left of the second needle text object ("Free"). Visual relations searches and validates that the second needle ("Free") is within a visual relation to the first needle ("PayPal"). Visual relations can be used with both images and texts; the first and second needles can be either an image or a text.

To use visual relations:

  1. Find the reference object (First Needle) - Use the Find command to search for an image or text and store it in memory.
  2. Select a command (e.g. button.click) used to find the target object (Second Needle).
  3. In the Label parameter of the command, enter the text or image you want to find. In this example, you would enter FREE.
  4. Define a relationship to the first needle using the following visual analysis parameters:
    1. Visual relation inline - indicates the type of alignment between the first needle (PayPal in the example) and the second needle (FREE).
      Values: horizontal or vertical
    2. Visual relation direction - indicates the direction relative to the first needle to look for the second needle.
      Values: right, left, above or below

The following code sample shows how to use the first method to find the reference object (first needle) and then the second method to find the main object (second needle), along with two additional parameters that define the relationship between the two searches. Perfecto automatically saves the result from the first method and passes it to the second method when the visual relations parameters are used.

Copy 
Map<String, Object> params = new HashMap<>();
params.put("content", "PayPal");
params.put("threshold", 80);
driver.executeScript("mobile:text:find",params);
params.clear();
params.put("content", "PUBLIC:Free-Text-Screenshot.png");
params.put("threshold", 70); // For an image, consider using a ~70% match in pixels.
params.put("operation", "single");
params.put("relation.direction", "Right");
params.put("relation.inline", "Horizontal");
String res = (String)driver.executeScript("mobile:image:find",params);