Perform text or image analysis

The Perfecto Visual Analysis functionality facilitates advanced verification of visual or textual elements on the screen. This feature is based on a deep analysis of a screenshot of the actual device screen that serves as a 100% reflection of what the user sees. Visual analysis is always slower than object analysis, due to heavier processing, but it is helpful when object analysis is not an option. This may include cases where objects are not well-defined, making it difficult to arrive at a conclusive validation, or where it is required to verify based on what is actually displayed.

Text analysis is based on optical character recognition (OCR) and provides the ability to validate the presence (or absence) of any textual element in different forms (full, partial, and so on).

Perfecto translates any visual analysis action into a valuable code snippet that you can embed into any test automation script in different coding flavors.

This section walks you through the steps of performing text or image analysis. Both visual analysis options support:

  • Providing values for parameters.
  • Testing the effect of parameters on the analysis. The result indicates the percentage of confidence to find the so-called needle in the haystack.
  • Creating the command to use in your automation script, in the language of your choice. Available options are Java (default), JavaScript, Python, Ruby, C Sharp, and PHP.

To learn how to use text analysis, watch this video or review the instructions following the video. 

For more conceptual information, see the Visual analysis section.

Step-by-step instructions

Expand the procedure you want to perform to view its steps.

  1. On the device toolbar, click the text analysis icon .
  2. In the Find text dialog box, on the TEST PARAMETERS tab, do the following:
    1. From the Command list, select the command you want to insert into your script. For more information, see Available commands.
    2. If applicable, provide the text you expect to find.
    3. (Optional) Click Add Parameters, select any required parameters, and then click ADD. Enter the parameter values as needed.
      For information on the available parameters, see Available parameters.
    4. (Optional) Click FIND to see how likely it is that the test will find the text on the screen with the current configuration. The test result appears as a percentage that indicates how likely it is that the text was found on the page.
    5. Click NEXT to create the command for your automation script.
  3. On the CREATE COMMAND tab, do the following:
    1. Select the programming language for the command.
    2. Click COPY TO CLIPBOARD, and then paste the command into your script. 
Restriction: The image file cannot exceed 150 MB.
  1. On the device toolbar, click the image analysis icon .

  2. In the Find image dialog box, on the UPLOAD IMAGE tab, select the location of your image file:

    Repository (default) or Computer

  3. Depending on your selection in step 2, do the following:

    • Repository: Search for and select the repository file you want to use.
    • Computer:
      1. Click ADD IMAGE.
      2. Browse to the file you want to use.
      3. Select the file and clickOpen.
        The file loads.
  4. Click NEXT to specify parameters.
  5. On the TEST PARAMETERS tab, do the following:
    1. From the Command list, select the command you want to insert into your script. For more information, see Available commands.
    2. (Optional) Click Add Parameters, select any required parameters, and then click ADD. Enter the parameter values as needed.
      For information on the available parameters, see Available parameters.
    3. (Optional) Click FIND to see how likely it is that the test will find the text on the screen with the current configuration. The test result appears as a percentage that indicates how likely it is that the text was found on the page.
    4. Click NEXT to create the command for your automation script.
  6. On the CREATE COMMAND tab, do the following:
    1. Select the programming language for the command.
    2. Click COPY TO CLIPBOARD, and then paste the command into your script. 

Available parameters

The following table describes the parameters available in the UI and their values. These parameters address the most common use cases. For more information about these parameters and about additional parameters that apply to more advanced use cases, see General Visual Analysis Parameters.

Parameter Description Values

Haystack

Use to limit the area of the screenshot (haystack) examined by providing the coordinates of the top-left corner of the haystack rectangle and the rectangle's dimensions. This is especially useful if the image or text that you are looking for is known to appear in a particular area of the screen, such as the header or footer.

If you specify a value other than 0 in the Top or Left field without adjusting the height or width accordingly (which are, by default, set to 100%), an error occurs.

To define the haystack, you can:

  • Drag the pointer over the device screen and, if needed, fine-tune the area using the Left & Top (%) and Width & Height (%) fields in the dialog box.
  • Specify the offset of the search area from the top and left as well as the width and height of the search area, in percent.

For example, in the following image, the search area is defined as offset by 25% from the left and 0% from the top, with a width of 50% and a height of 10%.

The offset from top and left as well as width and height, in percent or pixels

Relative position

Only available with the Button command selected. Use to specify the position of text relative to the button and the text distance from the button, in percent. 

Above, Below, Left, or Right

Distance in percent

Timeout (secs)

Use to define the time, in seconds, to wait for a needle to appear on the device screen. This is useful with a dynamic screen where the page loading time should be considered when searching for the needle.

When enabling scrolling, the timeout must be zero. Scroll cannot be used with a positive timeout value because this command dictates a different behavior when the needle is not found. Also, using a scroll results in longer execution times.

Time, in seconds

Accurate timer

Add to specify the accuracy level of the timer. Turn on for a measurement in sub-seconds (0.1 seconds, which is accurate). Turn off for normal accuracy, in seconds (considered rough). 

On/off

Threshold (%)

Use to specify the acceptable match level percentage, between 20 and 100.
Values that are too low can lead to a false positive result, while values that are too high can lead to a false negative result.

20-100

OCR Profile

(Text analysis only) Use to set optimal values for a large number of OCR parameters to match a particular scenario. We highly recommend setting a profile due to profile complexity and the large number of parameters involved.

For details, see What are OCR profiles?.
  • Default
    Document Conversion Accuracy
  • Document Conversion Speed
  • Document Archiving Accuracy
  • Document Archiving Speed
  • Book Archiving Accuracy
  • Book Archiving Speed
  • Text Extraction Accuracy
  • Text Extraction Speed
  • Field Level Recognition
  • Barcode Recognition

Languages

(Text analysis only) Use to specify the OCR language to use. You can select multiple values.

This parameter is irrelevant when using the native screen source.

 English, Spanish, Japanese, German, French, Dutch, Italian, Chinese, Russian, Korean, Portuguese

Scroll and Search

Turn on to specify whether to scroll until the label is found. Scrolling results in a longer execution time.

You cannot use this parameter together with a positive timeout value. When scrolling is enabled, the timeout value must be zero.

On/off

Ignore Case

(Text analysis only) Use to specify case sensitivity of the search pattern. When turned on, the search is not case sensitive.

On/off

Whole Word

(Text analysis only) Turn on to match only whole words in the haystack. Otherwise, also part of other words are found.

On/off

Ignore White Space

(Text analysis only) Turn on to ignore spaces within search pattern words.

On/off

Match

(Image analysis only) Use to specify the needle comparison method, where:

  • Identical means that the needle and haystack are from the same origin (device screen) and size. This makes matching faster and simpler.
  • Similar means that needle and haystack are not from the same origin (needle is from another device or other source) or not the same size (different ratio or size). This makes matching slower and more complex. Because analysis time increases, make sure to adjust timeout values when using this value.
  • Bounded means the same as Similar, but with a restriction on the possible needle sizes. This value allows you to decrease the Similar analysis time by using preexisting knowledge about the needle size or by decreasing the size of the haystack.

Identical, Similar, or Bounded 

Available commands

The following table describes the commands available in the UI. For more information on these commands, see Visual analysis functions.

Command Description Code Snippet (Java)

Find

Finds the text or image (needle) on the device screen (haystack) and stores the coordinates for future commands.

See also Find text (FR) and Find image (FR).

 
mobile:text:find
mobile:image:find

Button

Identifies a button, based on a text or image label, and clicks it. This command includes an integrated platform-specific OCR customization.

See also Text button click (FR) and Image Button Click (FR).

 
mobile:button-text:click
mobile:button-image:click

Edit-get

Identifies an edit field, based on a text or image label, and retrieves its text value. The value is returned as a string.

See also Edit-text get (FR) and Edit-image get (FR).

 
mobile:edit-text:get
mobile:edit-image:get

Edit-set

Identifies an edit field, based on a text or image label, and inserts the specified text or image in the value parameter into the field. The field is in relation to the found label. It is defined by the label position and offset parameters.

See also Edit-text set (FR) and Edit-image set (FR).

 
mobile:edit-text:set
mobile:edit-image:set

Screen

(Text analysis only) Returns the text that appears on the screen of the device. It does not select the text. 

See also Screen text (FR).

 
mobile:screen:text