Perform AI-driven user actions

The AI User Action Assistant can navigate the device screen like a user would. Phrase your prompts like you would when describing a test step to a manual tester.

To have the test interact with the application, you no longer need to identify object locators or learn a scripting language. The AI Assistant understands natural language like "select London as travel destination." The assistant interprets your instructions accordingly as button clicks, menu selections, typing values in fields, and so on. The prompt is then included as the value for the Action parameter in the AI User Action command.

For examples and limitations, see Best practices for working with AI user actions.

The AI Command command unifies and supersedes the AI User Action and AI Validation commands. AI Command is the recommended choice for all new scripts. AI User Action and AI Validation continue to function for backward compatibility.

Supported devices

This task is supported on the following devices:

  • Mobile

  • Virtual

  • Web browser

  • Desktop app

Steps

To add a user action:

To add a User Action, open a device.

  1. In the left sidebar, select the AI tab.

  2. Click and drag or double-click the User Action command to add it to the test editor.

  3. Double-click the action in the test editor to enter the action.

  4. In the AI Input assistant pane, at the bottom, describe the logical goal of a user action to perform, such as "Log in as Joe with password 123."

  5. (Optional) To include variables in your AI User Action:

    • To use an existing test variable, type ${ and select a variable. You can open the variable picker this way in several locations:

      • In the AI assistant dialog

      • In the Test editor when editing the AI User Action inline

      • In the Action field in the AI User Action parameters tab if Usage is set to Constant;
        if Usage is set to Variable, the variable picker opens immediately.

      Example of existing variable use: Log in as ${username} with password ${pwd}

    • To create a variable in the AI Assistant, enter a name in ${variable-name} format and provide an initial value when prompted.
      Example of variable creation: A request such as Open the ${app} app. asks you for a value and creates the app variable.

    • To create a variable from a value on the screen at runtime, tell the assistant the new variable name and describe its value. For example: 

      Put the maximum temperature shown on the screen into ${max_temp}.
      Identify the exchange rate shown and save it to ${rate}.
      Get the value of the account balance and put it into ${balance}.

    Values of variables entered as Secured String parameters are not shown in reporting steps.

    To learn more about variables in AI Scriptless, see Configure test variables.

  6. The Assistant confirms by repeating the instructions as it understood them and adds them in the test editor. Verify that the Assistant interpreted the command as intended.

    • If your input could be more specific, the Assistant asks for details or suggests an improved phrasing.
    • If it needs time to process, the Assistant will say "Working on it..."
    • If needed, cancel the processing by starting to type another prompt or by clicking the Stop button.

  7. (Optional) Add more AI commands and repeat the process.

  8. (Optional) To copy the suggested user action text to the clipboard, hover over the text and click the Copy to clipboard button next to the text.

  9. Save and run the test.