AI User Action command

Utilizes AI to navigate the open device screen using semantic and contextual information provided in natural language. Use this command to perform user actions without stating each click, and to interact with UI elements without knowing their locators.

An AI User Action command simulates a user interaction with the graphical interface seen on the device, such as selecting a menu item and confirming a dialog. Perfecto AI empowers you to perform a variety of UI navigation actions by letting you describe the process like you would talk to a person sitting next to you.

For instance, if you are testing online shopping for books, you give the assistant natural instructions like "Search for an Italian cookbook", "Order the cheapest book from the list", "Pay with voucher code ABC123". The assistant can identify the referenced buttons and fields. You do not need to break the scenario down into individual clicks, nor explicitly name UI elements nor locators. Use a series of navigation commands to describe more complex scenarios, such as filling in forms.

For more usage best practices, see Perform AI-driven user actions.

Each User Action command can contain up to 30 steps (such as clicking, scrolling, and so on) and the execution limit per call is 3 minutes. For optimal performance, phrase your User Actions so they do not resolve to more than 15 steps.

Name	Type	Default	Description
Device ID*	Device		The device for this command. By default, this is the device under test (DUT).
Action*	String		A natural-language description of an interaction that a user performs on the device screen. To learn more, see Execute User Actions in the UI using the AI Assistant.
Comment	String		Adds a comment for reference.
On-fail Result*	Condition	Ignore | Break | Continue | Abort | Catch	The behavior when the user action fails.

^* Mandatory