iOS configuration parameters for the Gradle Plugin | Virtual devices

When running your XCTest or XCUITest tests with the Perfecto Gradle Plugin, you can add configuration information for the test execution at the following levels:

  • Configuration file (recommended): A JSON-formatted file listing parameter assignments (see below).
  • build.gradle file: In the perfectoGradleSettings clause, include Authentication parameters and indicate the location of the configuration file.
  • Command-line: Include any of the parameters in the following format: -P<paramName>="<paramvalue>"

The configuration parameters are grouped into the following categories:

Perfecto lab authentication parameters

Parameter

Possible Values

Meaning

cloudURL


The URL of the Perfecto Lab to connect to, but without the .app notation.

For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com)

securityToken

Security Token

Tester's personal security token for the Perfecto Lab.

These parameters may be set in the format shown in the following sample code snippets:

Copy
Configuration file format
"cloudURL": "mobilecloud.perfectomobile.com",
"securityToken": "<your-security-token>",
Copy
build.gradle file format
perfectoGradleSettings { 
    cloudURL "mobilecloud.perfectomobile.com" 
    securityToken "<your-security-token>" 
} 
Copy
Command line format
> gradle perfecto-xctest-vd -PcloudURL="mobilecloud.perfectomobile.com" -PsecurityToken="<your-security-token>"

Device selection parameters

You can select specific devices or specify a number of default devices.

Select specific devices

You can select to use the following parameters to specify a list of devices. Obtain the values for the following parameters based on the matrix of supported simulators. For details, see Supported simulator devices and OS versions.

If no device selection parameters are set, the plugin will select the default Android device.

Parameter

Possible Values

Meaning

model


Device model - for example: iPhone-7

platformName

iOS

Device operating system

platformVersion


Operating system version - for example 11.0.3

These parameters may be set in the configuration file, in the following format:

Copy
Single device selection
"devices": {
    "platformVersion": "15.*",
    "model": "iPhone 13"
}

In the following example, two devices are selected. The second device in this example ("{}") is the default iOS Simulator.

Copy
Multiple device selection
"devices" : [
    {
        "model": "iPhone 13",
        "platformVersion": "16.2",
    },
    {}
]

Specify the number of random devices

Instead of the devices clause, you can use the following parameter to indicate that the specified number of iOS devices should be selected and the tests run on the devices in parallel.

Parameter

Possible Values

Meaning

numOfDevices

number in range [1,100]

Number of iOS devices to select. If numOfDevices is outside the range, the value will default to select a single device.

This parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:

Copy
"numOfDevices": 45

Application parameters

Parameter

Possible Values

Meaning

instrumentationEnvVars


Environment Variable values to pass to the Instrumentation Runner.

Format:

  • Configuration file:
    "instrumentationEnvVars" : ["key=value","key=value",...]
  • Command line:
    -PinstrumentationEnvVars="key=value;key=value;..."

failBuildOnFailure

 true | false

Fail the Gradle build if any test fails or there is a device error. Default is false.

xctestProductsPath

 

The path to the .xctestproducts file, which contains all test products

shard

true | false

Indicates that test runs should use the Sharding tests capability. The number of shards to create is equal to the number of devices selected by the configuration.

Default: false

Format of the xctestProductsPath value

The path parameter indicates the location of the .xctestproducts file. The file may be located in either the local workstation storage or in the Perfecto Repository. To differentiate between these locations, use the following format to indicate a location in the Repository:

repository://[PUBLIC | PRIVATE]:/<path to item>

Repository item must have .zip type indicator.

The "repository://" prefix is mandatory for Repository location.

Selective testing parameters

In addition, there are filter parameters that limit the test scenarios executed during the run, as listed in the following table.

Parameter

Possible Value

Meaning

testClassNames


Array of class names that you wish to run. If not specified, all the classes will run.

format:

  • Configuration file -
    "testClassNames":["Class","Class",...]
  • Command line -
    -PtestClassNames="Class;Class;..."

testMethodNames


Array of method names that you wish to run. If not specified, all methods will run. See the following code snippets for the format.

Copy

Configuration file

"testMethodNames":["Class#Method","Class#Method",...]
Copy

Command line

-PtestMethodNames="Class#Method;Class#Method;..."

runUITests

true | false

Indicates if the XCUITest runner application should be installed and executed. If the value is false, the XCUITests will not be run.

Default is true

runUnitTests

true | false

Indicates if the XCTest methods should be executed.

Default is true

testTimeout

number of milliseconds

An amount of time allocated to each test case. This is the number of milliseconds, rounded up to the nearest minute, for a test to run before it fails with a timeout error. For details, see the Apple documentation.

testPlanName

 

Specifies the name of the Xcode Test Plan, which specifies the tests to execute and adds custom configurations to your tests. You can create multiple test plans and select a different plan for each execution. For more information on test plans, see Apple's documentation.

These parameters may be set in the format shown in the following sample code snippets:

Copy
Configuration file format
"xctestProductsPath": "Users/usern/test.xctestproducts",
"testMethodNames": [
      "DemoClass#demoMethod",
      "DemoClass#demoMethod2"
]
Copy
Format of the perfectoGradleSettings clause in the build.gradle file
perfectoGradleSettings { 
    ...
    xctestProductsPath "Users/usern/test.xctestproducts"
} 
Copy
Command line format
gradle perfecto-xctest-vd ... -PxctestProductsPath="Users/usern/test.xctestproducts" -PrunUnitTests="false" 

Screenshot parameters

Screenshots are an important tool for analyzing the test actions. The parameters listed in the following table affect XCUITest test executions.

These parameters may be set in the configuration file, build.gradle file, or the command line.

Restriction: These configuration parameters are applicable only to XCUITest test methods and not to XCTest unit tests.

Parameter

Possible Values

Meaning

takeScreenshotOnTestEnd

true | false

Save all screenshots collected by XCTest during the test at end of the test executions, regardless of the test result status.

takeScreenshotOnTestFailure

true | false

Save all screenshots collected by XCTest during the test at end of the test executions if the test result status is a failure status.

Important: You must turn on the Automatic Screenshots option in the Test Plan (or in the scheme options if you do not use Test Plan). By default, the Automatic Screenshots option only saves screenshots if the test fails. Therefore, if you set takeScreenshotOnTestEnd to true, make sure to change this setting to save all screenshots in the test configuration.

Reporting parameters

The following parameters are used by Smart Reporting to classify the execution report and make it easy to identify the reports for the execution.

Parameter Possible Values Meaning

tags


Set of tags to associate with the execution

jobName


CI identification of the build, used for classification of the report in the CI Dashboard.

jobNumber


CI Job Number of the build

projectName


Name of the project - for classification

projectVersion


Version number assigned to the project for this build

branch


Branch name. like additional tag.

These parameters may be set in the format shown in the following sample code snippets:

Copy
Configuration file format
"tags" : [
       "plugin",
       "unit-test",
       "demo"
],
"projectName": "playground",
"projectVersion": "1.5",
"jobName": "newFeature",
"jobNumber": "45"
Copy
Format of the perfectoGradleSettings clause in the build.gradle file
perfectoGradleSettings { 
    ...
    projectName "playground"
    projectVersion "1.5"
} 
Copy
Command line format
gradle perfecto-xctest-vd ... -Ptags="plugin;uiTest;demo" -PjobName="newFeature"

Export the test execution report

The Perfecto toolset generates a test execution report that can be exported for analysis. It is recommended to:

  1. Use these configuration parameters to define tagsjobName, and jobNumber for the test execution.
  2. Export the report data using these information items with the Smart Reporting Public API.

Location of the configuration file

The plugin execution reads the configuration file whose location is indicated by the configFileLocation parameter that may be supplied in the format shown in the following code snippets

Copy
Format of the perfectoGradleSettings clause in the build.gradle file
perfectoGradleSettings { 
    ...
    configFileLocation "C:\temp\XcuiTest\ConfigFile.json" 
} 
Copy
Command line format
gradle perfecto-xctest-vd ... -PconfigFileLocation="C:\temp\XcuiTest\ConfigFile.json"

If the parameter is not supplied:

  • Tests will run on the default device.
  • The .xctestproducts file must be supplied in either the build.gradle file or on the command line.