iOS configuration parameters for the Gradle Plugin

Important: This document only applies to real devices. For virtual devices, see the relevant article under Virtual mobile devices.

When running your XCTest or XCUITest tests with the Perfecto Gradle Plugin, you can add configuration information for the test execution at the following levels:

Configuration file (recommended): A JSON-formatted file listing parameter assignments (see below).
build.gradle file: In the perfectoGradleSettings clause, include Authentication parameters and indicate the location of the configuration file.
Command-line: Include any of the parameters in the following format: -P<paramName>="<paramvalue>"

The configuration parameters are grouped into the following categories:

Perfecto lab authentication parameters

Parameter Possible Values Meaning

Parameter	Possible Values	Meaning
cloudURL		The URL of the Perfecto Lab to connect to, but without the `.app` notation. For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com)
securityToken	Security Token	Tester's personal security token for the Perfecto Lab.

cloudURL

The URL of the Perfecto Lab to connect to, but without the .app notation.

For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com)

securityToken

Security Token

Tester's personal security token for the Perfecto Lab.

These parameters may be set in the format shown in the following sample code snippets:

Copy

Configuration file format

"cloudURL": "mobilecloud.perfectomobile.com",
"securityToken": "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff",

Copy

build.gradle file format

perfectoGradleSettings { 
    cloudURL "mobilecloud.perfectomobile.com" 
    securityToken "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff" 
}

Copy

Command line format

> gradle perfecto-xctest -PcloudURL="mobilecloud.perfectomobile.com" -PsecurityToken="AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"

Device selection parameters

You can select specific devices or specify a number of random devices.

Select specific devices

You can select to use the following parameters to specify a list of devices. Obtain the values for the following parameters based on the information on the My Devices tab in the Manual Testing view, as listed for the specific devices to select, where:

Device model is the value for model
OS is the value for platformName and platformVersion
Location is the value for location
Device ID is the value for deviceName

To retrieve the value for network, select a device and click the information icon to view device details in the right pane. Then click More details to view network information.

Parameter	Possible Values	Meaning
deviceName		The device ID
description		The device description as defined in the Perfecto Lab.
location		Physical location of the device.
manufacturer		For mobile devices manufacturer of the model. For example: Apple.
model		Device model - for example: iPhone-7
network		Device network, For example AT&T, Verizon
platformName	iOS	Device operating system
platformVersion		Operating system version - for example 11.0.3
resolution		Screen resolution of device

Important: If no device selection parameters are set, the plugin selects a random device from the devices connected to the Perfecto Lab.

These parameters may be set in the configuration file, in the following format:

Copy

Single device selection

"devices": {
    "platformVersion": "10.*",
    "manufacturer" : "Apple"
}

In the following example, four devices are selected. The fourth device in this example ("{}") is chosen randomly from all available iOS devices.

Copy

Multiple device selection

"devices" : [
    {
        "deviceName" : "FA6BR0304878"
    },
    {
        "platformVersion": "11.*",
        "manufacturer" : "Apple"
    },
    {
        "model": "iPad 4",
        "location": "US-MA-BOS",
        "network": "AT&T, Verizon"
    },
    {}
]

Specify the number of random devices

Instead of the devices clause, you can use the following parameter to indicate that the specified number of iOS devices should be selected and the tests run on the devices in parallel.

Parameter	Possible Values	Meaning
numOfDevices	number in range [1,100]	Number of iOS devices to select. If numOfDevices is outside the range, the value will default to select a single device.

This parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:

Copy

"numOfDevices" : 45

Selection retry parameters

If the plugin was unsuccessful in allocating a device, you can use the following parameters to configure a retry mechanism built into the plugin:

Parameter	Possible Values	Meaning
acquireDeviceRetryNumber	default = 0	Number of retries that plugin should attempt to allocate the device.
acquireDeviceRetryInterval	default = 30	Number of seconds to wait between each retry attempt

These parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:

Copy

Configuration file format

"acquireDeviceRetryNumber" : 5,
"acquireDeviceRetryInterval" : 45

Copy

build.gradle file

perfectoGradleSettings { 
    ...
    acquireDeviceRetryNumber  5
    acquireDeviceRetryInterval  45 
}

Copy

Comment line format

gradle perfecto ... -PacquireDeviceRetryNumber=5 -PacquireDeviceRetryInterval=45

Network virtualization parameters

The Gradle plugin supports configuration of a virtual network environment to execute the XCUITest/XCTest tests.

Use the networkVirtualization clause in your configuration file or build.gradle file to define the parameters to define the characteristics of the emulated network. You can configure the virtual network using the same parameters supported by the Perfecto Network virtualization start command. The following table lists the parameters supported.

Sub-Parameter	Possible Values	Meaning
latency	In the range 0-8000 ms.	Latency applied on packets in the Network.
packetLoss	In the range 0-100%	Network packet loss. A reasonable packet loss value should not exceed 5%.
bandwidthIn	In range 3-100,000 Kbps or unlimited.	Limitation on the allowed download network bandwidth into the device.
bandwidthOut	In range 3-100,000 Kbps or unlimited.	Limitation on the allowed upload network bandwidth from the device.
packetCorruption	In the range 0-100%	Network packet corruption. A reasonable packet loss value should not exceed 5%
packetReordering	In the range 0-100%	The percentage of network packets sent immediately without any delay. Used alongside the latency parameter. Moreover, the packets sent immediately will arrive earlier than the packets that were delayed by the defined latency value, essentially creating a packet reordering. A reasonable packet reordering value should not exceed 10%.
packetDuplication	In the range 0-100%	Network packets duplicated. A reasonable packet duplication value should not exceed 3%.
delayJitter	In the range 0-8000 ms.	Random latency variation; the actual latency between latency +- jitter. Used alongside the Latency parameter. For example, if the latency is defined to be 100 ms and the jitter is 10 ms, this causes the added delay to be 100ms - 10ms.
correlation	In the range 0-100%	Network packet value correlation, affecting the Latency, Corruption, Reordering and Duplication. For Latency, Corruption, Reordering and Duplication, the current packet n value will be correlated by this % to previous packet n-1 value. In other words, the current packet value is correlated to the previous packet value. For example, if the correlation is defined to be 25%, the packet loss is 5%, and the previous packet was lost, then the updated packet loss value (for the current packet) would be 75% * 5% + 25%, which equals 28.75%. However, if the previous packet was not lost, then the packet loss value would be 75% * 5%, which equals 3.75%.
blockedDestinations	List of Strings	Network packet block, to specific destinations, defined by domain name, IP address, and IP range destinations in IP Prefix (Slash) notation.
blockedPorts	List of Strings	Network packet block, to specific ports. For example, to block http, define port number 80. To unblock, prefix the value with a '-'. For example, -80.
generateHarFile	true \| false	Indicates if a HTTP Archive (HAR) file should be generated for each test to analyze the traffic of the virtual network. To use the generateHarFile parameter, first install the certificate and configure the device using Perfecto Automation. HAR file generation is currently only supported in enterprise clouds. For details, see Generate and analyze HAR files.
profile		Suggested network virtualization profiles. For supported values, see Network Conditions.

These parameters may be set in the format shown in the following sample code snippets:

Copy

Configuration file format

"networkVirtualization": {
    "latency": 89,
    "packetLoss": 7,
    "bandwidthIn": 5,
    "bandwidthOut": 40
}

Copy

Format of the perfectoGradleSettings clause in the build.gradle file

perfectoGradleSettings { 
    ...
    networkVirtualization {
        packetLoss 5
        profile    4g_lte_good
    }
    ...
}

Location mock

The Gradle plugin supports providing a location for the device when executing XCUITest/XCTest and Espresso tests.

Use the locationMock clause in your configuration file or build.gradle file to define the parameters to define the characteristics of the emulated location. The following is the list of parameters supported.

Sub-Parameter	Possible Values	Meaning
address	Valid address	Address to be set on the device. The address will be automatically translated to latitude, longitude coordinates by the system.
latitude	max/min `+90` to `-90`	Latitude of the location
longitude	max/min `+180` to `-180`	Longitude of the location

Important: Only one of the options should be provided. Use either the "address" clause or the "latitude", "longitude" clauses, but not both.

Examples:

For setting the location of the device to Amal St 13, Rosh Haayin, Israel, use the following configuration:
Copy
```
"locationMock" : {"address" : "Amal St 13, Rosh Haayin, Israel"}
```

For setting the location of the device to the explicit coordinates 32.106710, 34.969384, use the following configuration
Copy
```
"locationMock" : {"latitude" : "32.106710", "longitude" : "34.969384"}
```

Use Google Maps to easily get the coordinates of a place.

Troubleshooting

If the Set device location command does not work, open, close, and reopen the application in which the location is set.

Application parameters

Parameter	Possible Values	Meaning
instrumentationArgs		Custom arguments to pass to the Instrumentation Runner. Format: Configuration file: "instrumentationArgs" : ["value","value",...] Command line: -PinstrumentationArgs="value;value;..." The instrumentationArgs indicates that the test methods use an external framework and the associated screenshot tool. For example: With the KIF framework, use: "instrumentationArgs" : ["KIF"] With the EarlGrey tool, use: "instrumentationArgs" : ["EARLGREY"]
instrumentationEnvVars		Environment Variable values to pass to the Instrumentation Runner. Format: Configuration file: "instrumentationEnvVars" : ["key=value","key=value",...] Command line: -PinstrumentationEnvVars="key=value;key=value;..."
failBuildOnFailure	true \| false	Fail the Gradle build if any test fails or there is a device error. Default is false.
tunnelId		Identifier of the tunnel created by Perfecto Connect. Value should be taken from the response of the perfectoConnect command.
debug		Run the task in debug mode to output more verbose messages.
appPath		Path to the ipa or app file for the application and unit tests. If the parameter links to an .app file (XCode output), the plugin will first convert this to an .ipa file before installing on the device. For details, see XCUITest. Restriction: `.xctestproducts` files are not supported for real devices.
testAppPath		Path to the UI test application Runner IPA or app file. If the parameter links to an .app file (XCode output), the plugin first converts this to an .ipa file before installing on the device. For details, see XCUITest. Restriction: `.xctestproducts` files are not supported for real devices.
shard	true \| false	Indicates that test runs should use the Sharding tests capability of the iOS JUnitRunner. The number of shards to create is equal to the number of devices selected by the configuration. Default: false

Format of app values

The path parameters indicate the location of the .ipa files of the application and the test application. These may be located in either the local workstation storage or in the Perfecto Lab Repository. To differentiate between these locations, use the following format to indicate a location in the Repository:

repository://[PUBLIC | PRIVATE]:/<path to item>

Repository item must have .ipa type indicator.

The "repository://" prefix is mandatory for Repository locations.

Important: The additional forward slash (/) before the application name is required. The path will not work without it.

Selective testing parameters

In addition, there are filter parameters that limit the test scenarios executed during the run, as listed in the following table.

Parameter	Possible Value	Meaning
testClassNames		Array of class names that you wish to run. If not specified, all the classes will run. format: Configuration file - "testClassNames":["Class","Class",...] Command line - -PtestClassNames="Class;Class;..."
testMethodNames		Array of method names that you wish to run. If not specified, all methods will run. format: Configuration file - "testMethodNames":["Class#Method","Class#Method",...] Command line - -PtestMethodNames="Class#Method;Class#Method;..."
runUITests	true \| false	Indicates if the XCUITest Runner application should be installed and executed. If the value is false, the XCUITests will not be run. Default is true.
runUnitTests	true \| false	Indicates if the XCTest methods should be executed. Default is true
testTimeout	number of milliseconds	An amount of time allocated to each test case. If a test case exceeds the timeout, it will be reported as "failed". The full test set will continue with the next test case.

When activating the plugin, with runUITests and runUnitTests set to true, supply both the appPath and testAppPath parameters.

These parameters may be set in the format shown in the following sample code snippets:

Copy

Configuration file format

"testAppPath": "Users/usern/AppProjects/Playground/app/build/outputs/app-Runner.ipa",
"appPath": "Users/usern/AppProjects/Playground/app/build/outputs/app.ipa",
"testMethodNames": [
      "DemoClass#demoMethod",
      "DemoClass#demoMethod2"
]

Copy

Format of the perfectoGradleSettings clause in the build.gradle file

perfectoGradleSettings { 
    ...
    testAppPath "Users/usern/AppProjects/Playground/app/build/outputs/app-Runner.app"
    appPath "Users/usern/AppProjects/Playground/app/build/outputs/app.ipa" 
}

Copy

Command line format

gradle perfecto-xctest ... -PappPath="Users/usern/AppProjects/Playground/app/build/outputs/app-target.ipa" -PrunUnitTests="false"

KIF/EarlGrey tests

When running test methods from external frameworks, for example Earl Grey or KIF, based on the XCTest framework, the plugin considers these to be unit tests, even though they may in fact perform UI automation and testing. Therefore:

Configure the runUnitTests parameter value to true.
If not including any XCUITest test methods, set runUITestsparameter value to false.
Configure the instrumentationArgs parameter to identify the framework (see note above).
Configure the framework to save the screenshots as described here.

Pre-execution and post-execution parameters

The following parameters control whether the application and test applications are uninstalled from the devices, either prior to installing the latest version or at the end of the tests:

Parameter	Sub-parameter	Possible Values	Meaning
installationDetails			The parameters in this Clause affect the installation phase of the test run.
-	preCleanUp	true \| false	If the parameter is set (true) then the ipa files of the application and test-application are uninstalled before installing the new version (supplied by above parameters)
-	resign	true \| false	If the parameter is set (true) then the application will be signed with the Perfecto developer signature. If the parameter is false, developer must sign the application with a developer certificate and supply the UDID of the Perfecto lab device. Learn more about signing the iOS applications. Default is true.
preExecution			The parameters in this clause affect the device used for the test run.
-	reboot	true \| false	If the parameter is set (true) then the device will perform a reboot operation prior to installing the ipa files. Take into account that this is a time-consuming action that may delay the execution of the tests.
postExecution			The parameters in this Clause affect the post-execution phase of the test run.
-	uninstall	true \| false	If the parameter is set (`true`) then the .ipa files of the application and test-Runner are uninstalled after the test has completed execution.

These parameters may be configured in the format shown in the following sample code snippets.

Copy

Configuration file format

"installationDetails" : {
                            "preCleanUp" : true,
                            "resign" : true
                        },
"postExecution" : {"uninstall" : false},
"preExecution" : {"reboot" : "true"}

Copy

Format of the perfectoGradleSettings clause in the build.gradle file

perfectoGradleSettings { 
    ...
    postExecution {
        uninstall false
    }
    installationDetails {
        preCleanUp true
    }
 }

Screenshot parameters

Screenshots are an important tool for analyzing the test actions. The following parameters affect XCUITest test executions.

Parameter	Possible Values	Meaning
takeScreenshotOnTestEnd	true \| false	Save screenshot of device at end of the test executions, regardless of the test result status.
takeScreenshotOnTestFailure	true \| false	Save screenshot of device at end of the test executions, if the test result status is a failure status.
takeScreenshotOnTestStep	true \| false	Save screenshot of device at every logical step in your tests automatically.

These parameters may be set in the configuration file, build.gradle file, or the command-line.

Restriction: These configuration parameters are applicable only to XCUITest test methods. For XCTest unit tests, these may be snapped by the test code, using third-party tools and retrieved by the Gradle plugin for the execution report.

Reporting parameters

The following parameters are used by Smart Reporting to classify the execution report and make it easy to identify the reports for the execution.

Parameter	Possible Values	Meaning
tags		Set of tags to associate with the execution
jobName		CI identification of the build, used for classification of the report in the CI Dashboard.
jobNumber		CI Job Number of the build
projectName		Name of the project - for classification
projectVersion		Version number assigned to the project for this build
branch		Branch name. like additional tag.

These parameters may be set in the format shown in the following sample code snippets:

Copy

Configuration file format

"tags" : [
       "plugin",
       "unit-test",
       "demo"
],
"projectName": "playground",
"projectVersion": "1.5",
"jobName": "newFeature",
"jobNumber": "45"

Copy

Format of the perfectoGradleSettings clause in the build.gradle file

perfectoGradleSettings { 
    ...
    projectName "playground"
    projectVersion "1.5"
}

Copy

Command line format

gradle perfecto-xctest ... -Ptags="plugin;uiTest;demo" -PjobName="newFeature"

Export the test execution report

The Perfecto toolset generates a test execution report that can be exported for analysis. It is recommended to:

Use these configuration parameters to define tags, jobName, and jobNumber for the test execution.
Export the report data using these information items with the Smart Reporting Public API.

Location of the configuration file

The plugin execution reads the configuration file whose location is indicated by the configFileLocation parameter that may be supplied in the format shown in the following code snippets

Copy

Format of the perfectoGradleSettings clause in the build.gradle file

perfectoGradleSettings { 
    ...
    configFileLocation "C:\temp\XcuiTest\ConfigFile.json" 
}

Copy

Command line format

gradle perfecto-xctest ... -PconfigFileLocation="C:\temp\XcuiTest\ConfigFile.json"

If the parameter is not supplied:

Tests will run on a randomly selected available device.
The application and Runner identification parameters (appPath and testAppPath) must be supplied in either the build.gradle file or on the command line.