Android configuration parameters for the Gradle Plugin
When running your Android instrumentation tests with the Perfecto Gradle Plugin, you can add configuration information for the test execution at three levels:
- Configuration file (recommended): A JSON-formatted file listing parameter assignments. For a full configuration file example, see the fullConfigFile.json in our Espresso project on GitHub.
- Within the module's build.gradle file: In the perfectoGradleSettings clause, include Authentication parameters and indicate the location of the configuration file.
- On the command-line: Include any of the parameters in the format:
-P<paramName>="<paramvalue>"
The following sections list the parameters for the different categories:
Perfecto Lab authentication parameters
Parameter | Possible Values | Meaning |
---|---|---|
cloudURL |
|
The URL of the Perfecto Lab to connect to, but without the For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com) |
securityToken |
Security Token |
Tester's personal security token for the Perfecto Lab. |
These parameters may be set:
-
In the configuration file, in the following format:
Copy"cloudURL": "mobilecloud.perfectomobile.com",
"securityToken": "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff", -
In the build.gradle file, in the following format:
CopyperfectoGradleSettings {
cloudURL "mobilecloud.perfectomobile.com"
securityToken "AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"
}
-
In the command-line in the following format:
Copy> gradle perfecto-android-inst -PcloudURL="mobilecloud.perfectomobile.com" -PsecurityToken="AAABNg0ODAoPeNqtkT1PwzAQhnf/CkssM...JxQ3HEI8NsX02ff"
Device selection parameters
You can select specific devices or specify a number of random devices.
Select specific devices
You can select to use the following parameters to specify a list of devices. Obtain the values for the following parameters based on the information on the My Devices tab in the Manual Testing view, as listed for the specific devices to select, where:
- Device model is the value for
model
- OS is the value for
platformName
andplatformVersion
- Location is the value for
location
- Device ID is the value for
deviceName
To retrieve the value for network
, select a device and click the information icon to view device details in the right pane. Then click More details to view network information.
Parameter | Possible Values | Meaning |
---|---|---|
deviceName |
|
The device ID |
description |
|
The device description as defined in the Perfecto Lab. |
location |
|
Physical location of the device. |
manufacturer |
|
For mobile devices manufacturer of the model. For example: Samsung, HTC. |
model |
|
Device model - for example: Galaxy S III, HTC One |
network |
|
Device network, For example AT&T, Verizon |
platformName |
Android |
Device operating system |
platformVersion |
|
Operating system or browser version |
resolution |
|
Screen resolution of device or browser |
These parameters may be set in the configuration file, in the following format:
"devices": {
"platformName" : "Android",
"platformVersion": "^[678].*",
"manufacturer" : "Samsung"
}
In the following example, four devices are selected. The fourth device in this example ({}) is chosen randomly from all available Android devices.
"devices" : [
{
"deviceName" : "FA6BR0304878"
},
{
"platformName" : "Android",
"platformVersion": "^[678].*",
"manufacturer" : "Samsung"
},
{
"model": "HTC One",
"location": "US-MA-BOS",
"network": "AT&T, Verizon"
},
{}
]
Specify a number of random devices
When using Espresso, you can use the following parameter, in place of the devices clause, to indicate that the specified number of Android devices should be selected and the Espresso tests run on the devices in parallel.
Parameter | Possible Values | Meaning |
---|---|---|
numOfDevices |
number in range [1,100] |
Number of Android devices to select. If numOfDevices is outside the range, will default to select a single device. |
This parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:
"numOfDevices" : 45
Selection Retry parameters
If the plugin was unsuccessful in allocating the device, you can use the following parameters to configure a retry mechanism built into the plugin:
Parameter | Possible Values | Meaning |
---|---|---|
acquireDeviceRetryNumber |
default = 0 |
Number of retries that plugin should attempt to allocate the device. |
acquireDeviceRetryInterval |
default = 30 |
Number of seconds to wait between each retry attempt |
These parameters may be set in either the configuration file, the perfectoGradleSettings clause of the build.gradle file, or on the command line, in the following format:
build.gradle file
perfectoGradleSettings {
...
acquireDeviceRetryNumber 5
acquireDeviceRetryInterval 45
}
Command line format
gradle perfecto-android-inst ... -PacquireDeviceRetryNumber=5 -PacquireDeviceRetryInterval=45
Network virtualization parameters
The Gradle plugin supports configuration of a virtual network environment to execute the Espresso/Robotium tests.
Use the networkVirtualization clause in your configuration file or build.gradle file to define the parameters to define the characteristics of the emulated network. The virtual network can be configured using the same parameters supported by the Perfecto Network virtualization start command. The following is the list of parameters supported:
Parameter | Possible Values | Meaning |
---|---|---|
latency |
In the range 0-8000 ms. |
Latency applied on packets in the Network. |
packetLoss |
In the range 0-100%. |
Network packet loss. A reasonable packet loss value should not exceed 5%. |
bandwidthIn |
In range 3-100,000 Kbps or unlimited. |
Limitation on the allowed download network bandwidth into the device. |
bandwidthOut |
In range 3-100,000 Kbps or unlimited. |
Limitation on the allowed upload network bandwidth from the device. |
packetCorruption |
In the range 0-100%. |
Network packet corruption. A reasonable packet loss value should not exceed 5% |
packetReordering |
In the range 0-100%. |
The percentage of network packets sent immediately without any delay. Used alongside the latency parameter. Moreover, the packets sent immediately will arrive earlier than the packets that were delayed by the defined latency value, essentially creating a packet reordering. A reasonable packet reordering value should not exceed 10%. |
packetDuplication |
In the range 0-100%. |
Network packets duplicated. A reasonable packet duplication value should not exceed 3%. |
delayJitter |
In the range 0-8000 ms. |
Random latency variation; the actual latency between latency +- jitter. Used alongside the Latency parameter. For example, if the latency is defined to be 100 ms and the jitter is 10 ms, this causes the added delay to be 100ms - 10ms. |
correlation |
In the range 0-100% |
Network packet value correlation, affecting the Latency, Corruption, Reordering and Duplication. For Latency, Corruption, Reordering and Duplication, the current packet n value will be correlated by this % to previous packet n-1 value. In other words, the current packet value is correlated to the previous packet value. For example, if the correlation is defined to be 25%, the packet loss is 5%, and the previous packet was lost, then the updated packet loss value (for the current packet) would be 75% * 5% + 25%, which equals 28.75%. However, if the previous packet was not lost, then the packet loss value would be 75% * 5%, which equals 3.75%. |
blockedDestinations |
List of Strings |
Network packet block, to specific destinations, defined by domain name, IP address, and IP range destinations in IP Prefix (Slash) notation. |
blockedPorts |
List of Strings |
Network packet block, to specific ports. For example, to block http, define port number 80. To unblock, prefix the value with a '-'. For example, -80. |
generateHarFile |
true | false |
Indicates if a HTTP Archive (HAR) file should be generated for each test to analyze the traffic of the virtual network. To use the generateHarFile parameter, first install the certificate and configure the device using Perfecto Automation. HAR file generation is currently only supported in enterprise clouds. For details, see Generate and analyze HAR files. |
profile |
|
Suggested network virtualization profiles. See the network conditions for supported values. |
These parameters may be set:
-
In the configuration file, in the following format:
Copy"networkVirtualization": {
"latency": 89,
"packetLoss": 7,
"bandwidthIn": 5,
"bandwidthOut": 40,
"generateHarFile": true
}
-
In the perfectoGradleSettings clause of the build.gradle file:
CopyperfectoGradleSettings {
...
networkVirtualization {
packetLoss 5
profile 4g_lte_good
}
...
}
Location mock
The Gradle plugin supports providing a location for the device when executing Espresso tests.
Use the locationMock clause in your configuration file or build.gradle file to define the parameters to define the characteristics of the emulated location. The following is the list of parameters supported. Only one of the options should be provided. Use either the "address" clause or the "latitude", "longitude" clauses, but not both.
Sub-Parameter | Possible Values | Meaning |
---|---|---|
address |
Valid address |
Address to be set on the device. The address will be automatically translated to latitude, longitude coordinates by the system. |
latitude |
max/min |
Latitude of the location |
longitude |
max/min +180 to -180 |
Longitude of the location |
Examples:
-
For setting the location of the device to Amal St 13, Rosh Haayin, Israel, use the following configuration:
Copy"locationMock" : {"address" : "Amal St 13, Rosh Haayin, Israel"}
-
For setting the location of the device to the explicit coordinates 32.106710, 34.969384, use the following configuration:
Copy"locationMock" : {"latitude" : "32.106710", "longitude" : "34.969384"}
Troubleshooting
If the Set device location command does not work:
- Open, close, and reopen the application in which the location is set.
On Android devices, verify the following settings in the device:
-
Allow mock locations:
Go to: Settings > Developer options > Allow mock locations
-
High accuracy:
Go to: Setting > Location > Mode > High accuracy
Application parameters
Parameter | Possible Values | Meaning |
---|---|---|
androidAppPath |
The name and path of the Android application bundle to be uploaded. During the execution, Perfecto creates the relevant APK from the bundle according to the requested device. Must be used in conjunction with the testApkPath parameter. Mutually exclusive with the apkPath parameter. |
|
androidOrchestrator |
true | false |
Indicates that plugin should execute each of the app tests in a separate process using the Android Test Orchestrator. Find JUnitRunner's Orchestrator official documentation here. Default: false |
androidX |
true | false |
Indicates that plugin should use androidx versions of test.services and test.orchestrator. By default, the plugin recognizes if androidx is used. |
apkName |
|
The name of the application to test. The plugin will assume that the application's apk file is located in the default location for the build, as defined by the gradle build process. |
apkPath |
|
Path to the application apk file. Use this field if the run is not part of your application build or apk file is output to a non-standard folder (other than "app") by the build process. See Format of apkPath, testApkPath, and androidAppPath values below. Must be used in conjunction with the testApkPath parameter. Mutually exclusive with the androidAppPath parameter. |
debug |
|
Run the task in debug mode to output more verbose messages. |
failBuildOnFailure |
true | false |
Report a failure to the Gradle console if any test fails. Default is false. |
instrumentationArgs |
|
Custom arguments to pass to the Instrumentation Runner. Format: |
tunnelId |
|
Identifier of the tunnel created by Perfecto Connect. Value should be taken from the response of the perfectoConnect command. Perfecto Connect is a limited access feature. Contact Perfecto Support to add to your Perfecto Lab. |
shard |
true | false |
Indicates that test runs should use the Sharding tests capability of the Android JUnitRunner. The number of shards to create is equal to the number of devices selected by the configuration. Default: false |
testApkPath |
|
Path to the test APK. Use this field if the run is not part of your application build or your APK is output to a different locationlocation than the default. See Format of apkPath, testApkPath, and androidAppPath values below. |
useTestApkOnly |
true | false |
By default, the Gradle plugin loads two APK files: the original APK file and the Test APK file. If this parameter is set to true, only the Test APK file is uploaded and used for testing. Default: false |
When activating the plugin:
- Within an Android Studio project, supply the apkName parameter.
- As a standalone activation, supply both the apkPath and testApkPath parameters or the androidAppPath and testApkPath parameters, depending on whether you work with an APK file or an Android application bundle.
Format of apkPath, testApkPath, and androidAppPath values
The path parameters indicate the location of the APK or AAB files of the application and the test-application. These may be located in either the local workstation storage or in the Perfecto Repository. To differentiate between these locations, use the following format to indicate a location in the Repository:
repository://[PUBLIC | PRIVATE]:/<app name>
Repository items must have .apk or .aab type indicator.
The "repository://" prefix is mandatory for Repository locations.
/
) before the application name is required. The path will not work without it. Selective testing parameters
In addition, there are filter parameters that limit the test scenarios executed during the run:
Parameter | Possible Values | Meaning |
---|---|---|
testSize | large | medium | small | Runs a test method annotated by size. The annotations are @LargeTest, @MediumTest, and @SmallTest. See examples below. |
testClassNames |
|
Array of class names that you wish to run. If not specified, all the classes will run. Use either testClassNames or testMethodNames, not both. |
testMethodNames |
|
Array of method names that you wish to run. If not specified, all methods will run. Use either testClassNames or testMethodNames, not both.
|
testTimeout | number of milliseconds |
An amount of time allocated to each test case. If a test case exceeds the timeout, it will be reported as "failed". The full test set will continue with the next test case. |
These parameters may be set as shown in the following sample codes snippets:
"testApkPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug-AndroidTest.apk",
"apkPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/debug/app-debug.apk",
"testSize": "small",
"testMethodNames": [
"com.perfectomobile.com.DemoClass#demoMethod",
"com.perfectomobile.com.DemoClass#demoMethod2"
],
"androidOrchestrator" : true
perfectoGradleSettings {
...
testApkPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug-AndroidTest.apk"
apkPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug.apk"
}
gradle perfecto-android-inst ... -PapkPath="Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug.apk" -PtestSize="large" -PandroidOrchestrator=true
"testApkPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug-AndroidTest.apk",
"androidAppPath": "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/bundle/debug/app-debug.aab",
"testSize": "small",
"testMethodNames": [
"com.perfectomobile.com.DemoClass#demoMethod",
"com.perfectomobile.com.DemoClass#demoMethod2"
],
"androidOrchestrator" : true
perfectoGradleSettings {
...
testApkPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/apk/androidTest/debug/app-debug-AndroidTest.apk"
androidAppPath "Users/usern/AndroidStudioProjects/Playground/app/build/outputs/bundle/debug/app-debug.aab"
}
gradle perfecto-android-inst ... -PandroidAppPath="Users/usern/AndroidStudioProjects/Playground/app/build/outputs/bundle/debug/app-debug.aab" -PtestSize="large" -PandroidOrchestrator=true
Pre-execution and post-execution parameters
The following parameters control whether the application and test apk files are uninstalled from the devices, either prior to installing the latest version or at the end of the tests:
Parameter | Sub-parameter | Possible Values | Meaning |
---|---|---|---|
installationDetails |
|
|
The parameters in this Clause affect the installation phase of the test run. |
- |
preCleanUp |
true | false |
If the parameter is set (true) then the apk files of the application and test-application are uninstalled before installing the new version (supplied by above parameters) |
|
grantAll |
true | false |
If the parameter is set (true) then the application is granted all Android permissions listed in the application manifest Restriction: Only applicable to Android 6.0 or later.
|
preExecution |
|
|
The parameters in this Clause affect the device used for the test run. |
- |
reboot |
true | false |
If the parameter is set (true) then the device will perform a reboot operation prior to installing the apk files. Take into account that this is a time-consuming action that may delay the execution of the tests. |
- |
screenOrientation |
landscape| portrait |
Configures how to orient the device when executing the tests. This affects the application display, test execution, dashboard display, and report video. |
postExecution |
|
|
The parameters in this Clause affect the post-execution phase of the test run. |
- |
uninstall |
true | false |
If the parameter is set (true) then the apk files of the application and test-application are uninstalled after the test has completed execution. |
These parameters may be configured as shown in the following sample code snippets:
"installationDetails" : { "preCleanUp" : true},
"postExecution" : { "uninstall" : false },
"preExecution" : { "screenOrientation" : "PORTRAIT" }
perfectoGradleSettings {
...
postExecution {
uninstall false
}
installationDetails {
preCleanUp true
}
}
Screenshot parameters
Screenshots are an important tool for analyzing the test actions, these may be snapped by the Espresso test code, or the plugin can be configured to add screenshots when the test run ends. The following parameters specify when the plugin takes a screenshot:
Parameter | Possible Values | Meaning |
---|---|---|
takeScreenshotOnTestEnd |
true | false |
Save screenshot of device at end of the test executions, regardless of the test result status. |
takeScreenshotOnTestFailure |
true | false |
Save screenshot of device at end of the test executions, if the test result status is a failure status. |
These parameters may be set in the configuration file, build.gradle file, or the command-line.
Reporting parameters
The following parameters are used by Smart Reporting to classify the execution report and make it easy to identify the reports for the execution.
Parameter | Possible Values | Meaning |
---|---|---|
tags |
|
Set of tags to associate with the execution |
jobName |
|
CI identification of the build, used for classification of the report in the CI Dashboard. |
jobNumber |
|
CI Job Number of the build |
projectName |
|
Name of the project - for classification |
projectVersion |
|
Version number assigned to the project for this build |
branch |
|
Branch name. like additional tag. |
These parameters may be set as shown in the following sample code snippets:
"tags" : [
"plugin",
"espresso",
"demo"
],
"projectName": "playground",
"projectVersion": "1.5",
"jobName": "newFeature",
"jobNumber": "45"
perfectoGradleSettings {
...
projectName "playground"
projectVersion "1.5"
}
gradle perfecto-android-inst ... -Ptags="plugin;espresso;demo" -PjobName="newFeature"
Export the execution report
The Perfecto toolset generates a test execution report that can be exported for analysis. It is recommended to:
- Use these configuration parameters to define tags, jobName, and jobNumber for the test execution.
- Export the report data using these information items with the Smart Reporting Public API.
Location of the configuration file
The plugin execution reads the configuration file whose location is indicated by the configFileLocation parameter that may be supplied as follows:
perfectoGradleSettings {
...
configFileLocation "C:\temp\Espresso\ConfigFile.json"
}
gradle perfecto-android-inst ... -PconfigFileLocation="C:\temp\Espresso\ConfigFile.json"
If the parameter is not supplied:
- If running in Android Studio project - default location is <project dir>/src/main/assets/perfecto/instrumentedTest/configFile.json
- If running in Standalone mode or the file is not found at the default location, then tests will run on a randomly selected available Android device.
- The application and test-application identification parameters (apkPath and testApkPath) must be supplied in either the build.gradle file or on the command line.