XCUITest sharding

Perfecto supports splitting your test suite into different tests run separately, in parallel on all selected devices. This enables time optimization in seeing and analyzing the test results and shortening the turnaround.

To learn how to implement XCUITest sharding, read the following sections.

On this page:

Split tests

The XCUITest Gradle Plugin supports XCUITest sharding, running each test shard on a separate device. Using the gradle plugin Configuration file:

Select a number of devices (either selecting specific parameters or random selection).
Configure the shard parameter to true (can be configured in either the Configuration File, build.gradle file, or on the command line).
Plugin will split the tests into a number of shards equal to the number of selected devices.
The following example splits the test into three shards on randomly selected devices:
Copy
```
"numOfDevices": 3,
"shard": true,
...
```

Each shard will run on a separate device and the summary report will identify the index of the shard for that device.

Device selection works on a best effort approach – the number of shards will be determined by the number of devices the system was able to launch. For example, if the configuration file is configured to select 5 devices, but only 3 devices were available – the tests will be split among those three into three different shards.

Shard test output

The console output of the plugin will show the tests running on each device - identifying the device, the test class name, method under test, status.

At the completion of the full test bundle, the summary will identify the results for each device, and provide a link to the DigitalZoom Report Library listing for the full test. In the Report Library view, each shard will be listed on a separate line.

The following is an example of the console output for a test bundle split between two devices:

Copy

Device: 00008020-001424A83650003A Apple iPhone-XS: TestResult{className: SimpleIosAppForTestingUITests.UITestsNegative, methodName: testDefaultValueNotDouble, status: PASS}
> :perfecto-xctest
Device: 00008020-001424A83650003A Apple iPhone-XS: TestResult{className: SimpleIosAppForTestingUITests.UITestsNegative, methodName: testStaticTextNotExistsFailureTest, status: FAIL}
> :perfecto-xctest
Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: TestResult{className: SimpleIosAppForTestingUITests.UITestsPositive, methodName: testTypeText, status: PASS}

Device: 00008020-001424A83650003A Apple iPhone-XS: TestResult{className: SimpleIosAppForTestingUITests.UITestsPositive, methodName: testButtonSetDefaultTextExists, status: PASS}
> :perfecto-xctest
Device: 00008020-001424A83650003A Apple iPhone-XS: TestResult{className: SimpleIosAppForTestingUITests.UITestsPositive, methodName: testButtonSetNewTextExists, status: PASS}
> :perfecto-xctest
Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: TestResult{className: SimpleIosAppForTestingUITests.UITestsPositive, methodName: testTypeTextReturnDefaultValue, status: PASS}

Device: 00008020-001424A83650003A Apple iPhone-XS: TestResult{className: SimpleIosAppForTestingUITests.UITestsPositive, methodName: testStaticTextExists, status: PASS}

Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: TestResult{className: SimpleIosAppForTestingUITests.UITestsNegative, methodName: testButtonNotExists, status: PASS}
> :perfecto-xctest
Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: TestResult{className: SimpleIosAppForTestingUITests.UITestsNegative, methodName: testEnterEmptyValue, status: PASS}
> :perfecto-xctest
Device: 00008020-001424A83650003A Apple iPhone-XS: Disabling instrumented test mode

Device: 00008020-001424A83650003A Apple iPhone-XS: Ending test execution

Device: 00008020-001424A83650003A Apple iPhone-XS: Stopping video recording

Device: 00008020-001424A83650003A Apple iPhone-XS: Closing device

Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: TestResult{className: SimpleIosAppForTestingUITests.UITestsNegative, methodName: testTextDeleted, status: PASS}
> :perfecto-xctest
Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: Disabling instrumented test mode

Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: Ending test execution

Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: Stopping video recording

Device: 9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7: Closing device

----------------------------------------------------------------------------

TOTAL SUMMARY

Your task finished with errors.
FAILED TO RUN ON 0 devices
RAN SUCCESSFULLY ON 2 devices
----------------------------------------------------------------------------

DEVICE 1
Status: OK

Device:
00008020-001424A83650003A Apple iPhone-XS

Results:
1 FAILED
4 PASSED
----------------------------------------------------------------------------

DEVICE 2
Status: OK

Device:
9AC585A8AD1E7B1FD326761706F5642B4EEB2F44 Apple iPad 9.7

Results:
0 FAILED
5 PASSED
----------------------------------------------------------------------------

View the detailed report at:
https://qatestlab.reporting.perfectomobile.com/reporting/library?tags[0]=7521c8d7-05a3-42d9-a302-3453494c79c0&startExecutionTime[0]=1547640448049&endExecutionTime[0]=1547640722484

Finished flow execution


BUILD SUCCESSFUL in 5m 0s