XCUITest tests for virtual devices

Executing XCUITest and XCTest tests on iOS simulators involves creating an .xctestproducts file that contains all test products.

You can prepare this file using the Xcode IDE or the xcodebuild command-line tool. Because of a restriction that applies when peparing the files using the Xcode IDE (details are included in the procedure), consider peparing them using the command-line tool.

Click the relevant procedure to expand the steps.

To create the .xctestproducts file using the xcodebuild command-line tool

• Run the following command:

  Copy

  xcodebuild build-for-testing [-project *.xcodeproj | -workspace *.xcworkspace] -destination generic/platform='iOS Simulator' -scheme <YOUR_SCHEME> -testProductsPath <PATH>

To create the .xctestproducts file using the Xcode IDE

1. In Apple Xcode, in the Navigator area on the left, click your project.

2. On the Build Settings tab, under Architectures, set Build Active Architectures Only to No. This will build your project to support all architectures.

   

3. On the toolbar at the top of the project window, select an iOS simulator device. This sets the scheme destination.

   

4. From the Product menu, select Perform Action > Build and Export Test Products.

   

   Restriction: If your project is configured to use Test Plan, when using the Xcode IDE method to create the test products, only the selected Test Plan will be included in the test products. Therefore, if you have multiple test plans, first choose which test plan to include in the test product (Product > Test Plan), and then create the .xctestproducts file.

5. Select the export destination for the .xctestproducts file, and then click Build and Export.

Optionally, you can upload your test products (the .xctestproducts file) to the Perfecto repository. This is helpful when you later specify the repository path in the xctestProductsPath parameter.

To upload test products to the repository:

1. On a macOS and Linux machine, zip the .xctestproducts file.

   Restriction: Windows machines cannot properly compress the .xctestproducts file.

2. Upload the ZIP file to the repository. For details on uploading the file through the Perfecto UI, see Manage the repository.

In this step, we create a build configuration script called build.gradle that defines the project and its tasks.

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
    repositories {
        jcenter()
            maven {
          url "https://repo1.perfectomobile.com/public/repositories/maven"        }
    }

    dependencies {
        classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"    }
}

apply plugin: 'com.perfectomobile.instrumentedtest.gradleplugin'

perfectoGradleSettings {
    configFileLocation "configFile.json"}

1. Create a new file named build.gradle.

2. Add the lines that define the location of the plugin library and the dependency on the plugin to the build.gradle file. Gradle will look to verify that the plugin is installed before performing the task.

   This is the recommended configuration method. However, if there are problems using the automatic download method described here, you can download the Perfecto Gradle plugin manually.

   Do one of the following, depending on whether the plugin library is already downloaded or you want to locate and download the plugin library automatically:

   • To configure Gradle to automatically locate and download the plugin library, add the following lines to the build.gradle file:

     Copy

     buildscript {
         repositories {
             jcenter()
             maven {
                 url "https://repo1.perfectomobile.com/public/repositories/maven"        }
         }
       
         dependencies {
             classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"    }
     }

   • If the plugin library is already downloaded to a folder (for example the libs sub-folder), add the following lines to the build.gradle file:

     Copy

     buildscript {
         repositories {
             jcenter()
       
             flatDir dirs: 'libs'
         }
       
         dependencies {
             classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"    }
     }

3. Add the line that defines the plugin task:

   Copy

   apply plugin: 'com.perfectomobile.instrumentedtest.gradleplugin'

4. Add the following line to load any configurations from the configFile.json configuration file.

   Copy

   perfectoGradleSettings {
       configFileLocation "configFile.json"}

In this step, you create the JSON text file that contains all configuration settings. This is the recommended practice. Configurations include the URL of the Perfecto Lab, the security token to use, which devices to select, which tags to use, and so on.

{
  "cloudURL": "<<cloud name>>",
  "securityToken": "<<SECURITY TOKEN>>",
  "xctestProductsPath": "Test.xctestproducts",
  "devices": [
      {
              "platformVersion": "15.*",
              "model": "iPhone13"          },
          {
                  "platformVersion": "16.*",
                  "model": "iPhone 14"              }
  ],
  "tags": [
    "xcuitest",
    "sample"  ],
  "projectName": "IOS-XCUITest",
  "projectVersion": "v10.9-test",
  "appName": "Sample_App",
  "jobName": "some_job",
  "jobNumber": 1,
  "branch": "master",
  "failBuildOnFailure": false,
  "runUnitTests": true,
  "runUiTests": true,
  "shard": false,
  "takeScreenshotOnTestEnd": true,
  "testTimeout": "60000"
}

1. Create a configuration file (configFile.json) as a JSON text file in a known folder.

2. Add the connection parameters to the configuration file.

   Copy

     "cloudURL": "<your cloud url>",
     "securityToken": "<your cloud security token>"

   where:

   • cloudURL is the URL of the Perfecto Lab to connect to, but without the .app notation and without protocol information (http:// or https://). For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com)

   • securityToken is the tester's personal security token for the Perfecto Lab. See also Generate security tokens.
     

3. Add the device selection parameters to the configuration file, as shown in the following examples. For details on platform-specific parameters, see iOS configuration parameters for the Gradle Plugin.

   If a device has not been explicitly selected, the system selects the default iOS device.

   The following code samples illustrate how to select any iOS device, specific devices, and a number of default iOS devices.

   Copy

   Any available iOS device (based on the gradle task)

   "devices": [
       {}
     ],

   Copy

   Specific devices, such as two devices where one is an iPhone 13 device of iOS version 16 (using a regular expression) and one is a default iOS simulator.

   "devices": [
       {
           "model":"iPhone 13",
           "platformVersion":"16.*"    },
       {}
   ]

   Copy

   A number of default iOS devices

   "numOfDevices": 20

4. Add the reporting parameter settings to add tags to the execution report. For more information on adding tags, see Tag-driven reports (RTDD workflow).

   Copy

   "tags": ["plugin", "xctest", "demo"],
   "projectName": "playground",
   "projectVersion": "1.5",
   "jobName": "newFeature",
   "jobNumber": "45",

   where:

   • tags is the set of tags to associate with the execution.

   • projectName is the name of the project, for classification.

   • projectVersion is the version number assigned to the project for this build.

   • jobName is the CI identification of the build, used for classification of the report in the CI Dashboard.

   • jobNumber is the CI job number of the build.

5. Add the test products parameter to identify where the .xctestproducts file is located, as follows:

   Copy

   "xctestProductsPath": "<path-to-xctestproducts>"

   where xctestProductsPath is the path to the .xctestproducts containing the test products.

6. Save the configuration file.

1. Open a command-line (terminal) window in the folder where you want to execute the plugin. 

2. Execute the plugin using the following command:

   Here we also supply the full path to the configuration file created in step 3. You can add other configuration parameters (except for device selection) to the command as needed.

   Copy

   gradle perfecto-xctest-vd -PconfigFileLocation=configFile.json

   This will:

   • Select the devices as specified in the Device selection parameters of the configuration file (or a random device if no specification is provided).

   • Install the application and test the .app files on the device.
     

   • Run the test methods (based on the configuration parameters).
     

   • Send output to the console window.
     

   • Generate an execution report that can be viewed in the Test analysis with Smart Reporting interface.

   The command-line parameters can set any of the configuration parameters, except for device selection parameters. For more information, see iOS configuration parameters for the Gradle Plugin.

   For information on running the Gradle plugin over a proxy connection, see Proxy connection.

   During the execution, the plugin reports on the progress of the execution and the completion of each test method to the command-line window.

   Copy

   Starting a Gradle Daemon (subsequent builds will be faster)
   :perfecto-xctest
   Parsing configuration file: configFile.json
    
   Parsed configuration file configFile.json successfully
    
   Starting Execution
    
   Files uploaded
   
   Your session id: ***
   
   [71cc7562ccf6] [iPhone 13 - 15.5] [2023-05-16 10:45:52] Executing test. 
   
   [71cc7562ccf6] [iPhone 13 - 15.5] [2023-05-16 10:46:23] [FAIL] Class: TestUITests.SampleTests, Method: testFailure
   
   [71cc7562ccf6] [iPhone 13 - 15.5] [2023-05-16 10:46:29] [PASS] Class: TestUITests.TestUITests, Method: testPositiveCase
   
   [71cc7562ccf6] [iPhone 13 - 15.5] [2023-05-16 10:46:33] [PASS] Class: TestUITests.TestUITests, Method: testSample
   
   [71cc7562ccf6] [iPhone 13 - 15.5] [2023-05-16 10:46:50] Execution finished
   
   Test execution finished

   At the end of the execution, the command-line window displays a high-level summary report of the completion status for each device. The report includes:

   • Resolution of configuration settings: The configuration file used and the list of the configuration settings for the test run
   • Progress notifications as the test is configured and executed, including notifications of the start and completion of:
     • Allocating the device
     • Installing the application and UI runner files
     • Executing the test
   • Completion status for each test method on each device

   At the end of the summary report, the plugin provides the URL of the single test report (STR) for the execution run.

   Copy

   Total Summary
   
   Results:
   Total: 3
   Passed: 2
   Failed: 1
   Skipped: 0
   
   Run successfully on 1 devices
   
   -----------------------------------------------------
   
   Device: iPhone 13 - 15.5
   
   ExecutionId: 71cc7562ccf6
   
   Results:
   Total: 3
   Passed: 2
   Failed: 1
   Skipped: 0
   
   -----------------------------------------------------
   
   Report url: https://demo.app.perfectomobile.com/reporting/library?tags%5B0%5D%3D=***

   If allocating a device fails, the following error message appears:

   Copy

   Failed to run on 1 devices
   The following executions exit with error: 
   ExecutionId: 66b4a4ff31ef. Message: failed to execute message: Error message

To access the execution report, copy the report URL from the summary report on your console and open the URL in your browser.