Espresso tests for virtual devices

In this step, we work with build configuration scripts called build.gradle. Android Studio projects are integrated with Gradle and include several build.gradle files – one at the project level and one for each application. In our sample project:

• The project-level build.gradle file is located here:

  https://github.com/PerfectoMobileSA/PerfectoEspressoProject/blob/master/PerfectoEspresso/build.gradle

  Copy

  Perfecto sample project build.gradle file

  // Top-level build file where you can add configuration options common to all sub-projects/modules.
  
  buildscript {
      repositories {
          jcenter()
          google()
          maven {
              url "https://repo1.perfectomobile.com/public/repositories/maven"
              }
      }
      dependencies {
          classpath 'com.android.tools.build:gradle:3.2.1'
          classpath 'com.perfectomobile.instrumentedtest.gradleplugin:plugin:+'
  
          // NOTE: Do not place your application dependencies here; they belong
          // in the individual module build.gradle files
      }
  }
  
  allprojects {
      repositories {
          jcenter()
          google()
      }
  }
  
  task clean(type: Delete) {
      delete rootProject.buildDir
  }

• The app-level build.gradle file is located here:

  https://github.com/PerfectoMobileSA/PerfectoEspressoProject/blob/master/PerfectoEspresso/app/build.gradle

  Copy

  Perfecto sample app build.gradle file

  apply plugin: 'com.android.application'
  apply plugin: 'com.perfectomobile.instrumentedtest.gradleplugin'
  
  perfectoGradleSettings {
      configFileLocation "configFile.json"}
  
  android {
      compileSdkVersion 26
      buildToolsVersion '28.0.3'
  
      defaultConfig {
          applicationId "com.example.perfecto.tipcalculator"        minSdkVersion 14
          targetSdkVersion 26
          versionCode 1
          versionName "1.0"        testInstrumentationRunner 'android.support.test.runner.AndroidJUnitRunner'
      }
      buildTypes {
          release {
              minifyEnabled false
              proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
          }
      }
  }
  
  dependencies {
      implementation fileTree(dir: 'libs', include: ['*.jar'])
      testImplementation 'junit:junit:4.12'
      implementation 'com.android.support:appcompat-v7:24.2.0'
      androidTestImplementation 'com.android.support.test.espresso:espresso-core:2.2.2', {
          exclude group: 'com.android.support', module: 'support-annotations'
      }
  }

To add the Perfecto plugin dependency to your build.gradle file:

1. In Android Studio, open your project-level build.gradle file

2. Add the lines that define the location of the plugin library and the dependency on the plugin to the build.gradle file. Gradle will look to verify that the plugin is installed before performing the task.

   Tip: This is the recommended configuration method. However, if there are problems using the automatic download method described here, you can download the Perfecto Gradle plugin manually.

   Do one of the following, depending on whether the plugin library is already downloaded or you want to locate and download the plugin library automatically:

   • To configure Gradle to automatically locate and download the plugin library, add the following lines to the build.gradle file:

     Copy

     buildscript {
         repositories {
             google()
             jcenter()
             maven {
                 url "https://repo1.perfectomobile.com/public/repositories/maven/"        
                 }
         }
     
         dependencies {
             classpath 'com.android.tools.build:gradle:3.2.1'
             classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"        // NOTE: Do not place your application dependencies here; they belong
             // in the individual module build.gradle files
         }
     }

   • If the plugin library is already downloaded to a folder (for example the libs sub-folder), add the following lines to the build.gradle file:

     Copy

     buildscript {
         repositories {
             google()
             jcenter()
       
             flatDir dirs: 'libs'
         }
       
         dependencies {
             classpath "com.perfectomobile.instrumentedtest.gradleplugin:plugin:+"    
         }
     }

3. Save the file.

4. Open the app-level build.gradle file and add the lines that defines the plugin task.

   Copy

   apply plugin: 'com.perfectomobile.instrumentedtest.gradleplugin'

5. Add the following line to load any configurations from the configFile.json configuration file.

   Copy

   perfectoGradleSettings {
       configFileLocation "configFile.json"
       }

6. Save the file.

In this step, you create the JSON text file that contains all configuration settings. This is the recommended practice. Configurations include the URL of the Perfecto lab, the security token to use, which devices to select, which tags to use, and so on.

{
  "cloudURL": "<<cloud name>>",
  "securityToken": "<<SECURITY TOKEN>>",
  "devices": [
      {},
      {
            "platformName": "Android",
            "platformVersion": "13",
            "model": "pixel.*"
      }
  ],
  "jobName": "some_job",
  "jobNumber": 1,
  "espresso", "plugin"
  "branch": "some_branch",
  "projectName": "My_Espresso_project",
  "projectVersion": "v1.0",
  "tags": [
    "espresso", "plugin"  ],
  "apkPath": "app/build/outputs/apk/debug/app-debug.apk",
  "testApkPath": "app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk",
  "installationDetails" : {"grantAll" : "true"},
  "postExecution" : {"uninstall" : "false" },
  "debug": false,
  "failBuildOnFailure": false,
  "takeScreenshotOnTestFailure": true,
  "shard": false,
  "testTimeout" : 60000
}

1. Create a configuration file (configFile.json) as a JSON text file in a known folder.

2. Add the connection parameters to the configuration file.

   Copy

     "cloudURL": "<cloud name>",
     "securityToken": "<SECURITY TOKEN>"

   where:

   • cloudURL is the URL of the Perfecto Lab to connect to, but without the .app notation. For example: mobilecloud.perfectomobile.com (but not mobilecloud.app.perfectomobile.com)

   • securityToken is the tester's personal security token for the Perfecto lab. See also Generate security tokens.

3. Add the device selection parameters to the configuration file, as shown in the following examples. For details on platform-specific parameters, see Android configuration parameters for the Gradle Plugin | Virtual devices.

   If a device has not been explicitly selected, the system selects the default Android device.

   For example, add the following to select:

   • Default Android device:

     Copy

     "devices": [
         {}
       ],

   • Specific devices, such as two devices where one is a Google Pixel device version 12 and one is the default Android device:

     Copy

      "devices": [
           {},
           {
                 "model": "pixel.*",
                 "platformVersion": "12"       
           }
       ],

   • A number of default Android devices (useful for sharding):

     Copy

     "numOfDevices": 20

4. Add the reporting parameter settings to add tags to the execution report. For more information on adding tags, see Tag-driven reports (RTDD workflow).

   Copy

     "jobName": "some_job",
     "jobNumber": 1,
     "branch": "some_branch",
     "projectName": "My_Espresso_project",
     "projectVersion": "v1.0",
     "tags": [
       "espresso", "plugin"

   where:
   

   • jobName is the CI identification of the build, used for classification of the report in the CI Dashboard.
     

   • jobNumber is the CI job number of the build.
     

   • branch is the job branch as reported in the execution context.
     

   • projectName is the name of the project, for classification.
     

   • projectVersion is the version number assigned to the project for this build.


   • tags is the set of tags to associate with the execution.

5. Add the application parameters to identify where the application files are located.
   For our Android application, these are apkPath and testApkPath:

   Copy

     "apkPath": "app/build/outputs/apk/debug/app-debug.apk",
     "testApkPath": "app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk",

   where:

   • apkPath refers to the path of the .apk file for the application.
     

   • testApkPathis the path to the UI test application runner .apk file.
     

   Important: When testing an Android (aar or jar) library, set both the apkPath and testApkPath fields to the location specified for the assembleAndroidTest gradle task. By default, this location is: <project folder>/app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk. You can supply additional parameters if you need to limit the test application to only execute specific methods or classes.

6. Save the configuration file.

1. Open a command-line (terminal) window in the folder where you want to execute the plugin. 

2. Execute the plugin using the following command:
   

   Here we also supply the full path to the configuration file created in step 3. You can add other configuration parameters (except for device selection) to the command as needed.

   Copy

   gradlew perfecto-android-inst-vd -PconfigFileLocation=configFile.json

   This will:
   

   • Select the devices as specified in the Device selection parameters of the configuration file (or a random device if no specification is provided).
     

   • Install the application and test .apk files onto the device.
     

   • Run the test methods (based on the configuration parameters).
     

   • Send output to the console window.
     

   • Generate an execution report that can be viewed in the Test analysis with Smart Reporting interface.

   The command-line parameters can set any of the configuration parameters, except for device selection parameters. For more information, see Android configuration parameters for the Gradle Plugin | Virtual devices.

   For information on running the Gradle plugin over a proxy connection, see Proxy connection.
   During the execution, the plugin reports on the progress of the execution and the completion of each test method to the command-line window.

   Copy

   Sample progress report

   Task :app:perfecto-android-inst-vd
   Parsing configuration file: configFile.json
   Parsed configuration file configFile.json successfully
   
   Starting Execution
   Uploading files
   Files uploaded
   Your session id: c97a6958-a21d-43a1-961d-ad91b379c230
   [cddc16a32e9e] [pixel 5 - 12] [2023-03-21 08:12:10] Installing app-debug.apk
   [cddc16a32e9e] [pixel 5 - 12] [2023-03-21 08:12:10] Installing app-debug-androidTest.apk
   [cddc16a32e9e] [pixel 5 - 12] [2023-03-21 08:12:12] Executing test
   [aaff75bf6175] [pixel 6 - 13] [2023-03-21 08:12:14] Installing app-debug.apk
   [aaff75bf6175] [pixel 6 - 13] [2023-03-21 08:12:14] Installing app-debug-androidTest.apk
   [cddc16a32e9e] [pixel 5 - 12] [2023-03-21 08:12:18] [PASS] Class: com.example.myfirstapp.AllFailedButOneTest, Method: test4
   [aaff75bf6175] [pixel 6 - 13] [2023-03-21 08:12:18] Executing test
   [cddc16a32e9e] [pixel 5 - 12] [2023-03-21 08:12:25] [FAIL] Class: com.example.myfirstapp.AllFailedButOneTest, Method: test5
   [aaff75bf6175] [pixel 6 - 13] [2023-03-21 08:12:28] [PASS] Class: com.example.myfirstapp.AllFailedButOneTest, Method: test4 
   Test execution finished

   At the end of the execution, the command-line window displays a high-level summary report of the completion status for each device. The report includes:

1. • Resolution of configuration settings: The configuration file used and the list of the configuration settings for the test run
   • Progress notifications as the test is configured, installed, and executed, including notifications of the start and completion of:
     • Installing the application and UI runner files
     • Executing the test
   • Completion status for each test method on each device

   At the end of the summary report, the plugin provides the URL of the single test report (STR) for the execution run.

2. Copy

   Sample summary report

   -----------------------------------------------------
   
   Total Summary
   
   Results:
   Total: 14
   Passed: 4
   Failed: 10
   Skipped: 0
   
   Run successfully on 1 devices
   
   -----------------------------------------------------
   
   Device: pixel 6 - 13
   
   ExecutionId: aaff75bf6175
   
   Results:
   Total: 7
   Passed: 2
   Failed: 5
   Skipped: 0
   
   -----------------------------------------------------
   
   Report url: https://demo.reporting.perfectomobile.com/reporting/library? tags%5B0%5D%3D==**************************

To access the execution report, copy the report URL from the summary report on your console and open the URL in your browser. For example:

Report url:: 
https://demo.reporting.perfectomobile.com/reporting/library? tags%5B0%5D%3D==**************************

For more information on Smart Reporting, see Test analysis with Smart Reporting. For details on the STR, see Single test report (STR).