Smart Reporting Public API

Use the Smart Reporting HTTP API to retrieve your entire test execution content in JSON format. You can retrieve the following types of test execution data:

  • Execution Driver Report - presents a list of all executions performed. This report includes links to the artifacts (can be downloaded via the links to them): video, device logs, PCAP files, vitals.

  • Single Test Commands Report - presents the test commands of a particular test execution. The test commands are presented according to their execution order and includes the metadata (time and length of execution) for the test execution. This report includes links to all artifacts (can be downloaded via the links to them): video, screenshots, device logs, PCAP files, vitals and PDF reports (Summary PDF, Single Test Report PDF).

Smart Reporting API terminology

Using the Smart Reporting API requires that you be acquainted with the following information:

  • CQL_NAME - The "private" name of your Perfecto CQ Lab.
    For example: If you use the lab at: https://demo.perfectomobile.com then the CQL_NAME=demo
  • Reporting Server - Perfecto Smart Reporting information is stored in a repository on the reporting server. The URI of the reporting server is the same as your Perfecto CQ Lab with ".app" appended to the CQL_NAME.
    For example: If you access your Perfecto Lab using the URL https://demo.perfectomobile.com then the corresponding reporting server would be accessed at: https://demo.app.perfectomobile.com
  • Driver Execution ID - Each execution has a unique identifier. To obtain the id for a particular execution use the following code sequence in your automation script:

    Copy
    Capabilities capabilities = driver.getCapabilities();
    String executionId = (String) capabilities.getCapability("executionId");

Demo video

The following video illustrates how you can use the Public API to retrieve test execution reports in JSON format.

Code samples

To see the full Java code sample see the export-api-sample in the Perfecto Git repository. Similar samples in other languages can be found in the Reporting-Samples folder (look for the export-api-sample in each language folder) in the Git repository.

Obtain a personal security token

The API requires the use of the Perfecto Security Token. Follow the procedure detailed in the Security Token documentation before proceeding.

Artifacts links are secure links. Use of the Perfecto Security Token is required.

Note

The security token is associated with the specific Perfecto Lab and the personal registered user.

Use the desired API to export data

The API provides two options for retrieving Report information:

  • Retrieve the Execution Driver Report data
  • Retrieve the Single Test Report data.
Important

The API calls are only valid after closing the Automation Driver, by executing the driver.close() method.

Retrieve the execution driver report

The following HTTP GET command is used to retrieve a list of the test executions in your lab

Method

API endpoint

GET

https://<reporting-server>/export/api/v3/test-executions

Header parameters

Parameter name

Description

Perfecto-Authorization

Security Token

Content-Type

The content type (application/json)

Perfecto-TenantId

The ID of your Perfecto cloud. This is your cloud URL with dashes instead of periods and no .app notation. For example: <your-cloud>-perfectomobile-com

Path parameters

The following table lists some of the URL query parameters that can be added to the URL to filter the number of returned executions. All filter parameters are supported in formatted PDF reports and API requests. However, Perfecto only retrieves the first 500 results, sorted by test start time.

Tip

We recommend that you limit the query to the last 45 days by using the startExecutionTime[0] field. If you do not limit the query by time, it will automatically be limited to the last 90 days.

Note

Perfecto uses AND logic between different fields and OR logic within the same field. For example, if you specify more than one tag, then if one of the tags matches, the test will appear in the results. The following query brings up results of test reports that have either a status of Failed or Blocked and are at the same time tagged with tag1 or tag2:

Copy
status[0]=FAILED&status[1]=BLOCKED&tags[0]=tag1&tags[1]=tag2
Important

When transmitting URL parameters that include special characters, such as the bracket characters ('[', ']'), the URL should be URL-encoded. We suggest using tools that encode URL parameters, like those included in the Apache HTTP Client component.

Name

Value

Description

startExecutionTime[0]

<Unix time>

The start time, in milliseconds from midnight January 1, 1970 GMT - Epoch/Unix Time. If this parameter is set, Perfecto makes sure that the test start time is less or equal to the startExecutionTime[0] parameter.

endExecutionTime[0]

<Unix time>

The end time, in milliseconds from midnight January 1, 1970 - GMT Epoch/Unix Time. If this parameter is set, Perfecto makes sure that the test start time is greater or equal to the endExecutionTime[0] parameter.

externalId[0]

<driver execution id>

Filters the data to show reports that match a specific driver execution

_page

integer

The page of the report to return. A page is defined to include up-to 500 items (execution/report units of information).

Default value: 1 - the first 500 items are returned.

tags[0]

<reporting tag>

Filters the data to show reports that match a specific tag.

You can query for several tags by adding the following:

Copy
tag[0]=XXX&tag[1]=YYY&tag[2]=ZZZ

To exclude a tag so that tests containing it will not be included in the results, use the following:

Copy
xtags[0]=AAA
Note

The x prefix also works to exclude other fields from the report.

status[0]

PASSED, FAILED, BLOCKED, UNKNOWN

Filters the data to show reports that match a specific test status

Id[0]

<test ID>

Filters the data to show reports that match a specific test ID

executionId[0] or externalId[0]

<execution ID>

Filters the data to show reports that match a specific execution ID

os[0]

ANDROID, BLACKBERRY, IOS, MAC, WINDOWS, LINUX

Filters the data to show reports that match a specific operating system

browserType[0]

CHROME, FIREFOX, INTERNET_EXPORER, SAFARI, MICROSOFT_EDGE

Filters the data to show reports that match a specific browser type

browserVersion[0]

<browser version>

Filters the data to show reports that match a specific browser version. For example:

Copy
browserVersion[0]=90.0.4430.212

screenResolution[0]

<width>x<height>

Filters the data to show reports that match a specific screen resolution. For example:

Copy
screenResolution[0]=1024x768

deviceType[0]

MOBILE, DESKTOP, SIMULATOR

Filters the data to show reports that match a specific device type

deviceModel[0]

<device model>

Filters the data to show reports that match a specific mobile device model. For example:

Copy
deviceModel[0]=iPhone-X

deviceId[0]

<device ID>

Filters the data to show reports that match a specific device ID

jobName[0]

<CI Job Name>

Filters the data to show reports that match a specific job name

jobNumber[0]

<CI Job Number>

Filters the data to show reports that match a specific job number

jobBranch[0]

<CI job branch>

Filters the data to show reports that match a specific job branch

projectVersion[0]

<project version>

Filters the data to show reports that match a specific project version

projectName[0]

<project name>

Filters the data to show reports that match a specific project name

failureReason[0]

<failure reason ID>

Filters the data to show reports that match a specific failure reason ID

customField.FIELD_NAME[0]

<custom field value>

Filters the data to show reports that match the value of a specific custom field. For example:

Copy
customField.myFieldName[0]=some-value
triggerType[0] Manual, Scheduled, CI/CD Filters the data to show reports that match a specific trigger type

Notes regarding time units:

  • The endExecutionTime parameter is only relevant if the startExecutionTime is included.
  • If using the Epoch Converter tool to generate the Unix time - convert the resulting time to milliseconds by multiplying the result by 1000.
  • To programmatically convert human readable dates to Unix time use the information provided at the end of the Epoch Converter page under the heading Convert from human readable date to epoch, but do not divide the result by 1000 to maintain the millisecond Unix time value.

Notes regarding other filter parameters:

  • Request can combine any number of the filter parameters (externalId, tags, jobName, jobNumber) to achieve the filtered list.
  • To check if there are additional pages in the response - the metadata field includes a truncated item with value true when there are additional resource items stored for the Execution Driver Report. Use the _page parameter to access the additional pages. See an example of using the _page parameter in GitHub.

Headers

Add the following header parameters to the request:

Name

Value

Perfecto-Authorization

<Your personal security token>

For example:

Response format

The Execution Driver Report is returned as a JSON structure with two sections:

  • resources - that provide information regarding the different information items included in the report. Each "page" of the report response is limited to 500 information items.
  • metadata - provides high-level information regarding the report. Two fields may be important for further processing:

    • truncated - indicates whether there are additional resource information items not included in the current page.

    • processingStatus - indicates if all resource information items have completed processing at time of report retrieval.

The following is an example response for test executions. The information for each test is listed as a "resource". In particular:

  • The "id" field below identifies the particular test (the TEST_ID) and can be used to retrieve the Single Test Report for the test.
  • The "reportURL" field provides the link to the Single Test Report for the test associated with the "id". The format of the URL is: https://<reporting-server>/test/<TEST_ID>
  • Includes information on the device "platform" of the test,
  • Provides the Jenkins job-name and build-number of the test and the user's project name and version information.
  • The "executionEngine" field provides the Perfecto MCM version used for the execution.
  • Provides links to the video segments included for the test.
  • Provides links to the artifacts generated for this execution:
    • VITALS - links to the vitals report generated if script activates the Start Vitals command.
    • NETWORK - links to the pcap file generated for tracking network activity.
    • DEVICE_LOGS - links to the logs file generated for the execution.
    • APP_CRASH_REPORT - links to the iOS crash report if detected during the execution.

Note

The processing of many of the components of the final report, e.g. videos and other artifacts, may require additional time. The processingStatus field (in the metadata) indicates when all processing is complete by changing from PROCESSING (some artifacts not available) to PROCESSING_COMPLETE.

The following code snippet shows an example JSON response for the request to retrieve test executions data.

Copy

Example JSON response for test executions

{
    "resources": [
        {
            "id": "5ad4888888c00080dd75a",
            "externalId": "888888@devtools.com_RemoteWebDriver_08-08-16_06_66_36",
            "name": "com.example.playground",
            "owner": "instrumentedtest@devtools.com",
            "startTime": 1523858478650,
            "endTime": 1523668484428,
            "uxDuration": 2164,
            "status": "PASSED",
            "platforms": [
                {
                    "deviceId": "#1234DEB31",
                    "deviceType": "MOBILE",
                    "os": "ANDROID",
                    "osVersion": "6.0.1",
                    "screenResolution": "1080*1920",
                    "location": "US-BOS",
                    "mobileInfo": {
                        "manufacturer": "Samsung",
                        "model": "Galaxy S5 SM-G900A",
                        "distributor": "ATT-US",
                        "description": "Central-vod",
                        "firmware": "klteuc-user 6.0.1 MMB88M G888AUCS4DPH4 release-keys",
                        "operator": "TT&T-United States of America"                    },
                    "selectionCriteria": {
                        "platformName": "Android"                    },
                    "customFields": []
                }
            ],
            "videos": [
                {
                    "startTime": 1588858467015,
                    "endTime": 152385888894,
                    "format": "MP4",
                    "downloadUrl": "https://perfectomobile.com/download-01/report-video/reporting-automation/video/sample_video_ios_landscape_phone_1_min_44_sec.mp4?fileName=sample_video_ios_landscape_phone_1_min_44_sec.mp4&signature=2931143c863507fc7a76b1118f3a48422e6ba7db6f47a64577b58b0dab13b5f8",                
                    "screen": {
                        "width": 1080,
                        "height": 1920
                    }
                }
            ],
            "tags": [
                "a55582871-c524-4f31-a41f-f5888541a61",
                "7a22587b-ac2a-4956-8328-865469838"            ],
            "executionEngine": {
                "version": "v1"            },
            "artifacts": [
                {
                    "type": "DEVICE_LOGS",
                    "path": "https://perfectomobile.com/download-01/report-artifact/artifacts-01/text-01/10000003/sample-data/browser-console.txt.zip?fileName=browser-console.txt.zip&signature=9f1d55a1d583f1d53acae19857dc6aa585743b49a91df7468f5f3968a5c8c75b"                }
            ],
            "reportURL": "https://reporting.perfectomobile.com/test/5ad4888880dd75a",
            "triggerType": "Manual"
            },
        {
            "id": "5ad6668005aadef1",
            "externalId": "1_2874520F-B013-B3AAFB1E6B64--60D14568107A16E2C6FB5--@@100.66.239.67@@1848",
            "name": "CHECK",
            "owner": "saes@gmail.com",
            "startTime": 15238557905579,
            "endTime": 1523857445638,
            "uxDuration": 4568,
            "status": "PASSED",
            "job": {
                "name": "Sample Job",
                "number": 45
            },
            "platforms": [
                {
                    "deviceId": "1_2548D-CA0F-420F-B013-B3456E6B64",
                    "deviceType": "DESKTOP",
                    "os": "WINDOWS",
                    "osVersion": "10",
                    "screenResolution": "1024x768",
                    "location": "US East",
                    "browserInfo": {
                        "browserType": "INTERNET_EXPLORER",
                        "browserVersion": "latest (11)"                    },
                    "selectionCriteria": {
                        "driverVersion": "N/A",
                        "platformVersion": "10",
                        "browserVersion": "11",
                        "browserName": "Internet Explorer",
                        "location": "US East",
                        "seleniumServerVersion": "3.4.0",
                        "platformName": "Windows",
                        "resolution": "1024x768"                    },
                    "customFields": []
                }
            ],
            "videos": [
                {
                    "startTime": 1523845681,
                    "endTime": 1523845610053,
                    "format": "MP4",
                    "downloadUrl": "https://perfectomobile.com/download-01/report-video/reporting-automation/video/sample_video_andriod_phone_6_min_02_sec.mp4?fileName=sample_video_andriod_phone_6_min_02_sec.mp4&signature=2931143c863507fc7a76b1118f3a48422e6ba7db6f47a64577b58b2j9qr5ngm3",
                    "screen": {
                        "width": 1024,
                        "height": 768
                    }
                }
            ],
            "tags": [
                "Native",
                "Java",
                "Sample"            ],
            "executionEngine": {
                "version": "18.4.0.4"            },
            "artifacts": [],
            "reportURL": "https://reporting.perfectomobile.com/test/4564849684adef1",
            "project": {
                "name": "Sample Script",
                "version": "1.0"            }
        },
        
    ],
     "metadata": {
        "createdAt": 1523859648413,
        "apiVersion": "v1",
        "userId": "reporting@perfectomobile.com",
        "tenantId": "bperfectomobile-com",
        "processingStatus": "PROCESSING_COMPLETE",
        "page": 1,
        "truncated": false
    }
}

The following code lists all fields that may be exported using this API. It describes the JSON schema that is returned in the response.

- resources (array) - list of objects that represent tests
    - id (string) - test ID
    - externalId (String) - execution driver ID
    - name (String) - name of the test. For example: "5ad4888888c00080dd75a"
    - owner (string) - user ID of the user who executed the test
    - startTime (number) - the start time of the test. Epoch time in millis (millis since 1/1/1970)
    - endTime (number) - the end time of the test. Epoch time in millis (millis since 1/1/1970)
    - uxDuration (number) - duration in milliseconds
    - status: PASSED, FAILED, UNKNOWN, BLOCKED (enum)
    - job (object)
        - name (string) - job name
        - number (number) - job number
        - branch (string) - job branch
    - platforms (array) - a list of objects that represents devices that took part in the test execution
        - deviceId (string) - device ID in case of mobile device. Will not be present for DESKTOP devices
        - deviceType: MOBILE, DESKTOP, SIMULATOR (enum)
        - os: ANDROID, BLACKBERRY, IOS, MAC, WINDOWS, LINUX (enum)
        - osVersion (string) - the version device's operating system. For example "10"
        - screenResolution (string) - <WIDTH>x<HEIGHT> string. For example "1170x2532"
        - location (string) - device physical location. For example: "EU Frankfurt"
        - mobileInfo (object) - will be present for MOBILE & SIMULATOR device types
            - imei (string) - device imei
            - imsi (string) - device imsi
            - manufacturer (string) - device manufacturer. For example "Apple"
            - model (string) - device model. For example: "Galaxy Tab A 8.0 (2019)"
            - phoneNumber (string) - device phone number. For example: "+1 (781) 123-4567"
            - distributor (string) - device's distributor. For example "MacCam14"
            - description (string) - device description
            - firmware (string) - firmware version. For example "gta3xlwifixx-user 9 PPR1.180610.011 T510XXU2ASK5 release-keys"
            - operator (string) - carrier. For example "AT&T"
            - operatorCountry (string) - carrier's country
            - hasAudio (boolean) - if this device's audio is available in perfecto lab
        - browserInfo (object) - will be present for DESKTOP device types
            - browserType: CHROME, INTERNET_EXPLORER, SAFARI, MICROSOFT_EDGE (enum)
            - browserVersion (string) - version of the browser. For example: "90"
        - alias (string) - the name that was given to the driver (relevant for the multiple executions in one report feature)
        - selectionCriteria (object) - deprecated. Use "selectionCriteriaV2".
        - selectionCriteriaV2 (array) - a list objects, each representing the desired capabilities when opening a device
            - name (string) - name of the capability
            - value (string) - the value of the capability
        - customFields (array) - list of an objects, each represents device's custom fields
            - name (string) - name of the device's custom field
            - value (string) - value of the device's custom field
    - videos (array)
        - startTime (number) - the start time of the video recording. Epoch time in millis (millis since 1/1/1970)
        - endTime (number) - the end time of the video recording. Epoch time in millis (millis since 1/1/1970)
        - videoFormat: FLV, MP4 (enum)     
        - downloadUrl (string) - a URL for video downloading
        - screen (object) - an object representing video size
            - width - video width. Number
            - height - video height. Number
    - tags (array[string]) - a list of test tags
    - executionEngine (object) - represents perfecto engine which executed the test
        - version (string) - engine version. For example: "21.1"
    - artifacts (array) - list of objects that represent artifacts that were attached to the test
        - type: IMAGE, DEVICE_LOGS, CONSOLE_LOG, NETWORK, VITALS, TEXT, APP_CRASH_REPORT (enum) - artifact type
        - path (string) - artifact's download URL
        - contentType (string) - MIME type. For example "image/png". For more information, visit: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types
        - fileName (string) - name of the file. If the file is zipped, then it will represent the file name that is inside the zip file .String
        - zipped (boolean) - if this artifact is zipped
    - reportURL (string) - a URL to the report of the execution to which this test belongs
    - project (object) - an object that represents project details
        - name (string) - project name
        - version (string) - project version
    - message (string) - a message that was attached to the "test stop" command. Can also contain an error message, or a stack trace. String
    - customFields (array) - each object represents test's custom field
        - name (string) - name of the custom field.
        - value (string) - value of the custom field.
    - failureReasonName (string) - name of the failure reason. For example "Element not found"
    - automationFramework (string) - the name of the automation framework that was used to execute the test. String. For example "SELENIUM"
    - parameters (array) - list of objects that represent the test parameters
        - name (string) - parameter name
        - value (string) - parameter value

Retrieve Single Test Report (STR) data

The following HTTP GET command is used to retrieve the list of commands associated with a single test execution:

Method API
GET https://<reporting-server>/export/api/v3/test-executions/<TEST_ID>/commands

The <TEST_ID> is taken from the Execution Driver Report, in the "id" field of the resource. In the example response, above, the <TEST_ID> would be:  5ad4888888c00080dd75a

Add the following parameters to the header section of the request:

Name

Value

Perfecto-Authorization

<Your personal security token>

For example:

The following code snippet shows an example JSON response for the request to retrieve STR data:

Copy

Example JSON response for request to retrieve STR data

{  
   "resources":[  
      {  
         "testExecutionId":"57f612f1c6c46111b02db94f",
         "commands":[  
            {  
               "name":"Command_vhsc2",
               "status":"SUCCESS",
               "startTime":1475744497372,
               "endTime":1475744498152,
               "uxDuration":551
            },
            {  
               "name":"Command_iofcw",
               "status":"FAILURE",
               "message":"Stacktrace:\n0...",
               "startTime":1475744498157,
               "endTime":1475744499611,
               "uxDuration":0,
               "screenshots":[  
                  "https://...sample-data/jpg.jpeg"               ],
               "expectedData":[  
                  {  
                     "type":"TEXT",
                     "value":"TEXT_0"                  }
               ],
               "resultData":[  
                  {  
                     "type":"TEXT",
                     "value":"TEXT_0"                  }
               ]
            },
            {  
               "name":"Command_hi dsffsdf dsfdsssb",
               "status":"FAILURE",
               "message":"Stacktrace...",
               "startTime":1475744499618,
               "endTime":1475744521762,
               "uxDuration":0,
               "screenshots":[  
               "https://perfectomobile.com/download-01/report-artifact/artifacts-01/image-01/10000003/sample-data/screenshot1.png?fileName=screenshot1.png&signature=e153e917b61ab867471fe218d37cb46d68e62c70a819a8283ae9a9b9b2c88911",
               "https://perfectomobile.com/download-01/report-artifact/artifacts-01/image-01/10000003/sample-data/tabletLandscape1.jpg?fileName=tabletLandscape1.jpg&signature=ee9da81dc988d8401eab319da63fc505982007c6c6a7d15d339953d6d0d2524e",
               "https://perfectomobile.com/download-01/report-artifact/artifacts-01/image-01/10000003/sample-data/landscape2.jpg?fileName=landscape2.jpg&signature=aac103bdc63dc2b5c7c33c98d42ac3c003b062e92a4b996cb727adbdec250e09",
               "https://perfectomobile.com/download-01/report-artifact/artifacts-01/image-01/10000003/sample-data/apps.jpg?fileName=apps.jpg&signature=f37fd48938078e6877d181d5a7ae7a9335662801458d0f1197420b204dcdc0dc",
             
                                 ],
               "expectedData":[  
                  {  
                     "type":"TEXT",
                     "value":"TEXT_0"                  },
                  {  
                     "type":"IMAGE_KEY",
                     "value":"https://...sample-data/sample64.jpeg"                  },
                  {  
                     "type":"TEXT",
                     "value":"TEXT_3"                  }
               ],
               "resultData":[  
                  {  
                     "type":"TEXT",
                     "value":"TEXT_0"                  },
                  {  
                     "type":"IMAGE_KEY",
                     "value":"https://...sample-data/sample65.jpeg"                  },
                  {  
                     "type":"TEXT",
                     "value":"TEXT_3"                  }
               ]
            }
         ]
      }
   ],
   "metadata":{  
      "createdAt":"1452419076769",
      "apiVersion":"v1",
      "userId":"MyUserID@perfectomobile.com",
      "tenantId":"MyTenantID"   }
}

The following code lists all fields that may be exported using this API. It describes the JSON schema that is returned in the response.

- resources (array) - list of objects which represent test steps
    - commands (array) - list of objects that represent commands of the step
       - name (string) - command name. For example "Find element"
       - status: SUCCESS, FAILURE (enum)
 	- message (string) - text field that can contain command result or error message
       - startTime (number) - the start time of the command. Epoch time in millis (millis since 1/1/1970)
	- endTime (number) - the end time of the command. Epoch time in millis (millis since 1/1/1970)
       - uxDuration (number) - duration in milliseconds (I actually have no idea what this means exactly. It is something set by the engine)
       - expectedData (array) - a list of objects that represents the expected command data
           - type: TEXT (enum)
           - value (string) - expected text
       - resultData (array) - a list of objects that represents the actual command data
           - type: TEXT, REFERENCE (enum)
              - TEXT - in that case the value will contain the actual text
              - REFERENCE - relevant only for old native IDE. The value will be an execution ID of a sub script that was called by this command
           - value (string) - actual value
       - screenshots (array[string]) - URLs to command's screenshots
       - expectedImages (array) - list of objects that represents the expected image. Can represent the needle in "image analysis"
           - imagePath (string) - URL of the expected image
       - parameters (array) - list of objects which represent the command parameters
           - name (string) - name of the parameter
           - value (string) - value of the parameter
           - attributes: SENSITIVE, HIDDEN (enum) - additional parameter attributes. This is an optional field
    - testExecutionId (string) - test ID this step belongs to. For example: "5ad4888888c00080dd75a"
    - name (string) - step name
    - startTime (number) - the start time of the test step. Epoch time in millis (millis since 1/1/1970)
    - endTime (number) - the end time of the test step. Epoch time in millis (millis since 1/1/1970)