API for external test information

Important:

This feature is available under a separate license. For more information on implementing this feature in your Perfecto Lab, contact Perfecto Support.

To use the functionality described in this document, do the following:

  • Download the Reporting SDK the Smart Reporting SDK: reportium-java-<version>.jar

  • Download the additional jar file: reportium-import-java-<version>.jar

  • (Optional) Download the additional jar file (reportium-import-selenium-<version>.jar) that supports Appium/Selenium scripts that will use the event-firing driver described here.

The Import SDK is an additional Reporting SDK that supports generating Single Test Reports (STR) for test executions executed on external servers or emulators - not connected to the Perfecto Lab. The basic use of the API exported by the SDK is:

  • Establish a connection to the Reporting Server. This requires a Security Token.
  • Provide basic information regarding the test:
    • Description of platform used to run the test
    • Tags, Job description, etc.
    • The custom fields that identify the script source code to allow the test report to display the test code
  • Activate the test start, end, and steps in the testing script code (use the basic SDK API.)
  • Retrieve the link to the Single Test Report.

Basic information units

The SDK defines a small set of classes that provide the interface (detailed below) to perform the process outlined above:

Class Description
ReportiumImportClient Specifies the Reporting client for passing test information for tests executed on an external server.
ImportExecutionContext Provides the context within which the test report information is generated.
ReportiumImportClientFactory Generates the Reporting client instance, based on the reporting server, context (generated by ImportExecutionContext), and additional test information.

This SDK extension is supported for SDK version 1.1.19 or later.

Establish a connection

To turn any execution into a report, pass the information to the Perfecto Reporting server. The test information is stored in a repository on the Reporting server.

Establish the connection by using the classes of the SDK. The following code snippet performs the following steps to establish a very basic and simple connection:

  • Creates a new instance of the context class for the data-import functionality.
  • Prepares the connection parameters.
  • Creates a new instance of the Reportium client, ready to receive the data regarding the external test.
Copy
        ImportExecutionContext executionContext = new ImportExecutionContext.Builder().build();
        Connection connection = new Connection(new URI(REPORTIUM_URL), SECURITY_TOKEN);
        ReportiumImportClient reportiumClient = new ReportiumImportClientFactory().createReportiumImportClient(connection, executionContext);

REPORTIUM_URL

Use the following format for the URI of the reporting server: https://<CQL_NAME>.app.perfectomobile.com
where <CQL_NAME> is the name of your Perfecto Lab

The <CQL_NAME> string will be used to keep the test report information from your company's tests separate from all other test information in a secure repository accessible to your personnel.

SECURITY_TOKEN

Your personal authentication token provides secure access to the Reporting server. To generate the token, follow the procedure presented here.

Basic test information

When creating the test report for execution, the STR includes basic information regarding the test. In particular, the report identifies the test with:

  • The device/platform the test was run on
  • The job name and number that the test is part of
  • Tags to be associated with the test

Platform information

The SDK includes a Platform class that encapsulates the information regarding the device. The Platform class includes the following information:

Field Values Description
Device Type MOBILE | BROWSER The type of device used. This is an enum value that supports only the documented values.
Device Device instance An instance of the Device class (see below) that describes the device used. The class comes in two flavors.
Platform OS String OS of the platform, normally Android or iOS, or Windows or MacOS.
Platform Version String The version of the OS used.
Screen Resolution String The screen resolution used for the test.

Mobile device information

The MobileInfo class provides the following information related to the mobile device:

Field Values Description
Model String The model name of the device, such as Nexus6P or iPhone8
Manufacturer String The maker of the device, such as LG or Samsung
Device ID String The identifier of the device

The following code snippet creates a Platform instance that includes a mobile device:

Copy
        MobileInfo mobileInfo = new MobileInfo.Builder()
                .withManufacturer("Samsung")
                .withModel("Galaxy Note 8")
                .withOperator("Sprint")
                .build();

        Platform platform = new Platform.Builder()
                .withMobileInfo(mobileInfo)
                .withOs("Android")
                .withDeviceType(DeviceType.MOBILE)
                .withScreenResolution("1440x2960")
                .build();

Browser device information

The BrowserInfo class provides the following information related to the mobile device:

Field Values Description
BrowserType

CHROME |

FIREFOX |

SAFARI

The type of browser used in the test
Version String The version number of the browser used for the test

The following code snippet creates a Platform instance that includes a browser device:

Copy
        BrowserInfo browserInfo = new BrowserInfo.Builder()
                .withBrowserType(BrowserType.CHROME)
                .withBrowserVersion("60")
                .build();

        Platform platform = new Platform.Builder()
                .withBrowserInfo(browserInfo)
                .withOs("Windows")
                .withOsVersion("10")
                .withDeviceType(DeviceType.DESKTOP)
                .withScreenResolution("1366x768")
                .build();

Context parameters

The ImportExecutionContext class supports all test labeling parameters supported by the PerfectoExecutionContext class. These include:

  • Tags
  • Job Number and Name
  • Project information
  • Platform information

The following snippet shows how to set the parameters of the ImportExecutionContext instance:

Copy
        ImportExecutionContext executionContext = new ImportExecutionContext.Builder()
                .withJob(new Job("my job name", 123))
                .withProject(new Project("my project name", "ver 123"))
                .withContextTags("tag1", "tag2", "tag3")
                .withPlatforms(platform)
                .build();

        Connection connection = new Connection(new URI(REPORTIUM_URL), SECURITY_TOKEN);
        ReportiumImportClient reportiumClient = new ReportiumImportClientFactory().createReportiumImportClient(connection, executionContext);

Add test start, steps, and stop

After creating the ReportiumImportClient instance, use the testStart, stepStart, stepEnd, and testStop methods as they are used for the ReportiumClient instance. Supply the test result as part of activating the testStop method.

The testStop() method of the ReportiumImportClient accepts a list of TextAttachment objects that will be attached to the test report (see below for more details).

Your test may be embedded between the steps. Do not pass the driver variable to the ReportiumClient instance because this is not linked to the Perfecto system.

Copy
        reportiumClient.testStart("my test name", new TestContext("tag4", "tag5"));
        // test initialization code
        reportiumClient.stepStart("my step");
        // add test step code
        reportiumClient.stepEnd();
        // repeat the stepStart and stepEnd as needed
        
        // supply the test result value as part of the testStop
        reportiumClient.testStop(TestResultFactory.createFailure("it was a failure"));

Collect execution information

Normally, a test step (code between the stepStart and stepEnd methods) includes automation operations whose results should be reported. This SDK for external automation testing offers the following options for reporting the operations executed by the script:

  • Using Perfecto extension of the Selenium EventFiringWebDriver class to automatically generate reporting items from each Selenium/Appium operation
  • Using a ReportiumImportClient method to generate entries in the report collector, specifying the actions performed and the results

By default, the execution information is sent asynchronously to the Smart Reporting server by a background thread, to minimize the overhead on the test execution. If there is a need to have the execution information sent by the foreground test script, use the setAsyncUpload() method of the ReportiumImportClient instance. To turn off the asynchronous event transfer:

Copy
reportiumClient.setAsyncUpload(false);

Turning off the AsyncUpload feature may affect the execution time of the test script because each command transmits an event to the Reporting server.

PerfectoEventFiringWebDriver Class

Perfecto supplies an extension of the Selenium EventFiringWebDriver class that activates the basic automation driver of the script, generates a reporting event for each action, and communicates information to the ReportiumImportClient as report units. Using this class allows the report to be generated together with information regarding any Selenium/Appium actions in a transparent methodology.

To use the PerfectoEventFiringWebDriver class:

  • Add a project dependency on the reportium-import-selenium jar file.
  • Add the following code to your automation script, after creating the ReportiumImportClient instance:

    Copy
    // create the RemoteWebDriver instance - or an AppiumDriver flavor
    RemoteWebDriver rwdDriver = createWebDriver();
    // use the automation driver and the reportium client to create an instance of the Event firing driver.
    EventFiringWebDriver perfectoDriver = new PerfectoEventFiringWebDriver(rwdDriver, reportiumClient);

From this point on, use the perfectoDriver instance in place of your rwdDriver, for example:

Copy
// use the following
perfectoDriver.findElement(...).click();
// in place of any rwdDriver.findElement().click(); command;

Use the ReportiumImportClient.command() method

The ReportiumImportClient class supports a command() method whereby you can explicitly add information regarding an operation performed by the automation script. This is particularly useful in cases where:

  • The automation performs some operations that are not part of Selenium/Appium, for example an internal framework.
  • You do not need to report all automation operations and wish to focus only on a small set of logical operations.

The command() method accepts a Command class instance configured with information to add to the report entry. The Command class supports the following report entry information units:

  • Command name (withName method)
  • Command status (withStatus method)
  • Execution description (withMessage method)
  • Start and End times (withStartTime and withEndTime methods)
  • Add parameter values (addParameter method)
  • Add screenshot artifact (addScreenshotAttachment method)

The following snippets demonstrate the use of this class and the command() method:

Copy
// create the Command object that reports execution of a command with its resulting status
        Command command = new Command.Builder()
                .withName("my command name")
                .withMessage("my command message")
                .withStatus(CommandStatus.SUCCESS)
                .addParameter(new CommandParameter("name1", "value1"))
                .addParameter(new CommandParameter("name2", "value2"))
                .build();

        reportiumClient.command(command);

// Add a screenshot previously stored in a file to the command report entry
        File screenshotFile = new File();
        Command command = new Command.Builder()
                .withName("my command name")
                .withStatus(CommandStatus.SUCCESS)
                .withStartTime(commandStartTime)
                .withEndTime(commandEndTime)
                .addScreenshotAttachment(new ScreenshotAttachment.Builder()
                        .withAbsolutePath(screenshotFile.getAbsolutePath())
                        .build())
                .build();

        reportiumClient.command(command);

// Add a screenshot taken from the live input stream provided
        InputStream screenshotInputStream = …;
        Command command = new Command.Builder()
                .withName("my command name")
                .withStatus(CommandStatus.SUCCESS)
                .withStartTime(commandStartTime)
                .withEndTime(commandEndTime)
                .addScreenshotAttachment(new ScreenshotAttachment.Builder()
                        .withInputStream(screenshotInputStream)
                        .withContentType(ScreenshotAttachment.IMAGE_JPEG)
                        .build())
                .build();

        reportiumClient.command(command);

Limitations of dataIn command fields

The limitations specified in the table apply to the datain command fields according to the following values:

Copy
FIELD_MAX_LENGTH = 256;
URL_MAX_LENGTH = 1024;
MAX_NUM_SCREENSHOTS = 1024;
MESSAGE_MAX_LENGTH = 12000;
 

Field name

Limit
Command fields

Name

FIELD_MAX_LENGTH

Message

MESSAGE_MAX_LENGTH

Number of screenshots limited

MAX_NUM_SCREENSHOTS

Each CommandParameter property

Name

FIELD_MAX_LENGTH

Value

FIELD_MAX_LENGTH

Each ExpectedImageData property

imagePath

URL_MAX_LENGTH

Each commandData property

All fields of type text

FIELD_MAX_LENGTH

Attach files to the test report

If the test script generates or accesses files or streamed information that is important to attach to the test report, to provide context or additional information in analyzing the results, you can perform this by doing the following:

  1. Create an Attachment instance by using the Attachment.Builder() method (see example below) that provides the vital metadata for the attachment.
  2. Use the addArtifacts() method of the ReportiumImportClient  instance.

The API supports adding any type of attachment. When creating the Attachment object, supply the following:

  • Content Type: the MIME type that best describes how to display the file.
  • Type: A string used by the Smart Reporting server to classify the data file. (This is optional for Text or Screenshot attachments.)
  • File Name: Name of the file to save.
  • Supply at least one of the following:
    • Input Stream: For streamed data.
    • Absolute Path: For non-streamed data. Indicates where the data is taken from.

Additional optional information includes:

  • Extension: Extension of the data file. Smart Reporting will try to determine this if not supplied.
  • Zipped: An indication if the attachment is a condensed file. Smart Reporting will determine this if the file extension is .zip.

The following shows how to use the Attachment type to attach a generic file attachment:

Copy
        String pdfFile = "TestDescription.pdf";
        Attachment pdfAttachment = new Attachment.Builder<>()
                .withType("Test Design")
                .withContentType("application/pdf")
                .withFileName(pdfFile)
                .withExtension(FilenameUtils.getExtension(pdfFile))
                .withZipped(false)
                .build();

        reportiumClient.addArtifacts(pdfAttachment);

Special attachment types

The Reporting SDK supports the following special classes of Attachments:

  • Text attachments: Use the dedicated TextAttachment Builder() method
  • Screenshot attachments: Use the dedicated ScreenshotAttachment Builder() method (see example above)

If the attachment can be represented as text (regardless of the format, for example JSON, XML, or plain text),  you can attach these to the test report for the execution by:

  1. Preparing the file's metadata for the reporting server with the TextAttachment's build() method
  2. Providing the set of TextAttachment instances to the ReportiumImportClient's addArtifacts() method.

The TextAttachment class supports methods to define either a static file's path or a connection to an incoming text stream.(see the following example snippet):

Copy
        File xmlFile = new File(<file-path>);
        Attachment xmlAttachment = new TextAttachment.Builder()
                .withAbsolutePath(xmlFile.getAbsolutePath())
                .build(); // no need to pass content type and file name, since we can guess them from the provided path

        InputStream textFileInputStream = new ClassPathResource(<stream-identifier>).getInputStream();
        Attachment txtAttachment = new TextAttachment.Builder()
                .withInputStream(textFileInputStream)
                .withContentType(TextAttachment.TEXT_PLAIN)
                .withFileName("text_file.txt")
                .build();

        reportiumClient.addArtifacts(xmlAttachment, txtAttachment)

Complete the test script

When using asynchronous reporting events (default, see here), you should add the following command to the script after closing the automation driver (for example RemoteWebDriver, ChromeDriver, or AppiumDriver):e

Copy
reportiumClient.close();

This allows the asynchronous events to complete before closing the test program, which will guarantee that your report includes all test information.

Retrieve the report link

Use the getReportURL method to retrieve the link to the STR.

Copy
System.out.println(reportiumClient.getReportUrl());

Code Samples

See simple code samples in the GitHub Repository.