API for external test information

The Import SDK is an additional Reporting SDK that supports generating Single Test Reports (STR) for test executions executed on external servers or emulators - not connected to the Perfecto Lab. The basic use of the API exported by the SDK is:

• Establish a connection to the Reporting Server. This requires a Security Token.
• Provide basic information regarding the test:
  • Description of platform used to run the test
  • Tags, Job description, etc.
  • The custom fields that identify the script source code to allow the test report to display the test code
• Activate the test start, end, and steps in the testing script code (use the basic SDK API.)
• Retrieve the link to the Single Test Report.

To use the functionality described in this document, do the following:

• Download the Reporting SDK the Smart Reporting SDK: reportium-java-<version>.jar

• Download the additional jar file: reportium-import-java-<version>.jar

• (Optional) Download the additional jar file (reportium-import-selenium-<version>.jar) that supports Appium/Selenium scripts that will use the event-firing driver described here.


Basic information units

The SDK defines a small set of classes that provide the interface (detailed below) to perform the process outlined above:

This SDK extension is supported for SDK version 1.1.19 or later.

Establish a connection

To turn any execution into a report, pass the information to the Perfecto Reporting server. The test information is stored in a repository on the Reporting server.

Establish the connection by using the classes of the SDK. The following code snippet performs the following steps to establish a very basic and simple connection:

• Creates a new instance of the context class for the data-import functionality.
• Prepares the connection parameters.
• Creates a new instance of the Reportium client, ready to receive the data regarding the external test.

        ImportExecutionContext executionContext = new ImportExecutionContext.Builder().build();
        Connection connection = new Connection(new URI(REPORTIUM_URL), SECURITY_TOKEN);
        ReportiumImportClient reportiumClient = new ReportiumImportClientFactory().createReportiumImportClient(connection, executionContext);

REPORTIUM_URL

Use the following format for the URI of the reporting server: https://<CQL_NAME>.app.perfectomobile.com
where <CQL_NAME> is the name of your Perfecto Lab

The <CQL_NAME> string will be used to keep the test report information from your company's tests separate from all other test information in a secure repository accessible to your personnel.

SECURITY_TOKEN

Your personal authentication token provides secure access to the Reporting server. To generate the token, follow the procedure presented here.

Basic test information

When creating the test report for execution, the STR includes basic information regarding the test. In particular, the report identifies the test with:

• The device/platform the test was run on
• The job name and number that the test is part of
• Tags to be associated with the test

Platform information

The SDK includes a Platform class that encapsulates the information regarding the device. The Platform class includes the following information:

Mobile device information

The MobileInfo class provides the following information related to the mobile device:

The following code snippet creates a Platform instance that includes a mobile device:

        MobileInfo mobileInfo = new MobileInfo.Builder()
                .withManufacturer("Samsung")
                .withModel("Galaxy Note 8")
                .withOperator("Sprint")
                .build();

        Platform platform = new Platform.Builder()
                .withMobileInfo(mobileInfo)
                .withOs("Android")
                .withDeviceType(DeviceType.MOBILE)
                .withScreenResolution("1440x2960")
                .build();

Browser device information

The BrowserInfo class provides the following information related to the mobile device:

The following code snippet creates a Platform instance that includes a browser device:

        BrowserInfo browserInfo = new BrowserInfo.Builder()
                .withBrowserType(BrowserType.CHROME)
                .withBrowserVersion("60")
                .build();

        Platform platform = new Platform.Builder()
                .withBrowserInfo(browserInfo)
                .withOs("Windows")
                .withOsVersion("10")
                .withDeviceType(DeviceType.DESKTOP)
                .withScreenResolution("1366x768")
                .build();

Context parameters

The ImportExecutionContext class supports all test labeling parameters supported by the PerfectoExecutionContext class. These include:

• Tags
• Job Number and Name
• Project information
• Platform information

The following snippet shows how to set the parameters of the ImportExecutionContext instance:

        ImportExecutionContext executionContext = new ImportExecutionContext.Builder()
                .withJob(new Job("my job name", 123))
                .withProject(new Project("my project name", "ver 123"))
                .withContextTags("tag1", "tag2", "tag3")
                .withPlatforms(platform)
                .build();

        Connection connection = new Connection(new URI(REPORTIUM_URL), SECURITY_TOKEN);
        ReportiumImportClient reportiumClient = new ReportiumImportClientFactory().createReportiumImportClient(connection, executionContext);

Add test start, steps, and stop

After creating the ReportiumImportClient instance, use the testStart, stepStart, stepEnd, and testStop methods as they are used for the ReportiumClient instance. Supply the test result as part of activating the testStop method.

The testStop() method of the ReportiumImportClient accepts a list of TextAttachment objects that will be attached to the test report (see below for more details).

Your test may be embedded between the steps. Do not pass the driver variable to the ReportiumClient instance because this is not linked to the Perfecto system.

        reportiumClient.testStart("my test name", new TestContext("tag4", "tag5"));
        // test initialization code
        reportiumClient.stepStart("my step");
        // add test step code
        reportiumClient.stepEnd();
        // repeat the stepStart and stepEnd as needed
        
        // supply the test result value as part of the testStop
        reportiumClient.testStop(TestResultFactory.createFailure("it was a failure"));

Collect execution information

Normally, a test step (code between the stepStart and stepEnd methods) includes automation operations whose results should be reported. This SDK for external automation testing offers the following options for reporting the operations executed by the script:

• Using Perfecto extension of the Selenium EventFiringWebDriver class to automatically generate reporting items from each Selenium/Appium operation
• Using a ReportiumImportClient method to generate entries in the report collector, specifying the actions performed and the results

By default, the execution information is sent asynchronously to the Smart Reporting server by a background thread, to minimize the overhead on the test execution. If there is a need to have the execution information sent by the foreground test script, use the setAsyncUpload() method of the ReportiumImportClient instance. To turn off the asynchronous event transfer:

reportiumClient.setAsyncUpload(false);

Turning off the AsyncUpload feature may affect the execution time of the test script because each command transmits an event to the Reporting server.

PerfectoEventFiringWebDriver Class

Perfecto supplies an extension of the Selenium EventFiringWebDriver class that activates the basic automation driver of the script, generates a reporting event for each action, and communicates information to the ReportiumImportClient as report units. Using this class allows the report to be generated together with information regarding any Selenium/Appium actions in a transparent methodology.

To use the PerfectoEventFiringWebDriver class:

• Add a project dependency on the reportium-import-selenium jar file.

• Add the following code to your automation script, after creating the ReportiumImportClient instance:

  Copy

  // create the RemoteWebDriver instance - or an AppiumDriver flavor
  RemoteWebDriver rwdDriver = createWebDriver();
  // use the automation driver and the reportium client to create an instance of the Event firing driver.
  EventFiringWebDriver perfectoDriver = new PerfectoEventFiringWebDriver(rwdDriver, reportiumClient);

From this point on, use the perfectoDriver instance in place of your rwdDriver, for example:

// use the following
perfectoDriver.findElement(...).click();
// in place of any rwdDriver.findElement().click(); command;

Use the ReportiumImportClient.command() method

The ReportiumImportClient class supports a command() method whereby you can explicitly add information regarding an operation performed by the automation script. This is particularly useful in cases where:

• The automation performs some operations that are not part of Selenium/Appium, for example an internal framework.
• You do not need to report all automation operations and wish to focus only on a small set of logical operations.

The command() method accepts a Command class instance configured with information to add to the report entry. The Command class supports the following report entry information units:

• Command name (withName method)
• Command status (withStatus method)
• Execution description (withMessage method)
• Start and End times (withStartTime and withEndTime methods)
• Add parameter values (addParameter method)
• Add screenshot artifact (addScreenshotAttachment method)

The following snippets demonstrate the use of this class and the command() method:

// create the Command object that reports execution of a command with its resulting status
        Command command = new Command.Builder()
                .withName("my command name")
                .withMessage("my command message")
                .withStatus(CommandStatus.SUCCESS)
                .addParameter(new CommandParameter("name1", "value1"))
                .addParameter(new CommandParameter("name2", "value2"))
                .build();

        reportiumClient.command(command);

// Add a screenshot previously stored in a file to the command report entry
        File screenshotFile = new File(…);
        Command command = new Command.Builder()
                .withName("my command name")
                .withStatus(CommandStatus.SUCCESS)
                .withStartTime(commandStartTime)
                .withEndTime(commandEndTime)
                .addScreenshotAttachment(new ScreenshotAttachment.Builder()
                        .withAbsolutePath(screenshotFile.getAbsolutePath())
                        .build())
                .build();

        reportiumClient.command(command);

// Add a screenshot taken from the live input stream provided
        InputStream screenshotInputStream = …;
        Command command = new Command.Builder()
                .withName("my command name")
                .withStatus(CommandStatus.SUCCESS)
                .withStartTime(commandStartTime)
                .withEndTime(commandEndTime)
                .addScreenshotAttachment(new ScreenshotAttachment.Builder()
                        .withInputStream(screenshotInputStream)
                        .withContentType(ScreenshotAttachment.IMAGE_JPEG)
                        .build())
                .build();

        reportiumClient.command(command);

Limitations of dataIn command fields

The limitations specified in the table apply to the datain command fields according to the following values:

FIELD_MAX_LENGTH = 256;
URL_MAX_LENGTH = 1024;
MAX_NUM_SCREENSHOTS = 1024;
MESSAGE_MAX_LENGTH = 12000;

Attach files to the test report

If the test script generates or accesses files or streamed information that is important to attach to the test report, to provide context or additional information in analyzing the results, you can perform this by doing the following:

1. Create an Attachment instance by using the Attachment.Builder() method (see example below) that provides the vital metadata for the attachment.
2. Use the addArtifacts() method of the ReportiumImportClient  instance.

The API supports adding any type of attachment. When creating the Attachment object, supply the following:

• Content Type: the MIME type that best describes how to display the file.
• Type: A string used by the Smart Reporting server to classify the data file. (This is optional for Text or Screenshot attachments.)
• File Name: Name of the file to save.
• Supply at least one of the following:
  
  • Input Stream: For streamed data.
  • Absolute Path: For non-streamed data. Indicates where the data is taken from.

Additional optional information includes:

• Extension: Extension of the data file. Smart Reporting will try to determine this if not supplied.
• Zipped: An indication if the attachment is a condensed file. Smart Reporting will determine this if the file extension is .zip.

The following shows how to use the Attachment type to attach a generic file attachment:

        String pdfFile = "TestDescription.pdf";
        Attachment pdfAttachment = new Attachment.Builder<>()
                .withType("Test Design")
                .withContentType("application/pdf")
                .withFileName(pdfFile)
                .withExtension(FilenameUtils.getExtension(pdfFile))
                .withZipped(false)
                .build();

        reportiumClient.addArtifacts(pdfAttachment);

Special attachment types

The Reporting SDK supports the following special classes of Attachments:

• Text attachments: Use the dedicated TextAttachment Builder() method
• Screenshot attachments: Use the dedicated ScreenshotAttachment Builder() method (see example above)

If the attachment can be represented as text (regardless of the format, for example JSON, XML, or plain text),  you can attach these to the test report for the execution by:

1. Preparing the file's metadata for the reporting server with the TextAttachment's build() method
2. Providing the set of TextAttachment instances to the ReportiumImportClient's addArtifacts() method.

The TextAttachment class supports methods to define either a static file's path or a connection to an incoming text stream.(see the following example snippet):

        File xmlFile = new File(<file-path>);
        Attachment xmlAttachment = new TextAttachment.Builder()
                .withAbsolutePath(xmlFile.getAbsolutePath())
                .build(); // no need to pass content type and file name, since we can guess them from the provided path

        InputStream textFileInputStream = new ClassPathResource(<stream-identifier>).getInputStream();
        Attachment txtAttachment = new TextAttachment.Builder()
                .withInputStream(textFileInputStream)
                .withContentType(TextAttachment.TEXT_PLAIN)
                .withFileName("text_file.txt")
                .build();

        reportiumClient.addArtifacts(xmlAttachment, txtAttachment); 

Complete the test script

When using asynchronous reporting events (default, see here), you should add the following command to the script after closing the automation driver (for example RemoteWebDriver, ChromeDriver, or AppiumDriver):e

reportiumClient.close();

This allows the asynchronous events to complete before closing the test program, which will guarantee that your report includes all test information.

Retrieve the report link

Use the getReportURL method to retrieve the link to the STR.

System.out.println(reportiumClient.getReportUrl());

Code Samples

See simple code samples in the GitHub Repository.