Perfecto with JMeter | Performance testing

Apache JMeter™ is open-source software and 100% pure Java, designed to load test functional behavior and measure performance. It was originally intended for testing web applications but has since expanded to other test functions.

Perfecto has e2e integration with JMeter to perform SUT (Single User Performance) testing, with JUnit supporting both web and mobile applications.

In this exercise, we will perform client-side performance testing while stressing the server with JMeter threads for the Etihad web application. While measuring responses on the few real devices, we will ramp up virtual users to increase the load on the system. Following is an architecture diagram.

We have selected the following transactions:

  • Launch App: The time it takes to load the app
  • Search Flights: The time it takes to list the available flights on the given itinerary

The following thread groups are required:

Perform the following steps to set up JMeter and a Java Perfecto environment:

Java-JUnit setup

In this example, we have implemented a sample Java script to perform SUT Performance Testing.

  1. Download the Sample Java project here: https://github.com/PerfectoCode/Community-Samples/tree/master/RemoteWebDriverJmeterSample.
  2. Import the project as a normal Java project in Eclipse or IntelliJ.

  3. Export the project as a JAR file.

  4. Place the generated JAR file into JMeter's lib/JUnit.
  5. Download the Java dependencies here:  https://github.com/PerfectoCode/Community-Samples/tree/master/JMeterSamples/Dependencies/libs
  6. Place the downloaded dependencies into JMeter’s lib folder.

JMeter setup

  1. Download the latest JMeter.
  2. Extract the downloaded JMeter file.
  3. Download the Plugins Manager JAR file and put it into JMeter's lib/ext.

  4. In JMeter, from the Options menu, select Plugins Manager.

  5. Under Available Plugins, search for Dummy Sampler. Then click Apply Changes and Restart JMeter.

  6. Download the Etihad Perfecto User Experience Jmx file - EtihadPerformance.jmx and open it in JMeter.

  7. Get the Perfecto security token and, under User Defined Variables, assign the value to the securityToken, host and reportingUrl parameters.

  8. For each JUnit request, select the ClassName and respective TestMethod, for example:

    • Get Driver > setUpBeforeClass

    • Get App Launch Time > launchApp

    • Get Search Flight Time > searchFlights

    • Close Driver > oneTimeTearDown

  9. Under User Parameters, enter the device and platform details as a separate user.

  10. On the toolbar, click the (green) Run button to execute the tests on the given device.

    JMeter gets the Perfecto user experience time (UXT) for the given transactions from the device.

    In this case, Android took on average of 2s to launch the app and 7.9s to search the flights. iOS took on average of 5.5 seconds to launch the app and 7.9s seconds to search the fights.

To load the server, we will use JMeter threads. Virtual users need network traffic or APIs to run the script. Perform the following steps to capture the traffic.

Method 1 | Use Perfecto Network Traffic Capture

  1. As part of the client-side tests SUT Performance Testing, we already captured the network traffic with the HAR file. Use this file and convert it into JMX using the following tool: https://converter.blazemeter.com

  2. Load the converted JMX file.

  3. Add the same transaction names as given in client-side performance testing (Launch AppandSearch Flight).

Method 2 | Capture traffic directly from the web or mobile device

  1. Perform the steps provided at the following link to record the traffic: https://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.html
  2. Add the same transaction names as given in client-side performance testing (Launch App and Search Flight).

Merge both Perfecto user experience and server-side scripts in the test plan. A sample e2e JMX test plan file is available here: EtihadPerformance.jmx

Because Etihad is the production website, let’s run Performance testing with 7 users: 2 on the client-side with a real device (in Perfecto) and 5 on the server-side.

The following image shows virtual user configuration (server-side).

The following image shows Perfecto user configuration (server-side).

To execute the performance tests:

On the toolbar, click the (green) Run button. The test will load the server and execute client-side tests in Perfecto. The summary report contains the transaction time of both server- and client-side.

From the summary, we see that as per JMeter threads (server-side), the system took an average of 28s to load the launch page and 7.8s to search the flights.

From the Perfecto user experience side (client-side), it took 2s on Android and 6s on iOS devices. More time was reported on the server-side because the server page load time depends on the processing time of all the APIs (including third-party APIs that are not visible on the screen, whereas the Perfecto page load time depends on specific text to display on the page, ignoring background and third-party APIs.

The following video shows a demo execution.