Web penetration testing with Perfecto

Important: Do not try techniques described here on public sites or your customer or client’s websites without approval. It may lead to blocks and potential legal complications.

Software security testing reveals possible vulnerabilities in the system, making sure that data and resources are protected. For a common breakout of software security testing, see ZAP's Getting Started documentation.

Fortunately, with security testing tools available in the market, you can perform vulnerability assessment and penetrating testing for web applications and websites on devices and browsers in Perfecto using both manual and automation testing.

This article provides a quick overview of the penetration testing process and then dives into an end-to-end demo of how you can perform penetration testing with Perfecto in combination with the following tools:

Zed Attack Proxy (ZAP): A free, open-source penetration testing tool. It serves as a proxy located between the browser and the web application, intercepting and inspecting messages before they reach their destination.
Burp Suite: A platform for vulnerability scanning, penetration testing, and web app security. It servers as a proxy located between the target web application and the web server, intercepting ongoing HTTP requests before they reach their destination.

Both tools make it possible to pause the traffic and manipulate the intercepted items, if needed, before forwarding them to their destination.

For a comparison of Burp Suite and ZAP, click here.

Vulnerabilities according to OWASP

In this article, we refer to the Open Web Application Security Project^® (OWASP), a foundation that works to improve the security of software. It also maintains ZAP.

OWASP periodically reviews the top security risks of web applications and lists the top 10 risks that developers and web security experts should be aware of. According to their website, this list represents a broad consensus about the most critical security risks to web applications.

Pen testing process

Penetration testing often combines manual and automated testing methods to test servers, networks, devices, and endpoints. The process commonly involves 3 stages in which you first explore the system, then attack the system using known or suspected vulnerabilities, and, last, report test results. For more information about the stages of pen testing, click here.

Penetration testing aims at searching for, revealing, and fixing vulnerabilities. It can also verify if a system is not vulnerable to a specific defect. When vulnerabilities reported earlier are supposedly fixed, the system gets retested to verify that the issues are truly resolved.

Most web penetration tests are performed manually. Many teams outsource this kind of testing to third-party services. When done, it is usually not part of the cycle, and often, it is not executed as frequently as it should be. This leaves the risk of escaped defects.

A security degradation or outage impacts user experiences and creates significant business issues. Shifting security testing to the left to perform it earlier in the DevOps cycle is therefore a best practice. This way, it is possible to collaborate on requirements, test for code vulnerabilities within existing CI/CD pipelines, and also implement other security practices in software development.

Fortunately, with Perfecto, web penetration testing is a seamless and free add-on to existing test automation cycles, for web apps on devices and in desktop browsers. Every regression test or smoke run proves functionality and checks how apps are vulnerable to hackers.

ZAP and Burp Suite are just two examples of tools that automate the task of scanning websites for content and vulnerabilities.

The process includes the following:

Web spider: Searches for new resources (URLs) on a website. The task begins with a list of URLs to visit, called seeds. The spider then visits these URLs, identifies all hyperlinks on the page, and adds them to the list of URLs to visit. This is a constant process that continues until new resources get identified.

For more comprehensive information and additional definitions, see here.
Scanning: When the spider has identified all links on the page, the scanner attempts to find potential vulnerabilities by using known attacks. Different types of scanning are used for different purposes. Active scanning searches for vulnerabilities by using known attacks against the system or application. Passive scanning scans all HTTP messages (requests and responses) sent to the web application under test. The requests or responses are not hampered by passive scanning, making it safe to use.

Important: Perform an active scan only if you have permission to test the application.

Pen testing prerequisites

Sample application

This exercise uses a sample application called WebGoat. WebGoat is a deliberately insecure web application maintained by OWASP and designed to teach lessons about web application security. The app is a demonstration of common server-side application flaws. The exercises are intended to be used by people to learn about application security and penetration testing techniques.

Warning: While running this program, your machine will be extremely vulnerable to attack. You should kill the running server immediately after test completion.

To install WebGoat:

Download the latest WebGoat version from here.
Run the download .jar file with the following command:
Copy
```
Java -jar <<xxxx.jar>>
```
(Optional) By default, it binds to localhost and 8080 port. To change the server address and port number before running the standalone .jar file.
Copy
Mac/UNIX
```
export WEBGOAT_HOST=xx.xx.xx.xx
export WEBGOAT_PORT=9000
```
Copy
Windows
```
set WEBGOAT_HOST=xx.xx.xx.xx
set WEBGOAT_PORT=9000
```
Access the application using any of the following URLs:
- http://192.xx.xx.xx:9000/WebGoat/login
- http://localhost:9000/WebGoat/login

Perfecto Connect

Perfecto Connect opens a direct connection between devices and browsers in the Perfecto lab and the local network where security testing tools (ZAP or Burp Suite) get installed. You need to generate a tunnelID with your machine's IP address and port to which the respective security testing tool is listening.

For example:

Copy

perfectoconnect start -c <<cloudname>>.perfectomobile.com -s <<SecurityToken>> --bridgeproxyip=<<XXX.XXX.XX.XXX>> --bridgeproxyport=8084 -fd

For details, seePerfecto Connect.

Pen testing steps

This section walks you through the testing steps. Click a pen testing tool to view the relevant steps.

Web security with ZAP

Zed Attack Proxy (ZAP) is an open-source penetration testing tool that is available for free. It gets maintained under the Open Web Application Security Project (OWASP). ZAP is flexible and extensible and used for testing web applications.

ZAP is situated between the tester’s browser and the web application. It intercepts and inspects messages between the browser and web application, modifies messages if needed, and sends them to the destination. ZAP can act as a stand-alone application and as a daemon process.

Manual pen testing

When the Perfecto device and ZAP are ready to capture the traffic for penetration testing, specify a breakpoint in ZAP, receive the request, and play around with the API to penetrate.

In the below example (WebGoat Challenge), the user resets the password by providing random security questions. In the actual API, the parameters secQuestion0 and secQuestion1 changed to secQuestionA and secQuestionB. The system ignores the security questions and allows users to change the password.

Automated pen testing

This exercise uses the Quantum framework. The sample integrated framework is available here. However, you can use the same method and steps in any other Java-based framework or port it to other language frameworks.

For more information on Quantum, see our Quantum documentation.

Copy

Sample test scenario (ZAP)

@ZAPSecurityTest
Feature: Automated Security Scan Using ZAP tool Feature

 @WebGoatSecurityTest_LoginPage
 Scenario: Verify WebGoat login Page with Automated Security Scan using ZAP
   Given I Start the ZAPScanner
   Then I launch WebGoat Login page
   And I run automated scans, report security issues and download html ZAP report
   Then I Close the ZAPScanner

 @WebGoatSecurityTest_HomePage
 Scenario: Verify WebGoat Home Page with Automated Security Scan using ZAP
   When I start the ZAPScanner
   Then I login the WebGoat app with "guestuser" and "guest123" credentials
   And Verify the home page
   Then I run automated scans, report security issues and download html ZAP report
   Then I Close the ZAPScanner

For filtering execution reports in the Perfecto Report Library, you can create a custom failure reason called Security Vulnerability fnd. For more information, see Work with failure reasons and Manage failure reasons.

Perfecto Smart Reporting

In Perfecto, the Test Analysis view provides access to reporting information. The following image shows a sample CI Dashboard.

The Report Library presents a detailed overview of the tests run within a selected period, including:

Statistical information for all listed tests
A historical breakdown of when tests failed
A table of all selected tests, including information on the parameters of the test run

If any security violations occur, the test fails with the custom failure reason Security Vulnerability fnd. This makes it easy to distinguish between functional and security violation errors.

Inside the Single Test Report (STR), the assert steps indicate the number of security violations with severity, details, and recommended solutions to mitigate the violation.

ZAP generates a detailed HTML report for each test if any security violations occur. The development team can analyze and follow the recommended solution provided in the report. You can use the API for external test information to have all information in one place and upload the reports Perfecto Smart Reporting.

Important: This requires a separate license for the Perfecto Lab.

Perfecto Insights

PerfectoInsights helps you jump-start your root cause analysis efforts, improve your testing success over time, and accelerate the identification of real bugs in the tested application. Insights can help you improve your success rate and efficiently find the real bug in the tested application. The latter goal may be hard to achieve when many tests fail for various reasons. Which error should you focus on first? Which test is the one that deserves immediate attention?

Insights clearly outlines that most tests do not fail due to functional issues but due to security vulnerabilities. It gives a clear idea to management, product owners, architects, and developers how the product evolves with subsequent regression tests.

For more information on Insights and how to work with it, see our Insights documentation.

Web security with Burp Suite

Burp Suite is an intercepting proxy that acts as middleman between the target web application and the web server. It captures the ongoing HTTP requests so that you can pause, replay, and even manipulate them before sending them on to the destination server.

Manual pen testing

When the device and Burp Suite are ready to capture the traffic for penetration testing, click the Intercept Is On button under the Proxy menu to receive the request and play around with the API to penetrate. In the following example (WebGoat Challenge), a hacker tries to reset the password by providing random security questions. In the actual API, the parameters secQuestion0 and secQuestion1 changed to secQuestionA and secQuestionB. The system ignores the security questions and allows users to change the password.

Automation pen testing

Important: For automated scanning, you must have a professional version of BurpSuite. For more details, click here.

For more information on Quantum, see our Quantum documentation.

Copy

Sample test scenario (Burp Suite)

@BurpSecurityTest
Feature: Automated Security Scan Using Burp tool Feature

   @WebGoatSecurityTest_LoginPage
   Scenario: Verify WebGoat login Page with Automated Security Scan using Burp
      Given I Start the BurpScanner
      Then I launch WebGoat Login page
      And I run automated scans, report security issues and download html Burp report
      Then I Close the BurpScanner

   @WebGoatSecurityTest_HomePage
   Scenario: Verify WebGoat Home Page with Automated Security Scan using Burp
      When I start the BurpScanner
      Then I login the WebGoat app with "guestuser" and "guest123" credentials
      And Verify the home page
      Then I run automated scans, report security issues and download html Burp report
      Then I Close the BurpScanner

For filtering execution reports in the PerfectoReport Library, you can create a cCustom failure reason called Security Vulnerability fnd can be created in the Perfecto Smart Reporting for filtering execution reports. For more information, see Work with failure reasons and Manage failure reasons.

Perfecto Smart Reporting

In Perfecto, the Test Analysis view provides access to reporting information. The following image shows a sample CI Dashboard.

The Report Library presents a detailed overview of the tests run within a selected period, including:

Statistical information for all listed tests
A historical breakdown of when tests failed
A table of all selected tests, including information on the parameters of the test run

If any security violations occur, the test fails with the custom failure reason Security Vulnerability fnd. This makes it easy to distinguish between functional and security violation errors.

Inside the Single Test Report (STR) report, the assert steps indicate the number of security violations with severity, details, and recommended solutions to mitigate the violation.

Burp Suite generates a detailed HTML report for each test if any security violations occur. The development team can analyze and follow the recommended solution provided in the report. You can use the API for external test information to have all information in one place and upload the reports to Perfecto Smart Reporting.

Important: This requires a separate license for the Perfecto Lab.

PerfectoInsights

Insights clearly outlines that most tests do not fail due to functional issues but instead due to Ssecurity Vvulnerabilities. It gives a clear idea to management, product owners, architects, and developers how the product evolves with subsequent regression tests.

For more information on Insights and how to work with it, see our Insights here documentation.