Pritpal Panesar

Platform Engineer

OWASP ZAP

OWASP ZAP (Zed Attack Proxy) is a free open source penetration testing tool. It requires Java 8+ to run. It acts as a proxy between your browser session and the web site/application allowing it to capture (an modify if you want) traffic between the two. You can actively or passively scan a site, or run it interactively. You can run it via the app (with a ui), cli, docker, as a daemon process or via the api. Scan your site, generate a report and fix any issues…

The Problem

All sounds great, the app is open source and very flexible…But our issue came from the site being hidden behind an external authentication source. There’s no way to get around this and you can’t get into any of the site without authenticating yourself first. Reading the docs I just couldn’t find a simple solution to get around this. Just pointing the app at the site to spider it just wouldn’t work and there’s limited options to “log in” in the config.

Solution

After some experimenting and reading, I ended up concluding that the only way to do what I needed was to run ZAP in headless proxy mode and automate my journey through the site via Selenium using ZAP as a proxy. Selenium allowed me to visit the site, log in and fill in the forms on the site to complete the user journey.

In testing this solution, I created a Python script to trace a journey through the site. If you’ve ever used Selenium, you’ll know how time consuming (and repetitive) this is. Our site, as with most Government sites, was simply a collection of pages where you answer a series of questions to progress. We were already using Selenium to run a series of browser tests, so it was relatively simple to add another switch to that code to switch to passing the tests through the ZAP proxy.

After testing the collection of data in ZAP when using it as a proxy, the next step was to generate a report. This can be done from the UI relatively easily, but we need to be able to do it from a command line. Luckily there is the API that we can use. After some testing, I was able to put together a Python script that called the API with the right options to generate a report.

So individually I was able to hit my site via the proxy and then generate a report. The next step was to put this all into a pipeline so we could run this once a week without user intervention.

The Pipeline

The steps for setting up the pipeline are as follows:

  • Start ZAP
  • Run tests using ZAP as a proxy
  • Generate a report from ZAP
  • Upload the report
  • Stop/Close ZAP

We use self hosted build agents, so I installed the latest version of OWASP ZAP to the build agent, as well as some Python libraries for generating the results.

Running ZAP in daemon mode, you have specify a api key at launch time. This was an important step to remember as you need to use the api key to get it to generate a report (without setting the api key, you won’t be able to generate a report via the command line). The following is the pipeline step to launch OWASP ZAP. Note that we use the supplied batch file to launch the app using start to launch it in the background in such a way that it should keep running when the step completes.

start "" /B zap.bat -silent -config api.key=YourAPIKeyHere

In the next step of the pipeline, I checked that OWASP ZAP had launched correctly and was still running. The batch file launches a Java process and it was the only Java process running on the server.

tasklist /fi "ImageName eq java.exe" /fo csv 2>NUL

Once the tests are run, you need to generate the report. This is done via a call to the api on the running instance of OWASP ZAP. We’re running it in daemon mode using the default port of 8080, so we can create a report by sending a request to localhost on 8080. There’s plenty of documentation on the OWASP ZAP site. The following is an example of the request to generate a report.

http://localhost:8080/JSON/reports/action/generate/?apikey=myAPIKeyHere&title=test&template=traditional-html&theme=&description=&contexts=&sites=https%3A%2F%2Fwww.mywebsite.com&sections=&includedConfidences=&includedRisks=&reportFileName=&reportFileNamePattern=&reportDir=&display=

To run this in the pipeline, I created a parameterized Python script passing in various values (see below). This produces a html report that you can then upload back into Azure Devops.

import requests
import argparse
import os

# add argument for reportname
parser = argparse.ArgumentParser()
parser.add_argument('--reportname', default='myreport')
parser.add_argument('--reportdir', default=os.getcwd())
parser.add_argument('--sites', default='https://mywebsite.com')
parser.add_argument('--zapkey', default='randomAPIKeyHere')
args = parser.parse_args()

# This is the command we want to run once ZAP is running on daemon mode
# http://localhost:8080/JSON/reports/action/generate/?apikey=randomAPIKeyHere&title=test&template=traditional-html&theme=&description=&contexts=&sites=https%3A%2F%2Fmywebsite.com&sections=&includedConfidences=&includedRisks=&reportFileName=&reportFileNamePattern=&reportDir=&display=

zapAddress = "http://localhost:8080/JSON/reports/action/generate/"
zapAPIKey = args.zapkey
title = "My OWASP ZAP Report"
template = "traditional-html"
sites = args.sites
reportFileName = args.reportname
reportDir = args.reportdir

try:
   # check that we get a response from ZAP
   response = requests.get("http://localhost:8080", stream=False, timeout=None)
   print(response.status_code)
   query = {'apikey':zapAPIKey,'title':title,'template':template,'sites':sites,'reportFileName':reportFileName,'reportDir':reportDir}
   headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0',
    'ACCEPT' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'ACCEPT-ENCODING' : 'gzip, deflate, br',
    'ACCEPT-LANGUAGE' : 'ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7'
   }
   print(query)
   print(zapAddress)
   print(sites)
   print(reportFileName)
   print(reportDir)
   response = requests.get(zapAddress, query, headers=headers, stream=False, timeout=None)
   print(response.status_code)
   print(response.json())
except requests.exceptions.HTTPError as errh:
    print(errh)
except requests.exceptions.ConnectionError as errc:
    print(errc)
except requests.exceptions.Timeout as errt:
    print(errt)
except requests.exceptions.RequestException as err:
    print(err)

Once this is done, your free to kill the OWASP ZAP process and free up the resources.

taskkill /f /im java.exe

Again, it’s the only Java process running on the server.

You May Also Enjoy