Merge pull request #85 from veracode/dast-api

Add support for DAST Essentials API

.github/workflows/sca.yml

@@ -12,8 +12,7 @@ on:
   workflow_dispatch:
   push:
     branches:
-      - master
-      - main      
+      - '**'   
   pull_request:
     branches:
       - master

docs/dast.md

@@ -0,0 +1,73 @@
+# DAST
+
+The following methods call Veracode REST APIs and return JSON.
+
+## Targets
+
+Manage information and settings related to scan targets.
+
+- `DASTTargets().get_all()`: get a list of DAST targets to which you have access.
+- `DASTTargets().get(target_id)`: get the DAST target identified by `target_id`.
+- `DASTTargets().get_by_name(target_name)`: get a list of DAST targets whose name contains `target_name`.
+- `DASTTargets().search(target_name(opt), url(opt), search_term(opt), target_type)`: get a list of DAST targets to which you have access based on the search terms provided:
+  - `target_name`: finds targets whose name contains `target_name`
+  - `url`: finds targets whose url contains `url`
+  - `search_term`: finds targets whose name or URL contains `search_term`
+  - `target_type`: use to restrict the search to a `WEB_APP` or an `API`
+- `DASTTargets().create(name, description, protocol, url, api_specification_file_url, target_type, scan_type,is_sec_lead_only,teams(opt))`: create a DAST target. Note that this will also create an analysis profile for the target. Arguments include:
+  - `name`: the name of the target.
+  - `description`: the long description of the target.
+  - `protocol`: the protocol of the main URL for the target (`HTTP`, `HTTPS`).
+  - `url`: the main URL for the target. Must be specified for `target_type` = `WEB_APP`.
+  - `api_specification_file_url`: the API specification URL for the target. Must be specified for `target_type` = `API`. 
+  - `target_type`: use to specify that the target is a `WEB_APP` or an `API`.
+  - `scan_type`: use to specify the type of scan (`QUICK` or `FULL`).
+  - `is_sec_lead_only`: set to `False` if the target should be accessed only by one or more `teams`.
+  - `teams` (opt): an array of team GUIDs for whom access to the target should be restricted.
+- `DASTTargets().update(target_id, name, description, protocol, url, api_specification_file_url, target_type, scan_type,is_sec_lead_only,teams(opt))`: update the DAST target identified by `target_id`.
+- `DASTTargets().delete(target_id)`: delete the DAST target identified by `target_id`.
+
+
+## Analysis Profiles
+
+Configure analysis options for a scan.
+
+- `DASTAnalysisProfiles().get_all(target_id(opt),type(opt))`: Retrieve the analysis profiles for the account, optionally filtered by `target_id` or `type`.
+  - `target_id`: Retrieve the analysis profiles associated with the target identified by `target_id`. Note that this returns an array, though currently DAST Essentials only supports a single analysis profile per target.
+  - `type`: One of `TARGET` or `SYSTEM`.
+- `DASTAnalysisProfiles().get(analysis_profile_id)`: Retrieve the details for the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().update(self, analysis_profile_id, allowed_urls(opt),denied_urls(opt), seed_urls(opt), grouped_urls(opt), crawler_mode(opt), rate_limit(opt), max_duration(opt), max_crawl_duration(opt))`: Update the analysis profile identified by `analysis_profile_id` with one or more settings:
+  - `allowed_urls`: an array of the URLs the scanner is allowed to scan.
+  - `denied_urls`: an array of the URLs the scanner is not allowed to scan.
+  - `seed_urls`: an array of [seed URLs](https://docs.veracode.com/r/advanced-scan-configuration#seed-urls) that the scanner can use as starting points to crawl the target. Use this to include URLs that are not linked from the application but should be scanned.
+  - `grouped_urls`: an array of [grouped URLs](https://docs.veracode.com/r/advanced-scan-configuration#grouped-urls). Define this parameter to improve scanning speed on sites that have a large number of similar pages.
+  - `crawler_mode`: one of `SMART`, `EXHAUSTIVE`.
+  - `rate_limit`: an integer that limits the number of attacks the crawler makes in an interval.
+  - `max_duration`: an integer that specifies the maximum duration for the scan.
+  - `max_crawl_duration`: an integer that specifies the maximum duration for crawling the target.
+- `DASTAnalysisProfiles().update_parent(analysis_profile_id, parent_analysis_profile_id)`: identifies a new parent analysis profile for the analysis profile identified by `analysis_profile_id`. This allows inheriting analysis profile settings from the parent.
+- `DASTAnalysisProfiles().get_authentications(analysis_profile_id)`: Retrieve the authentication options for the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().update_system_auth(analysis_profile_id, username, password)`: Set the username and password used for basic (HTTP) authentication for the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().update_app_auth(analysis_profile_id, username, password, login_url)`: Set the username and password used for application authentication on the login page at `login_url`, for the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().update_parameter_auth(analysis_profile_id, id, title, type, key, value)`: Set the options for paraemeter authentication for the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().get_scanners(analysis_profile_id)`: get the scanners associated with the analysis profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().update_scanners(analysis_profile_id, scanner_id, scanner_value)`: For the analysis profile identified by `analysis_profile_id`, enable or disable the scanner identified by `scanner_id`. Allowed values include: [ 'fingerprinting', 'ssl', 'http_header', 'portscan', 'fuzzer', 'sql_injection', 'xss', 'file_inclusion', 'deserialization', 'xxe', 'command_injection', 'csrf', 'ldap_injection']
+- `DASTAnalysisProfiles().get_schedules(analysis_profile_id)`: Get the schedules associated with the application profile identified by `analysis_profile_id`.  
+- `DASTAnalysisProfiles().get_schedule(analysis_profile_id, schedule_id)`: Get the schedule identified by `schedule_id` and associated with the application profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().create_schedule(analysis_profile_id, frequency, day=1, weekday=1, timezone='America/New York',time="00:00"))`: Create a schedule for the application profile identified by `analysis_profile_id`. Options include:
+  - `frequency`: one of [`daily`, `weekly`, `monthly`]
+  - `day`: integer identifying the day of the month to perform a scan with monthly frequency
+  - `weekday`: integer identifying the day of the week to perform a scan with weekly frequency
+  - `timezone`: time zone identifier for scheduling the scan
+  - `time`: timestamp at which to start the scan
+- `DASTAnalysisProfiles().update_schedule(analysis_profile_id, schedule_id, frequency, day=1, weekday=1, timezone='America/New York',time="00:00"))`: Update the schedule identified by `schedule_id` for the application profile identified by `analysis_profile_id`.
+- `DASTAnalysisProfiles().delete_schedule(analysis_profile_id, schedule_id))`: Delete the schedule identified by `schedule_id` for the application profile identified by `analysis_profile_id`. 
+
+## Analysis Runs
+
+Begin or check the status of an analysis run.
+
+- `DASTAnalysisRuns().start(target_id)`: start an analysis run for the target identified by `target_id`.
+- `DASTAnalysisRuns().get(target_id)`: get the PDF report for the target identified by `target_id`. Returns a 400 if the scanning report is not ready. Save the response to a file to use.
+
+[All docs](docs.md)

docs/docs.md

@@ -13,7 +13,8 @@ See the topics below for more information on how to use this library.
 * [Findings and Annotations](findings.md) - retrieve findings and propose, accept, and reject mitigations.
 * [Collections](collections.md) - (EARLY ACCESS) create, update, access, and delete collections.
 * [SCA Agent](sca.md) - access information about SCA workspaces, projects, issues, vulnerabilities, libraries, and licenses.
-* [Dynamic Analysis](dynamic.md) - configure, schedule and start dynamic analyses.
+* [Dynamic Analysis](dynamic.md) - configure, schedule and start dynamic analyses (use with the Veracode Dynamic Analysis product).
+* [DAST](dast.md) - configure, schedule, and run DAST Essentials scans (use with the Veracode DAST Essentials product).
 * [Analytics](analytics.md) - request and retrieve reports across your Veracode organization.
 
 ## Administration

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = 'veracode_api_py'
-version = '0.9.48'
+version = '0.9.49'
 authors = [ {name = "Tim Jarrett", email="tjarrett@veracode.com"} ]
 description = 'Python helper library for working with the Veracode APIs. Handles retries, pagination, and other features of the modern Veracode REST APIs.'
 readme = 'README.md'
@@ -22,4 +22,4 @@ dependencies = {file = ["requirements.txt"]}
 [project.urls]
 "Homepage" = "https://github.com/veracode/veracode-api-py"
 "Bug Tracker" = "https://github.com/veracode/veracode-api-py/issues"
-"Download" = "https://github.com/veracode/veracode-api-py/archive/v_0948.tar.gz"
+"Download" = "https://github.com/veracode/veracode-api-py/archive/v_0949.tar.gz"

requirements.txt

@@ -2,3 +2,4 @@ requests>=2.31.0
 veracode-api-signing>=22.3.0
 urllib3>= 2.2.2
 Pygments>= 2.9.0
+idna>=3.7

samples/dast_sample.py

@@ -0,0 +1,53 @@
+import time
+import sys
+import requests
+from veracode_api_py import DASTAnalysisProfiles, DASTAnalysisRuns, DASTTargets, VeracodeAPIError
+
+# The most simple workflow to set up a DAST Essentials scan is as follows: 
+# 1. Create the target
+# 2. Configure the analysis profile
+# 3. Run the scan
+# 4. Get results
+# See the Veracode Docs (https://docs.veracode.com/r/DAST_Essentials_REST_API) for more info.
+
+#Update the values in this example for your own application
+target = DASTTargets().create(name='Example', description='Example Target', protocol='HTTP',
+                                url='www.example.com', target_type='WEB_APP', scan_type='QUICK', is_sec_lead_only=True)
+
+#optionally, assign the target to a team during the create
+
+analysisprofile = DASTAnalysisProfiles().get_all(target_id=target['target_id'])[0]
+
+# update with actual credentials and login info using other authentication methods
+DASTAnalysisProfiles().update_system_auth(analysis_profile_id=analysisprofile['analysis_profile_id'], 
+                                          username='admin',password='smithy') 
+
+DASTAnalysisProfiles().update(analysis_profile_id=analysisprofile['analysis_profile_id'], allowed_urls=[],
+                              denied_urls=[], seed_urls=[], grouped_urls=[], crawler_mode='SMART')
+
+scan = DASTAnalysisRuns().start(target_id=target['target_id'])
+
+print('Scan id {} started on url {} at {}'.format(scan['id'], scan['url'], scan['started_at']))
+
+# To fetch the report afterwards, use the following and save to a file.
+done = False
+
+while not done:
+    try:
+        report = DASTAnalysisRuns().get(target_id=target['target_id'])
+        done = True
+    except requests.exceptions.RequestException as re:
+        data = re.response.json()
+        if data['status']==400 and data['errors'][0]['code']=='INVALID_TARGET_STATE':
+            print('Report for target {} is not ready. Checking again in 10 seconds…'.format(target['target_id']))
+            time.sleep(10)
+            continue
+        else:
+            print("An error occurred: HTTP Error {}, {}, {}".format(data['status'],data['detail'],data['errors']))
+    except Exception as e:
+        ex_type, ex_value, ex_traceback = sys.exc_info()
+        print('A general exception occurred:{} {}'.format(ex_type.__name__, ex_value))
+        done=True
+
+print('Got the report')
+print(report) # save the report to a file to actually do something with it!

setup.py

@@ -7,15 +7,15 @@
 setup(
   name = 'veracode_api_py',         
   packages = ['veracode_api_py'],   
-  version = '0.9.48',      
+  version = '0.9.49',      
   license='MIT',        
   description = 'Python helper library for working with the Veracode APIs. Handles retries, pagination, and other features of the modern Veracode REST APIs.',   
   long_description = long_description,
   long_description_content_type="text/markdown",
   author = 'Tim Jarrett',                  
   author_email = 'tjarrett@veracode.com',      
   url = 'https://github.com/tjarrettveracode',   
-  download_url = 'https://github.com/veracode/veracode-api-py/archive/v_0930.tar.gz',    
+  download_url = 'https://github.com/veracode/veracode-api-py/archive/v_0949.tar.gz',    
   keywords = ['veracode', 'veracode-api'],   
   install_requires=[            
           'veracode-api-signing'

veracode_api_py/__init__.py

@@ -3,11 +3,13 @@
 from veracode_api_py.applications import Applications, Sandboxes, CustomFields
 from veracode_api_py.collections import Collections
 from veracode_api_py.dynamic import Analyses, Scans, CodeGroups, Configuration, ScannerVariables, ScanCapacitySummary, Occurrences, DynUtils
+from veracode_api_py.exceptions import VeracodeAPIError, VeracodeError
 from veracode_api_py.findings import Findings, SummaryReport, ManualScans
 from veracode_api_py.healthcheck import Healthcheck
 from veracode_api_py.identity import Users, Teams, BusinessUnits, APICredentials, Roles
 from veracode_api_py.sca import Workspaces, ComponentActivity, SBOM, SCAApplications
 from veracode_api_py.exceptions import VeracodeError, VeracodeAPIError
 from veracode_api_py.xmlapi import XMLAPI
 from veracode_api_py.analytics import Analytics
-from veracode_api_py.static import StaticCLI
+from veracode_api_py.static import StaticCLI
+from veracode_api_py.dast import DASTTargets, DASTAnalysisProfiles, DASTAnalysisRuns

veracode_api_py/apihelper.py

@@ -103,7 +103,7 @@ def _rest_request(self, url, method, params=None, body=None, fullresponse=False,
                 raise VeracodeAPIError("Unsupported HTTP method")
         except requests.exceptions.RequestException as e:
             logger.exception("Error: {}".format(self.connect_error_msg))
-            raise VeracodeAPIError(e.text)
+            raise VeracodeAPIError(e.text) from e
 
         if r.status_code != requests.codes.ok:
             logger.debug("API call returned non-200 HTTP status code: {}".format(r.status_code))
@@ -119,7 +119,11 @@ def _rest_request(self, url, method, params=None, body=None, fullresponse=False,
             else:
                 logger.exception("Error [{}]: {} for request {}".
                                  format(r.status_code, r.text, r.request.url))
-            raise requests.exceptions.RequestException()
+                re = requests.exceptions.RequestException()
+                re.response = r
+                re.errno = r.status_code
+                re.request = r.request
+            raise re
 
         if fullresponse:
             return r

veracode_api_py/applications.py

@@ -66,7 +66,7 @@ def _create_or_update(self,method,app_name: str,business_criticality, business_u
             return
 
         if business_criticality not in Constants().BUSINESS_CRITICALITY:
-            raise ValueError("{} is not in the list of valid buinsess criticalities ({})".format(business_criticality,Constants().BUSINESS_CRITICALITY))
+            raise ValueError("{} is not in the list of valid business criticalities ({})".format(business_criticality,Constants().BUSINESS_CRITICALITY))
 
         app_def = {'name':app_name, 'business_criticality':business_criticality}
 

veracode_api_py/constants.py

@@ -42,4 +42,16 @@ class Constants():
 
      DEV_STAGE = [ 'DEVELOPMENT', 'TESTING', 'RELEASE']
 
-     BUSINESS_CRITICALITY = [ 'VERY HIGH', 'HIGH', 'MEDIUM', 'LOW', 'VERY LOW']
+     BUSINESS_CRITICALITY = [ 'VERY HIGH', 'HIGH', 'MEDIUM', 'LOW', 'VERY LOW']
+
+     DAST_TARGET_TYPE = [ 'WEB_APP', 'API']
+
+     DAST_CRAWLER_MODE = [ 'SMART', 'EXHAUSTIVE' ]
+
+     DAST_PROTOCOL = [ 'HTTP', 'HTTPS']
+
+     DAST_SCAN_TYPE = [ 'QUICK', 'FULL']
+
+     DAST_SCANNERS = [ 'fingerprinting', 'ssl', 'http_header', 'portscan', 'fuzzer', 'sql_injection',
+                      'xss', 'file_inclusion', 'deserialization', 'xxe', 'command_injection', 
+                      'csrf', 'ldap_injection']