This is a stub service for testing and integration with the HMRC upscan services and AWS. It removes the need to have the full suite of upscan services (upscan-initiate, upscan-verify, upscan-notify) running locally, and also replaces the need for local virus scanning, and communication with AWS from a local machine.
upscan-stub is intended for local testing only. i.e. smoke testing rather than acceptance testing. For acceptance tests, running in a Jenkins hosted environment, teams are strongly encouraged to test against the real suite of Upscan microservices instead of using upscan-stub. The benefits of this are:
- True reflection of Production functionality
- More test coverage for the Upscan services themselves
upscan-stub does not provide 100% of the functionality available in the Upscan microservices suite. (If it did, we would have effectively rewritten Upscan again!) Specifically, the following functionality is missing from upscan-stub:
- Verification that all required form fields are present when uploading a file
- Observability: metrics and logging
- Enforcement of correct field ordering when uploading files.
If you are unclear about the full functionality of the upscan services (including AWS and ClamAV), please read the documentation here.
By running this service locally, the following interactions can take place:
- POST to initiate a request and receive JSON parameters for an upload form. This stubs upscan-initiate
- POST a form to upload a file. The file will be stored in local temporary storage while the application runs. This stubs AWS S3.
- Receive a callback to a URL passed in during Step 2. The will be triggered automatically via an Actor when Step 2 completes. This stubs upscan-verify and upscan-notify
- Download the file POSTED in step 2 from local URL contained in the callback of Step 3. This stubs AWS S3.
Additionally, a specific file can be passed in at Step 2 which will cause the application to quarantine the file and notify of the quarantine in Step 3.
| Path | Supported Methods | Description |
|---|---|---|
/upscan/initiate |
POST | Endpoint to retrieve parameters for building upload form. Documented here: upscan-initiate |
/upscan/v2/initiate |
POST | Endpoint to retrieve parameters for building upload form offering the additional capability to redirect on upload error. Documented here: upscan-initiate-v2 |
/upscan/upload |
POST | Endpoint to upload a file, replacing the endpoint for uploading to an AWS S3 bucket. Documented here: upscan-initiate |
/upscan/download/:reference |
GET | Retrieve a file from local storage, as uploaded via the /upscan/upload endpoint |
Additionally, the service will make a callback in the format documented in upscan-notify.
It is possible to easily force different upscan errors by simply renaming the uploaded file according to the following schema:
reject.S3_ERROR_CODE.EXT, e.g. reject.UnexpectedContent.png, see https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#ErrorCodeListinfected.VIRUS_NAME.EXT, e.g. infected.MyDoom.jpeginvalid.REASON.EXT, e.g. invalid.ZipInDisguise.txtunknown.REASON.EXT, e.g. unknown.SpookyCookie.pdf
It is possible to test the upload of a virus-infected file using a test file included in the upscan-stub project. Uploading the following file will trigger a quarantined file callback:
test/resources/eicar-standard-av-test-file.txt
If you're using the Sophos Home antivirus application on your laptop, then you may find the software automatically quarantines the virus infected test file mentioned above.
Each time you pull from git, Sophos may delete test/resources/eicar-standard-av-test-file.txt.
To avoid this, you can allowlist the file on the Sophos Dashboard.
Note: you will need to allowlist the copy of the file built to /target as well. i.e. both of:
test/resources/eicar-standard-av-test-file.txt
target/scala-2.11/test-classes/eicar-standard-av-test-file.txt
Some of the upscan-stub integration tests may also fail due to collisions with Sophos.
Currently there is no workaround other than to disable the tests, or Sophos, when running locally.
Start your upscan-stub service on port 9570 with the following command: sbt "run 9570"
Alternatively, the Service Manager profile for Upscan can be started with:
sm -r --start UPSCAN
The flow of calls relies upon a "listening" endpoint within your service to receive asynchronous notification once your file has been virus scanned and is ready to downloaded within your service.
If you require a service to capture callbacks whilst developing your own service, you can use the helper service upscan-listener.