chore(nvd): use NVD API v2

[RTann]

NVD plans to retire the 1.1 JSON feeds on December 15, 2023. This PR swaps our usage of those feeds with the v2 API.

For backwards compatibility purposes with previous Scanner versions, which may still be deployed somewhere and may still be officially supported, we convert the v2 API JSON responses to the 1.1 JSON formatting. This allows us to continue serving NVD data to live Scanners without any downtime nor required updates.

Before, there was a JSON file per year (2002 - Present year). Now, there will be a JSON file for every 20,000 vulnerabilities. The maximum file size, uncompressed, is about 60MB, and the genesis-dump.zip is about 164MB. The genesis-dump.zip associated with the 4.3.0 release is about 166MB, so these changes result in negligible differences in the genesis-dump.

The change in file names will not affect older Scanners (at least no Scanners which are still supported), as each Scanner looks inside of a directory called nvd/ (which has not changed) for all .json files. As long as the file extension is .json, the file name does not matter. See https://github.com/stackrox/scanner/blob/2.28.7/cpe/nvdtoolscache/load.go, which (as of writing this), has not been touched since well before 3.74 (Scanner 2.28) was released.

See stackrox/nvdtools#9 for the related changes to the nvdtools library.

NOTE: Once this is merged, all live Scanners will be getting this data, so this can be a potentially dangerous change.

[ghost]

Images are ready for the commit at 7d44713.

To use the images, use the tag 2.31.x-54-g7d44713ce0.

[RTann]

/retest slim-e2e-tests

[openshift-ci]

@RTann: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

• /test images
• /test push-images
• /test style-checks
• /test unit-tests

The following commands are available to trigger optional jobs:

• /test db-integration-tests
• /test diff-dumps
• /test e2e-tests
• /test scale-tests
• /test slim-e2e-tests
• /test store-db-dump
• /test store-genesis-dump

Use /test all to run all jobs.

In response to this:

/retest slim-e2e-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[RTann]

/test slim-e2e-tests

[RTann]

/retest

[RTann]

Getting 403s, which is concerning. Perhaps we should look into getting an API key...

[RTann]

/retest

[RTann]

/retest

[RTann]

Failing e2e test is unrelated: #1322

[RTann]

will merge tomorrow morning

[RTann]

Verified via these tests, checking for specific known ones, and ensuring StackRox 3.74.7 can handle the vulns without panicking and return scan results. Merging