sprint production release v2022.03.14

[harshad16]

Hello, Thoth-station!

This Issue would be used for the current sprint cycle production release.
By the end of the sprint cycle, we will consolidate the information of thoth-station components features upgrade and fixes in this issue.

/kind documentation
/area release-eng
/triage accepted
/milestone 2022.03.14
/priority important-soon

[fridex]

Notify users when they use PyTorch index

Newly, the resolver notifies users if they consume releases from the PyTorch index.

• Notify when using PyTorch index adviser#2279

[fridex]

Memory optimizer in adviser

If Thoth's adviser consumes too much memory and is killed on OOM in a deployment, a memory optimizer can be turned on. The memory optimizer removes data from some of the internal data structures to optimize memory consumption. This can slow down finding a resolved set of dependencies or making the search not that optimal. Users, however, can get results instead of OOM.

• Advise does not give a stack_info when OOM is thrown adviser#2203
• Implement memory optimizer for resolver adviser#2280
• Set up adviser memory optimizer #2395

[harshad16]

we have completed the release for v2022.03.14 🎉 🎊 🥳

Features

Notify users when they use PyTorch index

Newly, the resolver notifies users if they consume releases from the PyTorch index.

• Notify when using PyTorch index adviser#2279

Memory optimizer in adviser

If Thoth's adviser consumes too much memory and is killed on OOM in a deployment, a memory optimizer can be turned on. The memory optimizer removes data from some of the internal data structures to optimize memory consumption. This can slow down finding a resolved set of dependencies or making the search not that optimal. Users, however, can get results instead of OOM.

• Advise does not give a stack_info when OOM is thrown adviser#2203
• Implement memory optimizer for resolver adviser#2280
• Set up adviser memory optimizer #2395

Created new handler that uses image analysis results and generate new prescriptions

Automatically propagated from the container image analyses - the container image analysis would provide how Pipfile looked like on image build and that information will be part of the image analyses information derived by package-extract (similarly as we propagate information about RPM packages). In this case, we implemented a handler in prescriptions-refresh-job that:

• checks what ps images are hosted on quay
• ask what is the container image analysis result and how the Pipfile looked like during the build
• automatically create prescriptions based on direct dependencies in Pipfile

References:

• Use an automated script in a pipeline to generate prescriptions from image analyzed core#343
• Create new handler that uses image analysis results and generate new prescriptions prescriptions-refresh-job#87
• https://thoth-station.ninja/docs/developers/adviser/prescription/wraps.html#prescription-wraps

Automatically bump base image versions to latest available on Quay

A new script integrated in the aicoe-ci pipeline allows to automatically update the base image versions present in configuration files such as .aicoe-ci.yaml to the latest available on Quay. The pipeline is triggered by an action on a repository such as the opening of a pull request or an issue. If the base image versions are not up-to-date compared on available versions on Quay, a pull request is automatically opened for the corresponding update.
For an example, see: thoth-station/package-releases-job#637 which was triggered by thoth-station/package-releases-job#636 on the package-releases-job repository.

• Add script to bump base image versions to latest pipeline-helpers#45
• Added Automatic base image bump pipeline AICoE/aicoe-ci#170

Fix method to iterate on thoth files

slo-reporter was not able to provide an analysis of adviser results. advise-reporter logic was not able to read adviser files, due to a method in thoth-storages. Iterating over files in a bucket gives empty lists even if files exist because the prefix used to identify them is using a default value that is not updated. The value RESULT_TYPE is fixed.

• Fix method to iterate on thoth files storages#2609

Component Updates

• adviser: v0.52.4

  • Implement memory optimizer for resolver adviser#2280
  • Notify when using PyTorch index adviser#2279

• prescriptions-refresh-job: v0.6.0

  • Add handler for thoth image analysis command prescriptions-refresh-job#97

• integration-tests: v0.11.0

  • Fix scenarios related to dependency monkey integration-tests#287
  • Check available runtime environments integration-tests#283
  • Provide provenance-check that results in errors integration-tests#269

Thanks for the amazing work everyone. 💯

[mayaCostantini]

Automatically bump base image versions to latest available on Quay

A new script integrated in the aicoe-ci pipeline allows to automatically update the base image versions present in configuration files such as .aicoe-ci.yaml to the latest available on Quay. The pipeline is triggered by an action on a repository such as the opening of a pull request or an issue. If the base image versions are not up-to-date compared on available versions on Quay, a pull request is automatically opened for the corresponding update.
For an example, see: thoth-station/package-releases-job#637 which was triggered by thoth-station/package-releases-job#636 on the package-releases-job repository.

• Add script to bump base image versions to latest pipeline-helpers#45
• Added Automatic base image bump pipeline AICoE/aicoe-ci#170