diff --git a/CHANGES.rst b/CHANGES.rst index ef69e01c..538446fc 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -1,13 +1,66 @@ Changes ======= -Version 0.9.0 (UNRELEASED) +Version 0.9.0 (2023-01-26) -------------------------- +- Users: + - Adds support for Rucio authentication for workflow jobs. + - Adds support for Kerberos authentication for workflow orchestration. + - Adds support for specifying ``slurm_partition`` and ``slurm_time`` for Slurm compute backend jobs. + - Adds support for XRootD remote file locations in Snakemake workflow specification definitions. + - Adds support for Python 3.11. + - Adds Launch on REANA page allowing the submission of workflows via badge-clicking. + - Adds notifications to inform users when critical levels of quota usage is reached. + - Adds 404 Not Found error page. + - Adds tab titles to all the pages. + - Adds the ``REANA_WORKSPACE`` environment variable to jupyter notebooks and terminals. + - Adds option to sort workflows by most disk and cpu quota usage to the workflow list endpoint. + - Adds support for specifying and listing workspace file retention rules. + - Adds support for ``.gitignore`` and ``.reanaignore`` to specify files that should not be uploaded to REANA. + - Adds ``retention-rules-list`` command to list the retention rules of a workflow. + - Changes REANA specification to allow specifying ``retention_days`` for the workflow. + - Changes default Slurm partition to ``inf-short``. + - Changes GitLab integration to also retrieve user's projects that are in groups and subgroups. + - Changes the workflow-details page to show the logs of the workflow engine. + - Changes the workflow-details page to show file sizes in a human-readable format. + - Changes the workflow-details page to show the workspace's retention rules. + - Changes the workflow-details page to show the duration of the workflow's jobs. + - Changes the workflow-details page to display a label of the workflow launcher URL remote origin. + - Changes the workflow-details page to periodically refresh the content of the page. + - Changes the workflow-details page to refresh after the deletion of a workflow. + - Changes the workflow-list page to add a way to hide deleted workflows. + - Changes the workflow-list page to add new workflows sorting options by most used disk and cpu quota. + - Changes the deletion of a workflow to always clean up the workspace and to update the user disk quota usage. + - Changes the CWD of jupyter's terminals to the directory of the workflow's workspace. + - Changes percentage ranges used to calculate the health status of user resource quota usage. + - Changes ``create`` and ``restart`` commands to always upload the REANA specification file. + - Changes ``delete`` command to always delete the workflow's workspace. + - Changes ``delete_workflow`` Python API function to always delete the workflow's workspace. + - Changes ``download`` command to add the possibility to write files to the standard output via ``-o -`` option. + - Changes ``list`` command to hide deleted workflows by default. + - Changes ``list`` command to allow displaying deleted workflows via ``--all`` and ``--show-deleted-runs`` options. + - Changes ``list`` and ``status`` commands to allow displaying the duration of workflows with the ``--include-duration`` option. + - Changes ``mv`` command to allow moving files while a workflow is running. + - Changes ``upload`` command to prevent uploading symlinks. + - Changes ``validation --environment`` command to use Docker registry v2 APIs to check that a Docker image exists in DockerHub. + - Fixes ``list`` command to highlight the workflow specified in ``REANA_WORKON`` correctly. + - Fixes ``secrets-delete`` command error message when deleting non existing secrets. + - Fixes ``start`` command to report failed workflows as errors. + - Fixes ``start`` and ``run`` commands to correctly follow the execution of the workflow until termination. + - Fixes ``status`` command to respect output format provided by the ``--format`` option. + - Fixes ``upload`` command to report when input directories are listed under the ``files`` section in the REANA specification file and vice versa. + - Fixes ``validate --environment`` command to detect illegal whitespace characters in Docker image names. + - Fixes Kerberos authentication for long-running workflows by renewing the Kerberos ticket periodically. + - Fixes status reporting for failed CWL and Snakemake jobs that were incorrectly considered successful. + - Fixes redirection chain for non-signed-in CERN SSO users to access the desired target page after sign-in. + - Fixes the ordering by size of the files showed in the ``Workspace`` tab of the workflow-details page. + - Fixes CERN OIDC authentication to possibly allow eduGAIN and social login users. + - Fixes wrong numbering of restarted workflows by limiting the number of times a workflow can be restarted to nine. - Administrators: + - Adds new configuration environment variable ``reana_server.environment.REANA_SCHEDULER_REQUEUE_COUNT`` to set workflow requeue count in case of scheduling errors or busy cluster situations. - Adds "infinity" option to ``REANA_SCHEDULER_REQUEUE_COUNT`` to disable requeue count. - Adds support for Kubernetes clusters 1.22, 1.23, 1.24, 1.25. - - Removes support for Kubernetes version prior to 1.19. - Adds new configuration option ``workspaces.retention_rules.maximum_period`` to set a default period for workspace retention rules. - Adds new configuration option ``workspaces.retention_rules.cronjob_schedule`` to set how often pending retention rules should be applied. - Adds configuration environment variable ``reana_server.environment.REANA_RATELIMIT_SLOW`` to limit API requests to some protected endpoints e.g launch workflow. @@ -16,17 +69,57 @@ Version 0.9.0 (UNRELEASED) - Adds new configuration option `ingress.tls.secret_name` to specify the name of the Kubernetes secret containing the TLS certificate to be used. - Adds support for configuring an additional volume to be used by the database and the message broker. - Adds new configuration option `maintenance.enabled` to scale down the cluster for maintenance. - - Changes default consumer prefetch count to handle 10 messages instead of 200 in order to reduce the probability of 406 PRECONDITION errors on message acknowledgement. + - Adds support for Unicode characters inside email body. + - Adds ``queue-consume`` command that can be used by REANA administrators to remove specific messages from the queue. + - Adds ``retention-rules-apply`` command that can be used by REANA administrators to apply pending retention rules. + - Adds ``retention-rules-extend`` command that can be used by REANA administrators to extend the duration of active retentions rules. + - Adds ``check-workflows`` command that can be used by REANA administrators to check for out-of-sync workflows and interactive sessions. - Changes configuration option ``quota.workflow_termination_update_policy`` to deactivate workflow termination accounting by default. - Changes Helm template to use PostgreSQL 12.13 version. + - Changes the base image for most of the components to Ubuntu 20.04 LTS and reduces final Docker image size by removing build-time dependencies. + - Changes ``reana-auth-vomsproxy`` sidecar to the latest stable version to support client-side proxy file generation technique and ESCAPE VOMS. + - Changes OAuth configuration to enable the new CERN SSO. + - Changes job status consumer to improve logging for not-alive workflows. + - Changes the deployment of interactive sessions to improve security by not automounting the Kubernetes service account token. + - Changes the deployment of job-controller to avoid unnecessarily mounting the database's directory. + - Changes the announcements to support limited HTML markup. + - Changes REANA specification loading functionality to allow specifying different working directories. + - Changes global setting of maximum number of parallel jobs to 300 for Snakemake workflow engine. + - Fixes job status consumer by discarding invalid job IDs. + - Fixes GitLab integration error reporting in case user exceeds CPU or Disk quota usage limits. + - Fixes issue when irregular number formats are passed to ``REANA_SCHEDULER_REQUEUE_COUNT`` configuration environment variable. + - Fixes quota updater to reduce memory usage. + - Fixes conversion of possibly-negative resource usage values to human-readable formats. + - Fixes disk quota updater to prevent setting negative disk quota usage values. + - Removes support for Kubernetes version prior to 1.19. - Developers: + - Adds OpenAPI specification support for ``launch`` endpoint that allows running workflows from remote sources. + - Adds OpenAPI specification support for ``get_workflow_retention_rules`` endpoint that allows to retrieve the workspace file retention rules of a workflow. + - Adds the remote origin of workflows submitted via Launch-on-REANA (``launcher_url``) to the workflow list endpoint. + - Adds common utility functions for managing workspace files to ``reana-commons``. + - Changes default consumer prefetch count to handle 10 messages instead of 200 in order to reduce the probability of 406 PRECONDITION errors on message acknowledgement. - Changes `git-upgrade-shared-modules` to generate the correct upper-bound in `setup.py`. - -Version 0.8.2 (UNRELEASED) --------------------------- - -- Administrators: - - Adds new configuration environment variable ``reana_server.environment.REANA_SCHEDULER_REQUEUE_COUNT`` to set workflow requeue count in case of scheduling errors or busy cluster situations. + - Changes REANA specification loading and validation functionalities by porting some of the logic to ``reana-commons``. + - Changes OpenAPI specification to include missing response schema elements. + - Changes the Kubernetes Python client to use the ``networking/v1`` API. + - Changes the deployment of interactive sessions to use ``networking/v1`` Kubernetes API. + - Changes to Flask v2. + - Changes ``/api/info`` endpoint to also include the kubernetes maximum memory limit, the kubernetes default memory limit and the maximum workspace retention period. + - Changes ``start_workflow`` endpoint to validate the REANA specification of the workflow. + - Changes ``create_workflow`` endpoint to populate workspace retention rules for the workflow. + - Changes ``start_workflow`` endpoint to disallow restarting a workflow when retention rules are pending. + - Changes API rate limiter error messages to be more verbose. + - Changes workflow scheduler to allow defining the checks needed to assess whether the cluster can start new workflows. + - Changes workflow list endpoint to add the possibility to filter by workflow ID. + - Changes the ``move_files`` endpoint to allow moving files while a workflow is running. + - Changes the k8s specification of interactive sessions' pods to remove the environment variables used for service discovery. + - Changes GitLab integration to use ``reana`` as pipeline name instead of ``default`` when setting status of a commit. + - Changes the loading of Snakemake specifications to preserve the current working directory. + - Changes the Invenio dependencies to the latest versions. + - Fixes the submission of jobs by stripping potential leading and trailing whitespaces in Docker image names. + - Fixes ``fetchWorkflow`` action to fetch a specific workflow instead of the entire user workflow list. (reana-ui) + - Fixes the download of files by changing the default MIME type to ``application/octet-stream``. + - Fixes the workflow list endpoint to correctly parse the boolean parameters ``include_progress``, ``include_workspace_size`` and ``include_retention_rules``. Version 0.8.1 (2022-02-15) -------------------------- diff --git a/helm/reana/Chart.yaml b/helm/reana/Chart.yaml index 2a3c1ae8..cc8a3eba 100644 --- a/helm/reana/Chart.yaml +++ b/helm/reana/Chart.yaml @@ -27,7 +27,7 @@ keywords: - reusable-science type: application # Chart version. -version: 0.9.0-alpha.8 +version: 0.9.0 kubeVersion: ">= 1.19.0-0 < 1.26.0-0" dependencies: - name: traefik diff --git a/helm/reana/values.yaml b/helm/reana/values.yaml index 8d709260..e647c6ce 100644 --- a/helm/reana/values.yaml +++ b/helm/reana/values.yaml @@ -68,12 +68,12 @@ components: docs_url: https://docs.reana.io forum_url: https://forum.reana.io imagePullPolicy: IfNotPresent - image: reanahub/reana-ui:0.9.0-alpha.7 + image: reanahub/reana-ui:0.9.0 reana_db: enabled: true reana_server: imagePullPolicy: IfNotPresent - image: reanahub/reana-server:0.9.0-alpha.8 + image: reanahub/reana-server:0.9.0 environment: REANA_MAX_CONCURRENT_BATCH_WORKFLOWS: 30 REANA_SCHEDULER_REQUEUE_SLEEP: 15 @@ -92,23 +92,23 @@ components: log_5xx: true reana_workflow_controller: imagePullPolicy: IfNotPresent - image: reanahub/reana-workflow-controller:0.9.0-alpha.8 + image: reanahub/reana-workflow-controller:0.9.0 environment: SHARED_VOLUME_PATH: /var/reana REANA_JOB_STATUS_CONSUMER_PREFETCH_COUNT: 10 reana_workflow_engine_cwl: - image: reanahub/reana-workflow-engine-cwl:0.9.0-alpha.5 + image: reanahub/reana-workflow-engine-cwl:0.9.0 reana_workflow_engine_yadage: - image: reanahub/reana-workflow-engine-yadage:0.9.0-alpha.5 + image: reanahub/reana-workflow-engine-yadage:0.9.0 reana_workflow_engine_serial: - image: reanahub/reana-workflow-engine-serial:0.9.0-alpha.5 + image: reanahub/reana-workflow-engine-serial:0.9.0 reana_workflow_engine_snakemake: - image: reanahub/reana-workflow-engine-snakemake:0.9.0-alpha.6 + image: reanahub/reana-workflow-engine-snakemake:0.9.0 reana_job_controller: - image: reanahub/reana-job-controller:0.9.0-alpha.8 + image: reanahub/reana-job-controller:0.9.0 reana_message_broker: imagePullPolicy: IfNotPresent - image: reanahub/reana-message-broker:0.9.0-alpha.1 + image: reanahub/reana-message-broker:0.9.0 notifications: enabled: false diff --git a/reana/version.py b/reana/version.py index 4053b17e..be2566a9 100644 --- a/reana/version.py +++ b/reana/version.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # This file is part of REANA. -# Copyright (C) 2017, 2018, 2019, 2020, 2021, 2022 CERN. +# Copyright (C) 2017, 2018, 2019, 2020, 2021, 2022, 2023 CERN. # # REANA is free software; you can redistribute it and/or modify it # under the terms of the MIT License; see LICENSE file for more details. @@ -13,4 +13,4 @@ from __future__ import absolute_import, print_function -__version__ = "0.9.0a8" +__version__ = "0.9.0" diff --git a/scripts/prefetch-images.sh b/scripts/prefetch-images.sh index 1c326c9b..dfda7423 100755 --- a/scripts/prefetch-images.sh +++ b/scripts/prefetch-images.sh @@ -11,15 +11,15 @@ for image in \ maildev/maildev:1.1.0 \ postgres:12.13 \ redis:5.0.5 \ - reanahub/reana-job-controller:0.9.0-alpha.8 \ - reanahub/reana-message-broker:0.9.0-alpha.1 \ - reanahub/reana-server:0.9.0-alpha.8 \ - reanahub/reana-ui:0.9.0-alpha.7 \ - reanahub/reana-workflow-controller:0.9.0-alpha.8 \ - reanahub/reana-workflow-engine-cwl:0.9.0-alpha.5 \ - reanahub/reana-workflow-engine-serial:0.9.0-alpha.5 \ - reanahub/reana-workflow-engine-snakemake:0.9.0-alpha.6 \ - reanahub/reana-workflow-engine-yadage:0.9.0-alpha.5 \ + reanahub/reana-job-controller:0.9.0 \ + reanahub/reana-message-broker:0.9.0 \ + reanahub/reana-server:0.9.0 \ + reanahub/reana-ui:0.9.0 \ + reanahub/reana-workflow-controller:0.9.0 \ + reanahub/reana-workflow-engine-cwl:0.9.0 \ + reanahub/reana-workflow-engine-serial:0.9.0 \ + reanahub/reana-workflow-engine-snakemake:0.9.0 \ + reanahub/reana-workflow-engine-yadage:0.9.0 \ ; do docker pull $image kind load docker-image $image