From bbbe83f20be951949b876d03beb075721ae73825 Mon Sep 17 00:00:00 2001 From: Andy Turner Date: Mon, 26 Aug 2024 12:11:07 +0100 Subject: [PATCH 1/2] Adds Sep 2024 capability days --- docs/user-guide/scheduler.md | 55 ++++++++++++++++++++++++++++++++++-- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/scheduler.md b/docs/user-guide/scheduler.md index 66bb6b7b0..79e8faaab 100644 --- a/docs/user-guide/scheduler.md +++ b/docs/user-guide/scheduler.md @@ -2286,8 +2286,11 @@ Your request will be checked by the ARCHER2 User Administration team and, if app ## Capability Days !!! important - We expect the next Capability Days session to be in August or September 2024. Once the dates have been - set, ARCHER2 users will be informed and this section updated. + The next Capability Days session will be from Tue 24 Sep 2024 to Thu 26 Sep 2024. + + - `pre-capabilityday` QoS: 0800-2000, Tue 24 Sep 2024 + - `NERCcapability` reservation: 0800-1600, Tue 24 Sep 2024 + - `capabilityday` QoS: 2000 Tue 24 Sep - 1400 Thu 26 Sep 2024 ARCHER2 Capability Days are a mechanism to allow users to run large scale (512 node or more) tests on the system free of charge. The motivations behind Capability Days are: @@ -2299,9 +2302,10 @@ on the system free of charge. The motivations behind Capability Days are: To enable this, a period will be made available regularly where users can run jobs at large scale free of charge. -Capability Days are made up of two parts: +Capability Days are made up of three parts: - pre-Capability Day session (`pre-capabilityday` QoS) to allow users to test scaling and job setup ahead of full Capability Day +- NERC Capability reservation (`NERCcapability` reservation) to allow NERC users to test at large scale - Capability Day session (`capabilityday` QoS) !!! tip @@ -2353,6 +2357,51 @@ srun --hint=multithread --distribution=block:block xthi > placement-${SLURM_JOBI srun --hint=multithread --distribution=block:block my_app.x ``` +### NERC Capability reservation + +The NERC Capability reservation is typically available directly before the full Capability Day session and allows +short test jobs to prepare for Capability Day. + +Submit to the `NERCcapability` *reservation*. Jobs can be submitted ahead of time and will start when the NERC Capability +reservatoin starts. + +`NERCcapability` reservation limits: + +- Only available to users in NERC projects +- Available for 8 hours +- 1024 nodes available +- Maximum job size: 1024 nodes +- Maximum walltime: 8 hours (reservation length) + - We will monitor use of the reservation to ensure multiple users get a chance to run + - Any long jobs blocking access for other users will be killed +- High memory nodes are not available +- Jobs are free + +#### Example NERC Capability reservation job submission script + +```slurm +#!/bin/bash +#SBATCH --job-name=NERC_capability_job +#SBATCH --nodes=256 +#SBATCH --ntasks-per-node=8 +#SBATCH --cpus-per-task=16 +#SBATCH --time=1:0:0 +#SBATCH --partition=standard +#SBATCH --reservation=NERCcapability +#SBATCH --qos=reservation +#SBATCH --account=t01 + +export OMP_NUM_THREADS=16 +export OMP_PLACES=cores +export SRUN_CPUS_PER_TASK=$SLURM_CPUS_PER_TASK + +# Check process/thread placement +module load xthi +srun --hint=multithread --distribution=block:block xthi > placement-${SLURM_JOBID}.out + +srun --hint=multithread --distribution=block:block my_app.x +``` + ### Capability Day session The Capability Day session is typically available directly after the pre-Capability Day session. From 5803006a49ff24b8133afb08b643aaf09521a47f Mon Sep 17 00:00:00 2001 From: Andy Turner Date: Mon, 26 Aug 2024 12:14:06 +0100 Subject: [PATCH 2/2] Comment out NERC reservation for now --- docs/user-guide/scheduler.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/scheduler.md b/docs/user-guide/scheduler.md index 79e8faaab..714cc97d2 100644 --- a/docs/user-guide/scheduler.md +++ b/docs/user-guide/scheduler.md @@ -2289,9 +2289,10 @@ Your request will be checked by the ARCHER2 User Administration team and, if app The next Capability Days session will be from Tue 24 Sep 2024 to Thu 26 Sep 2024. - `pre-capabilityday` QoS: 0800-2000, Tue 24 Sep 2024 - - `NERCcapability` reservation: 0800-1600, Tue 24 Sep 2024 - `capabilityday` QoS: 2000 Tue 24 Sep - 1400 Thu 26 Sep 2024 + + ARCHER2 Capability Days are a mechanism to allow users to run large scale (512 node or more) tests on the system free of charge. The motivations behind Capability Days are: @@ -2302,10 +2303,10 @@ on the system free of charge. The motivations behind Capability Days are: To enable this, a period will be made available regularly where users can run jobs at large scale free of charge. -Capability Days are made up of three parts: +Capability Days are made up of different parts: - pre-Capability Day session (`pre-capabilityday` QoS) to allow users to test scaling and job setup ahead of full Capability Day -- NERC Capability reservation (`NERCcapability` reservation) to allow NERC users to test at large scale + - Capability Day session (`capabilityday` QoS) !!! tip @@ -2357,6 +2358,8 @@ srun --hint=multithread --distribution=block:block xthi > placement-${SLURM_JOBI srun --hint=multithread --distribution=block:block my_app.x ``` + ### Capability Day session