Run test suite based on pr366 #70

trz42 · 2024-02-17T00:59:09Z

Derived from #69

Made so we can update the contents without modifying the source of #69

…mas' PR 366

…ger the bot build and test procedures and see if this PR actually works

…odulepath. It should be the same, and we no longer configure EasyBuild, since we don't use it when running the test suite

…fix layer

…ime fails to run the test suite (it is not an indication of tests failing themselves).

…perately by not quoting them

… prints something.

…ut file contains special characters in that pattern that are NOT whitespace

…ry sed call

…change to have the bot run...

…tly on GitHub in the report

…t to singlenode already here...). Also, make sure color is not used for printing the ReFrame results, to avoid color-characters making it into the slurm output file

…now that we got rid of color characters

eessi-bot-devel-trz42 · 2024-02-17T00:59:13Z

Instance dev-PR254 is configured to build:

arch x86_64/amd/zen2 for repo nessi-2022.11-swl-deb10
arch x86_64/amd/zen2 for repo nessi-2023.06-cl
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb10
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb11
arch aarch64/generic for repo nessi-2022.11-swl-deb10
arch aarch64/generic for repo nessi-2023.06-cl
arch aarch64/generic for repo nessi-2023.06-swl-deb10
arch aarch64/generic for repo nessi-2023.06-swl-deb11

trz42 · 2024-02-17T01:01:20Z

bot: build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic

eessi-bot-devel-trz42 · 2024-02-17T01:01:24Z

Updates by the bot instance dev-PR254 (click for details)

received bot command build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
handling command build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 131815, for details & status see Run test suite based on pr366 #70 (comment)

eessi-bot-devel-trz42 · 2024-02-17T01:01:28Z

New job on instance dev-PR254 for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/bot-devel/test_sync_feb24/jobs/2024.02/pr_70/131815

Might fail because it uses Python from the container? (/usr/bin/python3)

date	job status	comment
Feb 17 01:01:27 AM UTC 2024	submitted	job id `131815` awaits release by job manager
Feb 17 01:01:52 AM UTC 2024	released	job awaits launch by Slurm scheduler
Feb 17 01:02:54 AM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-131815.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1708131727.tar.gz` size: 0 MiB (230062 bytes) entries: 3 modules under 2023.06/software/linux/aarch64/generic/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/generic/software no software packages in tarball other under 2023.06/software/linux/aarch64/generic `.lmod/cache/spiderT.lua` `.lmod/cache/spiderT.luac_5.1` `.lmod/cache/timestamp`
Feb 17 01:02:54 AM UTC 2024	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-131815.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-02-17T01:10:59Z

bot: build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic

eessi-bot-devel-trz42 · 2024-02-17T01:11:02Z

Updates by the bot instance dev-PR254 (click for details)

received bot command build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
handling command build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 131816, for details & status see Run test suite based on pr366 #70 (comment)

eessi-bot-devel-trz42 · 2024-02-17T01:11:07Z

New job on instance dev-PR254 for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/bot-devel/test_sync_feb24/jobs/2024.02/pr_70/131816

date	job status	comment
Feb 17 01:11:06 AM UTC 2024	submitted	job id `131816` awaits release by job manager
Feb 17 01:11:59 AM UTC 2024	released	job awaits launch by Slurm scheduler
Feb 17 01:13:01 AM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-131816.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1708132333.tar.gz` size: 0 MiB (230063 bytes) entries: 3 modules under 2023.06/software/linux/aarch64/generic/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/generic/software no software packages in tarball other under 2023.06/software/linux/aarch64/generic `.lmod/cache/spiderT.lua` `.lmod/cache/spiderT.luac_5.1` `.lmod/cache/timestamp`
Feb 17 01:13:01 AM UTC 2024	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-131816.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-02-17T01:17:12Z

Run test suite in prefix env

bot: build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic

eessi-bot-devel-trz42 · 2024-02-17T01:17:15Z

Updates by the bot instance dev-PR254 (click for details)

received bot command build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
handling command build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 131817, for details & status see Run test suite based on pr366 #70 (comment)

eessi-bot-devel-trz42 · 2024-02-17T01:17:19Z

New job on instance dev-PR254 for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/bot-devel/test_sync_feb24/jobs/2024.02/pr_70/131817

date	job status	comment
Feb 17 01:17:18 AM UTC 2024	submitted	job id `131817` awaits release by job manager
Feb 17 01:18:05 AM UTC 2024	released	job awaits launch by Slurm scheduler
Feb 17 01:19:07 AM UTC 2024	running	job `131817` is running
Feb 17 01:29:22 AM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-131817.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1708132698.tar.gz` size: 0 MiB (230067 bytes) entries: 3 modules under 2023.06/software/linux/aarch64/generic/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/generic/software no software packages in tarball other under 2023.06/software/linux/aarch64/generic `.lmod/cache/spiderT.lua` `.lmod/cache/spiderT.luac_5.1` `.lmod/cache/timestamp`
Feb 17 01:29:22 AM UTC 2024	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite produced failures. ReFrame Summary [ FAILED ] Ran 5/5 test case(s) from 5 check(s) (2 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-131817.out` ❌ found message matching `ERROR:` ❌ found message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-02-17T20:53:01Z

Source /cvmfs/${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/bash script instead of running test suite in prefix env

bot: build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic

eessi-bot-devel-trz42 · 2024-02-17T20:53:04Z

Updates by the bot instance dev-PR254 (click for details)

received bot command build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
handling command build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 131889, for details & status see Run test suite based on pr366 #70 (comment)

eessi-bot-devel-trz42 · 2024-02-17T20:53:09Z

New job on instance dev-PR254 for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/bot-devel/test_sync_feb24/jobs/2024.02/pr_70/131889

date	job status	comment
Feb 17 08:53:08 PM UTC 2024	submitted	job id `131889` awaits release by job manager
Feb 17 08:53:56 PM UTC 2024	released	job awaits launch by Slurm scheduler
Feb 17 08:54:58 PM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-131889.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1708203254.tar.gz` size: 0 MiB (230059 bytes) entries: 3 modules under 2023.06/software/linux/aarch64/generic/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/generic/software no software packages in tarball other under 2023.06/software/linux/aarch64/generic `.lmod/cache/spiderT.lua` `.lmod/cache/spiderT.luac_5.1` `.lmod/cache/timestamp`
Feb 17 08:54:58 PM UTC 2024	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-131889.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

trz42 · 2024-02-17T21:13:34Z

Source ${EESSI_CVMFS_REPO}/versions/${EESSI_VERSION}/init/bash script instead of running test suite in prefix env

bot: build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic

eessi-bot-devel-trz42 · 2024-02-17T21:13:37Z

Updates by the bot instance dev-PR254 (click for details)

received bot command build inst:PR254 repo:nessi-2023.06-swl-deb10 arch:aarch64/generic from trz42
- expanded format: build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic
handling command build instance:PR254 repository:nessi-2023.06-swl-deb10 architecture:aarch64/generic resulted in:
- submitted job 131890, for details & status see Run test suite based on pr366 #70 (comment)

eessi-bot-devel-trz42 · 2024-02-17T21:13:41Z

New job on instance dev-PR254 for architecture aarch64-generic for repository nessi-2023.06-swl-deb10 in job dir /home/thomarob/bot-devel/test_sync_feb24/jobs/2024.02/pr_70/131890

date	job status	comment
Feb 17 09:13:40 PM UTC 2024	submitted	job id `131890` awaits release by job manager
Feb 17 09:14:03 PM UTC 2024	released	job awaits launch by Slurm scheduler
Feb 17 09:15:06 PM UTC 2024	running	job `131890` is running
Feb 17 09:25:24 PM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-131890.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1708204458.tar.gz` size: 0 MiB (230069 bytes) entries: 3 modules under 2023.06/software/linux/aarch64/generic/modules/all no module files in tarball software under 2023.06/software/linux/aarch64/generic/software no software packages in tarball other under 2023.06/software/linux/aarch64/generic `.lmod/cache/spiderT.lua` `.lmod/cache/spiderT.luac_5.1` `.lmod/cache/timestamp`
Feb 17 09:25:24 PM UTC 2024	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite produced failures. ReFrame Summary [ FAILED ] Ran 7/7 test case(s) from 7 check(s) (4 failure(s), 0 skipped, 0 aborted) Details ✅ job output file `slurm-131890.out` ❌ found message matching `ERROR:` ❌ found message matching `[\sFAILED\s].Ran . test case`

truib and others added 30 commits October 14, 2023 21:09

first version of bot/test.sh script

6cc2e3e

scripts to run tests after a build job has finished

9a32dc6

small adjustments to test script

17cfd02

Second attempt at having the bot run the test suite, now based on Tho…

0f3646e

…mas' PR 366

Merge upstream into old feature branch based on pr366

e8c91eb

Make a very simple change to an easystack file so we are able to trig…

34fca15

…ger the bot build and test procedures and see if this PR actually works

Make scripts executable

3faae9d

Check for existence of SLURM output first

081e2b1

Clarify return messages from the bot

9b6fa73

Use EESSI_SOFTWARE_PATH instead of EASYBUILD_INSTALLPATH to set the m…

cb48b36

…odulepath. It should be the same, and we no longer configure EasyBuild, since we don't use it when running the test suite

Do git clone in a seperate script, so that that can be run in the pre…

d1a0219

…fix layer

Should use the one from current dir

86d5d3d

Remove quotes, see if that helps

7861aec

Apparently, we need to remove curly braces too

a7b5ee1

Make sure this actually gets reported as failure, as the ReFrame runt…

f50e463

…ime fails to run the test suite (it is not an indication of tests failing themselves).

Fix the actual issue: make sure that two arguments are interpreted se…

f67df9b

…perately by not quoting them

See if we can make check-test formatting more fancy

3394851

Replace the env var for the result file

8e5682a

Add missing functions

6cf0cf9

Remove artefacts from reporting

7fe24e1

Add a reason for the failure

afc309a

Fixed missing bracket

b84e487

Add reporting of the ReFrame result

ef7bc01

Add missing bracket. Again

867681a

Add some echo for debugging

47e30ca

more debugging output

1bdadf8

Lets try some nicer formatting with headers etc. See if this actually…

5152f5e

… prints something.

Inverse check, it was incorrect

ea38c66

Remove debugging output

8edb6ff

Correct the regex pattern to account for the fact that the slurm outp…

26bd2cb

…ut file contains special characters in that pattern that are NOT whitespace

Caspar van Leeuwen and others added 8 commits February 8, 2024 19:16

Load default ReFrame version

79a9ad3

Point to docs for ReFrame config

2f280e4

Fix silly mistake: I was overwriting the ReFrame config file with eve…

e5d60b9

…ry sed call

See if @boegel is correct and if we don't actually need ANY software …

b5c60cc

…change to have the bot run...

Add escape character to star, so that the regex pattern prints correc…

69d6d2e

…tly on GitHub in the report

Add the list(SCALES.keys()) feature (though technically we could limi…

2dd048e

…t to singlenode already here...). Also, make sure color is not used for printing the ReFrame results, to avoid color-characters making it into the slurm output file

Try the original regex patterns, capturing spaces more specifically, …

0492b9e

…now that we got rid of color characters

fix minor bug + add debug output

8746a33

trz42 added the development label Feb 17, 2024

list loaded modules and available Python modules

2e23bf0

run test suite in prefix env

d56a9cd

source init script instead of running in compat layer

359a457

EESSI_CVMFS_REPO already contains '/cvmfs'

642a416

trz42 mentioned this pull request Feb 23, 2024

make upload directory in S3 bucket configurable EESSI/eessi-bot-software-layer#254

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run test suite based on pr366 #70

Run test suite based on pr366 #70

trz42 commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited by trz42

Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

Run test suite based on pr366 #70

Are you sure you want to change the base?

Run test suite based on pr366 #70

Conversation

trz42 commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited by trz42 Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

trz42 commented Feb 17, 2024

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 • edited Loading

trz42 commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited by trz42

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading

eessi-bot-devel-trz42 bot commented Feb 17, 2024 •

edited

Loading