Code

Run test suite based on pr366#70

Code

Open

trz42 wants to merge 69 commits intotrz42:2023.06-software.eessi.iotrz42/software-layer:2023.06-software.eessi.iofrom poksumdo:run_test_suite_based_on_pr366poksumdo/software-layer:run_test_suite_based_on_pr366Copy head branch name to clipboard

+751-22

Commits

Commits on Oct 14, 2023

• first version of bot/test.sh script

  truib

  committed
  6cc2e3e

  Copy full SHA for 6cc2e3e

Commits on Oct 15, 2023

• scripts to run tests after a build job has finished

  truib

  committed
  9a32dc6

  Copy full SHA for 9a32dc6

• small adjustments to test script

  truib

  committed
  17cfd02

  Copy full SHA for 17cfd02

Commits on Feb 5, 2024

• Second attempt at having the bot run the test suite, now based on Thomas' PR 366

  

  Caspar van Leeuwen

  committed
  0f3646e

  Copy full SHA for 0f3646e

• Merge upstream into old feature branch based on pr366

  

  Caspar van Leeuwen

  committed
  e8c91eb

  Copy full SHA for e8c91eb

• Make a very simple change to an easystack file so we are able to trigger the bot build and test procedures and see if this PR actually works

  

  Caspar van Leeuwen

  committed
  34fca15

  Copy full SHA for 34fca15

Commits on Feb 6, 2024

• Make scripts executable

  

  Caspar van Leeuwen

  committed
  3faae9d

  Copy full SHA for 3faae9d

• Check for existence of SLURM output first

  

  Caspar van Leeuwen

  committed
  081e2b1

  Copy full SHA for 081e2b1

• Clarify return messages from the bot

  

  Caspar van Leeuwen

  committed
  9b6fa73

  Copy full SHA for 9b6fa73

• Use EESSI_SOFTWARE_PATH instead of EASYBUILD_INSTALLPATH to set the modulepath. It should be the same, and we no longer configure EasyBuild, since we don't use it when running the test suite

  

  Caspar van Leeuwen

  committed
  cb48b36

  Copy full SHA for cb48b36

• Do git clone in a seperate script, so that that can be run in the prefix layer

  

  Caspar van Leeuwen

  committed
  d1a0219

  Copy full SHA for d1a0219

• Should use the one from current dir

  

  Caspar van Leeuwen

  committed
  86d5d3d

  Copy full SHA for 86d5d3d

• Remove quotes, see if that helps

  

  Caspar van Leeuwen

  committed
  7861aec

  Copy full SHA for 7861aec

• Apparently, we need to remove curly braces too

  

  Caspar van Leeuwen

  committed
  a7b5ee1

  Copy full SHA for a7b5ee1

• Make sure this actually gets reported as failure, as the ReFrame runtime fails to run the test suite (it is not an indication of tests failing themselves).

  

  Caspar van Leeuwen

  committed
  f50e463

  Copy full SHA for f50e463

• Fix the actual issue: make sure that two arguments are interpreted seperately by not quoting them

  

  Caspar van Leeuwen

  committed
  f67df9b

  Copy full SHA for f67df9b

• See if we can make check-test formatting more fancy

  

  Caspar van Leeuwen

  committed
  3394851

  Copy full SHA for 3394851

• Replace the env var for the result file

  

  Caspar van Leeuwen

  committed
  8e5682a

  Copy full SHA for 8e5682a

• Add missing functions

  

  Caspar van Leeuwen

  committed
  6cf0cf9

  Copy full SHA for 6cf0cf9

• Remove artefacts from reporting

  

  Caspar van Leeuwen

  committed
  7fe24e1

  Copy full SHA for 7fe24e1

• Add a reason for the failure

  

  Caspar van Leeuwen

  committed
  afc309a

  Copy full SHA for afc309a

• Fixed missing bracket

  

  Caspar van Leeuwen

  committed
  b84e487

  Copy full SHA for b84e487

• Add reporting of the ReFrame result

  

  Caspar van Leeuwen

  committed
  ef7bc01

  Copy full SHA for ef7bc01

• Add missing bracket. Again

  

  Caspar van Leeuwen

  committed
  867681a

  Copy full SHA for 867681a

• Add some echo for debugging

  

  Caspar van Leeuwen

  committed
  47e30ca

  Copy full SHA for 47e30ca

• more debugging output

  

  Caspar van Leeuwen

  committed
  1bdadf8

  Copy full SHA for 1bdadf8

• Lets try some nicer formatting with headers etc. See if this actually prints something.

  

  Caspar van Leeuwen

  committed
  5152f5e

  Copy full SHA for 5152f5e

• Inverse check, it was incorrect

  

  Caspar van Leeuwen

  committed
  ea38c66

  Copy full SHA for ea38c66

• Remove debugging output

  

  Caspar van Leeuwen

  committed
  8edb6ff

  Copy full SHA for 8edb6ff

• Correct the regex pattern to account for the fact that the slurm output file contains special characters in that pattern that are NOT whitespace

  

  Caspar van Leeuwen

  committed
  26bd2cb

  Copy full SHA for 26bd2cb

• Corrected typo

  

  Caspar van Leeuwen

  committed
  416b72d

  Copy full SHA for 416b72d

• Trigger failure in the workflow on purpose, to show what that looks like

  

  Caspar van Leeuwen

  committed
  1990093

  Copy full SHA for 1990093

• Change order of checking error codes. If the ReFrame tests ran and we find that pattern, that should take priority. Other errors might have come from errors in the build phase...

  

  Caspar van Leeuwen

  committed
  4b98856

  Copy full SHA for 4b98856

Commits on Feb 7, 2024

• Make sure we don't overwrite the grepped result from failures

  

  Caspar van Leeuwen

  committed
  0f26142

  Copy full SHA for 0f26142

• Fix plural/singular typo

  

  Caspar van Leeuwen

  committed
  94e0bb5

  Copy full SHA for 94e0bb5

• Be a bit more expressive about what we are doing

  

  Caspar van Leeuwen

  committed
  9e7ac1c

  Copy full SHA for 9e7ac1c

• Change to dedicated config file for testing by the bot in the build job

  

  Caspar van Leeuwen

  committed
  3a52bde

  Copy full SHA for 3a52bde

• Make it verbose, see if we can see why autodetect fails

  

  Caspar van Leeuwen

  committed
  159e384

  Copy full SHA for 159e384

• Add ReFrame config template. Not that four template variables need to be replaced for this to be a valid ReFrame config file

  

  Caspar van Leeuwen

  committed
  b92044e

  Copy full SHA for b92044e

• Use template config file and make replacements based on lscpu

  

  Caspar van Leeuwen

  committed
  02b0e31

  Copy full SHA for 02b0e31

• Fix incorrect variable name

  

  Caspar van Leeuwen

  committed
  58c4cfa

  Copy full SHA for 58c4cfa

• Missing comma

  

  Caspar van Leeuwen

  committed
  2fc121e

  Copy full SHA for 2fc121e

• Corrected typo

  

  Caspar van Leeuwen

  committed
  db08f4f

  Copy full SHA for db08f4f

• Remove verbose flags

  

  Caspar van Leeuwen

  committed
  6b3a6cc

  Copy full SHA for 6b3a6cc

• Pick name without spaces, to avoid issues with directories

  

  Caspar van Leeuwen

  committed
  a70b4f1

  Copy full SHA for a70b4f1

Commits on Feb 8, 2024

• Pick more meaningful name for SLURM variable, namely SLURM_OUTPUT_FOUND

  

  Caspar van Leeuwen

  committed
  ce24e6d

  Copy full SHA for ce24e6d

• Correct erroneous status

  

  Caspar van Leeuwen

  committed
  9881c2a

  Copy full SHA for 9881c2a

• Rename CoDeList variable to comment_details_list

  

  Caspar van Leeuwen

  committed
  1446055

  Copy full SHA for 1446055

• Remove commented section, it is not relevant

  

  Caspar van Leeuwen

  committed
  f80c0eb

  Copy full SHA for f80c0eb

• Update run_tests.sh

  Show description for ff5baac
  

  casparvl

  and

  boegel

  authored
  ff5baac

  Copy full SHA for ff5baac

• Fix mistake in replacing SLURM with SLURM_OUTPUT_FOUND

  

  Caspar van Leeuwen

  committed
  4adae18

  Copy full SHA for 4adae18

• Merge branch 'run_test_suite_based_on_pr366' of github.com:casparvl/software-layer into run_test_suite_based_on_pr366

  

  Caspar van Leeuwen

  committed
  848e387

  Copy full SHA for 848e387

• Change name of the template for the ReFrame config file to make clear it is a template. Then, alter the sed commands so that they output to a new file: the actual config file

  

  Caspar van Leeuwen

  committed
  9e6955f

  Copy full SHA for 9e6955f

• Fix typo in variable

  

  Caspar van Leeuwen

  committed
  08a7fd8

  Copy full SHA for 08a7fd8

• No longer needed

  

  Caspar van Leeuwen

  committed
  8b5bde5

  Copy full SHA for 8b5bde5

• Added header, rephrased some comments

  

  Caspar van Leeuwen

  committed
  5d383eb

  Copy full SHA for 5d383eb

• Fix headers

  

  Caspar van Leeuwen

  committed
  cdf7971

  Copy full SHA for cdf7971

• Load default ReFrame version

  

  Caspar van Leeuwen

  committed
  79a9ad3

  Copy full SHA for 79a9ad3

• Point to docs for ReFrame config

  

  Caspar van Leeuwen

  committed
  2f280e4

  Copy full SHA for 2f280e4

• Fix silly mistake: I was overwriting the ReFrame config file with every sed call

  

  Caspar van Leeuwen

  committed
  e5d60b9

  Copy full SHA for e5d60b9

• See if @boegel is correct and if we don't actually need ANY software change to have the bot run...

  

  Caspar van Leeuwen

  committed
  b5c60cc

  Copy full SHA for b5c60cc

Commits on Feb 14, 2024

• Add escape character to star, so that the regex pattern prints correctly on GitHub in the report

  

  Caspar van Leeuwen

  committed
  69d6d2e

  Copy full SHA for 69d6d2e

Commits on Feb 15, 2024

• Add the list(SCALES.keys()) feature (though technically we could limit to singlenode already here...). Also, make sure color is not used for printing the ReFrame results, to avoid color-characters …

  Show description for 2dd048e
  

  Caspar van Leeuwen

  committed
  2dd048e

  Copy full SHA for 2dd048e

• Try the original regex patterns, capturing spaces more specifically, now that we got rid of color characters

  

  Caspar van Leeuwen

  committed
  0492b9e

  Copy full SHA for 0492b9e

Commits on Feb 17, 2024

• fix minor bug + add debug output

  truib

  committed
  8746a33

  Copy full SHA for 8746a33

• list loaded modules and available Python modules

  truib

  committed
  2e23bf0

  Copy full SHA for 2e23bf0

• run test suite in prefix env

  truib

  committed
  d56a9cd

  Copy full SHA for d56a9cd

• source init script instead of running in compat layer

  truib

  committed
  359a457

  Copy full SHA for 359a457

• EESSI_CVMFS_REPO already contains '/cvmfs'

  truib

  committed
  642a416

  Copy full SHA for 642a416