list broken modules #690

MartinsNadia · 2024-09-02T07:24:38Z

Add script to list broken modules using --module-only

eessi-bot · 2024-09-02T07:24:42Z

Instance eessi-bot-mc-aws is configured to build for:

architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
repositories: eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-software, eessi.io-2023.06-compat

eessi-bot · 2024-09-02T07:24:43Z

Instance eessi-bot-mc-azure is configured to build for:

architectures: x86_64/amd/zen4
repositories: eessi.io-2023.06-compat, eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-software

ocaisa · 2024-09-11T07:03:48Z

@MartinsNadia This has two things inside, the missing modules script and the licencing PR, these need to be separated out.

ocaisa · 2024-09-11T07:06:36Z

modules/module_check.sh

+declare -A checked_modules
+
+# Locate all .eb files within the base dir
+easyconfig_files=$(find $base_dir -name "*.eb")


This is quite a hammer depending on how this is used. It will run on the entire stack, which can take a long time as sanity checks are also run when using --module-only (but not if coupled with --force I think)

It would be better if we can selectively do this for easyconfigs that are introduced by a PR.

One way to do this is to gather that information using --missing based on the easy stack files touched by a PR.

As for implementation, to me this would have to be done as part of the build script:

Gather the easyconfigs to be installed using --missing on the easystack file touched by the PR (which is already known in the build script)

Do the builds as currently implemented

Go through the missing easyconfig list with --module-only and the other necessary options, compare the two modules and make sure that they are identical

Specifically, I mean targeting the loop

software-layer/EESSI-install-software.sh

Lines 276 to 295 in 3944b3b

if [ -f ${easystack_file} ]; then

echo_green "Feeding easystack file ${easystack_file} to EasyBuild..."

${EB} --easystack ${TOPDIR}/${easystack_file} --robot

ec=$?

# copy EasyBuild log file if EasyBuild exited with an error

if [ ${ec} -ne 0 ]; then

eb_last_log=$(unset EB_VERBOSE; eb --last-log)

# copy to current working directory

cp -a ${eb_last_log} .

echo "Last EasyBuild log file copied from ${eb_last_log} to ${PWD}"

# copy to build logs dir (with context added)

copy_build_log "${eb_last_log}" "${build_logs_dir}"

fi

$TOPDIR/check_missing_installations.sh ${TOPDIR}/${easystack_file} ${TOPDIR}/${pr_diff}

else

fatal_error "Easystack file ${easystack_file} not found!"

fi

So add a step before 279 to gather the easyconfig files that will be built, then add an else clause to L283 (i.e., when the build succeeds) which rebuilds the module and checks the module contents against the original

This won't catch the existing software that fail --module-only but it will stop other being added in the future. You're existing script could (almost) be used to catch existing failures (but it is currently not checking the contents for the module to make sure they match).

ocaisa · 2024-09-12T07:28:49Z

modules/module_check.sh

+            echo "Testing module: $module_name"
+
+            # Try loading the module 
+            if module --ignore_cache load $module_name 2>/dev/null; then


This is checking if the module can be loaded, but we probably want more than that, we want to check that the resulting environment (apart from Lmod variables) is the same as when the original module was loaded.

You can do this easily enough on a module by module basis, just by changing the MODULEPATH:

module purge; module unuse $module_install_dir to reset for the next module

Load the original module, 'module --ignore_cache load $module_name'

store the environment using env, filtering out Lmod related content

module use $module_install_dir

reload the module (making sure it comes from where you expect)

Store the new environment (also filtering)

Make sure they are identical

ocaisa · 2024-09-12T07:38:01Z

modules/module_check.sh

+    for module_category in $(ls $module_install_dir/all); do
+        for module_version in $(ls $module_install_dir/all/$module_category); do
+            module_name="$module_category/$module_version"


Suggested change

for module_category in $(ls $module_install_dir/all); do

for module_version in $(ls $module_install_dir/all/$module_category); do

module_name="$module_category/$module_version"

for module_software in $(ls $module_install_dir/all); do

for module_version in $(ls $module_install_dir/all/$module_software); do

module_name="$module_software/$module_version"

ocaisa · 2024-11-21T11:43:51Z

licenses/check_licenses.yml

@@ -0,0 +1,57 @@
+name: Check and update licenses


This is still mixing up two PRs, the one for licenses and the one for checking --module-only

ocaisa · 2024-11-21T11:45:59Z

modules/module_check.sh

@@ -0,0 +1,203 @@
+#!/bin/bash


Does this work? Have you got sample output?

adding PR handling, build and comparison of modules and envs

update licenses

062b624

ocaisa requested changes Sep 11, 2024

View reviewed changes

ocaisa reviewed Sep 12, 2024

View reviewed changes

ocaisa requested changes Nov 21, 2024

View reviewed changes

deleting licenses

519a654

adding PR handling, build and comparison of modules and envs

MartinsNadia force-pushed the module-only branch from beec303 to 519a654 Compare December 3, 2024 09:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

list broken modules #690

list broken modules #690

MartinsNadia commented Sep 2, 2024

eessi-bot bot commented Sep 2, 2024

eessi-bot bot commented Sep 2, 2024

ocaisa commented Sep 11, 2024

ocaisa Sep 11, 2024 •

edited

Loading

ocaisa Sep 11, 2024

ocaisa Sep 11, 2024 •

edited

Loading

ocaisa Sep 11, 2024

ocaisa Sep 11, 2024

ocaisa Sep 12, 2024

ocaisa Sep 12, 2024 •

edited

Loading

ocaisa Sep 12, 2024

ocaisa Nov 21, 2024

ocaisa Nov 21, 2024

	if [ -f ${easystack_file} ]; then
	echo_green "Feeding easystack file ${easystack_file} to EasyBuild..."

	${EB} --easystack ${TOPDIR}/${easystack_file} --robot
	ec=$?

	# copy EasyBuild log file if EasyBuild exited with an error
	if [ ${ec} -ne 0 ]; then
	eb_last_log=$(unset EB_VERBOSE; eb --last-log)
	# copy to current working directory
	cp -a ${eb_last_log} .
	echo "Last EasyBuild log file copied from ${eb_last_log} to ${PWD}"
	# copy to build logs dir (with context added)
	copy_build_log "${eb_last_log}" "${build_logs_dir}"
	fi

	$TOPDIR/check_missing_installations.sh ${TOPDIR}/${easystack_file} ${TOPDIR}/${pr_diff}
	else
	fatal_error "Easystack file ${easystack_file} not found!"
	fi

list broken modules #690

Are you sure you want to change the base?

list broken modules #690

Conversation

MartinsNadia commented Sep 2, 2024

eessi-bot bot commented Sep 2, 2024

eessi-bot bot commented Sep 2, 2024

ocaisa commented Sep 11, 2024

ocaisa Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

ocaisa Sep 11, 2024

Choose a reason for hiding this comment

ocaisa Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

ocaisa Sep 11, 2024

Choose a reason for hiding this comment

ocaisa Sep 11, 2024

Choose a reason for hiding this comment

ocaisa Sep 12, 2024

Choose a reason for hiding this comment

ocaisa Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

ocaisa Sep 12, 2024

Choose a reason for hiding this comment

ocaisa Nov 21, 2024

Choose a reason for hiding this comment

ocaisa Nov 21, 2024

Choose a reason for hiding this comment

ocaisa Sep 11, 2024 •

edited

Loading

ocaisa Sep 11, 2024 •

edited

Loading

ocaisa Sep 12, 2024 •

edited

Loading