-
Notifications
You must be signed in to change notification settings - Fork 145
Conference call notes 20241218
Kenneth Hoste edited this page Dec 18, 2024
·
1 revision
(back to Conference calls)
Notes on the 261th EasyBuild conference call, Wednesday 18 Dec 2024 (16:00 UTC / 17:00 CET)
List of attendees (15):
- Simon Branford (Univ. of Birmingham, UK)
- Em Dragowsky (Case Western Reserve University, Ohio, US)
- Thomas Hoffman (EMBL, Germany)
- Jasper Grimm (University of York, UK)
- Alexander Grund (ZIH, Dresden, Germany)
- Kenneth Hoste (HPC-UGent, Belgium)
- Adam Huffman (Big Data Institute, Oxford, UK)
- Georgios Kafanas (University of Luxembourg)
- Bart Oldeman (McGill University, Digital Research Alliance of Canada)
- Jurij Pečar (EMBL, Germany)
- Andrea Pisercia (E4 consultancy company, Italy)
- Jan Reuter (JSC, Germany)
- Ãke Sandgren (Umeå University, Sweden)
- Jörg Saßmannshausen (Imperial College London, UK)
- Roberto Scipioni (Red Oak Consulting, UK)
- overview of recent developments
- outlook to EasyBuild 5.0 release
- 2024b update of common toolchains
- Q&A
- latest EasyBuild release: v4.9.4 (22 Sept 2024)
- next EasyBuild release: 5.0.0beta1 🔥
- any time now, just need to complete sync of
develop
into5.0.x
... - see (soon) also https://docs.easybuild.io/easybuild-v5/release-candidates/#beta-release-500beta1-18-dec-2024
-
feedback is welcome via mailing list,
#eb5
channel in Slack, conf calls, dedicated issue
- any time now, just need to complete sync of
- next (stable) EasyBuild release:
- very likely EasyBuild v5.0.0...
- additional EasyBuild 4.9.x versions could still be done via
4.9.x
branches, but none are planned currently - as it stands currently,
--install-latest-eb-release
would install5.0.0beta1
when easyconfigs PR #22049 gets merged
-
EasyBuild v5.0.0
- project board: https://github.com/orgs/easybuilders/projects/18/views/2
- detailed notes on latest developments in https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
- to test development version of EasyBuild 5.0:
# set up Python virtual environment, and jump into it python3 -m venv eb5 source eb5/bin/activate # install EasyBuild 5.0 development version into it pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz # go! eb --version
-
docs (merged PRs)
- ...
-
framework (merged PRs)
-
bug fixes
-
[5.0.x]
tweaksetup.py
to fall back to importingsetup
fromsetuptools
ifdistutils
is not available (PR #4478 ) -
[5.0.x]
improve portability of reproducible tarballs by replacing external tar command withtarfile
module from Python standard library (PR #4660) -
[develop]
fix dry-run output when usingmulti_deps
(PR #4678) -
[5.0.x]
make sure "type module
" is run during a dry run (fixes use ofeb -x
) (PR #4721)
-
-
enhancements
-
[5.0.x]
add--search-path-cpp-headers
configuration option to control how EasyBuild sets paths to headers at build time (PR #4645) -
[develop]
Addresolve_template
method toEasyConfig
class (PR #4677) -
[develop]
Allow templates incustom_paths
&custom_commands
sanity-check arguments (PR #4679) -
[5.0.x]
add--keep-debug-symbols
configuration option to set default value of 'debug
' toolchain option, and enable it by default so-g
is included in$CXXFLAGS
& co (PR #4688) -
[5.0.x]
add--search-path-linker
option to control linker options at build time (PR #4697) -
[5.0.x]
enhanceget_software_libdir
to return full paths if requested (PR #4699)
-
-
changes
-
[5.0.x]
don't allow unresolved templates in easyconfig parameters by default + add support for--allow-unresolved-templates
configuration option (PR #4516) -
[5.0.x]
Rename 'source
' step to 'extract
' (affectsskipsteps
easyconfig parameter +--stop
option) (PR #4629) -
[5.0.x]
Refactormake_extension_string
(PR #4690) -
[5.0.x]
changeToolchain.get_flag
so it doesn't automatically prepend a dash (-
) to compiler flags, add deprecation warning foroptarch
value without leading dash, renameCompiler.COMPILER*_FLAGS
toCompiler.COMPILER*_OPTIONS
(PR #4698) -
[5.0.x]
Renamepost_install_step
topost_processing_step
+ deprecate use ofpost_install_step
(PR #4715)- see also easyblocks PR #3525
-
[5.0.x]
run sanity checks commands from an empty temporary directory (rather than the software install directory) (PR #4723)
-
-
code cleanup
- ...
-
bug fixes
-
easyblocks (merged PRs)
- bug fixes
-
enhancements
-
[5.0.x]
enhanceConfigureMake
easyblock to error out on unknown configure options (PR #3025)
-
-
updates
-
[develop]
pass netCDF-Fortran path via$NETCDFF_DIR
in WPS easyblock (PR #3522)
-
- changes
-
new easyblocks
- ...
-
code cleanup
-
[5.0.x]
remove unused custom easyblock for TAU + clean up custom easyblock for Paraver, only support Paravar >= v4.7 (PR #3535)
-
-
easyconfigs (merged PRs)
- ~XXX easyconfig PRs were merged since last conf call
-
bug fixes/reports
- ...
-
enhancements
- ...
-
(noteworthy) new software
- ...
-
noteworthy software updates
- ...
-
cleanup
- ...
-
changes
- ...
-
docs (open PRs + issues)
- ...
-
framework (open PRs + issues)
- bug fixes
-
enhancements
- Problem using
$CPATH
in modulefiles overwriting system paths (issue #3331)-
[5.0.x]
refactor generation of required environment variables in module files + deprecatemake_module_req_guess
method inEasyBlock
class #4653 (PR #4653)- see also (initial) companion PR for easyblocks: PR #3513
-
[5.0.x]
addmodule-search-path-headers
configuration option to control how modules set search paths to header files (PR #4655)
-
- initial work towards integrating
easy_update
functionality (PR #4714)
- Problem using
-
code cleanup
-
[develop]
useenumerate
where applicable fixing_generate_multi_deps_list
(PR #4720)
-
-
changes
-
[5.0.x]
Deprecate use ofparallel
easyconfig parameter and fix updating the template value (PR #4580)
-
-
easyblocks (open PRs + issues)
- bug fixes
- enhancements
-
[develop -> 5.0.x]
enhance LLVM easyblock for compilation of clang/flang + other llvm-projects (PR #3373) -
[develop]
enhance genericBundle
easyblock to transfer module requirements of components, but do not create logfile in components (PR #3509) -
[develop]
enable hwloc and HeFFTe support in GROMACS easyblock (PR #3531)
-
-
updates
- ...
- changes
-
code cleanup
-
[5.0.x]
Remove code supporting obsolete versions fromIntelBase
family of easyblocks + remove unused custom easyblock for IPP (PR #3533)
-
-
new
- ...
-
easyconfigs (open PRs + issues)
-
over 1,000 open easyconfig PRs...
- we're due for a cleanup of backlog
- closing PRs that are only using unsupported/deprecated toolchains, see https://docs.easybuild.io/policies/toolchains/#current-situation
-
bug fixes/reports
- ...
-
enhancements
- ...
-
(noteworthy) new software
- ...
- software updates
-
[develop]
PyTorch v2.3.0 (PR #20489)- blocked by changes in how results of PyTorch test suite are reported...
- custom easyblock for PyTorch will need to be updated, Flamefire has started looking into it
-
[develop]
AlphaFold 3 (PR #21864)- Jörg is involved in meeting soon with DeepMind on their parameter DB
- only ~1GB of model files, so feasible to let users download themselves
- public database of data (couple of TBs) they also need can be installed centrally
- Jurij is interested in contacts related to this
- Thomas is actively looking into it
- jax builds (see open PR)
- Triton works (requires specific commit of LLVM...)
- dm-haiku (requires tensorstore), also ready (see open PR)
- non-working easyconfig for AlphaFold 3 available (missing deps)
- Jörg is involved in meeting soon with DeepMind on their parameter DB
-
-
changes
- ...
-
over 1,000 open easyconfig PRs...
- GCC 14.2 as a base (see easyconfigs PR #21114)
- we should define candidate toolchains using latest version of all components
- to discuss: skip
2024b
, jump straight from2024a
to2025a
to "catch up" on release cycle- EasyBuild v5.0 is ideal excuse for that break in continuity...
- Mikael is looked into Perl (see merged PR)
- Python 3.13.1 seems likely candidate for this toolchain...
- (Roberto) OS dependencies
- checked as RPM
- what about software installed in another way?
- could consider using external modules, see https://docs.easybuild.io/using-external-modules
- could also just kick out that dependency (via
--ignore-osdeps
) if the necessary libraries are available some other way
- (Roberto) support for a feature like
--continue
?- not trivial to implement since you need to checkpoint the state after a particular step
- attempts were made in the past, see also https://github.com/easybuilders/easybuild-framework/pull/2385
- Merry Christmas!