Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge dev into main for release 2.0 #806

Merged
merged 46 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
4658813
Add wrapper for fopen64
wangvsa May 16, 2023
12df216
Add new broadcast RPC call
rgmiller Jan 17, 2023
b053fbe
More work on the metaget_all broadcast RPC
rgmiller Mar 10, 2023
9fbb970
Clean up error handling for process_metaget_bcast_rpc()
rgmiller Mar 14, 2023
7bf8955
Error handling and bug fixes for the metaget_all RPC
rgmiller Mar 31, 2023
b72d7fa
Fix memory leak, update comments
rgmiller Apr 13, 2023
eb7b305
Minor changes to clean up some "TODO" comments
rgmiller May 5, 2023
53393be
Move the tree-walk out of process_metaget_bcast_rpc
rgmiller May 11, 2023
b01c1fb
Rename "Exit" label
rgmiller May 11, 2023
d011899
Change metaget_all broadcast RPC to include files owned by the caller
rgmiller May 16, 2023
33dcbf7
Fix error handling in git merge_metaget_all_bcast_outputs()
rgmiller May 19, 2023
5ac5da3
Add UnifyFS Citation sections to docs and readme
CamStan May 18, 2023
33aeb36
Add tests for stat(), lstat() and fstat()
wangvsa May 24, 2023
b8f75bc
Typo fix
wangvsa May 24, 2023
66ed258
Use --overlap for srun and remove mpiexec that was used for debug pur…
wangvsa May 24, 2023
aff6de4
Validate fd in fstat and _fxstat
wangvsa May 24, 2023
442753f
fstat after close should fail with EBADF
wangvsa May 24, 2023
99776d3
Add C and Fortran wrappers for MPI_Init_thread
wangvsa Jun 12, 2023
5f155ee
Add wrappers for stat64 family calls
wangvsa Jun 12, 2023
1650c70
Wrap the unifyfs_file_attr_to_stat marco in a while loop
wangvsa Jun 12, 2023
8163f16
Swap __fxstat64 and __fxstat in configure.ac
wangvsa Jun 12, 2023
589bf3a
Call PMPI_Finalize() before we unmount the client
wangvsa Jun 21, 2023
c4d97d6
avoid missing function prototype from newer C compilers
adammoody Jun 23, 2023
156ea71
make stat wrappers conditional on symbol existence
adammoody Jun 23, 2023
3e6f01d
avoid unused function error when no stat64 variant detected
adammoody Jul 6, 2023
b273f0e
doc: better describe need for ncmpi_sync
adammoody Jun 27, 2023
42d449e
Fix warning about too small buffer in write() call
rgmiller Jul 13, 2023
de63267
fix: remove potential double free
adammoody Jul 21, 2023
c6bccbf
Do not assume file must exist after an open call
wangvsa Jul 28, 2023
0250401
add pending metaget to reduce owner lookups
MichaelBrim Apr 21, 2023
a2e0f48
reduce sync requests to owner server
MichaelBrim Apr 25, 2023
470790b
Handle the case where h5cc -show returns multiple lines
wangvsa Aug 31, 2023
67c8ba9
Fortram mpi wrappers have wrong parameter types
wangvsa Aug 31, 2023
6b8f726
Use newer MD5 API function
rgmiller Aug 8, 2023
c5e7b6f
Add check for EVP_Digest() to configure script
rgmiller Aug 11, 2023
30922b4
Use newer MD5 API functions
rgmiller Sep 1, 2023
7857b7f
Docs: Update to use yaml file
CamStan Sep 14, 2023
9dcb590
Add resource manager support for flux
wangvsa Sep 19, 2023
0cc3a41
Remove ] when multiple nodes are returned
wangvsa Sep 25, 2023
188af17
Use OPENSSL_* variables
rgmiller Oct 6, 2023
25279b9
rework server bootstrap completion
MichaelBrim Oct 2, 2023
10ef8db
Docs: Update deps and enable reproducible builds
CamStan Oct 24, 2023
e0c95dd
Docs: Add UnifyFS videos and tutorial slides
CamStan Nov 7, 2023
ab3cf5e
Docs fix: add tutorial download image
CamStan Nov 20, 2023
738e687
Update Gitlab CI deps and jobs
CamStan Dec 18, 2023
de2c08e
Docs: Update for v2.0 release
CamStan Dec 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ stages:
script:
- cd unifyfs-build/t && $JOB_LAUNCH_COMMAND make check
after_script:
- rm -rf /tmp/unify* /tmp/tmp.* /tmp/mdhim* /tmp/na_sm | true
- rm -rf /tmp/unify* /tmp/tmp.* /tmp/na_sm | true

# Run the integration test suite with the options provided from the specific
# job.
Expand Down Expand Up @@ -156,7 +156,9 @@ before_script:
# Running with lsf tag here, but this may be too vague as the job may attempt to
# use a runner on a system the service user doesn't have permission to access.
# If so, move this job to a specific system.
full_clean:
# TODO: this fails when the rm tries to run on the path this job is running
# from. Rewrite to exclude current runner.
.full_clean:
stage: clean
extends: .base-template
variables:
Expand All @@ -166,12 +168,11 @@ full_clean:
tags:
- shell
before_script: []
script: rm -rf ${HOME}/.jacamar-ci/*
script: rm -rf ${WORKSPACE}/.jacamar-ci/*
needs: []

# System specific jobs
include:
- local: .gitlab/ascent.yml
- local: .gitlab/catalyst.yml
- local: .gitlab/lassen.yml
- local: .gitlab/quartz.yml
175 changes: 0 additions & 175 deletions .gitlab/catalyst.yml

This file was deleted.

73 changes: 73 additions & 0 deletions .gitlab/lassen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -173,3 +173,76 @@ lassen-gcc-8_3_1-integ-test-scheduled-stage:
CI_TEST_OPTIONS: ":: -s stage"
extends: [.lsf-multi-node-template, .lassen-scheduled-batch-template, .lassen-gcc-8_3_1-template, .integ-test-template]
needs: ["lassen-gcc-8_3_1-scheduled-build"]


### gcc@11.2.1 ###
.lassen-gcc-11_2_1-template:
variables:
COMPILER: gcc/11.2.1
CC_COMMAND: "which gcc"
FC_COMMAND: "which gfortran"
SPACK_ENV_NAME: "unifyfs-lsf-gcc11_2_1"

lassen-gcc-11_2_1-install-deps:
extends: [.lassen-shell-template, .lassen-gcc-11_2_1-template, .install-deps-template]

lassen-gcc-11_2_1-build:
extends: [.lassen-shell-template, .lassen-gcc-11_2_1-template, .build-template]
needs: ["lassen-gcc-11_2_1-install-deps"]

lassen-gcc-11_2_1-unit-test:
extends: [.lsf-single-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .unit-test-template]
needs: ["lassen-gcc-11_2_1-build"]

# Integration tests

lassen-gcc-11_2_1-integ-test-writeread-posix:
variables:
CI_TEST_OPTIONS: ":: -s writeread -t posix"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

lassen-gcc-11_2_1-integ-test-writeread-mpiio:
variables:
CI_TEST_OPTIONS: ":: -s writeread -t mpiio"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

lassen-gcc-11_2_1-integ-test-read-posix:
variables:
CI_TEST_OPTIONS: ":: -s read -t posix"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

lassen-gcc-11_2_1-integ-test-read-mpiio:
variables:
CI_TEST_OPTIONS: ":: -s read -t mpiio"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

lassen-gcc-11_2_1-integ-test-pc-all:
variables:
CI_TEST_OPTIONS: ":: -s pc -t all"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

lassen-gcc-11_2_1-integ-test-stage:
variables:
CI_TEST_OPTIONS: ":: -s stage"
extends: [.lsf-multi-node-template, .lassen-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-build"]

# Larger Scheduled Stage Job(s)

lassen-gcc-11_2_1-scheduled-install-deps:
extends: [.lassen-scheduled-shell-template, .lassen-gcc-11_2_1-template, .install-deps-template]

lassen-gcc-11_2_1-scheduled-build:
extends: [.lassen-scheduled-shell-template, .lassen-gcc-11_2_1-template, .build-template]
needs: ["lassen-gcc-11_2_1-scheduled-install-deps"]

lassen-gcc-11_2_1-integ-test-scheduled-stage:
variables:
CI_TEST_OPTIONS: ":: -s stage"
extends: [.lsf-multi-node-template, .lassen-scheduled-batch-template, .lassen-gcc-11_2_1-template, .integ-test-template]
needs: ["lassen-gcc-11_2_1-scheduled-build"]
Loading
Loading