Skip to content

Commit

Permalink
ci: use uv (#2424)
Browse files Browse the repository at this point in the history
Trialing uv to close #2416. Pixi is probably preferable, but for now, there is a blocker that I don't see a way around*. It's easy enough to switch once we are ready, and this gives us several CI benefits immediately:

- faster
- locked dependencies

I left the rtd.yml and examples.yml workflows untouched for two reasons:

- the model splitter needs pymetis, which is available for windows from conda-forge but not pypi
- the pypi vtk for windows crashed on one of the notebooks

These can be switched from micromamba to pixi later on, at which point we can consolidate on pixi for everything?

*Pixi has no way to tell uv to prerelease=allow, which is currently necessary because VTK has no valid 9.4+ official distribution for ARM macs on PyPI at the moment. I'd hoped PyPI dependencies could be overridden by conda dependencies, but pixi still tries (and fails) to solve them all for PyPI even if add them to pixi.tool.dependencies.
  • Loading branch information
wpbonelli authored Jan 21, 2025
1 parent f9da244 commit 732a55b
Show file tree
Hide file tree
Showing 8 changed files with 4,733 additions and 132 deletions.
37 changes: 17 additions & 20 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,18 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup Micromamba
uses: mamba-org/setup-micromamba@v2
- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
environment-file: etc/environment.yml
cache-environment: true
cache-downloads: true
create-args: >-
python=${{ matrix.python-version }}
init-shell: >-
bash
powershell
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: pyproject.toml

- name: Install FloPy
run: pip install .
run: uv sync --all-extras

- name: Install Modflow executables
uses: modflowpy/install-modflow-action@v1
Expand All @@ -52,7 +50,7 @@ jobs:
working-directory: autotest
run: |
mkdir -p .benchmarks
pytest -v --durations=0 --benchmark-only --benchmark-json .benchmarks/${{ matrix.os }}_python${{ matrix.python-version }}.json --keep-failed=.failed
uv run pytest -v --durations=0 --benchmark-only --benchmark-json .benchmarks/${{ matrix.os }}_python${{ matrix.python-version }}.json --keep-failed=.failed
ls .benchmarks
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -85,17 +83,16 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.12
cache: 'pip'
cache-dependency-path: pyproject.toml

- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install numpy pandas matplotlib seaborn
run: uv pip install numpy pandas matplotlib seaborn

- name: Download all artifacts
uses: actions/download-artifact@v4
Expand Down Expand Up @@ -132,7 +129,7 @@ jobs:
fi
# process benchmarks
python scripts/process_benchmarks.py $path $path
uv run python scripts/process_benchmarks.py $path $path
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down
22 changes: 10 additions & 12 deletions .github/workflows/codegen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,20 +23,18 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup Micromamba
uses: mamba-org/setup-micromamba@v1
- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
environment-file: etc/environment.yml
cache-environment: true
cache-downloads: true
create-args: >-
python=${{ matrix.python-version }}
init-shell: >-
bash
powershell
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: pyproject.toml

- name: Install FloPy
run: pip install .
run: uv sync

- name: Generate classes
run: python -m flopy.mf6.utils.generate_classes --ref ${{ matrix.ref }} --no-backup
run: uv run python -m flopy.mf6.utils.generate_classes --ref ${{ matrix.ref }} --no-backup
87 changes: 44 additions & 43 deletions .github/workflows/commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,26 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.9
cache: 'pip'
cache-dependency-path: pyproject.toml
python-version-file: pyproject.toml

- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install build twine
pip install .
python -c "import flopy; print(f'{flopy.__version__}')"
uv sync --extra build
uv run python -c "import flopy; print(f'{flopy.__version__}')"
- name: Build package
run: python -m build
run: uv build

- name: Check package
run: twine check --strict dist/*
run: uv run twine check --strict dist/*

lint:
name: Lint
Expand All @@ -52,32 +53,33 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.9
cache: 'pip'
cache-dependency-path: pyproject.toml
python-version-file: pyproject.toml

- name: Install Python dependencies
run: |
pip install .
pip install ".[lint]"
run: uv sync --extra lint

- name: Lint
run: ruff check .
run: uv run ruff check .

- name: Check format
run: ruff format . --check
run: uv run ruff format . --check

- name: Check spelling
run: codespell
run: uv run codespell

- name: Check CITATION.cff
run: |
cffconvert --validate
cffconvert -f apalike
cffconvert -f bibtex
uv run cffconvert --validate
uv run cffconvert -f apalike
uv run cffconvert -f bibtex
smoke:
name: Smoke test
Expand All @@ -93,25 +95,26 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
cache-dependency-path: pyproject.toml
python-version-file: pyproject.toml

- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install .
pip install ".[test, optional]"
# temp. allow prereleases for vtk on aarch64
run: uv sync --all-extras --prerelease=allow

- name: Install Modflow executables
uses: modflowpy/install-modflow-action@v1

- name: Smoke test
working-directory: autotest
run: pytest -v -n=auto --smoke --cov=flopy --cov-report=xml --durations=0 --keep-failed=.failed
run: uv run pytest -v -n=auto --smoke --cov=flopy --cov-report=xml --durations=0 --keep-failed=.failed
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand Down Expand Up @@ -146,20 +149,18 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Setup Micromamba
uses: mamba-org/setup-micromamba@v1
- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
environment-file: etc/environment.yml
cache-environment: true
cache-downloads: true
create-args: >-
python=${{ matrix.python-version }}
init-shell: >-
bash
powershell
python-version-file: pyproject.toml

- name: Install FloPy
run: pip install .
run: uv sync --all-extras

- name: Install Modflow-related executables
uses: modflowpy/install-modflow-action@v1
Expand All @@ -178,18 +179,18 @@ jobs:
subset: triangle

- name: Update package classes
run: python -m flopy.mf6.utils.generate_classes --ref develop --no-backup
run: uv run python -m flopy.mf6.utils.generate_classes --ref develop --no-backup

- name: Run tests
working-directory: autotest
run: |
pytest -v -m="not example" -n=auto --cov=flopy --cov-append --cov-report=xml --durations=0 --keep-failed=.failed --dist loadfile
uv run pytest -v -m="not example" -n=auto --cov=flopy --cov-append --cov-report=xml --durations=0 --keep-failed=.failed --dist loadfile
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Report coverage
working-directory: autotest
run: coverage report
run: uv run coverage report

- name: Upload failed test outputs
uses: actions/upload-artifact@v4
Expand Down
66 changes: 35 additions & 31 deletions .github/workflows/mf6.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,23 @@ jobs:
- name: Checkout flopy repo
uses: actions/checkout@v4

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.9
cache: 'pip'
cache-dependency-path: pyproject.toml
python-version-file: pyproject.toml

- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install https://github.com/modflowpy/pymake/zipball/master
pip install https://github.com/Deltares/xmipy/zipball/develop
pip install https://github.com/MODFLOW-USGS/modflowapi/zipball/develop
pip install .[test,optional]
pip install meson ninja
uv sync --all-extras
uv pip install https://github.com/modflowpy/pymake/archive/master.zip
uv pip install https://github.com/Deltares/xmipy/archive/develop.zip
uv pip install https://github.com/MODFLOW-USGS/modflowapi/archive/develop.zip
uv pip install meson ninja
- name: Setup GNU Fortran
uses: fortran-lang/setup-fortran@v1
Expand All @@ -55,27 +57,27 @@ jobs:
- name: Build and install MF6
working-directory: modflow6
run: |
meson setup builddir --buildtype=debugoptimized --prefix=$(pwd) --libdir=bin
meson install -C builddir
meson test --verbose --no-rebuild -C builddir
uv run meson setup builddir --buildtype=debugoptimized --prefix=$(pwd) --libdir=bin
uv run meson install -C builddir
uv run meson test --verbose --no-rebuild -C builddir
- name: Update package classes
working-directory: modflow6/autotest
run: python update_flopy.py
run: uv run python update_flopy.py

- name: Install executables
working-directory: modflow6/autotest
env:
GITHUB_TOKEN: ${{ github.token }}
run: pytest -v --durations=0 get_exes.py
run: uv run pytest -v --durations=0 get_exes.py

- name: Run tests
working-directory: modflow6/autotest
run: pytest -v --cov=flopy --cov-report=xml --cov-append --durations=0 -n auto -m "not repo and not regression"
run: uv run pytest -v --cov=flopy --cov-report=xml --cov-append --durations=0 -n auto -m "not repo and not regression"

- name: Print coverage report before upload
working-directory: ./modflow6/autotest
run: coverage report
run: uv run coverage report

- name: Upload coverage to Codecov
if:
Expand Down Expand Up @@ -107,22 +109,24 @@ jobs:
repository: MODFLOW-USGS/modflow6-examples
path: modflow6-examples

- name: Setup uv
uses: astral-sh/setup-uv@v5
with:
version: "0.5.18"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.9
cache: 'pip'
cache-dependency-path: pyproject.toml
python-version-file: pyproject.toml

- name: Install Python dependencies
run: |
pip install --upgrade pip
pip install https://github.com/modflowpy/pymake/zipball/master
pip install https://github.com/Deltares/xmipy/zipball/develop
pip install https://github.com/MODFLOW-USGS/modflowapi/zipball/develop
pip install .[test,optional]
pip install meson ninja
pip install -r modflow6-examples/etc/requirements.pip.txt
uv sync --all-extras
uv pip install https://github.com/modflowpy/pymake/archive/master.zip
uv pip install https://github.com/Deltares/xmipy/archive/develop.zip
uv pip install https://github.com/MODFLOW-USGS/modflowapi/archive/develop.zip
uv pip install meson ninja
uv pip install -r modflow6-examples/etc/requirements.pip.txt
- name: Setup GNU Fortran
uses: fortran-lang/setup-fortran@v1
Expand All @@ -136,15 +140,15 @@ jobs:
- name: Build and install MF6
working-directory: modflow6
run: |
meson setup builddir --buildtype=debugoptimized --prefix=$(pwd) --libdir=bin
meson install -C builddir
meson test --verbose --no-rebuild -C builddir
uv run meson setup builddir --buildtype=debugoptimized --prefix=$(pwd) --libdir=bin
uv run meson install -C builddir
uv run meson test --verbose --no-rebuild -C builddir
cp bin/* ~/.local/bin/modflow/
- name: Update package classes
working-directory: modflow6/autotest
run: python update_flopy.py
run: uv run python update_flopy.py

- name: Test MF6 examples
working-directory: modflow6-examples/autotest
run: pytest -v -n=auto --durations=0 test_scripts.py
run: uv run pytest -v -n=auto --durations=0 test_scripts.py
Loading

0 comments on commit 732a55b

Please sign in to comment.