Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI docs build and python dependency management #529

Draft
wants to merge 4 commits into
base: develop
Choose a base branch
from
Draft

Improve CI docs build and python dependency management #529

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a3e265d
python deprecation to setup_requires
vpipkt Dec 9, 2020
8b4e10d
Revert "python deprecation to setup_requires"
vpipkt Dec 9, 2020
ff8a8e4
explicit deployment of python project in sbt makeSite command, via py…
vpipkt Dec 10, 2020
f4cbd38
update readme for python setup
vpipkt Dec 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions pyrasterframes/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,17 @@ As a tip, know that `sbt` is much faster if run in "interactive" mode, where you
and subsequent commands are invoked via an interactive shell. But for context clarity, we'll prefix each command
example below with `sbt`.

## Basic Python environment

Recommend the following steps to set up a conda environment with a few requirements.

```bash
$ conda create -n $ENV_NAME python==3.7
$ conda activate $ENV_NAME
($ENV_NAME) $ conda install -c conda-forge --yes --file pyrasterframes/src/main/python/requirements-condaforge.txt
```

Run the various `sbt` commands below with this environment activated.

## Running Tests

Expand All @@ -98,8 +109,8 @@ sbt pyrasterframes/test # alias 'pyTest'
After running `sbt pyrasterframes/package`, you can run tests more directly in the `src/main/python` directory like this:

```bash
python -m unittest tests/RasterFunctionsTests.py
python -m unittest tests/RasterFunctionsTests.py -k test_rf_agg_overview_raster
($ENV_NAME) $ python setup.py test # all tests
($ENV_NAME) $ python setup.py test --addopts "-k rf_agg_overview_raster" # tests matching pattern
```

## Running Python Markdown Sources
Expand Down
4 changes: 3 additions & 1 deletion pyrasterframes/build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ Python / doc := (Python / doc / target).toTask.dependsOn(
Def.sequential(
assembly,
Test / compile,
pySetup.toTask(" pweave --quick True")
pySetup.toTask(" develop"),
pySetup.toTask(" pweave --quick True"),
pySetup.toTask(" develop --uninstall") // put environment back how it was
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it would be good to have something like this:

    pySetup.toTask(" develop"),
    Try(pySetup.toTask(" pweave")),
    pySetup.toTask(" develop --uninstall")

So that the environment is put back how it was (more or less?) after the pweave task runs even if it fails.

That's beyond my sbt fu though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility is a much larger refactor:

  1. set up tox in pyrasterframes
    1. separate environments for unit tests and docs builds
  2. sbt task runs tox -e $TOX_ENVIRONMENT instead of python setup.py $SETUP_COMMAND

This would afford a chance to also carve out many of the setup_requires and declare specific requirements files for the disctinct tox environments.

There is possibly some downside I am not thinking of (other than the work of refactoring), but it occurred to me that this might be a good direction

)
).value

Expand Down