Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some diagnostic plots + enhance phrasing in Demo notebook #183

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Add some diagnostic plots + enhance phrasing in Demo notebook #183

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cb60926
adds some diagnostic plots in demo
navidcy Aug 23, 2024
7909ff8
import matplotlib
navidcy Aug 23, 2024
2bc3dca
fix typo
navidcy Aug 23, 2024
b3e35a3
use first person to make it read more inclusive
navidcy Aug 23, 2024
2098081
drop matplotlib
navidcy Aug 24, 2024
94852b3
Update demos/reanalysis-forced.ipynb
navidcy Oct 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 104 additions & 33 deletions demos/reanalysis-forced.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,18 @@
"\n",
"Setting up a regional model in MOM6 can be a pain. The goal of this package is that users should spend their debugging time fixing a model that's running and doing weird things, rather than puzzling over a model that won't even start.\n",
"\n",
"In running this notebook, you'll hopefully have a running MOM6 regional model. There will still be a lot of fiddling to do with the `MOM_input` file to make sure that the parameters are set up right for your domain, and you might want to manually edit some of the input files. *But*, this package should help you bypass most of the woes of regridding, encoding and understanding the arcane arts of the MOM6 boundary segment files. "
"After running this notebook, we should have a running MOM6 regional model. There might still be a lot of fiddling needed related to the `MOM_input` file to ensure that the parameters are set up right for our domain, and we might also want to manually edit some of the input files. *But*, this package should help us bypass most of the woes of regridding, encoding, and understanding the arcane arts of the MOM6 boundary segment files. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What does this notebook do?\n",
"This notebook is designed to set you up with a working MOM6 regional configuration. First, try to get it running with our default Tasmania case, then you can clone the notebook and modify for your region of interest. \n",
"\n",
"This notebook is designed to set us up with a working MOM6 regional configuration.\n",
"\n",
"We urge users to first ensure that notebook runs on their machine with our default Tasmania case. Then users can start modifying the notebook to tailor it for the region of their interest. \n",
"\n",
"Input Type | Source | Subsets required\n",
"---|---|---\n",
Expand Down Expand Up @@ -78,9 +81,9 @@
"source": [
"## Step 1: Choose our domain, define workspace paths\n",
"\n",
"To make sure that things are working I'd recommend starting with the default example defined below. If this runs ok, then change to a domain of your choice and hopefully it runs ok too! If not, check the [README](https://github.com/COSIMA/regional-mom6/blob/main/README.md) and [documentation](https://regional-mom6.readthedocs.io/) for troubleshooting tips.\n",
"We recommend first ensuring that the default configuration works. Afterwards, users should feel free to change to a domain of their choice! If the default configuration does not work on the machine of your choice, check out the [README](https://github.com/COSIMA/regional-mom6/blob/main/README.md) and [documentation](https://regional-mom6.readthedocs.io/) for troubleshooting tips.\n",
"\n",
"You can log in and use [this GUI](https://data.marine.copernicus.eu/product/GLOBAL_MULTIYEAR_PHY_001_030/download) to find the lat/lon of your domain and copy paste below."
"To find the longitude-latitude coordinates of the region of interest we can use [the Copernicus GUI](https://data.marine.copernicus.eu/product/GLOBAL_MULTIYEAR_PHY_001_030/download) and then copy-paste the coordinates below."
]
},
{
Expand All @@ -96,16 +99,16 @@
"\n",
"date_range = [\"2003-01-01 00:00:00\", \"2003-01-05 00:00:00\"]\n",
"\n",
"## Place where all your input files go \n",
"## Place where all our input files go \n",
"input_dir = Path(f\"mom6_input_directories/{expt_name}/\")\n",
"\n",
"## Directory where you'll run the experiment from\n",
"## Directory where we'll run the experiment from\n",
"run_dir = Path(f\"mom6_run_directories/{expt_name}/\")\n",
"\n",
"## Directory where compiled FRE tools are located (needed for construction of mask tables)\n",
"toolpath_dir = Path(\"PATH_TO_FRE_TOOLS\")\n",
"\n",
"## Path to where your raw ocean forcing files are stored\n",
"## Path to where our raw ocean forcing files are stored\n",
"glorys_path = Path(\"PATH_TO_GLORYS_DATA\" )\n",
"\n",
"## if directories don't exist, create them\n",
Expand Down Expand Up @@ -147,21 +150,32 @@
"source": [
"We can now access the horizontal and vertical grid of the regional configuration via `expt.hgrid` and `expt.vgrid` respectively.\n",
"\n",
"Plotting the vertical grid with `marker = '.'` lets you see the spacing. You can use `numpy.diff` to compute the vertical spacings, e.g.,\n",
"```python\n",
"import numpy as np\n",
"np.diff(expt.vgrid.zl).plot(marker = '.')\n",
"```\n",
"shows you the vertical spacing profile.\n",
"\n",
"Plotting the vertical grid with `marker = '.'` lets us see the spacing."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"expt.vgrid.zl.plot(marker = '.', y='zl', yincrease=False, figsize=(4, 8))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Modular workflow!\n",
"\n",
"After constructing your `expt` object, if you don't like the default `hgrid` and `vgrid` you can simply modify and then save them back into the `expt` object. However, you'll then also need to save them to disk again. For example:\n",
"After constructing our `expt` object, if we don't like default horizontal and vertical grids (`hgrid` and `vgrid`) we can modify and then save them back into the `expt` object. However, we will also need to save them to disk again. For example:\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"After constructing our `expt` object, if we don't like default horizontal and vertical grids (`hgrid` and `vgrid`) we can modify and then save them back into the `expt` object. However, we will also need to save them to disk again. For example:\n",
"After constructing our `expt` object, if we don't like the default horizontal and vertical grids (`hgrid` and `vgrid`) we can modify and then save them back into the `expt` object. However, we will also need to save them to disk again. For example:\n",

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's easiest to just include matplotlib in the package. No, it's not needed by the package, but it's the first thing people will install when they make a new env and realise it's missing

Copy link
Contributor Author

@navidcy navidcy Oct 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should have matplotlib as a dependency.

But I was googling this a bit and saw that there is a way to have "optional dependencies"? See the extras_require kwarg:

https://stackoverflow.com/questions/41268863/what-is-the-difference-between-extras-require-and-install-requires-in-se/45043494#45043494

That might be a nice way? @angus-g have you seen this before? You familiar with how it works?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from chatGPT:

If you want to use optional dependencies with setuptools but without a setup.py file, you can still achieve this by using the pyproject.toml configuration. This is in line with the modern PEP 517 and PEP 518 standards, which define the use of pyproject.toml to manage build tools like setuptools.

Here’s how you can set it up using setuptools in pyproject.toml:

Step-by-Step Setup Using pyproject.toml with setuptools

  1. Create or Update pyproject.toml:

    You need to configure setuptools as your build backend in the pyproject.toml file.

    Here's an example:

    [build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "yourpackage"
version = "0.1.0"
description = "A package with optional dependencies"
authors = [
    { name = "Your Name", email = "you@example.com" }
]
dependencies = [
    # Add essential dependencies here, e.g.
    "requests",
]

[project.optional-dependencies]
visualization = ["matplotlib"]
    • [build-system] specifies that you're using setuptools and wheel for your package's build system.
    • [project] defines the core metadata of your project (name, version, etc.).
    • dependencies lists the required dependencies for your package.
    • [project.optional-dependencies] is where you define optional features like visualization with its corresponding dependencies.

  2. Instruct Users:

    After setting up pyproject.toml, users can install the package with optional dependencies as follows:

    pip install yourpackage[visualization]

    This command will install the core package plus the optional matplotlib dependency under the visualization group.

  3. Handle Conditional Imports:

    In your code, you can use try-except to conditionally import the optional dependency:

    try:
    import matplotlib.pyplot as plt
except ImportError:
    raise ImportError("Matplotlib is required for visualization. Please install it with `pip install yourpackage[visualization]`.")

Benefits of Using pyproject.toml with setuptools:

  • No setup.py needed: pyproject.toml serves as the modern alternative to setup.py and provides a clean way to configure optional dependencies.
  • Standardized build configuration: This method complies with PEP 517 and PEP 518, ensuring compatibility with modern Python tooling.
  • Optional dependencies in a clean format: You can define multiple optional groups (e.g., visualization, testing, docs), and users can choose which extras to install.

Example for Multiple Optional Dependencies

If you want to define multiple optional features (e.g., visualization, docs, dev), you can do it like this:

[project.optional-dependencies]
visualization = ["matplotlib"]
docs = ["sphinx", "myst-parser"]
dev = ["pytest", "black"]

Now, users can install specific extras by running:

pip install yourpackage[visualization]
# or
pip install yourpackage[dev,docs]

This setup with pyproject.toml using setuptools provides flexibility and avoids the need for setup.py while still giving you full control over your optional dependencies. Let me know if you need help with any specific part!

Copy link
Collaborator

@angus-g angus-g Oct 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should have matplotlib as a dependency.

Agreed.

That might be a nice way? @angus-g have you seen this before? You familiar with how it works?

I have used this a few times before, but I'm not sure how it interacts with conda... I don't think it's unreasonable that somebody would already have matplotlib installed if they wanted to plot stuff anyway? Plotting isn't really the job of this package?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm.... probably need to have different environment yml files and mix and match... oh well..

"\n",
"```python\n",
"new_hgrid = xr.open_dataset(input_dir + \"/hgrid.nc\")\n",
"```\n",
"Modify `new_hgrid`, ensuring that _all metadata_ is retained to keep MOM6 happy. Then, save your changes\n",
"\n",
"Modify `new_hgrid`, ensuring that _all metadata_ is retained to keep MOM6 happy. Then, save our changes:\n",
"\n",
"\n",
"```python\n",
"expt.hgrid = new_hgrid\n",
Expand All @@ -176,9 +190,9 @@
"source": [
"## Step 3: Prepare ocean forcing data\n",
"\n",
"We need to cut out our ocean forcing. The package expects an initial condition and one time-dependent segment per non-land boundary. Naming convention is `\"east_unprocessed\"` for segments and `\"ic_unprocessed\"` for the initial condition.\n",
"We need to cut out our ocean forcing. The package expects an initial condition and one time-dependent segment per non-land boundary. Naming convention is `\"east_unprocessed\"`, ... for the boundary segments and `\"ic_unprocessed\"` for the initial condition.\n",
"\n",
"In this notebook, we are forcing with the Copernicus Marine \"Glorys\" reanalysis dataset. There's a function in the `mom6-regional` package that generates a bash script to download the correct boundary forcing files for your experiment. First, you will need to create an account with Copernicus, and you'll be prompted for your username and password when you try to run the bash script.\n",
"In this notebook, we are forcing with the Copernicus Marine \"Glorys\" reanalysis dataset. There's a function in the package that generates a bash script to download the correct boundary forcing files for our experiment. First, we need to create an account with Copernicus; we will be prompted for our username and password when we try to run the bash script.\n",
"\n",
"The function is called `get_glorys_rectangular` because the fully automated setup is only supported for domains with boundaries parallel to lines of longitude and latitude. To download more complex domain shapes you can call `rmom6.get_glorys_data` directly."
]
Expand All @@ -201,9 +215,9 @@
"source": [
"## Step 4: Set up bathymetry\n",
"\n",
"Similarly to ocean forcing, we point the experiment's `setup_bathymetry` method at the location of the file of choice and also provide the variable names. We don't need to preprocess the bathymetry since it is simply a two-dimensional field and is easier to deal with. Afterwards you can inspect `expt.bathymetry` to have a look at the regional domain.\n",
"Similarly to ocean forcing, we point the experiment's `setup_bathymetry` method at the location of the file of choice and also provide the variable names. We don't need to preprocess the bathymetry since it is simply a two-dimensional field and is easier to deal with. Afterwards, we can inspect `expt.bathymetry` to have a look at the regional domain.\n",
"\n",
"After running this cell, your input directory will contain other bathymetry-related things like the ocean mosaic and mask table too. The mask table defaults to a 10x10 layout and can be modified later."
"After running this cell, our input directory contains other bathymetry-related things like the ocean mosaic and mask table too. The mask table defaults to a 10x10 layout and can be modified later."
]
},
{
Expand All @@ -225,7 +239,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Check out your domain:"
"### Check out our domain:"
]
},
{
Expand Down Expand Up @@ -269,11 +283,11 @@
"source": [
"## Step 5: Handle the ocean forcing - where the magic happens\n",
"\n",
"This cuts out and interpolates the initial condition as well as all boundaries (unless you don't pass it boundaries).\n",
"This cuts out and interpolates the initial condition as well as all boundaries (unless we don't pass it boundaries).\n",
"\n",
"The dictionary maps the MOM6 variable names to what they're called in your ocean input file. Notice how for GLORYS, the horizontal dimensions are `latitude` and `longitude`, vs `xh`, `yh`, `xq`, `yq` for MOM6. This is because for an 'A' grid type tracers share the grid with velocities so there's no difference.\n",
"The dictionary maps the MOM6 variable names to what they're called in our ocean input file. Notice how for GLORYS, the horizontal dimensions are `latitude` and `longitude` versus `xh`, `yh` or `xq`, `yq` for MOM6. This is because for an Arakawa type A grid, the tracers share the grid with velocities so there's no difference.\n",
"\n",
"If one of your segments is land, you can delete its string from the 'boundaries' list. You'll need to update MOM_input to reflect this though so it knows how many segments to look for, and their orientations."
"If one of our segments is land, we can simply delete its string from the `boundaries` keywarg argument list. We would need to update the `MOM_input` file to reflect this though so that MOM6 knows how many segments to look for and their corresponding orientations."
]
},
{
Expand Down Expand Up @@ -309,13 +323,68 @@
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can plot our the interpolated initial condition. It’s a good idea to check and ensure things look reasonable, especially near the region’s boundaries."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The initial condition for the free surface:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"expt.ic_eta.plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"the $u$ velocity at the surface:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"expt.ic_vels.u.sel(zl=0, method='nearest').plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and the $v$ velocity at the surface:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"expt.ic_vels.v.sel(zl=0, method='nearest').plot()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6: Run the FRE tools\n",
"\n",
"This is just a wrapper for the FRE tools needed to make the mosaics and masks for the experiment. The only thing you need to tell it is the processor layout. In this case we're saying that we want a 10 by 10 grid of 100 processors. "
"This is just a wrapper for the FRE tools needed to make the mosaics and masks for the experiment. The only thing we need to provide is the processor layout. In this case we're saying that we want a 10 by 10 grid of 100 processors. "
]
},
{
Expand Down Expand Up @@ -373,9 +442,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 8: Modify the default input directory to make a (hopefully) runnable configuration out of the box\n",
"## Step 8: Modify the default input directory to make an out-of-the-box runnable configuration\n",
"\n",
"This step copies the default directory and modifies the `MOM_layout` files to match your experiment by inserting the right number of x, y points and CPU layout.\n",
"This step copies the default directory and modifies the `MOM_layout` files to match our experiment by inserting the right number of x, y points and CPU layout.\n",
"\n",
"To run MOM6 using the [payu infrastructure](https://github.com/payu-org/payu), provide the keyword argument `using_payu = True` to the `setup_run_directory` method and an example `config.yaml` file will be appear in the run directory. The `config.yaml` file needs to be modified manually to add the locations of executables, etc."
]
Expand All @@ -395,21 +464,23 @@
"source": [
"## Step 9: Run and Troubleshoot!\n",
"\n",
"To run the regional configuration first navigate to your run directory in terminal and use your favourite tool to run the experiment on your system. \n",
"To run the regional configuration first navigate to our run directory in terminal and use our favourite tool to run the experiment on our machine of choice.\n",
"\n",
"Ideally, MOM6 runs. If not, the first thing you should try is reducing the timestep. You can do this by adding `#override DT=XXXX` to your `MOM_override` file. \n",
"Ideally, MOM6 runs. If not, the first thing we should try is reducing the timestep. We can do this by adding `#override DT=XXXX` to our `MOM_override` file. \n",
"\n",
"If there's strange behaviour on your boundaries, you could play around with the `nudging timescale` (an example is already included in the `MOM_override` file). Sometimes, if your boundary has a lot going on (like all of the eddies spinning off the western boundary currents or off the Antarctic Circumpolar current), it can be hard to avoid these edge effects. This is because the chaotic, submesoscale structures developed within the regional domain won't match the flow at the boundary. \n",
"If there's strange behaviour on our boundaries, we could play around with the `nudging timescale` (an example is already included in the `MOM_override` file). Sometimes, if our boundary has a lot going on (like all of the eddies spinning off the western boundary currents or off the Antarctic Circumpolar current), it can be hard to avoid these edge effects. This is because the chaotic, submesoscale structures developed within the regional domain won't match the flow at the boundary. \n",
"\n",
"Another thing that can go wrong is little bays that create non-advective cells at your boundaries. Keep an eye out for tiny bays where one side is taken up by a boundary segment. You can either fill them in manually, or move your boundary slightly to avoid them"
"Another thing that can go wrong is little bays that create non-advective cells at your boundaries. Keep an eye out for tiny bays where one side is taken up by a boundary segment. We can either fill them in manually, or move our boundary slightly to avoid them."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
"source": [
"client.close()"
]
}
],
"metadata": {
Expand All @@ -428,7 +499,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
"version": "3.10.6"
}
},
"nbformat": 4,
Expand Down
Loading