Switch spherical viz to use `uxarray` and `holoviews` #140

xylar · 2023-10-20T12:06:21Z

This merge introduces several new dependencies used for plotting:

antimeridian
geoviews
holoviews
hvplot
spatialpandas
uxarray

The approach used here is loosely based on the tutorials from uxarray provided here:
https://uxarray.readthedocs.io/en/latest/examples/005-to-geodataframe-for-holoviz.html
and here:
https://uxarray.readthedocs.io/en/latest/examples/004-working-with-mpas-grids.html

The MPAS mesh and an associated data array are converted to a SpatialPandas GeoDataFrame for visualization in holoviews using its matplotlib backend. (The alternative bokeh backend is nice for working in
Jupyter notebooks within a web browser but I did not find it useful for exporting images to files.)

At least for the mesh sizes and types of visualization we are currently doing on the sphere in Polaris, this new approach is much faster than generating mapping files, remapping the data to a regular grid, and then visualizing.

The to_geodataframe() method from uxarray currently only works reliably for DataArrays on cells, not on edges or vertices. Because of this, I have dropped plots of the normal velocity on edges from the geostrophic tasks. We can re-introduce this later if desired but it seems like a lot of work for some pretty noisy viz.

Checklist

User's Guide has been updated
Developer's Guide has been updated
API documentation in the Developer's Guide (api.md) has any new or modified class, method and/or functions listed
Documentation has been built locally and changes look as expected
Testing comment in the PR documents testing used to verify the changes

xylar · 2023-10-20T12:10:22Z

@cbegeman and @sbrus89, I could use your eyes on this when you have a chance. I am currently testing all the affected tasks on Chrysalis and can point you to a tarball with all the images once my testing is done.

So far, the approach seems better. It's faster, though not lightning fast. It has a few quirks like when it decides have a flat vs. pointy end on the colorbar. I'm sure we can manually interfere but I didn't want to take the trouble for now.

xylar · 2023-10-20T12:26:01Z

docs/developers_guide/ocean/api.md

+### inertial_gravity_wave 
+
+```{eval-rst}
+.. currentmodule:: polaris.ocean.tasks.inertial_gravity_wave
+


I think we want to keep these headings alphabetized here in the API so they mimic the directory structure in the Polaris package.

xylar · 2023-10-20T12:28:03Z

polaris/ocean/__init__.py


        # spherical: please keep these in alphabetical order
        add_cosine_bell_tasks(component=self)
        add_geostrophic_tasks(component=self)
+        add_sphere_transport_tasks(component=self)


We want to organize these tasks roughly alphabetically first by planar, single_column, spherical and then by specific subdirectory. Since this isn't easy to do with the 4 types of tasks in spherical transport, we'll put it at the end.

xylar · 2023-10-20T17:59:56Z

Testing

I was able to run all with_viz test cases on Chrysalis. The images that result are tarred up in:

/lcrc/group/e3sm/ac.xylar/polaris_0.2/chrysalis/test_20231020/spherical-convergence-uxarray-viz/images.tar.gz

xylar · 2023-10-20T18:06:34Z

I'm seeing some glitches along the antimeridian that I may need to work with the uxarray folks on fixing:

cbegeman

@xylar This looks great to me! I ran successfully on perlmutter with gnu, mpich. I also took a look at your tarred images and compared them with previous figs and they look good to me. Yes, it would be nice to have arrows on the colorbar but I agree that it doesn't need to hold us up.

This approach uses uxarray and holoviews to plot efficiently on the native grid.

Extend colorbar at both limits and make it skinnier.

xylar · 2023-10-24T16:24:19Z

I seem to have fixed the last 2 aesthetic issues with the colorbar:

sbrus89 · 2023-10-26T20:24:12Z

polaris/viz/spherical.py

+def plot_global_lat_lon_field(lon, lat, data_array, out_filename, config,
+                              colormap_section, title=None, plot_land=True,
+                              colorbar_label=None):


@xylar - I think it's good to leave in the existing capability to plot on a remapped lat/lon grid. I'm just wondering if the holowview approach is efficient enough to replace it more generally, i.e. for MPAS-Analysis purposes?

It might be. @cbegeman and I were talking about this. We currently do analysis on a comparison grid and that sometimes makes it easier to compare with observations or with other model runs on a different mesh. But there's not really a reason we couldn't do as much as possible on the native mesh.

Good point about using a comparison grid for in those cases. That makes sense. I was just curious.

sbrus89 · 2023-10-26T20:25:50Z

polaris/viz/spherical.py

@@ -2,17 +2,116 @@

 import cartopy
 import cmocean  # noqa: F401
+import geoviews.feature as gf
+import holoviews as hv
+import hvplot.pandas  # noqa: F401


What is the use for this import?

The short answer is that I'm following this example:
https://uxarray.readthedocs.io/en/latest/examples/005-to-geodataframe-for-holoviz.html
I believe the longer answer is that this import is what makes the later call to hvplot.polygons() on the GeoDataFrame possible.

Was there something you wanted me to do here or were you just curious?

Thanks for the info. I was just curious since it is an unused import.

sbrus89 · 2023-10-26T20:27:57Z

polaris/viz/spherical.py

-def plot_global_field(lon, lat, data_array, out_filename, config,
-                      colormap_section, title=None, plot_land=True,
-                      colorbar_label=None):
+def plot_global_mpas_field(mesh_filename, da, out_filename, config,


Would it be of interest to add an option to add the cell outline to the plot?

I could see that being useful and we do that with the planar plots.

I added this capability. However, it exposed two less-than-ideal issues:

There doesn't seem to be a way to set the color of the edges -- they are blue no matter what color I tell it to use. I didn't find a workaround.

There seems to be some kind of glitch in the handling of the antimeridian, which is much more visible when plotting with edges. I will see if this problem persists in a future release of uxarray and follow up with them if it does.

xylar · 2023-10-27T10:22:55Z

@sbrus89, could you let me know if you are happy with the handling of edge plotting, given the limitations I listed above? Is there anything else you'd like to see here?

sbrus89

@xylar - This is a great new feature to have! I tested all the with_viz cases on Anvil and other than the dateline artifacts you already mentioned, everything worked as expected. Thanks for adding the edge feature. I think it's fine to leave it as-is despite the current drawbacks. It seems like those are things that can/will be addressed in a future uxarray release.

xylar · 2023-10-27T14:32:48Z

@sbrus89 and @cbegeman, thank so much for you reviews! I really appreciate them!

xylar added enhancement New feature or request framework Changes relating to the polaris framework as opposed to individual tests or analysis ocean Related to ocean tests or analysis labels Oct 20, 2023

xylar requested review from sbrus89 and cbegeman October 20, 2023 12:06

xylar self-assigned this Oct 20, 2023

xylar commented Oct 20, 2023

View reviewed changes

xylar force-pushed the switch-spherical-viz-to-uxarray branch 3 times, most recently from 06f9182 to c05eccd Compare October 20, 2023 15:46

cbegeman approved these changes Oct 23, 2023

View reviewed changes

xylar added 8 commits October 24, 2023 13:30

Add uxarray, holoviews and related packages

61c3d1b

Add function for plotting global MPAS fields

66dc16a

This approach uses uxarray and holoviews to plot efficiently on the native grid.

Update cosine bell viz

dfa9f1b

Update geostrophic viz

686ebe9

Update sphere transport viz

e881236

Update the API docs

5eecede

Update developer's guide

aabdbc4

Update user's guide

5c41a86

xylar force-pushed the switch-spherical-viz-to-uxarray branch from c05eccd to 5c41a86 Compare October 24, 2023 11:30

Adjust colorbar

da32a1f

Extend colorbar at both limits and make it skinnier.

xylar force-pushed the switch-spherical-viz-to-uxarray branch from fcdbe82 to da32a1f Compare October 24, 2023 16:15

sbrus89 reviewed Oct 26, 2023

View reviewed changes

Support showing edges

0679279

sbrus89 approved these changes Oct 27, 2023

View reviewed changes

xylar merged commit ea309c8 into E3SM-Project:main Oct 27, 2023

xylar deleted the switch-spherical-viz-to-uxarray branch October 27, 2023 14:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch spherical viz to use `uxarray` and `holoviews` #140

Switch spherical viz to use `uxarray` and `holoviews` #140

xylar commented Oct 20, 2023 •

edited

Loading

xylar commented Oct 20, 2023

xylar Oct 20, 2023

xylar Oct 20, 2023

xylar commented Oct 20, 2023

xylar commented Oct 20, 2023

cbegeman left a comment

xylar commented Oct 24, 2023

sbrus89 Oct 26, 2023

xylar Oct 27, 2023

sbrus89 Oct 27, 2023

sbrus89 Oct 26, 2023

xylar Oct 27, 2023

xylar Oct 27, 2023

sbrus89 Oct 27, 2023 •

edited

Loading

sbrus89 Oct 26, 2023

xylar Oct 27, 2023

xylar Oct 27, 2023

xylar commented Oct 27, 2023

sbrus89 left a comment

xylar commented Oct 27, 2023

Switch spherical viz to use uxarray and holoviews #140

Switch spherical viz to use uxarray and holoviews #140

Conversation

xylar commented Oct 20, 2023 • edited Loading

xylar commented Oct 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xylar commented Oct 20, 2023

Testing

xylar commented Oct 20, 2023

cbegeman left a comment

Choose a reason for hiding this comment

xylar commented Oct 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbrus89 Oct 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xylar commented Oct 27, 2023

sbrus89 left a comment

Choose a reason for hiding this comment

xylar commented Oct 27, 2023

Switch spherical viz to use `uxarray` and `holoviews` #140

Switch spherical viz to use `uxarray` and `holoviews` #140

xylar commented Oct 20, 2023 •

edited

Loading

sbrus89 Oct 27, 2023 •

edited

Loading