Issue with 'dateutil' and running run_metplus.py

[typhonmike]

Greetings,

I am new to METplus and running through the tutorial here: https://dtcenter.org/metplus-practical-session-guide-version-4-0/session-1-metplus-setupgrid-grid/metplus-setup/metplus-user-configuration-settings. Everything has been going fine until I run the command:

run_metplus.py

And run into the issue:

ModuleNotFoundError: No module named 'dateutil'

The catch here is that NCO installed this METplus build for the Weather Prediction Center, which includes python. In other words, I do not have permissions to install dateutil for the python3 version currently used by METplus. I do have a local python3 version with dateutil.

My question is, is there anyway to point a previously installed METplus to my python3 version without building METplus from scratch locally? I suspect this is not the case, but I wanted to ask first.

Thanks,

Mike Erickson

[georgemccabe]

Hi Mike,

Which machine are you using here? The dateutil Python package is a requirement of the METplus wrappers (See https://metplus.readthedocs.io/en/latest/Users_Guide/installation.html#python-package-requirements) so the package should have be obtained for a successful install of the wrappers. Also, if the MET executables were built with Python embedding enabled, then the version of Python used to install should include the dateutil package. If this is not the case, then they may need to re-install MET. It may also be the case that the version of python3 that is in your path is not the same version used to install.

The METplus wrappers can easily be run using different versions of Python. Calling the "run_metplus.py" script directly will use the version of Python3 that is found in the user's path, so adding the bin directory that contains the python3 executable that you need to use to the beginning of the path will ensure that this version is used in this case. Here is an example bash command to prepend a directory to the path:

export PATH=/usr/local/python3/bin:$PATH

This assumes that the python3 executable to use is found in the /usr/local/python3/bin directory. You can check which executable is found first in the path by running:

which python3

You can also pass the run_metplus.py command as an argument to the version of Python3 that is needed. However, you will likely need to specify the full path to the run_metplus.py script:

/usr/local/python3/bin/python3 /path/to/METplus/ush/run_metplus.py

Let me know if that helps or if you have any other questions.

[typhonmike]

Hi George,

Thank you for your quick reply. I am working on a WPC machine. The METv10/METplusv4 install was done by NCO with assistance from your team on our netapp. It is mounted to several WPC machines.

I did some modifications to the path variable (note we use csh rather than bash) and things seem to work properly now. However, when I get to here in the tutorial, https://dtcenter.org/metplus-practical-session-guide-version-4-0/session-1-metplus-setupgrid-grid/metplus-setup/metplus-user-configuration-settings, I need to directly modify paths in ${METPLUS_BUILD_BASE}/parm/metplus_config/defaults.conf. I am only able to modify local files, but do not have permissions to modify files on the netapp. Is there anyway to still only use the one build of METplus (where we don't have edit permissions) without installing multiple METplus versions locally?

Thank you!

Mike

[georgemccabe]

Hi Mike,

It is common that users do not have permissions to modify the METplus default configurations in shared installations. You can create a user configuration file to override the settings in the default file. See this section in the METplus User's Guide for more information: https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#user-configuration-file

In the tutorial, we provide a file called tutorial.conf that will be used as the user configuration file. If the MET_INSTALL_DIR variable is not set to a valid value in the defaults.conf file, you will need to add it to your tutorial.conf file. See this section of the METplus User's Guide for information on how to set this value: https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#met-install-dir

[typhonmike]

Thank you George for your reply and I apologize for the delay.

I have not had any further issues in the tutorial, except for the python embedding portion of the tutorial last week. When I reach this portion of the tutorial, https://dtcenter.org/metplus-practical-session-guide-version-4-0/session-1-metplus-setupgrid-grid/met-tool-plot-data-plane/python-embedding, I get an error when running:

plot_data_plane
PYTHON_NUMPY
${METPLUS_TUTORIAL_DIR}/output/met_output/plot_data_plane/python_fcst.ps
'name = "${METPLUS_DATA}/met_test/scripts/python/read_ascii_numpy.py ${METPLUS_DATA}/met_test/data/python/fcst.txt Forecast";'

My entire error is below, but the issue appears to be that the wrong version of python is called (e.g not the one I specified at MET_PYTHON_EXE, but an older version on our WPC netapp). I would like python to be called from /export/hpc-lw-dtbdev5/merickson/anaconda3/bin/python rather than /opt/MET/external/python37/lib/python3.7/site-packages/. Perhaps the MET_PYTHON_EXE variables does not control this?

As always, thank you for your help!

Mike

Input File: '/export/hpc-lw-dtbdev5/merickson/code/python/METplus-4.0.0_Tutorial/data/met_test/data/python/fcst.txt'
Data Name : 'Forecast'
Data Shape: (129, 185)
Data Type: dtype('float64')
Attributes: {'valid': '20050807_120000', 'init': '20050807_000000', 'lead': '120000', 'accum': '120000', 'name': 'Forecast', 'long_name': 'Forecast_word', 'level': 'Surface', 'units': 'None', 'grid': {'type': 'Lambert Conformal', 'hemisphere': 'N', 'name': 'FooGrid', 'scale_lat_1': 25.0, 'scale_lat_2': 25.0, 'lat_pin': 12.19, 'lon_pin': -135.459, 'x_pin': 0.0, 'y_pin': 0.0, 'lon_orient': -95.0, 'd_km': 40.635, 'r_km': 6371.2, 'nx': 185, 'ny': 129}}
merickson@hpc-lw-dtbdev5> plot_data_plane
? PYTHON_NUMPY
? ${METPLUS_TUTORIAL_DIR}/output/met_output/plot_data_plane/python_fcst.ps
? 'name = "${METPLUS_DATA}/met_test/scripts/python/read_ascii_numpy.py ${METPLUS_DATA}/met_test/data/python/fcst.txt Forecast";'
DEBUG 1: Opening data file: PYTHON_NUMPY
Python Script: '/opt/MET/METPlus4_0/share/met/wrappers/write_tmp_dataplane.py'
User Command: '/export/hpc-lw-dtbdev5/merickson/code/python/METplus-4.0.0_Tutorial/data/met_test/scripts/python/read_ascii_numpy /export/hpc-lw-dtbdev5/merickson/code/python/METplus-4.0.0_Tutorial/data/met_test/data/python/fcst.txt Forecast'
Temporary File: '/tmp/tmp_met_nc_15209_0'
Input File: '/export/hpc-lw-dtbdev5/merickson/code/python/METplus-4.0.0_Tutorial/data/met_test/data/python/fcst.txt'
Data Name : 'Forecast'
Data Shape: (129, 185)
Data Type: dtype('float64')
Attributes: {'valid': '20050807_120000', 'init': '20050807_000000', 'lead': '120000', 'accum': '120000', 'name': 'Forecast', 'long_name': 'Forecast_word', 'level': 'Surface', 'units': 'None', 'grid': {'type': 'Lambert Conformal', 'hemisphere': 'N', 'name': 'FooGrid', 'scale_lat_1': 25.0, 'scale_lat_2': 25.0, 'lat_pin': 12.19, 'lon_pin': -135.459, 'x_pin': 0.0, 'y_pin': 0.0, 'lon_orient': -95.0, 'd_km': 40.635, 'r_km': 6371.2, 'nx': 185, 'ny': 129}}
Traceback (most recent call last):
File "/opt/MET/METPlus4_0/share/met/wrappers/read_tmp_dataplane.py", line 10, in
import numpy as np
File "/opt/MET/external/python37/lib/python3.7/site-packages/numpy/init.py", line 142, in
from . import core
File "/opt/MET/external/python37/lib/python3.7/site-packages/numpy/core/init.py", line 102, in
from . import _dtype_ctypes
File "/opt/MET/external/python37/lib/python3.7/site-packages/numpy/core/_dtype_ctypes.py", line 25, in
import _ctypes
ImportError: libffi.so.5: cannot open shared object file: No such file or directory
WARNING:
WARNING: tmp_nc_dataplane() -> an error occurred importing module "read_tmp_dataplane"
WARNING:
ERROR :
ERROR : plot_data_plane -> trouble getting field "name = "${METPLUS_DATA}/met_test/scripts/python/read_ascii_numpy.py ${METPLUS_DATA}/met_test/data/python/fcst.txt Forecast";" from file "PYTHON_NUMPY"
ERROR :

[georgemccabe]

Hi Mike,

If you set MET_PYTHON_EXE, then that version of Python is used to call the Python Embedding script to read the data into a temporary NetCDF file. Then the data is read into the MET tools using the version of Python that was used to compile the MET executables. It looks like the data was read properly using the version of Python you specified with MET_PYTHON_EXE, but there is an issue with the version of Python that was used to install MET. If you are able to, I would suggest re-compiling MET using the other version of Python that should have numpy properly installed. Information on how to do that can be found here: https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#compiling-python-support

[typhonmike]

Hi George,

Thank you for your response. Unfortunately we are unable to recompile MET since that is controlled by NCO. A recompile may take months. In your opinion, would the tutorial still be worthwhile without the Python Embedding utility?

Thanks,

Mike

[georgemccabe]

Hi Mike,

Yes, definitely. I don't think Python Embedding is used anywhere else in the tutorial instructions. Also, Python Embedding will still work fine if MET_PYTHON_EXE is unset, which should be used if the Python Embedding script does not include any additional Python package dependencies.

[typhonmike]

Hi George,

I have one follow up question. A coworker of mine was able to get the command to run just fine. We work out of our own local directories (e.g. METPLUS_DATA and MET_PYTHON_EXE point to our own local machines) but we use the same METplus/MET installation at /opt/MET. Please see below for his properly working command/output. I am a bit confused how his can work and mine does not given we use the same MET/METplus build? Please let me know if you have any questions.

Thanks again,

Mike

plot_data_plane
PYTHON_NUMPY
${METPLUS_TUTORIAL_DIR}/output/met_output/plot_data_plane/python_fcst.ps
'name = "${METPLUS_DATA}/met_test/scripts/python/read_ascii_numpy.py ${METPLUS_DATA}/met_test/data/python/fcst.txt Forecast";'

I get this as output:

DEBUG 1: Opening data file: PYTHON_NUMPY
Python Script: '/opt/MET/METPlus4_0/share/met/wrappers/write_tmp_dataplane.py'
User Command: '/export/wpc-lw-dtbdev8/mgreen/METplus-4.0.0_Tutorial/data/met_test/scripts/python/read_ascii_numpy /export/wpc-lw-dtbdev8/mgreen/METplus-4.0.0_Tutorial/data/met_test/data/python/fcst.txt Forecast'
Temporary File: '/tmp/tmp_met_nc_7347_0'
Input File: '/export/wpc-lw-dtbdev8/mgreen/METplus-4.0.0_Tutorial/data/met_test/data/python/fcst.txt'
Data Name : 'Forecast'
Data Shape: (129, 185)
Data Type: dtype('float64')
Attributes: {'valid': '20050807_120000', 'init': '20050807_000000', 'lead': '120000', 'accum': '120000', 'name': 'Forecast', 'long_name': 'Forecast_word', 'level': 'Surface', 'units': 'None', 'grid': {'type': 'Lambert Conformal', 'hemisphere': 'N', 'name': 'FooGrid', 'scale_lat_1': 25.0, 'scale_lat_2': 25.0, 'lat_pin': 12.19, 'lon_pin': -135.459, 'x_pin': 0.0, 'y_pin': 0.0, 'lon_orient': -95.0, 'd_km': 40.635, 'r_km': 6371.2, 'nx': 185, 'ny': 129}}
DEBUG 1: Creating postscript file: /export/wpc-lw-dtbdev8/mgreen/METplus-4.0.0_Tutorial/output/met_output/plot_data_plane/python_fcst.ps

[georgemccabe]

Hi Mike, Do either of you have a value set for the environment variable LD_LIBRARY_PATH?

[georgemccabe]

The error you received is a dynamic library load error. It appears that your colleagues environment is set up to properly find the dynamic library. I'm not sure how you set up your environments on this machine, but using module commands to set up a user's environment will often set environment variables like LD_LIBRARY_PATH to ensure dynamic libraries are found. I would suggest comparing what is loaded in each of your environments to see what you are missing.

[typhonmike]

Thanks George,

I have matched our paths for LD_LIBRARY_PATH, LIBS, and PATH in our .cshrc files and sourced them. Unfortunatetely, I still get the same error. May be a more complicated environmental variable issue on my machine.

Thanks,

Mike

[georgemccabe]

Are you both running the same OS? You can check by running this command:

cat /etc/os-release

If your versions differ, then that may be the cause of the difference in behavior.

[typhonmike]

Thanks George. Strange my coworker's works given he utilizes the same METplus software. We do have the same OS (Red Hat Enterprise Linux Version 7.9).

I appreciate you trying to help me out with this. Sounds like we may have hit a wall. I may submit a ticket to NCO but it may be better to wait until there is a newer version of METplus for them to install.

Mike

[georgemccabe]

Hi Mike,

@jprestop was able to log into a WPC machine and get past the libpython3.7m error by adding /opt/MET/external/python37/lib to the LD_LIBRARY_PATH. However, she did not get the same numpy error that you are seeing.

You could try to compare the value of LD_LIBRARY_PATH between you and your coworker's environment by running echo $LD_LIBRARY_PATH to double check that they are identical. There may be something in your cshrc file that is overwriting the value and ultimately giving you a different result. I suspect that if you were to both unset LD_LIBRARY_PATH in a terminal window and run the MET commands, then you would get the same result, but it is hard to say.

The good news is you can continue with the METplus tutorial without this functionality working properly. We unfortunately have limited support available for the rest of this month, but if you are interested then we could schedule a screen sharing session with you sometime in January to take a closer look. Let me know if you are interested and we can set something up.

[typhonmike]

Thank you George and Julie! I've done the following:

setenv LD_LIBRARY_PATH /opt/MET/external/python37/lib
source METplus-4.0.0_TutorialSetup.sh (returns /opt/MET/external/python37/lib)
plot_data_plane PYTHON_NUMPY ${METPLUS_TUTORIAL_DIR}/output/met_output/plot_data_plane/python_fcst.ps 'name = "${METPLUS_DATA}/met_test/scripts/python/read_ascii_numpy.py ${METPLUS_DATA}/met_test/data/python/fcst.txt Forecast";'

and got the same error. I think setting up screenshare in January would be great, thank you! I can email you directly if you would like.

Thanks,

Mike

[JohnHalleyGotway]

Hi @typhonmike, I'm just following up here to close the loop on this issue. To recap, during our screen sharing session on 1/20/22, we found the issue in your environment. Your machine is RHEL7 but you're running a version of MET that was compiled on RHEL6. This causes a runtime linker error for an undefined reference to libffi.so.5.

We found a temporary hack that does work but is very unsatisfying. You created a soft link named libffi.so.5 that simply points to the libffi.so.6 shared object file that does exist on your machine. After adding the directory that contains that libffi.so.5 link to your $LD_LIBRARY_PATH, the runtime linker error went away. There's no guarantee that this approach would work for any other linker problems. Long term, NCO will compile a newer version of MET for RHEL7, and that is preferred, permanent solution.

Big picture, the lesson is that its always advised to run MET on the same OS on which it was installed. And side-note, runtime linker errors are often solved by updating the contents of the $LD_LIBRARY_PATH environment variable. And cheating (by substituting in a newer library version) sometimes works, but will not always work.