key string CURRENT_OBS_LEVEL not passed for use

[hertneky]

I tried to put together a use-case that runs GridStat, then a number of instances of MODE for different variables that use different thresholds. MODE uses the difference field from GridStat to run.

When running MODE by itself in METplus, I was able to use the key string CURRENT_OBS/FCST_LEVEL, for my MODE output prefix. It doesn't seem to work in the use case (i.e. it is not listed in the keys that get passed to do_string_sub for MODE).

I am guessing it is because levels are set differently for GridStat and the MODE instances in the use-case. Is there another way to do this? Currently my output files at each level for a given variable get overridden due to level not being available.

My parm is on cheyenne: /glade/work/hertneky/airforce/validation/parm/GridStat_MODE_Validation.conf

[JohnHalleyGotway]

@hertneky I definitely see the behavior you expect listed in the documentation for CURRENT_FCST_LEVEL and CURRENT_OBS_LEVEL.

And I see your use of CURRENT_OBS_LEVEL in your config file:

MODE_OUTPUT_PREFIX = {MODEL}_vs_{OBTYPE}_TMP_{CURRENT_OBS_LEVEL}

Comparing that this mode.conf example:

MODE_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_vs_{OBTYPE}_{CURRENT_OBS_NAME}_{CURRENT_OBS_LEVEL}

Your usage looks to be totally consistent and in line with that usage.

I checked the (hopefully correct) log file and do see this error:

cat /glade/scratch/hertneky/airforce/validation/metprd/logs/metplus.log.20220225122104

TypeError: The key CURRENT_LEVEL was not passed to do_string_sub  for template: GALWEM-DF_vs_GALWEM-DA_TMP_{CURRENT_LEVEL}: {'now': datetime.datetime(2022, 2, 25, 12, 21, 4), 'today': '20220225', 'instance': 'tmp', 'lead': 0, 'offset': 0, 'init': datetime.datetime(2019, 4, 18, 0, 0), 'valid': datetime.datetime(2019, 4, 18, 0, 0), 'loop_by': 'init', 'da_init': datetime.datetime(2019, 4, 18, 0, 0), 'da_init_fmt': '20190418000000', 'valid_fmt': '20190418000000', 'init_fmt': '20190418000000', 'lead_string': '0 hours', 'offset_hours': 0, 'date': datetime.datetime(2019, 4, 18, 0, 0), 'cycle': datetime.datetime(2019, 4, 18, 0, 0), 'lead_hours': 0, 'lead_minutes': 0, 'lead_seconds': 0, 'custom': ''}

I see in your config file, you're using "LOOP_ORDER = processes" whereas that use case example uses "LOOP_ORDER = times".

You could try testing with "LOOP_ORDER = times" to see if that produces a different result?

@georgemccabe would be best suited to debug this behavior, but he's out of the office until Thursday. I suspect there's a bug here that will require fixing. I don't see any obvious problems with your setup.

[hertneky]

Thanks for the reply John. I tried 'LOOP_ORDER = times' and the result didn't change.

Ack - Please ignore the error in that log file - I purposefully changed the string to an incorrect parameter (CURRENT_LEVEL), just to see what was passed to do_string_sub as I know this info is printed out with that error.

LOOP_ORDER = times: As with 'LOOP_ORDER = processes' there is no error in the log file (/glade/scratch/hertneky/airforce/validation/metprd/logs/metplus.log.20220228140920). The only evidence of the issue is the fact that the output file name does not include the level, causing override issues when running multiple levels. Sorry for the confusion and for not stating the associated log file before - that was my bad.

This can wait for George.

[georgemccabe]

Hi @hertneky, it looks like the issue is likely occurring in the logic to handle reading the config variables from the instance sections. I ran your test config file and confirmed the behavior you are seeing. The issue goes away when I remove the [tmp] section and the tmp instance name from the process list, so you can temporarily get around this by calling run_metplus.py to run MODE for each instance separately.

I created issue #1471 to take care of this for the upcoming 4.1.0 release which should be completed this month. Please let me know if you will need a bugfix applied to METplus 4.0 as well.

[hertneky]

@georgemccabe Thanks George, that was my suspicion, that the instance was causing the issue. Yes I was previously running gridstat and mode for each variable with a separate metplus call, but thought I would get 'fancy' and combine everything into 1 nice use-case. It's not necessary to apply a bug-fix to METplus 4.0.