MetViewer batch job problem

[Ho-ChunHuang-NOAA]

This is a problem related to METViewer, please let me know if I need to submit it to different links.

I am running the similar script as in the mid-February for monthly graphics plot. I run into problem of
Running:
/opt/metviewer_python/METviewer_py3.6.3//bin/python
/opt/metviewer_python/METcalcpy//metcalcpy/agg_stat.py
/data/mv_data/ho-chun.huang/data/G246_HYSPLIT_DUST_CSI_DAY2_12Z_202202.agg_stat.info
Python script execution time 0:00:00.850

Running:
/opt/metviewer_python/METviewer_py3.6.3//bin/python
/opt/metviewer_python/METplotpy//metplotpy/plots/line/line.py
/data/mv_data/ho-chun.huang/data//plot_G246_HYSPLIT_DUST_CSI_DAY2_12Z.yaml
==== Start Python output ====

Invalid value of type 'builtins.float' received for the 'y' property of layout.legend
    Received value: 3.03

The 'y' property is a number and may be specified as:
  - An int or float in the interval [-2, 3]

==== End Python output ====
Python script execution time 0:00:01.192

Job 1 execution time 0:00:02.139

 Plots run: 1 of 1
Total time: 0:00:02.139

Avg plot time: 0:00:02.139

---- MVBatch Done ----

I can not find where is the error. My xml & sh can be found at
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/plot_hysplit_dust_csi.xml
https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/mv_batch_on_aws.sh

[j-opatz]

Hello,

And thank you for your question. Our resident expert for METviewer (@TatianaBurek) is currently out of the office, but I'll tag her on this issue and she'll respond as soon as she is able. Thank you for your patience in the meantime.

[TatianaBurek]

The latest version of METviewer - v4.1 - uses Python as a default method to calculate statistics and create plots. Even though we try our best to address the conversion of R-style margins, sizes, line widths and others to Python values sometimes it doesn't work.
Your XML was created to run with Rscript and some of positioning values can't be converted to Python. For example, legend_inset.
You have two options to make this work:

1. Continue to use Rscript. But Rscript is deprecated and will be removed in the next version of METviewer.
   In this case you need to add
   <execution_type>Rscript</execution_type>
   to your XML. You could place it before <event_equal> tag.
2. Adjust your XML to work with Python
   Changes:
   <legend_inset>c(0,-0.25)</legend_inset>
   c(1)
   <title_offset>-2</title_offset>
   If you find that these values are not optimal, please use the web page to find better options

Let us know if this works for you

[Ho-ChunHuang-NOAA]

Hi, Tatiana: Thanks. Can you be more specific on the "c(1)" ? What is the <abc> </abc> around it? Ho-Chun Huang, Ph.D. IMSG at NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Thu, Mar 31, 2022 at 4:28 PM Tatiana Burek ***@***.***> wrote: The latest version of METviewer - v4.1 - uses Python as a default method to calculate statistics and create plots. Even though we try our best to address the conversion of R-style margins, sizes, line widths and others to Python values sometimes it doesn't work. Your XML was created to run with Rscript and some of positioning values can't be converted to Python. For example, legend_inset. You have two options to make this work: 1. Continue to use Rscript. But Rscript is deprecated and will be removed in the next version of METviewer. In this case you need to add <execution_type>Rscript</execution_type> to your XML. You could place it before <event_equal> tag. 2. Adjust your XML to work with Python Changes: <legend_inset>c(0,-0.25)</legend_inset> c(1) <title_offset>-2</title_offset> If you find that these values are not optimal, please use the web page to find better options Let us know if this works for you — Reply to this email directly, view it on GitHub <#1527 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3GRMND7GWHBW2T47FDVCYDFXANCNFSM5RK65YIQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[TatianaBurek]

Sorry, GitHub did some unasked formatting on my response. Here are the XML tags:

<legend_inset>c(0,-0.25)</legend_inset>
<lwd>c(1)</lwd>
<title_offset>-2</title_offset>

[Ho-ChunHuang-NOAA]

Hi, Thanks. My plot change from [image: image.png] to [image: plot_plot_G246_HYSPLIT_DUST_CSI_DAY2_12Z_202202.png] My histogram disappeared, can you help? Ho-Chun Huang, Ph.D. IMSG at NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Thu, Mar 31, 2022 at 4:57 PM Tatiana Burek ***@***.***> wrote: Sorry, GitHub did some unasked formatting on my response. Here are the XML tags: <legend_inset>c(0,-0.25)</legend_inset> <lwd>c(1)</lwd> <title_offset>-2</title_offset> — Reply to this email directly, view it on GitHub <#1527 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3F4KAFPQDNOEAVIR4LVCYGU3ANCNFSM5RK65YIQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[TatianaBurek]

The images that you mentioned did not get sent. Could you please attache these images to the GitHub Discussion instead of the email.
Where can I find the XML for your histogram?

[Ho-ChunHuang-NOAA]

original
http://www.emc.ncep.noaa.gov/mmb/hchuang/web/fig/2022/202201/G236_HYSPLIT_DUST_CSI_DAY1_06Z_202201.png

New one case 20220401_141110
the setting is
c("h")

Both use similar xml except the changes suggested in your previous response.
original one (without your suggestion) is https://www.emc.ncep.noaa.gov/mmb/hchuang/transfer/plot_hysplit_dust_csi.xml

[TatianaBurek]

Python implementation does not support Series line type=histogram-like.
If you would like to create similar plot please use Bar plot with the same variable configurations.
20220401_174745
20220401_175516
Please note that Bar plot supports only summary stats.

[Ho-ChunHuang-NOAA]

Hi, Tatiana: I do not understand "Please note that Bar plot supports only summary stats.". Do you mean Bar plot can not be used for categorical score usch as CSI? This is the legency plot for the CSI for AQ group. Ho-Chun Huang, Ph.D. IMSG at NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Fri, Apr 1, 2022 at 1:48 PM Tatiana Burek ***@***.***> wrote: Python implementation does not support Series line type=histogram-like. If you would like to create similar plot please use Bar plot with the same variable configurations. 20220401_174745 Please note that Bar plot supports only summary stats. — Reply to this email directly, view it on GitHub <#1527 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3A7QSLB3GBACAXE32LVC4ZHDANCNFSM5RK65YIQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[TatianaBurek]

METviewer can calculate statistics using two different methods - Summary stats and Aggregation stats.
There are some explanations about the differences between these methods in this discussion:
#1081
Series plots support both methods but the Bar plot currently supports only Summary stats

[Ho-ChunHuang-NOAA]

Hi, Tatiana: Can you check 20220401_192247 *why is the x-axis labeling not following the order specified in *Independent Variable? Ho-Chun Huang, Ph.D. IMSG at NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Fri, Apr 1, 2022 at 2:44 PM Tatiana Burek ***@***.***> wrote: METviewer can calculate statistics using two different methods - Summary stats and Aggregation stats. There are some explanations about the differences between these methods in this discussion: #1081 <#1081> Series plots support both methods but the Bar plot currently supports only Summary stats — Reply to this email directly, view it on GitHub <#1527 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3FP525O6MTJ4LJ2WOLVC47WVANCNFSM5RK65YIQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[TatianaBurek]

The behavior that you described is a bug in METplotpy code.
I created a GItHub issue and will fix it in the next release:
dtcenter/METplotpy#218

[Ho-ChunHuang-NOAA]

Hi, Tatiana: Thanks for your effort to fix it. Ho-Chun Huang, Ph.D. IMSG at NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Mon, Apr 4, 2022 at 10:21 AM Tatiana Burek ***@***.***> wrote: The behavior that you described is a bug in METplotpy code. I created a GItHub issue and will fix it in the next release: dtcenter/METplotpy#218 <dtcenter/METplotpy#218> — Reply to this email directly, view it on GitHub <#1527 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3C4QFKJD35QWP66XB3VDL3GVANCNFSM5RK65YIQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>