NLDAS expansion #57
Replies: 6 comments 14 replies
-
|
With updated NLDAS drivers (see this issue), runs for 37 sites failed. There is one new failed site: > nrow(filter(p2_nldas_glm_uncalibrated_runs, glm_success==FALSE))
[1] 37
> p2_nldas_glm_uncalibrated_runs %>% filter(glm_success==FALSE) %>% select(site_id, driver, extraction_error, param_sim_stop, max_output_date, glm_code) %>% slice(1:20)
# A tibble: 20 × 6
site_id driver extraction_error param_sim_stop max_output_date glm_code
<chr> <chr> <chr> <chr> <chr> <int>
1 nhdhr_109946… NLDAS NA values 2022-04-11 2022-04-11 0
2 nhdhr_114545… NLDAS NA values 2022-04-11 2022-04-11 0
3 nhdhr_117648… NLDAS NA values 2022-04-11 2022-04-11 0
4 nhdhr_120019… NLDAS NA values 2022-04-11 2022-04-11 0
5 nhdhr_120020… NLDAS NA 2022-04-11 1989-02-23 1
6 nhdhr_122546… NLDAS NA values 2022-04-11 2022-04-11 0
7 nhdhr_123165… NLDAS NA values 2022-04-11 2022-04-11 0
8 nhdhr_124617… NLDAS NA values 2022-04-11 2022-04-11 0
9 nhdhr_133099… NLDAS NA values 2022-04-11 2022-04-11 0
10 nhdhr_144272… NLDAS NA values 2022-04-11 2022-04-11 0
11 nhdhr_148621… NLDAS NA values 2022-04-11 2022-04-11 0
12 nhdhr_151870… NLDAS NA values 2022-04-11 2022-04-11 0
13 nhdhr_151949… NLDAS NA values 2022-04-11 2022-04-11 0
14 nhdhr_151959… NLDAS NA values 2022-04-11 2022-04-11 0
15 nhdhr_152517… NLDAS NA values 2022-04-11 2022-04-11 0
16 nhdhr_155352… NLDAS NA values 2022-04-11 2022-04-11 0
17 nhdhr_155639… NLDAS NA values 2022-04-11 2022-04-11 0
18 nhdhr_157360… NLDAS NA 2022-04-11 2022-04-11 134
19 nhdhr_159101… NLDAS NA values 2022-04-11 2022-04-11 0
20 nhdhr_258611… NLDAS NA values 2022-04-11 2022-04-11 0
> p2_nldas_glm_uncalibrated_runs %>% filter(glm_success==FALSE) %>% select(site_id, driver, extraction_error, param_sim_stop, max_output_date, glm_code) %>% slice(21:37)
# A tibble: 17 × 6
site_id driver extraction_error param_sim_stop max_output_date glm_code
<chr> <chr> <chr> <chr> <chr> <int>
1 nhdhr_259216… NLDAS NA values 2022-04-11 2022-04-11 0
2 nhdhr_261524… NLDAS NA values 2022-04-11 2022-04-11 0
3 nhdhr_261577… NLDAS NA values 2022-04-11 2022-04-11 0
4 nhdhr_474867… NLDAS NA values 2022-04-11 2022-04-11 0
5 nhdhr_487840… NLDAS NA values 2022-04-11 2022-04-11 0
6 nhdhr_564130… NLDAS NA values 2022-04-11 2022-04-11 0
7 nhdhr_597459… NLDAS NA values 2022-04-11 2022-04-11 0
8 nhdhr_604470… NLDAS NA values 2022-04-11 2022-04-11 0
9 nhdhr_604559… NLDAS NA values 2022-04-11 2022-04-11 0
10 nhdhr_703344… NLDAS NA values 2022-04-11 2022-04-11 0
11 nhdhr_756519… NLDAS NA values 2022-04-11 2022-04-11 0
12 nhdhr_756545… NLDAS NA values 2022-04-11 2022-04-11 0
13 nhdhr_809976… NLDAS NA values 2022-04-11 2022-04-11 0
14 nhdhr_810037… NLDAS NA values 2022-04-11 2022-04-11 0
15 nhdhr_810037… NLDAS NA values 2022-04-11 2022-04-11 0
16 nhdhr_829559… NLDAS NA values 2022-04-11 2022-04-11 0
17 nhdhr_916842… NLDAS NA values 2022-04-11 2022-04-11 0
> dput(p2_nldas_glm_uncalibrated_runs %>% filter(glm_success==FALSE) %>% pull(site_id))
c("nhdhr_109946618", "nhdhr_114545137", "nhdhr_117648183", "nhdhr_120019262",
"nhdhr_120020716", "nhdhr_122546701", "nhdhr_123165634", "nhdhr_124617242",
"nhdhr_133099150", "nhdhr_144272629", "nhdhr_148621069", "nhdhr_151870592",
"nhdhr_151949010", "nhdhr_151959489", "nhdhr_152517749", "nhdhr_155352079",
"nhdhr_155639566", "nhdhr_157360156", "nhdhr_159101759", "nhdhr_25861152",
"nhdhr_25921640", "nhdhr_26152472", "nhdhr_26157758", "nhdhr_47486745",
"nhdhr_48784080", "nhdhr_56413007", "nhdhr_59745939", "nhdhr_60447081",
"nhdhr_60455965", "nhdhr_70334497", "nhdhr_75651917", "nhdhr_75654579",
"nhdhr_80997655", "nhdhr_81003721", "nhdhr_81003725", "nhdhr_82955915",
"nhdhr_91684249") |
Beta Was this translation helpful? Give feedback.
-
|
Oh no. That is unexpected and shocking. 😱 |
Beta Was this translation helpful? Give feedback.
-
|
Well...I did misread this originally and think that the count was now 37 failures when we originally had 11 failures (which I think was the MN count). But now I see this is for the whole footprint and we're instead going from 36 to 37. So not quite as bad, but still unexpected to me. I'd expect the failures to either stay the same or decrease with this input data modification. I'm in favor of keeping the change as-is regardless though, as having mixed precipitation in the inputs did cause odd issues in the past that weren't necessarily model failures caught by the codes, but were unrealistic. |
Beta Was this translation helpful? Give feedback.
-
|
Surface evaluation plots for the 5/25/2022 run (9,332 lakes succeeded) are here. There were 3,665 evaluation sites (sites with >= 10 |
Beta Was this translation helpful? Give feedback.
-
|
@lindsayplatt @jread-usgs Just ran the NLDAS models for the full footprint (with landsat Kd and updated WQP data). It took 130 minutes to run the models ( > p2_nldas_glm_uncalibrated_runs %>% group_by(extraction_error) %>% summarize(n=n())
# A tibble: 4 × 2
extraction_error n
<chr> <int>
1 NA temperature values 119
2 Number of layers drops to 1 67
3 Number of layers drops to 1, Maximum hice value exceeds lake depth 1
4 NA 14430Note that the 'NA temperature values' error is triggered by the extraction of GLM temperatures and precedes the generation of the other two extraction errors. So lakes for which temperature data extraction failed ( I'm going to generate the surface prediction evaluation plots for these runs now. |
Beta Was this translation helpful? Give feedback.
-
|
This is a 98.7% success rate 🎉 Nice job scaling up! When we're modeling 14k+ lakes and ~1% fail, I'm not sure it is worth the effort to really dive deep here (esp given our time constraint & the fact that we kind of know why they failed thanks to your error handling. I will also let @jread-usgs comment on that. Maybe do a quick check for the lake depth of these 189 failed sites to see if they fall in that shallow category. |
Beta Was this translation helpful? Give feedback.
-
|
It is a great sign for the modeling pipeline that nearly doubling the lakes modeled was accomplished relatively easily - nice work Hayley! I agree that we've got a pretty good sense of why these would fail and a good set of catches to avoid exporting them. I'm good with moving on after Lindsay's suggestion of checking the distribution of depths for the 189. |
Beta Was this translation helpful? Give feedback.
-
|
Ok great! The mean lake depth across the 189 lakes for which NLDAS runs failed is 0.7 meters. Median lake depth is 0.6 m, minimum lake depth is 0 m (4 sites - ^ Pretty consistent with the pattern we've seen so far of failures being concentrated in very shallow lakes. The NLDAS surface evaluation plots are here |
Beta Was this translation helpful? Give feedback.
-
|
NLDAS model evaluation plots for surface and bottom predictions are HERE. Now that we are only evaluating predictions for lakes within CASC states we're down to 3688 evaluation sites. |
Beta Was this translation helpful? Give feedback.
-
|
Now that we are also restricting model runs and therefore extracted output to lakes within CASC states, our success percentage is 98.6% (NLDAS models for 12687/12864 lakes succeeded, models for 177 lakes failed). |
Beta Was this translation helpful? Give feedback.
-
|
Using the latest NLDAS-driven GLM build completed on June 16, 2022, the full footprint of thermal metrics has been completed (built with |
Beta Was this translation helpful? Give feedback.
-
|
The code in this thread assumes you have the
library(tidyverse)
library(sf)
nldas_metrics_all <- readr::read_csv('annual_metrics_glm3_pb0nldas.csv')
select_metrics <- nldas_metrics_all %>%
select(year,
starts_with("max_surf_"),
starts_with("max_bot_")) %>%
pivot_longer(cols = c(!year)) %>%
separate(name, c("stat", "depth", "month_abb"), "_") %>%
filter(month_abb != "JulAugSep") %>%
mutate(month_num = as.numeric(factor(month_abb, levels = tolower(month.abb))))%>%
mutate(depth_f = factor(depth, levels=c('surf', 'bot'), labels=c('surface', 'bottom'), ordered=TRUE))
# Add groups by decade (the 2020 decade will have less data than the rest)
select_metrics_yearsgrp <- select_metrics %>%
mutate(decade = year - year%%10)
ggplot(select_metrics_yearsgrp, aes(y = value, group=decade)) +
geom_boxplot(outlier.colour = "darkgrey", outlier.shape = 1, outlier.alpha = 0.1) +
scale_x_discrete(breaks=1:12, labels = month.abb) +
facet_grid(depth_f ~ month_num) +
ylab('GLM-predicted temperature, deg C') +
ggtitle('Annual maximum temps at surface and bottom depths for each month of the year',
subtitle = ' There is a boxplot per decade') |
Beta Was this translation helpful? Give feedback.
-
|
Most of the 247 lakes with a really high winter temperature are toward the southern part of our footprint. There is one lake near the WI-IL border
library(tidyverse)
library(sf)
library(usmap)
nldas_metrics_all <- readr::read_csv('annual_metrics_glm3_pb0nldas.csv')
# Get data for mapping
site_info <- readr::read_csv('lake_metadata.csv') # You can download this from the same ScienceBase item
state_sf <- usmap::us_map(include = unique(site_info$state)) %>%
st_as_sf(coords = c('x', 'y'))
# Map out lakes that have unexpectedly high winter values. Does it make sense?
high_winter_temp_val <- 18
high_winter_temp_sites <- nldas_metrics_all %>%
filter(max_surf_jan >= high_winter_temp_val |
max_surf_feb >= high_winter_temp_val |
max_surf_dec >= high_winter_temp_val) %>%
select(site_id) %>%
left_join(site_info) %>%
st_as_sf(coords = c('centroid_lon', 'centroid_lat'), crs = 4326) %>%
st_transform(crs = usmap::usmap_crs())
usmap::plot_usmap(include = unique(site_info$state), color = 'darkgrey', size=0.15) +
geom_sf(data = high_winter_temp_sites, aes(color = lake_depth_m), alpha=0.7) +
scico::scale_color_scico(palette = 'batlow', direction = -1) +
ggtitle(sprintf('Lakes with at least one max surface temperature in Dec, Jan, or Feb above %s degC', high_winter_temp_val),
subtitle = sprintf('n = %s lakes', nrow(high_winter_temp_sites))) |
Beta Was this translation helpful? Give feedback.
-
|
Generally, the lakes that have really high summer temperatures are either in the southern part of the footprint and/or are shallow lakes.
library(tidyverse)
library(sf)
library(usmap)
nldas_metrics_all <- readr::read_csv('annual_metrics_glm3_pb0nldas.csv')
# Get data for mapping
site_info <- readr::read_csv('lake_metadata.csv') # You can download this from the same ScienceBase item
state_sf <- usmap::us_map(include = unique(site_info$state)) %>%
st_as_sf(coords = c('x', 'y'))
# Map out lakes that have unexpectedly high summer values. Does it make sense?
high_summer_temp_val <- 40
high_summer_temp_sites <- nldas_metrics_all %>%
filter(max_surf_jun > high_summer_temp_val |
max_surf_jul > high_summer_temp_val |
max_surf_aug > high_summer_temp_val) %>%
select(site_id) %>%
left_join(site_info) %>%
st_as_sf(coords = c('centroid_lon', 'centroid_lat'), crs = 4326) %>%
st_transform(crs = usmap::usmap_crs())
usmap::plot_usmap(include = unique(site_info$state), color = 'darkgrey', size=0.15) +
geom_sf(data = high_summer_temp_sites, aes(color = lake_depth_m), alpha=0.7) +
scico::scale_color_scico(palette = 'batlow', direction = -1) +
ggtitle(sprintf('Lakes with at least one max surface temperature in Jun, Jul, or Aug above %s degC', high_summer_temp_val),
subtitle = sprintf('n = %s lakes', nrow(high_summer_temp_sites))) |
Beta Was this translation helpful? Give feedback.
-
|
Okay I just re-ran the NLDAS models for the full footprint, following the fix to delete failed simulation directories and the update to the nml_list.rds. As Lindsay and I had anticipated, all of the NLDAS models rebuilt, but many of the extraction steps were skipped, as the output was identical for the majority of model runs. Model output changed for 12 sites across 5 states (WI, MI, MN, SD, IL): > output_summary <- readr::read_csv('../../../Downloads/GLM_NLDAS_summary.csv',col_types=cols())
> changed_output <- output_summary %>%
+ left_join(output_summary_old, by='site_id', suffix = c('_current','_old')) %>%
+ filter((export_fl_hash_current != export_fl_hash_old)) %>%
+ pull(site_id) %>%
+ dput()
c("nhdhr_{D8A9DD23-94AD-4C15-B242-FE519B5EDF8C}", "nhdhr_109946618",
"nhdhr_116542403", "nhdhr_120018832", "nhdhr_144272629", "nhdhr_151949010",
"nhdhr_152517749", "nhdhr_157360156", "nhdhr_48784080", "nhdhr_70333995",
"nhdhr_75651917", "nhdhr_80997655")As expected, the two WI sites are among those with changed output. And we have gained output for one additional site, in IN: > new_output <- output_summary %>%
+ left_join(output_summary_old, by='site_id', suffix = c('_current','_old')) %>%
+ filter(is.na(export_fl_hash_old)) %>%
+ pull(site_id) %>%
+ dput()
"nhdhr_123165634"So our success percentage is still 98.6% (NLDAS models for 12688/12864 lakes succeeded, models for 176 lakes failed) |
Beta Was this translation helpful? Give feedback.
-
|
I currently have a job submitted to re-run the |
Beta Was this translation helpful? Give feedback.
-
|
The updated evaluation plots and matched pred-obs are here. The # of evaluation sites didn't change, since the newly added site |
Beta Was this translation helpful? Give feedback.
-
|
The updated NLDAS output and matched pred-obs are on SB. Note that 5 of the NLDAS zips (for ND, MO, IA, AR, and OH) have not been updated, as no output changed for sites in those states. |
Beta Was this translation helpful? Give feedback.
-
|
Using the latest NLDAS-driven GLM build completed on August 29, 2022, the full footprint of thermal metrics has been rebuilt for the 12,688 successfully modeled lakes. These new NLDAS thermal metrics have been updated on SB. |
Beta Was this translation helpful? Give feedback.
-
Just ran the NLDAS models for the full footprint (9,369 lakes). It ran in just under an hour.
I'm generating the output files now.UPDATE: all 9333 NLDAS feather files are in3_extract/out. They are also zipped into a single zip file.Out of 9369 runs, 9333 succeeded. 36 failed. 99.6% success rate. The 11 runs that had failed in the MN run also failed this time around.
The mean lake depth across the 36 lakes for which NLDAS runs failed is 1.6 meters. Median lake depth is 0.96 m, minimum lake depth is 0 m (
nhdhr_151870592), maximum lake depth is 14 m (nhdhr_157360156).Details on the failed runs:
Beta Was this translation helpful? Give feedback.
All reactions