dOFV and AIC columns for summary_log()

[seth127]

We would like to add several columns to the summary_log() output.

AIC

This is a straightforward calculation. Consider if there are any edge cases to catch.

Example code

add_aic <- function(.runlog) {
  ## check summary log appended
  if(any(str_detect(names(.runlog), "param_count")) & 
     any(str_detect(names(.runlog), "ofv"))) {
    return(mutate(.runlog, aic=2*param_count + ofv))
  }
  ## if not, return unchanged
  warning("AIC could not be computed")
  return(.runlog)
}

dOFV

The change in OFV from the parent model. Several considerations:

• What to do if there are multiple models in based_on? Likely select the first?
• What to do if the based on model has NA for ofv? This could happen with Bayesian models, or the new bootstrap model type.

Example code

## compare ofv to that of the model in based_on
## if multiple models are in based_on, use the first
add_dofv <- function(.runlog) {
  .runlog %>%
    mutate(based_on_join = purrr::map_chr(based_on, function(x) {ifelse(length(x) > 0, x[1], NA)})) %>% 
    left_join(
      select(.runlog, based_on_join=run, based_on_ofv=ofv), 
      by="based_on_join"
    ) %>% 
    mutate(dofv = as.numeric(format(ofv - based_on_ofv, nsmall = 2))) %>% 
    select(-based_on_ofv, -based_on_join)
}

dAIC

If we're adding the above, should we also calculate the difference in AIC?

[seth127]

Consider also adding a method for easily comparing two model objects (as opposed to a log tibble), similar to the discussion in #401

[barrettk]

Once consideration worth talking about: summary logs dont have a based_on column, which is required for add_dofv. Do we:

1. want to only implement this for run_log() %>% add_summary(), or
2. add based_on to the summary_log (and maybe even the config_log()).
   • I dont think it would make sense to remove the based_on column when dofv is present (i.e. dont just add it to compute dofv, and then remove), as this could cause confusion.