[TASK] Template driver system should be more flexible for adding new workflows and features

[j-bryan]

---

Issue Description

Is your feature request related to a problem? Please describe.
The HERON template driver is quite cumbersome. At over 2000 lines, the HERON/templates/template_driver.py file does everything for preparing RAVEN XML files for the HERON case at hand by adding, removing, and editing nodes in the template outer.xml and inner.xml files. While effective, this file and the TemplateDriver class are in need of refactoring. This refactoring would ideally improve the template system in a number of ways, such as

• More readable and maintainable code
• Easier addition of new template XML files to the templating system
• Easier addition of new HERON workflows and features

A refactor of the template driver is motivated by plans to add resource adequacy/capacity expansion modeling capabilities to HERON. Adding a new workflow like this will require significant modification and expansion of the existing template driver.

Describe the solution you'd like
At a minimum, the template driver capabilities for the inner and outer workflows could be broken into separate files or classes. This would reduce the size of the template_driver.py file and the current TemplateDriver class. However, it may be more helpful and flexible to take a feature-based approach to building up a RAVEN workflow to make adding new capabilities, such as adding new samplers or optimizers to HERON, a more straightforward process. I think the template driver should also be able to determine if the HERON case could be run as a "flat" RAVEN workflow to avoid the additional overhead of starting RAVEN multiple times.

Describe alternatives you've considered
As stated earlier, a minimum refactoring of splitting the template driver into "inner" and "outer" template drivers could be sufficient for the time being. However, the concern with that approach is that is does nothing to make the template driver more extensible, only more readable. Adding more workflows after this simple refactor would likely require refactoring again or significant code duplication.

---

For Change Control Board: Issue Review

This review should occur before any development is performed as a response to this issue.

• 1. Is it tagged with a type: defect or task?
• 2. Is it tagged with a priority: critical, normal or minor?
• 3. If it will impact requirements or requirements tests, is it tagged with requirements?
• 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
• 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

---

For Change Control Board: Issue Closure

This review should occur when the issue is imminently going to be closed.

• 1. If the issue is a defect, is the defect fixed?
• 2. If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
• 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
• 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
• 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

[PaulTalbot-INL]

This is a great and much-needed task, in my opinion.

One thing to watch out for: if we weren't careful, we ended up with some gnarly decision trees based on many choices the user may have made in the HERON input that had nesting results on the RAVEN inputs. A simple example, choosing opt or sweep mode changes what Sampler (and hence what input options) are using in the outer RAVEN, which means "go add a variable to the sampler" has to include some way of knowing other decisions made in the input. While a few of these are no big deal, the "decision spaghetti" can quickly become unmanagable as we provide more decision points in the HERON input.

Looking forward to see what you come up with for a design on the refactor!