Extend manufactured solution to del2 and del4

[cbegeman]

This PR adds manufactured solution tests for del2 and del4 and renames the existing case to default. Both del2 and del4 tests depend on the corresponding source terms being added to the model. These terms are added to MPAS-Ocean here E3SM-Ocean-Discussion/E3SM#110. All tests depend on E3SM-Ocean-Discussion/E3SM#55 to achieve expected convergence.

Checklist

• User's Guide has been updated
• Documentation has been built locally and changes look as expected
• Testing comment in the PR documents testing used to verify the changes
• New tests have been added to a test suite

[cbegeman]

Testing

MPAS-Ocean tests have been run with E3SM-Ocean-Discussion/E3SM#110 and E3SM-Ocean-Discussion/E3SM#55.

Update 19 Dec: MPAS-Ocean has been re-run and tests pass

[cbegeman]

@xylar and @sbrus89 Would you like to look this over again and get this in before the IO streams changes? I think I'd rather do those changes to this test separately.

[xylar]

I'm not working much tomorrow but I can try to test this out. I agree that it would be good to get it in first without worrying about Omega, and then rework it for Omega support.

[xylar]

Just a note that this needs E3SM-Project/E3SM#6862 and and update to the E3SM-Project submodule.

[xylar]

I only have one other concern besides the Omega support and that is that the analysis plots for the 3 tests look exactly identical, which I wouldn't have expected.

Does that suggest that the del2 and del4 terms are negligible and we need to tweak the parameters?

[xylar]

@cbegeman, this change will break manufactured solution in Omega because the associated changes in forward.py don't yet include anything related to Omega. I know you don't want to address Omega here yet but maybe the solution is: 1) to leave these changes out for Omega and 2) to have setup fail for Omega for the del2 and del4 tasks for now.

[cbegeman]

@xylar I removed the whole Omega section from this file. Is that in line with what you were thinking?

[xylar]

@cbegeman, hmm, I'm hesitant to drop Omega support for manufactured_solution entirely. It's one of only 2 working tests we have.

[cbegeman]

@xylar Should I just go ahead and make the io streams changes then to support all 3?

[cbegeman]

@xylar I reverted the changes to this file. Should we merge this PR and then start adding support for Omega del2/del4?

[xylar]

Great, thanks! Yes, I think so. (Sorry, I somehow missed the earlier comment).

I think we need to rebase or otherwise resolve the conflict listed below before we can merge.

[cbegeman]

I only have one other concern besides the Omega support and that is that the analysis plots for the 3 tests look exactly identical, which I wouldn't have expected.

The convergence rates are different for each test. Is it important that the analysis plots look different?

[xylar]

The convergence rates are different for each test. Is it important that the analysis plots look different?

Hmm, let me check that that's true for my test. The plots are visually identical so I assumed the convergence rates would be, too. It seems like if the del2 and del4 terms are actually having an effect, the error would be more quantitatively different with and without the terms, wouldn't you think?

[cbegeman]

@xylar My memory was foggy. I should have said that the errors are different for each test. I didn't check the convergence rates to enough significant digits to tell.

[xylar]

cat */analysis/*.csv
default:
resolution,l2
200.0,0.31642597886910423
100.0,0.07877610404218326
50.0,0.019694390950707373
25.0,0.004945690456387108

del2:
resolution,l2
200.0,0.3164142762184392
100.0,0.0787731760256298
50.0,0.019693657273716133
25.0,0.00494550529879111

del4:
resolution,l2
200.0,0.316425984359642
100.0,0.07877611000352792
50.0,0.01969439697649461
25.0,0.004945696294213287

It seems like differences are in something like the 5th digit. To me, that suggests that the del2 and del4 terms aren't big enough to be affecting the convergence rate. We wouldn't notice if those terms don't have the expected convergence rate because all three tests are really only showing the convergence rate of the dominant terms.

[cbegeman]

@xylar That's a good point. We'd want that term to be large enough that you'd notice if you broke the del2/del4 operators. I'll test some other values shortly.

[xylar]

Thanks, that sounds good!

[cbegeman]

With these changes the convergence rates are:
Default: 2.044
Del2: 2.042
Del4: 2.039

I already had to decrease the time step by a factor of 2 to accommodate these higher viscosities. If we want to increase the del2 viscosity further, we will need to further decrease the time step. I would rather not do this because it's nice to have this be a quick (1 min) test.

[xylar]

@cbegeman, okay, that seems like a reasonable compromise. It seems like we're still going to have a hard time telling if we've messed up the either the del2 or del4 term using these tests but maybe that just suggests we would need a dedicated tests to determine that if it were a high priority for us. We can discuss more outside of this PR if need be.

[xylar]

Approving based on my previous testing and @cbegeman's newest testing.

[xylar]

In case my comment above gets lost, I think the remaining step is to rebase or merge main into this branch to address the conflict listed below.

[sbrus89]

@cbegeman, this is great! I tested on Perlmutter and got the same convergence results you did above. My one comment is that although the errors are clearly converging, the viz step seems to be showing incorrect differences in the left panel below:

[cbegeman]

@sbrus89 Thanks for catching that! I've added a fix here: d48ea7b

[sbrus89]

Thanks for fixing the viz step, @cbegeman. The changes look good.

[cbegeman]

@xylar Can you check that I've done the submodule update correctly before I merge? It's been a while since I've done one of those.

[xylar]

@cbegeman, yes, looks great to me!