From 826508ae91db2a35e2cd7fcef3a03a3d9ad31433 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Wed, 6 Dec 2023 11:18:46 -0500 Subject: [PATCH 1/7] Add design doc for mesh class --- components/omega/doc/design/MeshClass.md | 71 ++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 components/omega/doc/design/MeshClass.md diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/MeshClass.md new file mode 100644 index 000000000000..31105c8863d2 --- /dev/null +++ b/components/omega/doc/design/MeshClass.md @@ -0,0 +1,71 @@ +(omega-design-my-class)= +# Template: MyClassOrModuleName + +## 1 Overview + +A short overview of the functionality this provides. + +## 2 Requirements + +### 2.1 Requirement: xxx + +State requirement clearly and simply, followed by justification. +Avoid getting into design or implementation details + +### 2.2 Requirement: yyy + +State requirement clearly and simply, followed by justification. +Avoid getting into design or implementation details + +### 2.3 Desired: zzz + +Clear statement of desired feature and reason for including (e.g. design needs +to take this into account for future capability). + +## 3 Algorithmic Formulation + +Describe any algorithms or point to references that are important for implementation + +You can include math in Latex format like his: + +$$ +(a + b)^2 &= (a + b)(a + b) \\ + &= a^2 + 2ab + b^2 +$$ + +## 4 Design + +You can include code blocks like this: + +```c++ +int var = value; +``` + +### 4.1 Data types and parameters + +#### 4.1.1 Parameters + +List and define any configuration parameters or public constants. + +#### 4.1.2 Class/structs/data types + +Describe any public data types and/or the class definition + +### 4.2 Methods + +List and describe all public methods and their interfaces (actual code for +interface that would be in header file). Describe typical use cases. + +## 5 Verification and Testing + +### 5.1 Test xxx + +Describe test including conditions for pass/fail +List which requirements it tests: + - tests requirement xxx + +### 5.2 Test yyy + +Describe test including conditions for pass/fail +List which requirements it tests: + - tests requirement zzz From d847204d507b4be1a8b76456b5018d4a3802bfb2 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Wed, 6 Dec 2023 12:16:08 -0500 Subject: [PATCH 2/7] Add requirements --- components/omega/doc/design/MeshClass.md | 40 +++++++++++++----------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/MeshClass.md index 31105c8863d2..64b73871f4cf 100644 --- a/components/omega/doc/design/MeshClass.md +++ b/components/omega/doc/design/MeshClass.md @@ -1,37 +1,39 @@ -(omega-design-my-class)= -# Template: MyClassOrModuleName +(omega-design-mesh)= +# Mesh ## 1 Overview -A short overview of the functionality this provides. +The mesh class will contain the YAKL arrays which describe the mesh and are used in the computation of the tendency terms in the discrete governing equations. ## 2 Requirements -### 2.1 Requirement: xxx +### 2.1 Requirement: OMEGA will use the previously estabilished MPAS Mesh Spec -State requirement clearly and simply, followed by justification. -Avoid getting into design or implementation details +The OMEGA mesh information should be compatiable with the [MPAS Mesh Specification](https://mpas-dev.github.io/files/documents/MPAS-MeshSpec.pdf). -### 2.2 Requirement: yyy +### 2.2 Requirement: Functionality is needed to read the mesh on the host and transfer relevant data to the device for comutation -State requirement clearly and simply, followed by justification. -Avoid getting into design or implementation details +Not all mesh information is required in computing the tendency terms on the device, e.g. lonCell, latCell, etc. +However, other arrays will need to allocated and copied to the device for use in tendency computation. +One option is to template the mesh class so a mesh object can be created for both the host and device, with a method to perform the copy between the two object types. -### 2.3 Desired: zzz +### 2.3 Requirement: Zero-based cell, edge, and vertex numbering -Clear statement of desired feature and reason for including (e.g. design needs -to take this into account for future capability). +Although the existing MPAS Mesh spec uses a one-based mesh numbering, zero-based mesh numbering is required to be compatiable with the zero-based indexing used for YAKL arrays. -## 3 Algorithmic Formulation +### 2.4 Desired: Ability to support multiple independent mesh objects + +This flexibility is required to support future implmentations of spatially split barotropic and baroclinic modes that are computed on different resolution -Describe any algorithms or point to references that are important for implementation +### 2.5 Desired: OMEGA can read in a reduced number of mesh variables and compute the remaning array information online. -You can include math in Latex format like his: +Many of the mesh variables are not independent, e.g. areaCell, weightsOnEdge, etc., and can be computed from a reduced set of mesh variables. +This functionality could be used to reduce mesh/restart file size for high resoultion meshes. +Builiding in this flexibility would allow for all mesh-related calculations to be availiable within the code base instead of spreading them amonst various utility programs in MPAS-Tools. +The functions used to compute the dependent mesh information can also be used to verify mesh information provided in the mesh input stream. + +## 3 Algorithmic Formulation -$$ -(a + b)^2 &= (a + b)(a + b) \\ - &= a^2 + 2ab + b^2 -$$ ## 4 Design From 86c6ae27be9c825056cba756f97c9bfca8b32a18 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Mon, 11 Dec 2023 14:27:52 -0500 Subject: [PATCH 3/7] Add decomp requirement --- components/omega/doc/design/MeshClass.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/MeshClass.md index 64b73871f4cf..36bbfd5e5aad 100644 --- a/components/omega/doc/design/MeshClass.md +++ b/components/omega/doc/design/MeshClass.md @@ -21,11 +21,14 @@ One option is to template the mesh class so a mesh object can be created for bot Although the existing MPAS Mesh spec uses a one-based mesh numbering, zero-based mesh numbering is required to be compatiable with the zero-based indexing used for YAKL arrays. -### 2.4 Desired: Ability to support multiple independent mesh objects +### 2.4 Requirement: Work with Decomp class to decompose mesh + + +### 2.5 Desired: Ability to support multiple independent mesh objects This flexibility is required to support future implmentations of spatially split barotropic and baroclinic modes that are computed on different resolution -### 2.5 Desired: OMEGA can read in a reduced number of mesh variables and compute the remaning array information online. +### 2.6 Desired: OMEGA can read in a reduced number of mesh variables and compute the remaning array information online. Many of the mesh variables are not independent, e.g. areaCell, weightsOnEdge, etc., and can be computed from a reduced set of mesh variables. This functionality could be used to reduce mesh/restart file size for high resoultion meshes. From cdce46d834e51bf689b10e7a792e0cf563e96b98 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Mon, 18 Dec 2023 16:27:56 -0500 Subject: [PATCH 4/7] Update requirements based on comments and discussion --- components/omega/doc/design/MeshClass.md | 45 ++++++++++++++++-------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/MeshClass.md index 36bbfd5e5aad..75e94f843a12 100644 --- a/components/omega/doc/design/MeshClass.md +++ b/components/omega/doc/design/MeshClass.md @@ -1,39 +1,56 @@ -(omega-design-mesh)= +(omega-design-horz-mesh)= # Mesh ## 1 Overview -The mesh class will contain the YAKL arrays which describe the mesh and are used in the computation of the tendency terms in the discrete governing equations. +The mesh class will contain the YAKL arrays which describe the horizontal mesh and are used in the computation of the tendency terms in the discrete governing equations. OMEGA will separate the horizontal mesh variables from the vertical mesh information. ## 2 Requirements -### 2.1 Requirement: OMEGA will use the previously estabilished MPAS Mesh Spec +### 2.1 Requirement: OMEGA will use the previously established MPAS Mesh Spec -The OMEGA mesh information should be compatiable with the [MPAS Mesh Specification](https://mpas-dev.github.io/files/documents/MPAS-MeshSpec.pdf). +The OMEGA mesh information should be compatible with the [MPAS Mesh Specification](https://mpas-dev.github.io/files/documents/MPAS-MeshSpec.pdf). -### 2.2 Requirement: Functionality is needed to read the mesh on the host and transfer relevant data to the device for comutation +### 2.2 Requirement: Functionality is needed to read the mesh on the host and transfer relevant data to the device for computation Not all mesh information is required in computing the tendency terms on the device, e.g. lonCell, latCell, etc. -However, other arrays will need to allocated and copied to the device for use in tendency computation. -One option is to template the mesh class so a mesh object can be created for both the host and device, with a method to perform the copy between the two object types. +However, other arrays will need to be allocated and copied to the device for use in tendency computation. +The mesh class will explicitly include host and device YAKL arrays for each variable. +A class method will be included to copy the relevant mesh information to the device. ### 2.3 Requirement: Zero-based cell, edge, and vertex numbering -Although the existing MPAS Mesh spec uses a one-based mesh numbering, zero-based mesh numbering is required to be compatiable with the zero-based indexing used for YAKL arrays. +Although the existing MPAS Mesh spec uses a one-based mesh numbering, zero-based mesh numbering is required to be compatible with the zero-based indexing used for YAKL arrays. ### 2.4 Requirement: Work with Decomp class to decompose mesh +The mesh class will reference the partitioned connectivity arrays created by the Decomp class. -### 2.5 Desired: Ability to support multiple independent mesh objects +### 2.5 Requirement: Mesh variables will be associated with metadata to describe data -This flexibility is required to support future implmentations of spatially split barotropic and baroclinic modes that are computed on different resolution +Following the Metadata and IO designs, the YAKL arrays for the mesh variables will be associated with information about the represented values. -### 2.6 Desired: OMEGA can read in a reduced number of mesh variables and compute the remaning array information online. +### 2.6 Requirement: I/O to obtain mesh data + +The Mesh class will have a method to read in the mesh information not obtained by the Decomp class. + +### 2.7 Desired: Ability to support multiple independent mesh objects + +This flexibility is required to support future implementations of spatially split barotropic and baroclinic modes that are computed on different resolution meshes. +Additionally, this flexibility can be used to support separate domain decompositions for the barotropic and baroclinic meshes, which can help optimize the communication frequency for the barotropic subcycling via wide barotropic halos. + +### 2.8 Desired: OMEGA can read in a reduced number of mesh variables and compute the remaining array information online. Many of the mesh variables are not independent, e.g. areaCell, weightsOnEdge, etc., and can be computed from a reduced set of mesh variables. -This functionality could be used to reduce mesh/restart file size for high resoultion meshes. -Builiding in this flexibility would allow for all mesh-related calculations to be availiable within the code base instead of spreading them amonst various utility programs in MPAS-Tools. -The functions used to compute the dependent mesh information can also be used to verify mesh information provided in the mesh input stream. +This functionality could be used to reduce mesh/restart file size for high resolution meshes. +Building in this flexibility would allow for all mesh-related calculations to be available within the code base instead of spreading them amongst various utility programs in MPAS-Tools. +This will improve the ability to maintain and unit test the mesh calculations in MPAS-Tools. +The functions used to compute the dependent mesh information can also be used to verify any mesh information that is provided in the mesh input stream. + +As standard practice, all necessary internally computed mesh information will be output in a single file for post-processing purposes. +A checksumming strategy will be implemented to avoid situations where simulation data and mesh information are mismatched during post processing. + +Where appropriate, some additional derived quantities (e.g. reciprocals) will also be included to improve the performance of device-side calculations. ## 3 Algorithmic Formulation From 105d3b836369e2742b69294c383f429e9ee640e4 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Wed, 3 Jan 2024 16:49:02 -0500 Subject: [PATCH 5/7] Add Design section --- components/omega/doc/design/MeshClass.md | 71 ++++++++++++++++++------ 1 file changed, 53 insertions(+), 18 deletions(-) diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/MeshClass.md index 75e94f843a12..66f17dc0bd85 100644 --- a/components/omega/doc/design/MeshClass.md +++ b/components/omega/doc/design/MeshClass.md @@ -54,40 +54,75 @@ Where appropriate, some additional derived quantities (e.g. reciprocals) will al ## 3 Algorithmic Formulation +The algorithms required for computing dependent mesh quantities are currently implemented in the MPAS Mesh Converter utility. ## 4 Design -You can include code blocks like this: +### 4.1 Data types and parameters + +#### 4.1.1 Parameters ```c++ -int var = value; + ``` -### 4.1 Data types and parameters +#### 4.1.2 Class/structs/data types +The horizontal mesh information will be organized in a class with public YAKL arrays. +Arrays that require a device copy will have a explicit variable. +Connectivity arrays that are already contained in the Decomp class will be replicated in the horizontal mesh class via pointers. +```c++ +class HorzMesh { -#### 4.1.1 Parameters +public: -List and define any configuration parameters or public constants. + Array1DR8 AreaCell; + ArrayHost1DR8 AreaCellH; -#### 4.1.2 Class/structs/data types + Array2DI4 CellsOnCell; + ArrayHost2DI4 CellsOnCellH; -Describe any public data types and/or the class definition +} +``` ### 4.2 Methods -List and describe all public methods and their interfaces (actual code for -interface that would be in header file). Describe typical use cases. +There will be a constructor and destructor for the class with the constructor being responsible for calling several private methods. -## 5 Verification and Testing +#### 4.2.1 Constructor +The constructor will also be responsible for: + * use Decomp object to create reference to the decomposed connectivity arrays. + * reading the other local mesh information. + * computing and dependent mesh quantities. + * creating device copies of mesh information + * registering metadata with the I/O infrastructure + +```c++ +HorzMesh(Decomp decomp); +``` + +#### 4.2.2 Destructor +A destructor will be available to release memory. + +#### 4.2.2 Read +The mesh class requires a method to read in all other available mesh information that has been provided in the mesh file, but has not been initialized by the decomposition. This will be a private method called by the constructor. + +#### 4.2.3 Compute +The compute method will be a private method called by the constructor. It will be resonsible for calculating any dependent mesh information that is not provided in the mesh input file. + +#### 4.2.4 Device copy creation +This method will be repsonsible for creating the device copies of the required mesh information on the host. It will be a private method called by the constructor. -### 5.1 Test xxx +```c++ +AreaCell = AreaCellH.createDeviceCopy() + +``` -Describe test including conditions for pass/fail -List which requirements it tests: - - tests requirement xxx +#### 4.2.5 Metadata registration +The metadata associated with each mesh variable will be registred within the I/O infrastructure in a private method called by the constrctor. + + +## 5 Verification and Testing -### 5.2 Test yyy +### 5.1 Test mesh compute routines -Describe test including conditions for pass/fail -List which requirements it tests: - - tests requirement zzz +The sample domain used for the Decomp test will be used to test obtaining the correct local values and the mesh computation routines. From 89c91740b8fcb370eb306a12fcd058292343f7d1 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Wed, 3 Jan 2024 16:51:22 -0500 Subject: [PATCH 6/7] Rename MeshClass.md -> HorzMeshClass.md --- components/omega/doc/design/{MeshClass.md => HorzMeshClass.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename components/omega/doc/design/{MeshClass.md => HorzMeshClass.md} (100%) diff --git a/components/omega/doc/design/MeshClass.md b/components/omega/doc/design/HorzMeshClass.md similarity index 100% rename from components/omega/doc/design/MeshClass.md rename to components/omega/doc/design/HorzMeshClass.md From c186f4d9d2591227246384cfcf9df74140334247 Mon Sep 17 00:00:00 2001 From: Steven Brus Date: Mon, 22 Jan 2024 10:35:15 -0500 Subject: [PATCH 7/7] Add HorzMeshClass design doc to doc index --- components/omega/doc/design/HorzMeshClass.md | 2 +- components/omega/doc/index.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/components/omega/doc/design/HorzMeshClass.md b/components/omega/doc/design/HorzMeshClass.md index 66f17dc0bd85..5d4433f25eef 100644 --- a/components/omega/doc/design/HorzMeshClass.md +++ b/components/omega/doc/design/HorzMeshClass.md @@ -1,5 +1,5 @@ (omega-design-horz-mesh)= -# Mesh +# Horizontal Mesh ## 1 Overview diff --git a/components/omega/doc/index.md b/components/omega/doc/index.md index fb95431a40bc..e22df7385ec0 100644 --- a/components/omega/doc/index.md +++ b/components/omega/doc/index.md @@ -63,6 +63,7 @@ design/IOStreams design/Reductions design/ShallowWaterOmega0 design/TimeMgr +design/HorzMeshClass design/OmegaDesignTemplate ```