-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
61e5fa5
commit e64940d
Showing
2 changed files
with
228 additions
and
0 deletions.
There are no files selected for viewing
114 changes: 114 additions & 0 deletions
114
test/jarvis/jarvis_coeus/jarvis_coeus/adios2_producer_consumer/config/hermes.xml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| <?xml version="1.0"?> | ||
| <adios-config> | ||
|
|
||
| <!--============================================ | ||
| Configuration for Gray-Scott and GS Plot | ||
| ============================================--> | ||
|
|
||
| <io name="SimulationOutput"> | ||
| <engine type="Plugin"> | ||
| <parameter key="PluginName" value="hermes" /> | ||
| <parameter key="PluginLibrary" value="hermes_engine" /> | ||
|
|
||
| <parameter key="ppn" value='##PPN##'/> | ||
| <parameter key="VarFile" value="##VARFILE##"/> | ||
| <parameter key="OPFile" value="##OPFILE##"/> | ||
| <parameter key="db_file" value="##DBFILE##"/> | ||
| </engine> | ||
| </io> | ||
|
|
||
| <!--=========================================== | ||
| Configuration for PDF calc and PDF Plot | ||
| ===========================================--> | ||
|
|
||
| <io name="PDFAnalysisOutput"> | ||
| <engine type="Plugin"> | ||
| <parameter key="PluginName" value="hermes_out" /> | ||
| <parameter key="PluginLibrary" value="hermes_engine" /> | ||
|
|
||
| <parameter key="ppn" value='##PPN##'/> | ||
| <parameter key="VarFile" value="##VARFILE##"/> | ||
| <parameter key="OPFile" value="##OPFILE##"/> | ||
| <parameter key="db_file" value="##DBFILE##"/> | ||
| </engine> | ||
|
|
||
| <!-- Compress variables --> | ||
| <!-- | ||
| <variable name="U"> | ||
| <operation type="sz"> | ||
| <parameter key="accuracy" value="0.001"/> | ||
| </operation> | ||
| </variable> | ||
| <variable name="V"> | ||
| <operation type="sz"> | ||
| <parameter key="accuracy" value="0.001"/> | ||
| </operation> | ||
| </variable> | ||
| --> | ||
| </io> | ||
|
|
||
| <!-- example engines | ||
| <engine type="FileStream"/> | ||
| <engine type="SST"/> | ||
| <engine type="BP4"/> | ||
| <engine type="BP5"/> | ||
| <engine type="HDF5"/> | ||
| <engine type="File"/> | ||
| === SST === | ||
| SST can be set up to force blocking the producer on a consumer | ||
| or to discard unused data. Separately, it can be also set up | ||
| so that the producer is waiting for a first connection or | ||
| just starts running alone. | ||
| Producer start alone, and it does not keep data. | ||
| Consumer will get recent data when connects. | ||
| If consumer(s) goes away, producer runs alone and | ||
| discards data. | ||
| <engine type="SST"> | ||
| <parameter key="RendezvousReaderCount" value="0"/> | ||
| <parameter key="QueueLimit" value="1"/> | ||
| <parameter key="QueueFullPolicy" value="Discard"/> | ||
| </engine> | ||
| Producer will wait for 1 consumer to connect before proceeding. | ||
| Producer will buffer 5 output steps so a consumer may lag behind a bit. | ||
| If consumer(s) goes away, producer will block indefinitely after | ||
| the buffer is full. | ||
| <engine type="SST"> | ||
| <parameter key="RendezvousReaderCount" value="1"/> | ||
| <parameter key="QueueLimit" value="5"/> | ||
| <parameter key="QueueFullPolicy" value="Block"/> | ||
| </engine> | ||
| === BP4 === | ||
| BP4 is the default ADIOS2 file format. | ||
| 'NumAggregators ' parameter controls how many files | ||
| are created under the output folder. By default, each process | ||
| writes its own file (N-to-N pattern), which is fast but is | ||
| not scalable to tens of thousands of processes. The number of | ||
| substreams should be chosen for the capability of the underlying | ||
| filesystem (e.g. twice the number of OST servers on a Lustre file system). | ||
| 'OpenTimeoutSecs' parameter specifies how long to wait on a reading Open | ||
| for the file to appear. Useful for streaming when the reader starts up | ||
| faster than the writer produce any output. | ||
| <engine type="BP4"> | ||
| <parameter key="NumAggregators " value="4"/> | ||
| <parameter key="OpenTimeoutSecs" value="10.0"/> | ||
| </engine> | ||
| === FileStream === | ||
| FileStream is a token name for a file format that supports reading while writing | ||
| It refers to BP4 currently. Timeout for opening is set to 1 hour by default for readers. | ||
| 'NumAggregators ' parameter is the same as for BP4. | ||
| <engine type="FileStream"> | ||
| <parameter key="NumAggregators " value="4"/> | ||
| </engine> | ||
| === File === | ||
| FileStream is a token name for a file format, useful on reader side to open | ||
| any version of BP or an HDF5 file. | ||
| --> | ||
| </adios-config> | ||
|
|
114 changes: 114 additions & 0 deletions
114
test/jarvis/jarvis_coeus/jarvis_coeus/io_comp/config/hermes.xml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| <?xml version="1.0"?> | ||
| <adios-config> | ||
|
|
||
| <!--============================================ | ||
| Configuration for Gray-Scott and GS Plot | ||
| ============================================--> | ||
|
|
||
| <io name="SimulationOutput"> | ||
| <engine type="Plugin"> | ||
| <parameter key="PluginName" value="hermes" /> | ||
| <parameter key="PluginLibrary" value="hermes_engine" /> | ||
|
|
||
| <parameter key="ppn" value='##PPN##'/> | ||
| <parameter key="VarFile" value="##VARFILE##"/> | ||
| <parameter key="OPFile" value="##OPFILE##"/> | ||
| <parameter key="db_file" value="##DBFILE##"/> | ||
| </engine> | ||
| </io> | ||
|
|
||
| <!--=========================================== | ||
| Configuration for PDF calc and PDF Plot | ||
| ===========================================--> | ||
|
|
||
| <io name="PDFAnalysisOutput"> | ||
| <engine type="Plugin"> | ||
| <parameter key="PluginName" value="hermes_out" /> | ||
| <parameter key="PluginLibrary" value="hermes_engine" /> | ||
|
|
||
| <parameter key="ppn" value='##PPN##'/> | ||
| <parameter key="VarFile" value="##VARFILE##"/> | ||
| <parameter key="OPFile" value="##OPFILE##"/> | ||
| <parameter key="db_file" value="##DBFILE##"/> | ||
| </engine> | ||
|
|
||
| <!-- Compress variables --> | ||
| <!-- | ||
| <variable name="U"> | ||
| <operation type="sz"> | ||
| <parameter key="accuracy" value="0.001"/> | ||
| </operation> | ||
| </variable> | ||
| <variable name="V"> | ||
| <operation type="sz"> | ||
| <parameter key="accuracy" value="0.001"/> | ||
| </operation> | ||
| </variable> | ||
| --> | ||
| </io> | ||
|
|
||
| <!-- example engines | ||
| <engine type="FileStream"/> | ||
| <engine type="SST"/> | ||
| <engine type="BP4"/> | ||
| <engine type="BP5"/> | ||
| <engine type="HDF5"/> | ||
| <engine type="File"/> | ||
| === SST === | ||
| SST can be set up to force blocking the producer on a consumer | ||
| or to discard unused data. Separately, it can be also set up | ||
| so that the producer is waiting for a first connection or | ||
| just starts running alone. | ||
| Producer start alone, and it does not keep data. | ||
| Consumer will get recent data when connects. | ||
| If consumer(s) goes away, producer runs alone and | ||
| discards data. | ||
| <engine type="SST"> | ||
| <parameter key="RendezvousReaderCount" value="0"/> | ||
| <parameter key="QueueLimit" value="1"/> | ||
| <parameter key="QueueFullPolicy" value="Discard"/> | ||
| </engine> | ||
| Producer will wait for 1 consumer to connect before proceeding. | ||
| Producer will buffer 5 output steps so a consumer may lag behind a bit. | ||
| If consumer(s) goes away, producer will block indefinitely after | ||
| the buffer is full. | ||
| <engine type="SST"> | ||
| <parameter key="RendezvousReaderCount" value="1"/> | ||
| <parameter key="QueueLimit" value="5"/> | ||
| <parameter key="QueueFullPolicy" value="Block"/> | ||
| </engine> | ||
| === BP4 === | ||
| BP4 is the default ADIOS2 file format. | ||
| 'NumAggregators ' parameter controls how many files | ||
| are created under the output folder. By default, each process | ||
| writes its own file (N-to-N pattern), which is fast but is | ||
| not scalable to tens of thousands of processes. The number of | ||
| substreams should be chosen for the capability of the underlying | ||
| filesystem (e.g. twice the number of OST servers on a Lustre file system). | ||
| 'OpenTimeoutSecs' parameter specifies how long to wait on a reading Open | ||
| for the file to appear. Useful for streaming when the reader starts up | ||
| faster than the writer produce any output. | ||
| <engine type="BP4"> | ||
| <parameter key="NumAggregators " value="4"/> | ||
| <parameter key="OpenTimeoutSecs" value="10.0"/> | ||
| </engine> | ||
| === FileStream === | ||
| FileStream is a token name for a file format that supports reading while writing | ||
| It refers to BP4 currently. Timeout for opening is set to 1 hour by default for readers. | ||
| 'NumAggregators ' parameter is the same as for BP4. | ||
| <engine type="FileStream"> | ||
| <parameter key="NumAggregators " value="4"/> | ||
| </engine> | ||
| === File === | ||
| FileStream is a token name for a file format, useful on reader side to open | ||
| any version of BP or an HDF5 file. | ||
| --> | ||
| </adios-config> | ||
|
|