Uncomment cmake installs, update jarvis (#47)

* Make it so sqlite is always included

* Make the repo jarvis_coeus

* update the change from master

* remove globavariable.cpp, remove class default

* remove globalvaribale.cpp, fix the class construction

* change the output format, temporary remove the global variable

* change the spdlog and create two spdlog for put and get

* add macro in cmakelist and ifdef method to allow metadata collection

* add support for put and get count

* simpfy the logger code for globalvariable and metadata

* change the formate of spdlog output

* add header for metadata result file

* update the change from master

* remove globavariable.cpp, remove class default

* remove globalvaribale.cpp, fix the class construction

* change the output format, temporary remove the global variable

* change the spdlog and create two spdlog for put and get

* add macro in cmakelist and ifdef method to allow metadata collection

* add support for put and get count

* simpfy the logger code for globalvariable and metadata

* change the formate of spdlog output

* add header for metadata result file

* remove test/src folder to /test, add POSIX I/O support

* add flag for each function

* fix bug when use latest hermes

* fix the error caused by hermes upgrade

* add get put syn support

* add trace_func to each function for coeus adapter

* remove the destructor of Tracer and add std cout method

* add spdlog flush and rank support log

* add more function to trace

* debug process: comment out the endstep, database operation

* change the rank to getpid()

* add cout for doputdeferred method

* change format for cout

* add log file

* comment out metadata part

* comment out delete db

* SHM rank consensus added

* Changes to the CMake for the rank consensus

* Added the new task to the search path for includes of the engine

* Return the value through the task class

* minor change to the initialization of the atomic

* changes to return the ranl properly and delete the task

* changes to return the ranl properly and delete the task

* add rank with consensus

* fix error with double declear variable

* move hermes setup location

* Registering the new task on hermes

* print debug on the task

* more prints

* more prints

* small fix to rank print

* changed task state name

* changed task state name

* printing task id

* printing task id

* changing the init of the task id

adding changes to mdm and mdm constructor prints a message

* decommenting the metadata

* add code for metadata database

* change the db_creation and detach database create and set

* add endstep database support

* remove throw db_file not found

* Debug prints on the mdm

* Debug prints on the mdm

* Gettter for db name

* Gettter for db name

* more prints

* more prints

* More prints and use of Local

* Put is now synchronous

* put back to async

* More debug pritnitng

* more print debugs

* Gettter for db name

* movign to namespace

* adding deserialization

* uncomment db operation in putsync

* uncommented all database operation in put

* add wrf lammps jarvis package

* add more content to lammps and wrf

* Create README.md

* Update README.md

* Update README.md

* Update README.md

* Create README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* fix wrf and lammps jarvis package

* Update README.md

* add post_wrf support for jarvis

* Update README.md

* add post_wrf for support

* add openfoam jarvis

* Create README.md

* Update README.md

* remove test real_app, add new tracer.h, remove trace function in bucket

* Update README.md

* clean the code and remove debug cout

* clean the code and remove debug cout

* add variable name and cout in TRACE_FUNC()

* add information in TRACE_FUNC

* Update README.md

* Fix jarvis

* Add install to cmake

* Uncomment real apps

* Uncomment coeus mdm install

* Uncomment coeus mdm install

* Add hermes libraries

* Use rpath

* Add replacements to jarvis pkg

* Replacements in right file

* ppn 16

* gitignore

* gitignore

* Ignore logs

---------

Co-authored-by: linux <hxu40@hawk.iit.edu>
Co-authored-by: hxu65 <hxu40@hawk.iitedu>
Co-authored-by: unknown <wardhsu7@gmail.com>
Co-authored-by: jaime <jcernudagarcia@hawk.iit.edu>
Co-authored-by: Hua Xu <97185370+hxu65@users.noreply.github.com>

.gitignore

@@ -1,3 +1,4 @@
+cmake*
 .idea
 *.cmd
 *.symvers
@@ -10,11 +11,52 @@
 *.o.cmd
 *.o.d
 build
+
+build/
+benchmarks/HermesVFD
+
+hermes_shm/.git*
+
+GPATH
+GRTAGS
+GTAGS
+
+/compiler_data
+
+.gdb_history
+
+cmake-build-*
+.idea
+.clang-format
+__pycache__/
+/hrun/src/hermes_adapters/posix/cmake-build-debug-system/CMakeFiles/clion-log.txt
+/hermes_adapters/test/posix/Testing/Temporary/LastTest.log
+.cache
+
+logs
+
+wrapper/java/out
+
 Testing
-/build-docker/
-/build/
-/build-ares/
-/build-remote/
-/cmake-build-ares-coverage/
-/build-remote/
-/venv/
+
+.gradle
+**/build/
+!hrun/src/**/build/
+
+# Ignore Gradle GUI config
+gradle-app.setting
+
+# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
+!gradle-wrapper.jar
+
+# Avoid ignore Gradle wrappper properties
+!gradle-wrapper.properties
+
+# Cache of project
+.gradletasknamecache
+
+# Eclipse Gradle plugin generated files
+# Eclipse Core
+.project
+# JDT-specific (Eclipse Java Development Tools)
+.classpath

CMakeLists.txt

@@ -84,6 +84,9 @@ endif()
 #------------------------------------------------------------------------------
 # External libraries
 #------------------------------------------------------------------------------
+# This is for compatability with SPACK
+SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
+
 # Hermes
 find_package(Hermes CONFIG REQUIRED)
 message(STATUS "found Hermes at ${Hermes_DIR}")

include/common/SQlite.h

@@ -55,9 +55,13 @@ class SQLiteWrapper {
   }
 
   void createTables(){
+    std::cout << "MDM: table creation: " << getpid() << " "<< dbName << std::endl;
     createAppsTable();
+    std::cout << "MDM: table APPS: " << getpid() << " " << dbName << std::endl;
     createBlobLocationsTable();
+    std::cout << "MDM: table LOCATION: " << getpid() << " "<< dbName << std::endl;
     createVariableMetadataTable();
+    std::cout << "MDM: table METADATA: " << getpid() << " "<< dbName << std::endl;
   }
 
   ~SQLiteWrapper() {
@@ -83,6 +87,7 @@ class SQLiteWrapper {
   }
 
   void UpdateTotalSteps(const std::string& appName, int step) {
+    std::cout << "MDM: UpdateTotalSteps" << std::endl;
     sqlite3_stmt* stmt;
     const std::string insertOrUpdateSQL = "INSERT OR REPLACE INTO Apps (appName, TotalSteps) VALUES (?, ?);";
     sqlite3_prepare_v2(db, insertOrUpdateSQL.c_str(), -1, &stmt, 0);
@@ -133,6 +138,7 @@ class SQLiteWrapper {
   }
 
   void InsertBlobLocation(int step, int mpi_rank, const std::string& varName, const BlobInfo& blobInfo) {
+    std::cout << "MDM: InsertBlobLocation" << std::endl;
     sqlite3_stmt* stmt;
     const std::string insertOrUpdateSQL = "INSERT OR REPLACE INTO BlobLocations (step, mpi_rank, name, bucket_name, blob_name) VALUES (?, ?, ?, ?, ?);";
     sqlite3_prepare_v2(db, insertOrUpdateSQL.c_str(), -1, &stmt, 0);
@@ -207,6 +213,7 @@ class SQLiteWrapper {
     sqlite3_bind_text(stmt, 8, metadata.dataType.c_str(), -1, SQLITE_STATIC);
     sqlite3_step(stmt);
     sqlite3_finalize(stmt);
+    std::cout << "MDM: InsertVariableMetadata done" << std::endl;
   }
 
   VariableMetadata GetVariableMetadata(int step, int mpi_rank, const std::string& name) {

include/comms/Bucket.h

@@ -23,6 +23,7 @@ class Bucket : public IBucket {
   hapi::Bucket bkt;
 
   Bucket(const std::string &bucket_name, coeus::IHermes *h) {
+
     name = bucket_name;
     TRACE_FUNC(name);
     bkt = h->hermes->GetBucket(bucket_name);
@@ -59,14 +60,17 @@ class Bucket : public IBucket {
   };
 
   std::vector<hermes::BlobId> GetContainedBlobIds() override {
+      TRACE_FUNC();
     return bkt.GetContainedBlobIds();
   }
 
   hermes::BlobId GetBlobId(const std::string &blob_name) override {
+      TRACE_FUNC();
     return bkt.GetBlobId(blob_name);
   }
 
   std::string GetBlobName(const hermes::BlobId &blob_id) override {
+      TRACE_FUNC();
     return bkt.GetBlobName(blob_id);
   }
 };

src/CMakeLists.txt

@@ -9,6 +9,7 @@ set(HERMES_ENGINE_SRCS
 #------------------------------------------------------------------------------
 # Libraries
 #------------------------------------------------------------------------------
+add_definitions(-DMeta_enabled)
 add_library(hermes_engine ${HERMES_ENGINE_SRCS} ../include/coeus/MetadataSerializer.h)
 add_dependencies(hermes_engine spdlog)
 add_dependencies(hermes_engine coeus_mdm)
@@ -32,15 +33,15 @@ target_link_libraries(hermes_engine LINK_PUBLIC
 #-----------------------------------------------------------------------------
 # Add Target(s) to CMake Install
 #-----------------------------------------------------------------------------
-#install(
-#  TARGETS
-#    hermes_engine
-#  EXPORT
-#    ${COEUS_EXPORTED_TARGETS}
-#  LIBRARY DESTINATION ${COEUS_INSTALL_LIB_DIR}
-#  ARCHIVE DESTINATION ${COEUS_INSTALL_LIB_DIR}
-#  RUNTIME DESTINATION ${COEUS_INSTALL_BIN_DIR}
-#)
+install(
+  TARGETS
+    hermes_engine
+  EXPORT
+    ${COEUS_EXPORTED_TARGETS}
+  LIBRARY DESTINATION ${COEUS_INSTALL_LIB_DIR}
+  ARCHIVE DESTINATION ${COEUS_INSTALL_LIB_DIR}
+  RUNTIME DESTINATION ${COEUS_INSTALL_BIN_DIR}
+)
 #
 ##-----------------------------------------------------------------------------
 ## Add Target(s) to CMake Install for import into other projects

src/hermes_engine.cc

@@ -27,7 +27,6 @@ HermesEngine::HermesEngine(adios2::core::IO &io,//NOLINT
                            const adios2::Mode mode,
                            adios2::helper::Comm comm)
     : adios2::plugin::PluginEngineInterface(io, name, mode, comm.Duplicate()) {
-
   Hermes = std::make_shared<coeus::Hermes>();
 //  mpiComm = std::make_shared<coeus::MPI>(comm.Duplicate());
   Init_();
@@ -56,10 +55,7 @@ HermesEngine::HermesEngine(std::shared_ptr<coeus::IHermes> h,
 * Initialize the engine.
 * */
 void HermesEngine::Init_() {
-
-  // Logger setup
-  // Console log
-
+  // initiate the trace manager
   auto console_sink = std::make_shared<spdlog::sinks::stdout_color_sink_mt>();
   console_sink->set_level(spdlog::level::trace);
   console_sink->set_pattern("%^[Coeus engine] [%!:%# @ %s] [%l] %$ %v");
@@ -81,7 +77,7 @@ void HermesEngine::Init_() {
   meta_logger_get = std::make_shared<spdlog::logger>(logger2);
   meta_logger_get->info(
       "Name, shape, start, Count, Constant Shape, Time, selectionSize, sizeofVariable, ShapeID, steps, stepstart, blockID");
-      
+
   auto file_sink3 = std::make_shared<spdlog::sinks::basic_file_sink_mt>(
       "logs/metadataCollect_put.txt", true);
   file_sink3->set_level(spdlog::level::trace);
@@ -98,7 +94,6 @@ void HermesEngine::Init_() {
   logger.set_level(spdlog::level::debug);
   engine_logger = std::make_shared<spdlog::logger>(logger);
 
-
   // hermes setup
   if (!Hermes->connect()) {
     engine_logger->warn("Could not connect to Hermes", rank);
@@ -119,7 +114,6 @@ void HermesEngine::Init_() {
   comm_size = m_Comm.Size();
   pid_t processId = getpid();
 
-
   //Identifier, should be the file, but we don't get it
   uid = this->m_IO.m_Name;
 
@@ -167,7 +161,7 @@ void HermesEngine::Init_() {
   open = true;
 
 
-  
+
 }
 
 /**
@@ -195,7 +189,7 @@ adios2::StepStatus HermesEngine::BeginStep(adios2::StepMode mode,
   TRACE_FUNC(std::to_string(currentStep));
 
   IncrementCurrentStep();
- 
+
   if (m_OpenMode == adios2::Mode::Read) {
     if (total_steps == -1) total_steps = db->GetTotalSteps(uid);
 
@@ -303,8 +297,9 @@ void HermesEngine::ApplyElementMinMax(adios2::MinMaxStruct &MinMax,
 
 template<typename T>
 T *HermesEngine::SelectUnion(adios2::PrimitiveStdtypeUnion &u) {
+  TRACE_FUNC();
   return reinterpret_cast<T *>(&u);
-  
+
 
 }
 
@@ -457,4 +452,3 @@ coeus::HermesEngine *EngineCreate(adios2::core::IO &io,//NOLINT
 /** C wrapper to destroy engine */
 void EngineDestroy(coeus::HermesEngine *obj) { delete obj; }
 }
-

tasks/coeus_mdm/include/coeus_mdm/coeus_mdm.h

@@ -90,6 +90,7 @@ class Client : public TaskLibClient {
                             const TaskNode &task_node,
                             const DomainId &domain_id,
                             DbOperation db_op) {
+    std::cout << "MDM: insert fire" << std::endl;
     HRUN_CLIENT->ConstructTask<Mdm_insertTask>(
         task, task_node, domain_id, id_, db_op);
 

tasks/coeus_mdm/src/CMakeLists.txt

@@ -13,26 +13,26 @@ target_include_directories(coeus_mdm PRIVATE
 #------------------------------------------------------------------------------
 # Install Small Message Task Library
 #------------------------------------------------------------------------------
-#install(
-#        TARGETS
-#        coeus_mdm
-#        EXPORT
-#        ${HERMES_EXPORTED_TARGETS}
-#        LIBRARY DESTINATION ${HERMES_INSTALL_LIB_DIR}
-#        ARCHIVE DESTINATION ${HERMES_INSTALL_LIB_DIR}
-#        RUNTIME DESTINATION ${HERMES_INSTALL_BIN_DIR}
-#)
+install(
+        TARGETS
+        coeus_mdm
+        EXPORT
+        ${COEUS_EXPORTED_TARGETS}
+        LIBRARY DESTINATION ${HERMES_INSTALL_LIB_DIR}
+        ARCHIVE DESTINATION ${HERMES_INSTALL_LIB_DIR}
+        RUNTIME DESTINATION ${HERMES_INSTALL_BIN_DIR}
+)
 
 #-----------------------------------------------------------------------------
 # Add Target(s) to CMake Install for import into other projects
 #-----------------------------------------------------------------------------
 #install(
 #        EXPORT
-#        ${HERMES_EXPORTED_TARGETS}
+#        ${COEUS_EXPORTED_TARGETS}
 #        DESTINATION
 #        ${HERMES_INSTALL_DATA_DIR}/cmake/hermes
 #        FILE
-#        ${HERMES_EXPORTED_TARGETS}.cmake
+#        ${COEUS_EXPORTED_TARGETS}.cmake
 #)
 
 #-----------------------------------------------------------------------------
@@ -46,7 +46,7 @@ if(NOT HERMES_EXTERNALLY_CONFIGURED)
             TARGETS
             ${HERMES_EXPORTED_LIBS}
             FILE
-            ${HERMES_EXPORTED_TARGETS}.cmake
+            ${COEUS_EXPORTED_TARGETS}.cmake
     )
 endif()
 

tasks/coeus_mdm/src/coeus_mdm.cc

@@ -41,8 +41,9 @@ class Server : public TaskLib {
 
       db->InsertVariableMetadata(db_op.step, db_op.rank, db_op.metadata);
       db->InsertBlobLocation(db_op.step, db_op.rank, db_op.name, db_op.blobInfo);
-      
+
     } else if (db_op.type == OperationType::UpdateSteps) {
+      std::cout << "MDM: Hermes update MDM: "  << db->getName() << ", step:" << db_op.currentStep <<std::endl;
       db->UpdateTotalSteps(db_op.uid, db_op.currentStep);
 
     }

tasks/rankConsensus/src/CMakeLists.txt

@@ -8,26 +8,26 @@ target_link_libraries(rankConsensus ${Hermes_RUNTIME_LIBRARIES})
 #------------------------------------------------------------------------------
 # Install Small Message Task Library
 #------------------------------------------------------------------------------
-#install(
-#        TARGETS
-#        rankConsensus
-#        EXPORT
-#        ${HERMES_EXPORTED_TARGETS}
-#        LIBRARY DESTINATION ${HERMES_INSTALL_LIB_DIR}
-#        ARCHIVE DESTINATION ${HERMES_INSTALL_LIB_DIR}
-#        RUNTIME DESTINATION ${HERMES_INSTALL_BIN_DIR}
-#)
+install(
+        TARGETS
+        rankConsensus
+        EXPORT
+        ${COEUS_EXPORTED_TARGETS}
+        LIBRARY DESTINATION ${HERMES_INSTALL_LIB_DIR}
+        ARCHIVE DESTINATION ${HERMES_INSTALL_LIB_DIR}
+        RUNTIME DESTINATION ${HERMES_INSTALL_BIN_DIR}
+)
 
 #-----------------------------------------------------------------------------
 # Add Target(s) to CMake Install for import into other projects
 #-----------------------------------------------------------------------------
 #install(
 #        EXPORT
-#        ${HERMES_EXPORTED_TARGETS}
+#        ${COEUS_EXPORTED_TARGETS}
 #        DESTINATION
 #        ${HERMES_INSTALL_DATA_DIR}/cmake/hermes
 #        FILE
-#        ${HERMES_EXPORTED_TARGETS}.cmake
+#        ${COEUS_EXPORTED_TARGETS}.cmake
 #)
 
 #-----------------------------------------------------------------------------
@@ -41,7 +41,7 @@ if(NOT HERMES_EXTERNALLY_CONFIGURED)
             TARGETS
             ${HERMES_EXPORTED_LIBS}
             FILE
-            ${HERMES_EXPORTED_TARGETS}.cmake
+            ${COEUS_EXPORTED_TARGETS}.cmake
     )
 endif()
 

test/jarvis/jarvis_coeus/jarvis_coeus/adios2_gray_scott/pkg.py

@@ -40,7 +40,7 @@ def _configure_menu(self):
                 'name': 'ppn',
                 'msg': 'Processes per node',
                 'type': int,
-                'default': None,
+                'default': 16,
             },
             {
                 'name': 'L',
@@ -173,7 +173,7 @@ def _configure_menu(self):
 
     # jarvis pkg config adios2_gray_scott ppn=20 full_run=true engine=hermes db_path=/mnt/nvme/jcernudagarcia/metadata.db out_file=gs.bp nprocs=1
 
-    def configure(self, **kwargs):
+    def _configure(self, **kwargs):
         """
         Converts the Jarvis configuration to application-specific configuration.
         E.g., OrangeFS produces an orangefs.xml file.
@@ -221,7 +221,13 @@ def configure(self, **kwargs):
                                 self.adios2_xml_path)
         elif self.config['engine'].lower() == 'hermes':
             self.copy_template_file(f'{self.pkg_dir}/config/hermes.xml',
-                                    self.adios2_xml_path)
+                                    self.adios2_xml_path,
+                                    replacements={
+                                        'PPN': self.config['ppn'],
+                                        'VARFILE': self.var_json_path,
+                                        'OPFILE': self.operator_json_path,
+                                        'DBFILE': self.config['db_path'],
+                                    })
             self.copy_template_file(f'{self.pkg_dir}/config/var.yaml',
                                     self.var_json_path)
             self.copy_template_file(f'{self.pkg_dir}/config/operator.yaml',