Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimizations for Frontier Streaming template and for KelvinHelmholtz Streaming configs #4790

Merged
merged 2 commits into from
Jan 17, 2024
Merged

Optimizations for Frontier Streaming template and for KelvinHelmholtz Streaming configs #4790

merged 2 commits into from
Jan 17, 2024

Conversation

franzpoeschel
Copy link
Contributor

@franzpoeschel franzpoeschel commented Jan 16, 2024 •
edited
Loading

  1. As seen in the Frontier node diagram, the system has four Slingshot network cards per node. This sets the FABRIC_IFACE environment variable such that each rank uses the card pertaining the the L3 cache group that it was pinned to.
  2. The SST engine of ADIOS2 has some trouble handling lots of metadata at large scale due to MPI collectives restricted to 2GB. As long as metadata is treated collectively (which it is in PIConGPU), we can instruct the dev version of openPMD-api to consider metadata only from some ranks and discard the other. Note that since this is dev, the API is not stable yet.

TODO

  • Test run

@franzpoeschel
Set rank-specific FABRIC_IFACE
c7add0f 
This way, all four CXI devices cxi0 cxi1 cxi2 cxi3 per node are actually
used.
@franzpoeschel
Add attribute_writing_ranks parameter in streaming cfg
7392f3a
@psychocoderHPC psychocoderHPC added component: tools scripts, python libs and CMake machine/system machine & HPC system specific issues labels Jan 17, 2024
@psychocoderHPC psychocoderHPC added this to the 0.8.0 / Next stable milestone Jan 17, 2024
@psychocoderHPC psychocoderHPC merged commit 5100796 into ComputationalRadiationPhysics:dev Jan 17, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@steindev steindev steindev approved these changes

@psychocoderHPC psychocoderHPC psychocoderHPC approved these changes

Assignees
No one assigned
Labels
component: tools scripts, python libs and CMake machine/system machine & HPC system specific issues
Projects
None yet
Milestone
0.8.0
Development

Successfully merging this pull request may close these issues.
3 participants
@franzpoeschel @psychocoderHPC @steindev