Use the openPMD API for the PhaseSpace plugin (formerly libsplash) #3468

franzpoeschel · 2020-12-09T14:19:18Z

See issue #3357.

TODO:

Remove libsplash output. Currently, both versions of output are produced. I'll keep libsplash around for a while, this makes for easier comparison.
File naming? The current plugin encodes the species name in the file name and writes separate files per species (e.g. PhaseSpace_e_all_ypy_100.h5). Since that information is classically encoded in the dataset path in openPMD, I think we could also write everything into one file (e.g. PhaseSpace_400.h5 ). Would require some small refactoring to keep the Series handle open between different calls to the PhaseSpace plugin, much like the openPMD plugin already does. (Which is also necessary to support group-based iteration layout, e.g. for streaming).
EDIT: As discussed below, we'll stick with the old file naming.

Attributes. The PhaseSpace plugin writes some attributes, and for some of them I had to take a guess concerning their meaning.
Some, I just copied verbatim (sim_unit, p_unit, p_min, p_max, dr, dV).
Others, I took a guess on what they mean within openPMD:

movingWindowOffset → becomes gridGlobalOffset. Not too sure about this one, since the gridGlobalOffset is in terms of physical units, while the movingWindowOffset is solely in terms of indices. Alternatively, use a custom attribute.
movingWindowSize: dropped, since it's the same as the dataset size in dimension x
dr_unit → becomes gridUnitSI. I don't know whether the scaling is the same in all dimensions.
dt → becomes dt (iteration attribute)
dt_unit → becomes timeUnitSI

Some support by an actual user of the PhaseSpace plugin would be appreciated.

Also, some openPMD attributes are still left as default, need to fill in those, too.

I'll need to figure out how to express datasets with different physical units and scalings per dimension.

Old:

New:

float     /data/600/dt                                 attr   = 1
string    /data/600/meshes/e_all_ypy/axisLabels        attr   = {"x"}
float     /data/600/meshes/e_all_ypy/dV                attr   = 19.2232
string    /data/600/meshes/e_all_ypy/dataOrder         attr   = "C"
float     /data/600/meshes/e_all_ypy/dr                attr   = 1.06309
string    /data/600/meshes/e_all_ypy/geometry          attr   = "cartesian"
double    /data/600/meshes/e_all_ypy/gridGlobalOffset  attr   = {0, 0}
double    /data/600/meshes/e_all_ypy/gridSpacing       attr   = 1
double    /data/600/meshes/e_all_ypy/gridUnitSI        attr   = 4.16712e-08
float     /data/600/meshes/e_all_ypy/p_max             attr   = 0.00014378
float     /data/600/meshes/e_all_ypy/p_min             attr   = -0.00014378
double    /data/600/meshes/e_all_ypy/p_unit            attr   = 1.89938e-18
double    /data/600/meshes/e_all_ypy/position          attr   = 0
double    /data/600/meshes/e_all_ypy/sim_unit          attr   = 1.53994e+07
float     /data/600/meshes/e_all_ypy/timeOffset        attr   = 0
double    /data/600/meshes/e_all_ypy/unitDimension     attr   = {0, 0, 0, 0, 0, 0, 0}
double    /data/600/meshes/e_all_ypy/unitSI            attr   = 1
double    /data/600/time                               attr   = 0
double    /data/600/timeUnitSI                         attr   = 1.39e-16

Update the Python analysis scripts. EDIT: They seem to be fundamentally working, but I'll need to re-apply any changes there that are done to the attribute layout. Also some todos and further testing. Also, read from backends other than HDF5.
Documentation.
Cleanup

Related: openPMD/openPMD-standard#193. Necessary for expressing this data layout properly in openPMD, merged already in the standard, not yet available in the API.

psychocoderHPC · 2020-12-09T15:49:46Z

For the fule nameingn Can we close the file each time and open the file if required already existing. As you wrote you can call this pr for each species and for different directions. Since each species is an independent plugin instance it is not possible in PIC to figure out which instance mist close the file.

If this would be to complicated we should keep the file naming even if they have redundante information which are already in the file.

franzpoeschel · 2020-12-09T16:01:07Z

If this would be to complicated we should keep the file naming even if they have redundante information which are already in the file.

Then that's probably what we are going to have to do. Not all our backends support a ReadWrite mode, so this would not work in general.

psychocoderHPC · 2020-12-10T12:36:05Z

@PrometheusPi @steindev @wmles @Anton-Le Could you please help and encrypt the meanings of the sim_unit, p_unit, p_min, p_max, dr, dV

I would say dim_unit is to convert the value from the matrix into SI and should be applied to each array cell.
p_min minimum energy and p_max the maximum energy (Y axis of the array)

steindev · 2020-12-10T16:35:01Z

@PrometheusPi @steindev @wmles @Anton-Le Could you please help and encrypt the meanings of the sim_unit, p_unit, p_min, p_max, dr, dV

Phase space information is particle information which combines one component of a particle's momentum vector p = (p_x, p_y, p_z) with one component of its position vector r = (r_x, r_y, r_z) at a particular time.

The phase space is the aggregation of this information for all particles for the chosen momentum and position component.

But the phase space is not a pure scatter plot of (r_i, p_j) for all particles.
Rather it is a histogram, meaning that the individual particle information is binned.
In conclusion, a phase space provides the number of particles in an interval of a momentum and position space in form of a charge density on a grid whose axes are defined by the chosen spatial and momentum directions.

p_min and p_max are the lower and upper bound of the momentum axis, respectively.
These are configured in a simulation's *.cfg by setting the minEnergy and maxEnergy options of the histogram plugin, respectively.

dr is the spacing of the spatial axis in PIConGPU units, i.e. the cell size along the chosen spatial coordinate. Can differ depending on the chosen component.

dV is the volume of a phase space cell, i.e. dV = dp * dr, where dp would be the grid spacing along the momentum axis.

p_unit is the factor to convert momentum values from PIConGPU scaling to SI.

dr_unit is the factor to convert spatial coordinates from PIConGPU scaling to SI. Should be the same for all dimensions.

sim_unit (I have to guess here) is the factor to convert the charge density value of the phase space from PIConGPU scaling to SI (remember the value provided is charge/length/momentum which needs to be converted from PIConGPU to SI).

franzpoeschel · 2020-12-11T11:40:43Z

Thanks for the detailed description!

p_min and p_max are the lower and upper bound of the momentum axis, respectively.
These are configured in a simulation's *.cfg by setting the minEnergy and maxEnergy options of the histogram plugin, respectively.

In combination with the openPMD standard files, this would then mean that we would set: gridGlobalOfset = (<movingWindowOffset>, p_min), where <movingWindowOffset> is given not as an index, but as the point in simulation coordinates. p_max could theoretically be restored by calculating it from the dataset dimensions. Writing custom attributes p_min and p_max additionally to gridGlobalOffset seems good though.

dr is the spacing of the spatial axis in PIConGPU units, i.e. the cell size along the chosen spatial coordinate. Can differ depending on the chosen component.

dV is the volume of a phase space cell, i.e. dV = dp * dr, where dp would be the grid spacing along the momentum axis.

Hence, gridSpacing = (dr, dp).

p_unit is the factor to convert momentum values from PIConGPU scaling to SI.

dr_unit is the factor to convert spatial coordinates from PIConGPU scaling to SI. Should be the same for all dimensions.

I was not talking about the simulation dimensions, but the dimensions of the dataset (i.e. the momentum and the position dimension). p_unit and dr_unit both do roughly what openPMD calls gridUnitSI, but openPMD currently only allows one global value for all dataset dimensions, so we'll have to stick with custom attributes for now and see whether we should extend the standard for this use case. Similar issue with unitDimension.

sim_unit (I have to guess here) is the factor to convert the charge density value of the phase space from PIConGPU scaling to SI (remember the value provided is charge/length/momentum which needs to be converted from PIConGPU to SI).

From the implementation, that looks like you're right:

const float_64 unit = UNIT_CHARGE / UNIT_VOLUME;

I should additionally use custom attributes for windowMoving(Offset|Extent) probably.

steindev · 2020-12-11T12:55:21Z

p_min and p_max are the lower and upper bound of the momentum axis, respectively.
These are configured in a simulation's *.cfg by setting the minEnergy and maxEnergy options of the histogram plugin, respectively.

In combination with the openPMD standard files, this would then mean that we would set: gridGlobalOfset = (<movingWindowOffset>, p_min), where <movingWindowOffset> is given not as an index, but as the point in simulation coordinates.

p_min will usually be negative, while p_max will be positive. I would not offset the momentum values, just put them in the histogram as they are.

p_max could theoretically be restored by calculating it from the dataset dimensions. Writing custom attributes p_min and p_max additionally to gridGlobalOffset seems good though.

This requires to know dp, but currently this is not in the output. But will be with your addition of the value to gridSpacing 👍
However, these values are in the output just for convenience anyways.
They could be retrieved from the data itself.
But the python script for plotting uses these, so leaving them in seems to be the better option to me.

dr is the spacing of the spatial axis in PIConGPU units, i.e. the cell size along the chosen spatial coordinate. Can differ depending on the chosen component.
dV is the volume of a phase space cell, i.e. dV = dp * dr, where dp would be the grid spacing along the momentum axis.

Hence, gridSpacing = (dr, dp).

p_unit is the factor to convert momentum values from PIConGPU scaling to SI.
dr_unit is the factor to convert spatial coordinates from PIConGPU scaling to SI. Should be the same for all dimensions.

I was not talking about the simulation dimensions, but the dimensions of the dataset (i.e. the momentum and the position dimension). p_unit and dr_unit both do roughly what openPMD calls gridUnitSI, but openPMD currently only allows one global value for all dataset dimensions, so we'll have to stick with custom attributes for now and see whether we should extend the standard for this use case. Similar issue with unitDimension.

The upcoming 2.0.0. standard allows gridUnitSI to be an array, just as grid spacing. Something similar applies to gridUnitDimension. I suggest to directly implement what will be conform with the new standard.

sim_unit (I have to guess here) is the factor to convert the charge density value of the phase space from PIConGPU scaling to SI (remember the value provided is charge/length/momentum which needs to be converted from PIConGPU to SI).

From the implementation, that looks like you're right:
const float_64 unit = UNIT_CHARGE / UNIT_VOLUME;

If UNIT_VOLUME is the phase space volume.
But usage of sim_unit in the python plotting script suggest that my guess is correct, too.

I should additionally use custom attributes for windowMoving(Offset|Extent) probably.

I would love to have the original window size in the data.
As the particle distribution develops over time and can spatially spread out, the varying spatial axis scale is irritating when plotting with the current version where the axis extent is defined by the lowest and largest particle position.

The Offset is already in gridGlobalOffset, there is no need to put it again, is there?

franzpoeschel · 2020-12-11T13:08:08Z

p_min and p_max are the lower and upper bound of the momentum axis, respectively.
These are configured in a simulation's *.cfg by setting the minEnergy and maxEnergy options of the histogram plugin, respectively.

In combination with the openPMD standard files, this would then mean that we would set: gridGlobalOfset = (<movingWindowOffset>, p_min), where <movingWindowOffset> is given not as an index, but as the point in simulation coordinates.

p_min will usually be negative, while p_max will be positive. I would not offset the momentum values, just put them in the histogram as they are.

The gridGlobalOffset just states where the origin of the dataset is within the physical domain, so I wouldn't say this would be offsetting the momentum values, but rather just giving the right info for that attribute.

p_max could theoretically be restored by calculating it from the dataset dimensions. Writing custom attributes p_min and p_max additionally to gridGlobalOffset seems good though.

This requires to know dp, but currently this is not in the output. But will be with your addition of the value to gridSpacing +1
However, these values are in the output just for convenience anyways.
They could be retrieved from the data itself.
But the python script for plotting uses these, so leaving them in seems to be the better option to me.

Yep, I'll leave them in then. Although I need to touch those Python scripts anyway ;)

dr is the spacing of the spatial axis in PIConGPU units, i.e. the cell size along the chosen spatial coordinate. Can differ depending on the chosen component.
dV is the volume of a phase space cell, i.e. dV = dp * dr, where dp would be the grid spacing along the momentum axis.

Hence, gridSpacing = (dr, dp).

p_unit is the factor to convert momentum values from PIConGPU scaling to SI.
dr_unit is the factor to convert spatial coordinates from PIConGPU scaling to SI. Should be the same for all dimensions.

I was not talking about the simulation dimensions, but the dimensions of the dataset (i.e. the momentum and the position dimension). p_unit and dr_unit both do roughly what openPMD calls gridUnitSI, but openPMD currently only allows one global value for all dataset dimensions, so we'll have to stick with custom attributes for now and see whether we should extend the standard for this use case. Similar issue with unitDimension.

The upcoming 2.0.0. standard allows gridUnitSI to be an array, just as grid spacing. Something similar applies to gridUnitDimension. I suggest to directly implement what will be conform with the new standard.

Ah, that's good to know. I will need to check whether the openPMD API will let me implement it in that way ahead of time. If it doesn't (which I'm afraid it will), some custom solution will have to do in the meantime.

sim_unit (I have to guess here) is the factor to convert the charge density value of the phase space from PIConGPU scaling to SI (remember the value provided is charge/length/momentum which needs to be converted from PIConGPU to SI).

From the implementation, that looks like you're right:
const float_64 unit = UNIT_CHARGE / UNIT_VOLUME;
If UNIT_VOLUME is the phase space volume.
But usage of sim_unit in the python plotting script suggest that my guess is correct, too.

I should additionally use custom attributes for windowMoving(Offset|Extent) probably.

I would love to have the original window size in the data.

Can you specify what you mean by this? Make a global (non-window) dataset, but only write to those positions that are covered by the window?

As the particle distribution develops over time and can spatially spread out, the varying spatial axis scale is irritating when plotting with the current version where the axis extent is defined by the lowest and largest particle position.

The Offset is already in gridGlobalOffset, there is no need to put it again, is there?

The gridGlobalOffset is within the physical domain, while the movingWindowOffset are just indexes.

steindev · 2020-12-11T19:39:10Z

I would love to have the original window size in the data.

Can you specify what you mean by this? Make a global (non-window) dataset, but only write to those positions that are covered by the window? dcrt

As the particle distribution develops over time and can spatially spread out, the varying spatial axis scale is irritating when plotting with the current version where the axis extent is defined by the lowest and largest particle position.

Just the extent/size of the spatial grid. In order to set the extent of the axis of the spatial component to the original simulation window extent when plotting.

franzpoeschel · 2020-12-14T08:58:20Z

Ah, alright. Yeah that should be no problem.

franzpoeschel · 2020-12-14T13:17:54Z

Just the extent/size of the spatial grid. In order to set the extent of the axis of the spatial component to the original simulation window extent when plotting.

To get back to this again, since I don't have such in-depth knowledge on the internals of PIC:
Currently, the extent of the dataset is computed via:

            const SubGrid<simDim>& subGrid = Environment<simDim>::get().SubGrid();
            const std::uint64_t rGlobalSize = subGrid.getGlobalDomain().size[axis_element.space];
            splash::Dimensions globalPhaseSpace_size(hBuffer.size().x(), rGlobalSize, 1);

If I'm interpreting this correctly, there is no moving window involved in that?
If that's insufficient, I guess it would be the dimensions given via command line parameter -g that you'd like to see?
EDIT: But wouldn't those really just be the other entries in subGrid.getGlobalDomain().size, and they are irrelevant to the PhaseSpace?

franzpoeschel · 2020-12-14T15:01:48Z

Ah, that's good to know. I will need to check whether the openPMD API will let me implement it in that way ahead of time. If it doesn't (which I'm afraid it will), some custom solution will have to do in the meantime.

Update: It will not allow me to do this:

RuntimeError: Unexpected Attribute datatype for 'gridUnitSI'

steindev · 2020-12-17T22:20:10Z

If that's insufficient, I guess it would be the dimensions given via command line parameter -g that you'd like to see?

Yes, together with the gridGlobalOffset this would be the information one usually requires when plotting.

EDIT: But wouldn't those really just be the other entries in subGrid.getGlobalDomain().size, and they are irrelevant to the PhaseSpace?

Now I am not sure what you mean. As far as I remember, information on the simulation grid size has never been part of the information given in phase space output. One could just infer the extent of the phase space's spatial axis from the data, and this extent is given by the positions of the particles. But these will usually not be at the simulation volume boundary. Therefore there was no chance to infer the simulation grid size (along the chosen spatial axis) from the phase space output. Your questions sounds like the simulation grid size along the chosen spatial axis will be in the phase space output anyways when using openPMD-api. If this is the case, this would be all I asked for.

franzpoeschel · 2021-01-04T13:56:41Z

If that's insufficient, I guess it would be the dimensions given via command line parameter -g that you'd like to see?

Yes, together with the gridGlobalOffset this would be the information one usually requires when plotting.

EDIT: But wouldn't those really just be the other entries in subGrid.getGlobalDomain().size, and they are irrelevant to the PhaseSpace?

Now I am not sure what you mean. As far as I remember, information on the simulation grid size has never been part of the information given in phase space output. One could just infer the extent of the phase space's spatial axis from the data, and this extent is given by the positions of the particles. But these will usually not be at the simulation volume boundary. Therefore there was no chance to infer the simulation grid size (along the chosen spatial axis) from the phase space output. Your questions sounds like the simulation grid size along the chosen spatial axis will be in the phase space output anyways when using openPMD-api. If this is the case, this would be all I asked for.

I have now added four new attributes: globalDomainOffset, globalDomainSize, totalDomainOffset and totalDomainSize, as found in SubGrid.hpp. Hopefully this will be sufficient for your purposes?

A current comparison of the attributes written by the old PhaseSpace plugin:

…and the new one:

As stated in the updated documentation on the PhaseSpace plugin, the attributes unitSI and unitDimension are to be ignored.

steindev · 2021-01-11T10:15:54Z

Excellent! I don't understand the difference between the pair globalDomainSize, globalDomainOffset and totalDomainSize, totalDomainOffset, though. It is a bit of a pity, that we cannot give the axis labels for the globalDomainSize, since the axisLabels key clearly must define the axis labels of the actual phase space.
This would help to select the item or the correct axis from the globalDomainSize (and the others) arrays.
Or could we add an out-of-standard key globalDomainAxisLabels? 😅 Sorry for asking for even more now. You do a great job!

Btw. I would do a test simulation with the new plugin this week. Besides the source code from this PR, do I need anything else to run this updated plugin on hemera at HZDR?

franzpoeschel · 2021-01-11T11:15:21Z

I don't understand the difference between the pair globalDomainSize, globalDomainOffset and totalDomainSize, totalDomainOffset, though.

To be fair, I don't know what the difference is either. Hence why I simply forwarded the terms that PIConGPU uses there. If someone with more knowledge on the internals of PIConGPU could quickly check (@psychocoderHPC ?), that would be great.

Or could we add an out-of-standard key globalDomainAxisLabels?

More custom attributes can always be added, sure ;)

Btw. I would do a test simulation with the new plugin this week. Besides the source code from this PR, do I need anything else to run this updated plugin on hemera at HZDR?

Great! You will need openPMD API 0.12.0 or newer, and either ADIOS2 (2.6.0 or newer) or Parallel HDF5. HDF5 is the default output, so I suggest you use that (otherwise you will have to use --e_phaseSpace.ext bp and use the file_ext parameter in the Python scripts).

sbastrakov · 2021-01-11T11:53:54Z

@steindev @franzpoeschel PIConGPU domain notation is explained on this Wiki page.

franzpoeschel · 2021-01-11T12:13:44Z

Thanks for clarifying! I think, I'll leave both naming levels in the output then and link the Wiki page in the documentation?

psychocoderHPC · 2021-01-25T13:00:49Z

@franzpoeschel Do you currently wait for more input from our side? If not I would suggest to clean the pr to bring it into a finale state.

franzpoeschel · 2021-01-25T13:02:38Z

Yep, I'd need some feedback from testers @psychocoderHPC.

@steindev Did you do a test run yet?

psychocoderHPC · 2021-01-25T14:26:54Z

Yep, I'd need some feedback from testers @psychocoderHPC.

@steindev Did you do a test run yet?

@wmles @Anton-Le @PrometheusPi Please help with testing. If there is no feedback until 03.02.2021 we will merge this pr with the risk of breaking the plugin.

franzpoeschel · 2021-01-26T12:20:22Z

This branch removes the Splash output.

psychocoderHPC · 2021-01-26T19:02:14Z

This branch removes the Splash output.

Thanks, we need to push the changes into this PR.

psychocoderHPC · 2021-02-09T19:03:01Z

@franzpoeschel before we merge this PR we need to wait until #3516 is in the dev branch and rebase this PR against dev. Currently, the CI is not activating OpenPMD.

sbastrakov · 2021-02-10T10:43:30Z

@franzpoeschel #3516 is now merged, so no issues there.

Basic dataset dumping via openPMD Fix dataset dimensions const Write some more attributes Build PhaseSpace only if openPMD is available Fix integer signedness Use using Create a new file for each species, enforce file-based iteration layout Update naming once more Make filenames dumped by openPMD identical to the ones formerly dumped by splash. Give splash some legacy filename. Move Python scripts to openPMD: get_iterations() Move Python scripts to openPMD: _get_for_iteration Cleanup python to make CI happy Fix indexing Update attributes naming Formatting Write _global_start and _global_offset Go back to using scalar record components Write some otherwise-defaulted openPMD attributes Update documentation Allow reading from any openPMD backend in the python scripts Python formatting Add (global|total)Domain(Size|Offset) Attributes Write globalDomainAxisLabels Document custom attributes and fix some errors Remove libsplash output from phasespace plugin

franzpoeschel · 2021-02-10T10:44:42Z

Ok, I rebased it

steindev · 2021-02-10T20:21:00Z

Coming back to the error where h5 files are not written with the new plugin. (The one I received earlier.) Do you have a way to check whether this is an issue with the openPMD plugin, @franzpoeschel? The error only occurs when writing phase space output with the new plugin to *.h5 files. It does not occur when writing to *.bp files and it does not occur with the current dev and the old (libsplash based) plugin. To test this, I took the same setup and *.cfg file and compiled once against the current dev branch and once against your branch. The test ran on 2 k80 nodes at hemera, the partition you tried earlier yourself.
You can find and (hopefully) access the setup in /bigdata/hplsim/scratch/steinigk/projects/2021-01_phase-space-plugin-on-openPMD-PR/01_current-filamentation-instability

The stderr output is attached.
stderr.txt

Is there anything else I could try to pinpoint the problem?
@PrometheusPi are you currently writing *.h5 files with the openPMD plugin on hemera or did you switch to ADIOS too?

franzpoeschel · 2021-02-11T09:25:25Z

I'll try to see whether I can reproduce the error with this

steindev · 2021-02-11T09:26:03Z

I am afraid to say, but there are massive differences between the new and the old plugin. I suspected this earlier, and it is the reason why I rerun everything (see comment above) and all of this testing takes so long. Running the exact same simulation, the old and new plugin produce different results in y-px phase space created from all particles (I did not check others). See the comment above for the location of the simulation setup. The setup simulates a relativistic laser pulse impinging on an 8µm thick, thrice pre-ionized, cold Aluminum foil which starts at 12µm in the simulation domain. Ionization modules are not activated (so there are no new particles created during the simulation).

Total number of particles in phase space w/ openPMD PS plugin = 2657180450816.000000

Total number of particles in phase space w/ libsplash PS plugin = 6771643514880.000000

Difference in total number of particles in phase space between plugins = -4114463064064

Looking at the images, the differences concentrate at the beginning of the foil located around 12µm in the domain and the bulk of the foil (=electrons with no transverse momentum located in the region from 15-20µm).

Generally, I expect differences between the simulations, as the foil is initially cold and an instability is driven.
But the differences are too large to be attributed to this.
Especially the missing bulk electrons suggest an error in the new version of the phase space plugin.
Btw, I checked the total number of macro particles, too. It is equal in both simulations.

Is there anything else I can do to help finding the error?

franzpoeschel · 2021-02-11T10:47:02Z

I can reproduce the file accessing issues with the Hemera openPMD module, a self-built version runs fine. I will try to locate the issue more precisely.

EDIT: Self-built openPMD API + HDF5 from Module system crashes as well. Trying completely self-built next.

Thanks for the measurements, I will have a look at them.

franzpoeschel · 2021-02-12T09:13:11Z

Alright let's talk about debugging for a day just to find out I forgot to pass the MPI communicator

At least this should also explain the differences you saw, you only ever saw parts of your data at once

steindev · 2021-02-12T14:16:17Z

@franzpoeschel I think you need to trigger the CI again by git commit --amend; git push -f as the CI failed to download picongpu from github before actually starting compiling.

franzpoeschel · 2021-02-12T17:58:01Z

Thanks for the hint, trying that again

psychocoderHPC · 2021-02-15T07:01:19Z

@steindev feel free to merge this PR after your final review

steindev · 2021-02-17T16:02:33Z

Finally, tests finished. In order to see differences, I used that laser on Aluminum foil simulation described earlier but with 400keV initial electron temperature, made a checkpoint in the middle of the simulation and created phase spaces with the old plugin and the new plugin (ADIOS2 as well as HDF5) one step after restarting from the checkpoint.
Here are the results:

total electron phase space with the old libsplash plugin
difference of the total electron phase written the new openPMD PS plugin w/ h5 output to the old libsplash plugin

Difference in total number of particles in phase space between plugins = 0
Relative difference in total number of particles in phase space between plugins = 0.00e+00
difference of the total electron phase written the new openPMD PS plugin w/ bp output to the old libsplash plugin

Difference in total number of particles in phase space between plugins = 0
Relative difference in total number of particles in phase space between plugins = 0.00e+00
difference of the total electron phase written the new openPMD PS plugin w/ bp output to the new openPMD PS plugin w/ h5 output

Difference in total number of particles in phase space between plugins = 0
Relative difference in total number of particles in phase space between plugins = 0.00e+00

In essence, the difference is on the single particle level between both, new and old plugin as well as new and new plugin.
This leads me to the conclusion, that everything is fine now 😃

psychocoderHPC requested review from Anton-Le, PrometheusPi and steindev December 10, 2020 12:31

psychocoderHPC added the refactoring code change to improve performance or to unify a concept but does not change public API label Dec 10, 2020

psychocoderHPC added this to the 0.6.0 / 1.0.0: Next Stable milestone Dec 10, 2020

psychocoderHPC added the component: plugin in PIConGPU plugin label Dec 10, 2020

psychocoderHPC marked this pull request as draft December 10, 2020 12:36

franzpoeschel force-pushed the topic-phaseSpace-via-openPMD branch from 0eb22e4 to a040766 Compare January 4, 2021 12:55

franzpoeschel force-pushed the topic-phaseSpace-via-openPMD branch 3 times, most recently from eaf0ec9 to e67699d Compare February 9, 2021 15:17

franzpoeschel added 2 commits February 10, 2021 11:44

Fix includes: Include PhaseSpace.hpp is openPMD is available

325032b

franzpoeschel force-pushed the topic-phaseSpace-via-openPMD branch from e67699d to 325032b Compare February 10, 2021 10:44

franzpoeschel changed the title ~~[WIP] Use the openPMD API for the PhaseSpace plugin (formerly libsplash)~~ Use the openPMD API for the PhaseSpace plugin (formerly libsplash) Feb 10, 2021

franzpoeschel marked this pull request as ready for review February 10, 2021 15:22

franzpoeschel changed the title ~~Use the openPMD API for the PhaseSpace plugin (formerly libsplash)~~ [WIP] Use the openPMD API for the PhaseSpace plugin (formerly libsplash) Feb 11, 2021

franzpoeschel marked this pull request as draft February 11, 2021 09:27

franzpoeschel force-pushed the topic-phaseSpace-via-openPMD branch from 74a3610 to 8034b4f Compare February 12, 2021 09:16

Bugfix: Actually pass the MPI communicator lel

65891d3

franzpoeschel force-pushed the topic-phaseSpace-via-openPMD branch from 8034b4f to 65891d3 Compare February 12, 2021 17:57

franzpoeschel marked this pull request as ready for review February 13, 2021 10:51

franzpoeschel changed the title ~~[WIP] Use the openPMD API for the PhaseSpace plugin (formerly libsplash)~~ Use the openPMD API for the PhaseSpace plugin (formerly libsplash) Feb 13, 2021

psychocoderHPC approved these changes Feb 15, 2021

View reviewed changes

steindev merged commit 57a8301 into ComputationalRadiationPhysics:dev Feb 17, 2021

steindev mentioned this pull request Jul 3, 2023

Fix units of phase space plugin and in documentation #4618

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the openPMD API for the PhaseSpace plugin (formerly libsplash) #3468

Use the openPMD API for the PhaseSpace plugin (formerly libsplash) #3468

franzpoeschel commented Dec 9, 2020 •

edited

Loading

psychocoderHPC commented Dec 9, 2020

franzpoeschel commented Dec 9, 2020

psychocoderHPC commented Dec 10, 2020

steindev commented Dec 10, 2020

franzpoeschel commented Dec 11, 2020

steindev commented Dec 11, 2020

franzpoeschel commented Dec 11, 2020

steindev commented Dec 11, 2020

franzpoeschel commented Dec 14, 2020

franzpoeschel commented Dec 14, 2020 •

edited

Loading

franzpoeschel commented Dec 14, 2020

steindev commented Dec 17, 2020 •

edited

Loading

franzpoeschel commented Jan 4, 2021

steindev commented Jan 11, 2021

franzpoeschel commented Jan 11, 2021

sbastrakov commented Jan 11, 2021

franzpoeschel commented Jan 11, 2021

psychocoderHPC commented Jan 25, 2021

franzpoeschel commented Jan 25, 2021

psychocoderHPC commented Jan 25, 2021

franzpoeschel commented Jan 26, 2021

psychocoderHPC commented Jan 26, 2021

psychocoderHPC commented Feb 9, 2021

sbastrakov commented Feb 10, 2021

franzpoeschel commented Feb 10, 2021

steindev commented Feb 10, 2021 •

edited

Loading

franzpoeschel commented Feb 11, 2021

steindev commented Feb 11, 2021 •

edited

Loading

franzpoeschel commented Feb 11, 2021 •

edited

Loading

franzpoeschel commented Feb 12, 2021 •

edited

Loading

steindev commented Feb 12, 2021

franzpoeschel commented Feb 12, 2021

psychocoderHPC commented Feb 15, 2021

steindev commented Feb 17, 2021

Use the openPMD API for the PhaseSpace plugin (formerly libsplash) #3468

Use the openPMD API for the PhaseSpace plugin (formerly libsplash) #3468

Conversation

franzpoeschel commented Dec 9, 2020 • edited Loading

psychocoderHPC commented Dec 9, 2020

franzpoeschel commented Dec 9, 2020

psychocoderHPC commented Dec 10, 2020

steindev commented Dec 10, 2020

franzpoeschel commented Dec 11, 2020

steindev commented Dec 11, 2020

franzpoeschel commented Dec 11, 2020

steindev commented Dec 11, 2020

franzpoeschel commented Dec 14, 2020

franzpoeschel commented Dec 14, 2020 • edited Loading

franzpoeschel commented Dec 14, 2020

steindev commented Dec 17, 2020 • edited Loading

franzpoeschel commented Jan 4, 2021

steindev commented Jan 11, 2021

franzpoeschel commented Jan 11, 2021

sbastrakov commented Jan 11, 2021

franzpoeschel commented Jan 11, 2021

psychocoderHPC commented Jan 25, 2021

franzpoeschel commented Jan 25, 2021

psychocoderHPC commented Jan 25, 2021

franzpoeschel commented Jan 26, 2021

psychocoderHPC commented Jan 26, 2021

psychocoderHPC commented Feb 9, 2021

sbastrakov commented Feb 10, 2021

franzpoeschel commented Feb 10, 2021

steindev commented Feb 10, 2021 • edited Loading

franzpoeschel commented Feb 11, 2021

steindev commented Feb 11, 2021 • edited Loading

franzpoeschel commented Feb 11, 2021 • edited Loading

franzpoeschel commented Feb 12, 2021 • edited Loading

steindev commented Feb 12, 2021

franzpoeschel commented Feb 12, 2021

psychocoderHPC commented Feb 15, 2021

steindev commented Feb 17, 2021

franzpoeschel commented Dec 9, 2020 •

edited

Loading

franzpoeschel commented Dec 14, 2020 •

edited

Loading

steindev commented Dec 17, 2020 •

edited

Loading

steindev commented Feb 10, 2021 •

edited

Loading

steindev commented Feb 11, 2021 •

edited

Loading

franzpoeschel commented Feb 11, 2021 •

edited

Loading

franzpoeschel commented Feb 12, 2021 •

edited

Loading