Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Direct Output of Xarray DataSets #75

Open
medley56 opened this issue Sep 24, 2024 · 0 comments
Open

Support Direct Output of Xarray DataSets #75

medley56 opened this issue Sep 24, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@medley56
Copy link
Owner

Context

The API of space-packet-parser is relatively low level in that it makes few assumptions about what users will want to do with their data. The output from the packet parsing generator is basically a dict of parameter name keys and ParsedDataItem values. To do anything beyond that requires the user to do extra work.

We want to support easily converting packet files to Xarray DataSets where each parameter is a variable. This has been demonstrated by @greglucas on the IMAP project. Code here:
https://github.com/IMAP-Science-Operations-Center/imap_processing/blob/373f9f382260188d94366007853b673982ba1275/imap_processing/utils.py#L298-L332

This is also discussed in Discussion #73

The function essentially surmises the data types of parameters coming out of the generator and converts them to minimally small numpy data types to be memory efficient.

Implementation

The Xarray dependency should be specified as an extra in pyproject.toml.

The actual conversion function should be upstreamed from the IMAP repo for inclusion here.

  • Add guards around the imports in the module and raise an exception if someone tries to import the xarray abstraction without having the dependency extra installed.
  • Add unit tests to make sure it behaves correctly.
  • Add documentation to demonstrate how to use it and access the data in xarray.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment