Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default setting in imas2json takes forever to save #25

Open
eldond opened this issue Jul 25, 2024 · 3 comments
Open

Default setting in imas2json takes forever to save #25

eldond opened this issue Jul 25, 2024 · 3 comments
Assignees

Comments

@eldond
Copy link

eldond commented Jul 25, 2024

https://github.com/ProjectTorreyPines/IMASDD.jl/blob/7784a05b23b99b3d859c1fbc335dd0562049ffe8/src/io.jl#L347

The default of freeze=true with strict=false causes all the expressions to evaluate. A sparse IDS with freeze=false, strict=true can be saved rather quickly. A sparse IDS that tries to evaluate every expression while saving takes forever to save. I recommend changing the default behavior as the case for just dumping what's in the IDS to disk makes more sense. After all, that sparse dataset can be picked back up easily with json2imas and the expressions can then be evaluated.

@eldond
Copy link
Author

eldond commented Jul 25, 2024

BTW it's 5 seconds with strict=true/freeze=false and I killed the job after 5 minutes of it redlining my CPU fan the other way.

@orso82
Copy link
Member

orso82 commented Jul 26, 2024

@eldond I suspect the problem is that freeze was traveling down the grid_ggd data structure. We were already marking anything under grid_ggd as not having expressions, but that was not taken into account by freeze!. I have pushed some fixed. Can you please try again?

@eldond
Copy link
Author

eldond commented Jul 26, 2024

With my settings from before that made it fast:

imas2json timing:  6.416736 seconds (15.98 M allocations: 1.384 GiB, 36.41% gc time)

With default timing (no custom freeze or strict keywords):
killed it after a few minutes of CPU GO BRRRRR

To reproduce:

  1. Check out branch imas in SD4SOLPS
  2. Edit test/runtests.jl to remove freeze and strict keywords in @time IMASDD.imas2json(dd, filename * ".json", strict=true, freeze=false)
  3. Optional to save time: restrict test to the topic of interest with newARGS=["--preparation"]
  4. Run the test, such as with include("test/runtests.jl")

IMASDD is v1.0.1 @ 10d751f
IMAS @ 97ecd4ee65a31f0195ee48c45b22879179ed3d05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants