pyrex.ini: add /tmp to default [run]bind #95
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I observed anomalously high disk I/O on my root filesystem during an OpenEmbedded build which used pyrex. (In fact, *no* rootfs I/O should generally be happening on this machine during an OpenEmbedded build; both the sources and TMPDIR are on other disks, and `/tmp` is a tmpfs.) After some judicious debugging with fatrace(1) I noticed that virtually all of the I/O was happening against, e.g., `/var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o`. It appears that under pyrex, bitbake will use the `/tmp` in the container overlay, not the system's `/tmp`. I believe that this is a mistake: pyrex containers should *always* use `/tmp` directly through a bind mount. My reasoning is as follows. - On systems where `/tmp` is a tmpfs, using it will effect a major performance improvement, and potentially reduce root filesystem wear. - By using a tmpfs instead of the root filesystem, this change cannot introduce any OOM that wouldn't have happened anyway without pyrex. - The use of `/tmp` in recipes must already be robust in the presence of parallel builds; any collision between simultaneous builds under different containers represents a fault in the recipe, not in the bind-mounting. - No security problems can occur through bind-mounting `/tmp` that would not have happened anyway when building outside of pyrex.
|
Ya, I think that makes sense. Please update with an non RFC PR |
JoshuaWatt
changed the title
|
I guess I can do that :) |
JoshuaWatt
approved these changes
Dec 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I observed anomalously high disk I/O on my root filesystem during an OpenEmbedded build which used pyrex. (In fact, no rootfs I/O should generally be happening on this machine during an OpenEmbedded build; both the sources and TMPDIR are on other disks, and
/tmpis a tmpfs.) After some judicious debugging with fatrace(1) I noticed that virtually all of the I/O was happening against, e.g.,/var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o.It appears that under pyrex, bitbake will use the
/tmpin the container overlay, not the system's/tmp. I believe that this is a mistake: pyrex containers should always use/tmpdirectly through a bind mount. My reasoning is as follows./tmpis a tmpfs, using it will effect a major performance improvement, and potentially reduce root filesystem wear./tmpin recipes must already be robust in the presence of parallel builds; any collision between simultaneous builds under different containers represents a fault in the recipe, not in the bind-mounting./tmpthat would not have happened anyway when building outside of pyrex.I tested this change in my own OE build by editing
args, notbind; I'm marking this as an RFC insofar as I have not literally tested this change yet.