Consider additional measures for reproducible builds

[bollwyvl]

The new format sounds like it makes great strides for performance. The other end of moving the package landscape forward is taking any possible opportunities to increase the bit-for-bit reproducibility of the resulting artifacts.

A number of the measures in conda/conda-build#2140 were introduced at various stages, but there are still places where strange paths and file stamps get injected. Especially with the enhanced reproducibility of pyc in Python 3.7, this would be a great opportunity to look again at some of these issues.

Keep up the good work!

[msarahan]

Thanks for re-raising these. Some of them can be here, others that deal more with the metadata contents and generation of pyc files will need to be in conda-build.

The collection, for reference, is at https://github.com/conda/conda-build/labels/reproducibility

The ones I think we can do something about here are:

• Strip timestamp and user information from target archive conda-build#2234 (strip timestamp and user info from files)
• Add filename to order function so that filenames with identical filesizes are sorted alphabetically conda-build#2235 (After binsort or filesize sort, sort by filename in case two files are otherwise identical.)

[msarahan]

To be clear, this package has absolutely nothing to do with generating pyc files. I'm trying to keep the separation of concerns stronger. This library is strictly about packing stuff up and unpacking it. Removing timestamp and user info might be admissible, but pyc is definitely more than I want in here. The file sorting is exactly the right kind of thing for this package.

[bollwyvl]

👍 to separation of concerns. Cool label! My point with the pycs is that it was inconvenient to hit predictable builds for many of the python packages before, is easier now, and that this library shouldn't make anything worse.

.conda format should aspirationally (maybe with post-processing) be able to be verifiably reproducible, and eventually tracked and preferred.

Carry on, sir!

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!