Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making cross-platform wheels #40

Open
rth opened this issue Oct 2, 2019 · 3 comments
Open

Making cross-platform wheels #40

rth opened this issue Oct 2, 2019 · 3 comments
Labels
enhancement New feature or request

Comments

@rth
Copy link

rth commented Oct 2, 2019

As mentioned by @scopatz in #38,

Right now, we need to run conda-press on the target platform. [...] proper toolchain is not available to rewrite the binaries with [otherwise]

and

But then running it with --subdir win-64 on Linux seems to work?

think that is a short term thing because we are not yet stripping binaries for Windows packages (mostly because I don't know how). Once that is added, I wouldn't expect it to work on Linux or Mac anymore (ref #6)

Thanks for the explanations @scopatz !

I think in the use case of making wheels for a package to upload them on PyPi, it would have been really nice if was possible to do that from a single platform. Otherwise having to setup CI would remove part of advantage of conda-press.

Could you mention blockers for that in this issue (assuming that there would be an option to disable stripping of symbols once that's added)?

If I understand correctly one issue is that patchelf doesn't exist on MacOS while install_name_tool doesn't exist on Linux?

In this particular case, some people have managed to use patchelf on MacOS and the source code of install_name_tool can be found here. I wonder if that could be built on Linux (or if there is another alternative that would work with Mach-O files).

@rth rth changed the title Making cross-platform wheels Cross-platform wheels Oct 2, 2019
@rth rth changed the title Cross-platform wheels Making cross-platform wheels Oct 2, 2019
@scopatz
Copy link
Contributor

scopatz commented Oct 2, 2019

Thanks for opening this up @rth! I think there are a couple of big issues with cross-building wheels.

  1. Binary rewriting tools for changing RPATHS for .so, .lib, .dylib, and .dll files are needed (at least on Linux)
  2. There needs to be a way to strip symbols from the above binaries as well.

If you could rely on users having some of these tools on their own platforms, you might be able to get away with running some of these rewrite tools at install time or on-first-use time.

@ihnorton
Copy link
Contributor

ihnorton commented Oct 3, 2019

conda-build has similar needs, and they use LIEF, which is cross-platform, for some binary rewriting tasks.

(although apparently both LIEF and patchelf are buggy in different ways: https://github.com/conda/conda-build/blob/1bbfc0791bbb86cf719ff7a98313904dcd839805/conda_build/post.py#L424)

@scopatz
Copy link
Contributor

scopatz commented Oct 3, 2019

yep, moving to lief is a good step

@marcelotrevisani marcelotrevisani added the enhancement New feature or request label Oct 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants