Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a new package/project as a 2FA user without using Trusted Publisher #13852

Closed
ronaldoussoren opened this issue Jun 4, 2023 · 5 comments

Comments

@ronaldoussoren
Copy link

What's the problem this feature will solve?

As of recently 2FA users can only upload distributions using API tokens or using Trusted Publisher. API tokens cannot be used to create a new package on PyPI, only upload new distributions for an existing package. Hence Trusted Publisher is the only way to create a new package when using an account that has 2FA enabled.

I have some packages that basically cannot use Trusted Publisher in the way it is intended to be used. In particular the set of packages for the PyObjC project, which need to be compiled and build using a recent compiler and SDK whereas CI support on GitHub is often lagging. Running the full PyObjC testsuite in GitHub CI is also problematic due to the sheer amount of time needed for a complete test run. This is why test runs and package builds are done on a VMs on my own hardware.

Describe the solution you'd like

It would be nice if there were a way to register a new package/project on PyPI using the web interface, after which API tokens can be used to upload.

Additional context

I need to fill in some blanks, but I'm sure that I can work around this issue by way of helper that can use Trusted Publisher to upload a dummy release for any new packages/projects I create. That's just not ideal (IMHO).

And last but not least, I completely agree with the new policy of not allowing using the user password to upload to PyPI.

@ronaldoussoren ronaldoussoren added feature request requires triaging maintainers need to do initial inspection of issue labels Jun 4, 2023
@di
Copy link
Member

di commented Jun 4, 2023

Hi @ronaldoussoren, this is a duplicate of #11296. The way to work around this for you is to make your first release with a user-scoped API token, and then replace it in your release process with a project -scoped API token.

I am curious if you could provide some more details about your release process (e.g., where are the VMs hosted?), it's possible we could eventually support it via Trusted Publishing depending on some factors.

@miketheman miketheman removed the requires triaging maintainers need to do initial inspection of issue label Jun 4, 2023
@ronaldoussoren
Copy link
Author

#11296 indeed looks like the same problem, although that issue's fix seems to add a way to reserve a name for use with Trusted Publisher (if I read it correctly).

BTW. It seems I've misunderstood the restrictions on user tokens, I was pretty sure user scoped tokens couldn't be used to create a project on PyPI. I already use user scoped tokens for PyObjC because last time I checked I couldn't create a scoped token with rights on multiple project (#6292)

I don't think my use case is something that can easily be added to Trusted Publishing. The VM's I'm using are hosted by myself on a Mac mini in my study.

To expand on this: PyObjC provides bindings to system frameworks on macOS, as such it is a macOS-only project and needs to be build on macOS. Some of the bindings are created at runtime (think ctypes, but different), and some are created from extension modules (when the API cannot be described by the system for runtime bindings). For those extension modules, and for exposing all Objective-C protocols, it is necessary to build the release wheels using the latest stable release of Xcode. The Xcode available on public CI systems tends to lag, especially around the release of a new major release of macOS.

Because of this, and for my own convenience to be honest, I've always[*] used a clean VM on my own hardware to test releases and create release artefacts. There's also a set of VMs running older macOS releases to be able to test on those releases, but that's not relevant for this issue.

A full test run for PyObjC on this hardware takes about 8 hours, with extensive tests and all Python versions that still receive updates (currently 3.7 and later). That's way outside what's reasonable to run on public CI. Actually building the release takes a lot less time, and might be acceptable to run on public CI (except for the Xcode version problem).

Long term I might switch to a set up where my local VM is a self-hosted runner for GitHub CI for release builds, although I haven't thought about the security impact of that on my home network yet. That will have to wait until I've switched other, simpler, projects to a more automated release proces.

[*] "always" as in, "from the time that macOS could be run in a VM from Parallels or VMware"

@dstufft
Copy link
Member

dstufft commented Jun 5, 2023

I believe that you're correct it'll likely be difficult to support a VM running on your own hardware for trusted publishers, but a user scoped token should be perfectly capable of creating new projects.

@di
Copy link
Member

di commented Jun 5, 2023

Long term I might switch to a set up where my local VM is a self-hosted runner for GitHub CI for release builds

If you go down this path, my understanding is that the runner will be able to get an OIDC token (with some configuration) and you'll be able to use Trusted Publishers as-is, although I haven't seen this used yet.

@ronaldoussoren
Copy link
Author

Thanks for the feedback.

I'm closing this issue because it is a duplicate of #11296.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants