Request to mark pip 24.1.2 as broken.

[tribeiro]

Guidelines for marking packages as broken:

• We prefer to patch the repo data (see here)
  instead of marking packages as broken. This alternative workflow makes environments more reproducible.
• Packages with requirements/metadata that are too strict but otherwise work are
  not technically broken and should not be marked as such.
• Packages with missing metadata can be marked as broken on a temporary basis
  but should be patched in the repo data and be marked unbroken later.
• In some cases where the number of users of a package is small or it is used by
  the maintainers only, we can allow packages to be marked broken more liberally.
• We (conda-forge/core) try to make a decision on these requests within 24 hours.

What will happen when a package is marked broken?

• Our bots will add the broken label to the package. The main label will remain on the package and this is normal.
• Our bots will rebuild our repodata patches to remove this package from the repodata.
• In a few hours after the anaconda.org CDN picks up the new patches, you will no longer be able to install the package from the main channel.

Checklist:

• I want to mark a package as broken (or not broken):
  • Added a description of the problem with the package in the PR description.
  • Pinged the team for the package for their input.

I opened an issue in the package feedstock about the issue we are experiencing with this version of the package (conda-forge/pip-feedstock#124). The change was actually intentional and we are now discussing whether they are going to keep the change or not. I am opening the PR now to make sure it is ready to go if they decide to rollback the change.

[tribeiro]

I think we arrived at a consensus to undo the change for the time being. So I am opening this up again to mark the package as broken.

[jakirkham]

@conda-forge/core , do we want to...?

1. Mark these old packages as broken (done in this PR)
2. Repodata patch the old packages to depend on wheel & setuptools
3. Leave as-is (since newer packages don't have this issue)

[beckermr]

Marking broken is easiest.

[tsibley]

@beckermr wrote:

Marking broken is easiest.

FWIW, marking as broken broke our reproducible environments which incorporated this package build. I think patching the repo data (@jakirkham's option 2) would have avoided that. The guidelines from the PR description support that too:

We prefer to patch the repo data (see here) instead of marking packages as broken. This alternative workflow makes environments more reproducible.

It would have been nicer for us if the choice made was to patch the repo data instead of marking broken, even if marking broken was easier.

In any case, thank you for the work y'all do to keep the Conda and Pip/Python ecosystems up-to-date and moving forward! I know it's a job that's not easy and often thankless. 🙏

[beckermr]

Oh no! We can undo this. Reproducible envs made with tools like conda-lock, explicit conda list exports, and pixi do not have that issue. My appoligies and I will fix this later.

[beckermr]

I'm curious @tsibley about what workflow you use that this broke. If you can send me some links, that'd be helpful.

FWIW, environments exported via conda env export > environment.yml are inherently not reproducible. The reason is that envs made this way have to be resolved using the solver. That process depends on the repodata.json from anaconda.org which is updated every ~10 minutes or so. Thus this process is inherently not reproducible. If you instead use conda-lock, pixi or similar, this PR would leave things unaffected. Those tools directly download the packages without running the solver and so do not depend on the repodata.

[tsibley]

@beckermr Not a big deal, but I wanted to note the breakage. What we're doing is creating meta-packages that strictly pin their dependencies at build/render time, akin to pin_depends: strict I think but implemented differently due to performance constraints with conda at the time (I suspect better now!) that forced us to use boa. We then create new envs with just that meta-package installed, and yes, that uses the solver. But using meta-packages lets us easily version and release these meta-packages and upgrade envs without downloading packages that haven't changed. Most of the time the solver isn't an issue, but yes, this approach can be broken by upstream actions that rewrite history like yanking packages from the index or repo data patches that substantially alter them to cause conflicts (we've seen this before). You can see our build repo at https://github.com/nextstrain/conda-base.

[beckermr]

Right OK. I'd strongly suggest moving away from this workflow as we at conda-forge cannot promise it won't get broken by our actions. For you all, conda-lock would be a suitable, but somewhat different solution.

[tsibley]

Nod. Understood. I figured that would be your suggestion. :-) We'll have to consider the tradeoffs of publishing lock files ourselves instead of meta-packages via Anaconda. (We already use Docker images we produce when we do really want stricter reproducibility.)

[beckermr]

xref: conda-forge/conda-forge-repodata-patches-feedstock#810
xref: #1042

[jakirkham]

Thanks Matt! 🙏