pkgs/top-level: make package sets composable

[wolfgangwalther]

Description of changes

The various pkgsXYZ top-level package sets did not pass localSystem / crossSystem options to lower levels, so far. This change propagates the original arguments, i.e. localSystem0 / crossSystem0 to lower levels. Those include the overrides made by an upper package set.

Example: pkgsStatic adds crossSystem = { isStatic = true }. pkgsStatic.pkgsMusl previously lost this setting, because it didn't propagate crossSystem0, yet.

The first commit removes / rewrites some outdated comments, which referred to the cross and i686 package sets. Meanwhile, we have a lot more, so I rewrote those comments to not say anything about specific sets anymore.

The second commit adds some tests to confirm composability of package sets. This helped me coming up with the fixes, I'm not sure whether it should be kept. It takes about 8-9 seconds for me to run. It also helps showing what the remaining commits fix by uncommenting the previously broken tests.

The remaining commits fix those composability issues. This:

• will fix Build failure: pkgsCross.mingwW64.pkgsStatic.stdenv #281596 (@yshui, @wegank)
• could possibly fix pkgsStatic: Pass hostPlatform.gcc attribute #283460 with a more principal approach (@rodarima, @alyssais, @K900)
• supersedes the fix in pkgsStatic: make pkgsMusl a no-op #136549 (@r-burns, @rnhmjoj)

One open question for me is how to deal with pkgsXYZ.pkgsCross. This has practical challenges (#114510), but is also conceptually unsound: In a way pkgsCross expects to reset crossSystem from scratch, because the whole point about this layer is to start with "well known" example configurations. IMHO, it's fine to make modifications on top of that. e.g. pkgsCross.XYZ.pkgsLLVM or pkgsCross.XYZ.pkgsStatic, but it makes little sense to have pkgsLLVM.pkgsCross... or pkgsStatic.pkgsCross.... Probably completely unusable is pkgsCross.XYZ.pkgsCross.ABC.

One idea would be to just restrict pkgsCross to be used at the top-level, for example by adding something like this in each package set, including pkgsCross itself: [...]

That would also disallow pkgsMusl.pkgsCross.XYZ, which I think is about the only remotely reasonable use-case here: "cross compiling from a native musl environment". I'm not sure whether anyone uses that, though, and whether it's worth supporting at all.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[K900]

Eval is broken :(

[K900]

That said, this is generally good stuff and we should land this (though maybe after 24.05?)

[lf-]

I don't have time to review this right now but I just want to say, thank you so much for writing this, and your commit messages are great!

[wolfgangwalther]

Eval is broken :(

I haven't been able to find out why, yet :/

[wolfgangwalther]

Eval is broken :(

I haven't been able to find out why, yet :/

Will change to "draft" until I have the time to figure that out.

[wolfgangwalther]

I rebased on latest master and should have fixed eval along the way.

[wolfgangwalther]

Improved the commit messages, fixed another eval failure and added another commit to do the following, as mentioned above:

One idea would be to just restrict pkgsCross to be used at the top-level

Let's see where ofborg fails this time.

[wolfgangwalther]

Rebased and cherry-picked #314845. This makes ./outpaths.nix evaluate locally for me.

[wolfgangwalther]

eval passes now. Ready for review. Please ignore the first commit (see other comment).

[Ericson2314]

Thank for doing a very thorough job documenting your work, @wolfgangwalther, but I am a little worried about passing this information. it was originally very intentional that I transformed the ugly/bad localSystem and crossSystem to other better arguments things in the process of going from pkgs/top-level/default.nix -> pkgs/top-level/stage.nix.

I think I see why you are doing this, but another way to look at this is that having the other package sets mashed into stage.nix like this is --- while convenient --- and architectural abomination. The package sets are are supposed to be intended, and the proper way to get them is just to call nixpkgs with different arguments. Having some convience example calls is fine, but it was a mistake to ever make this part of the package set. pkgsCross being bad makes it unsurprising to me that supporting it forces further architectural imperfections. It's "virally" the wrong thing.

[wolfgangwalther]

@Ericson2314 I'm not sure what the takeaway from this is. Are you saying that package sets should not be composed? I.e. we should only have (example) pkgsStatic at the top-level, but not pkgsCross.XYZ.pkgsStatic?

Clearly the status-quo, where it's possible to call them, but they only compose in parts, but not in others, is a problem.

[wolfgangwalther]

Overall, I can find 5 different approaches how to do deal with those package sets:

• Get rid of all of them and force users to pass arguments to nixpkgs.
• Get rid of them when nested. pkgsStatic exists, but pkgsCross.XYZ.pkgsStatic is forbidden (either undefined or throwing an error).
• Status quo: Have them, but let them compose badly.
• Change them to always override all previous settings. So pkgsCross.XYZ.pkgsStatic would not be cross anymore, but just the same as pkgsStatic directly.
• Make them compose well. pkgsCross.XYZ.pkgsStatic would do exactly what you'd expect it to do.

Obviously, I'm in favor of the last option.

Once we agree on where this should go, we can look into how to make this nicer architecturally.

[wolfgangwalther]

but I am a little worried about passing this information. it was originally very intentional that I transformed the ugly/bad localSystem and crossSystem to other better arguments things in the process of going from pkgs/top-level/default.nix -> pkgs/top-level/stage.nix.

I have changed the approach from "pass localSystem and crossSystem to stage.nix" to "make nixpkgsFun itself composable". So "dealing with the bad/ugly localSystem/crossSystem" only happens in default.nix now - stage.nix is a lot cleaner than before.

@Ericson2314 Do you like this approach better?

[m-bdf]

pkgsMusl seems broken on NixOS:

$ nix repl github:wolfgangwalther/nixpkgs/compose-pkgs-sets
> system = lib.nixosSystem { system = "x86_64-linux"; modules = []; }
> system.pkgs.pkgsMusl.stdenv.hostPlatform.libc
"glibc"

nixpkgs master prints "musl" as expected.

[wolfgangwalther]

Hmm... I do this and get the expected result:

nix-repl> pkgs = import ./default.nix { }

nix-repl> pkgs.pkgsMusl.stdenv.hostPlatform.libc
"musl"

Not familiar too much with lib.nixosSystem, yet. That's flake-specific, right?

Unfortunately, nix repl github:wolfgangwalther/nixpkgs/compose-pkgs-sets doesn't work with my version of lix. Seems to be a newer nix feature to pass a github reference like that to repl?

[pwaller]

Not familiar too much with lib.nixosSystem, yet. That's flake-specific, right?

It's the same thing as evaluating a nixos system which you can do with nix-instantiate --eval from a nixpkgs checkout; I see the same issue there, which is interesting:

$ nix-instantiate --eval --arg configuration {} ./nixos -A pkgs.pkgsMusl.stdenv.hostPlatform.libc
glibc

[wolfgangwalther]

It's the same thing as evaluating a nixos system which you can do with nix-instantiate --eval from a nixpkgs checkout;

Ah, thanks. I can reproduce it with that, indeed. Really odd. Will look into it on the weekend.

[wolfgangwalther]

This can be reduced down to this:

% nix-instantiate --eval --strict -E 'with import ./. {}; let result = lib.systems.elaborate { config = "x86_64-unknown-linux-musl"; isMusl = false; }; in { inherit (result) config libc; }'
{ config = "x86_64-unknown-linux-musl"; libc = "glibc"; }

Imho, this is an existing bug in lib.systems.elaborate, that we're uncovering here. It doesn't return a valid system, because it doesn't know which of config or isMusl to take as the source of truth.

[wolfgangwalther]

@m-bdf thanks for your feedback, I applied the suggestion for the tests.

pkgsMusl seems broken on NixOS:

This works now:

% nix-instantiate --eval --arg configuration {} ./nixos -A pkgs.pkgsMusl.stdenv.hostPlatform.libc
"musl"

Imho, this is an existing bug in lib.systems.elaborate, that we're uncovering here.

Not much we can do about systems.elaborate, it's not designed to return "correct" systems with arbitrary overrides applied. pkgs/top-level/default.nix takes care to never pass on the elaborated system, but the original localSystem/crossSystem args for that purpose. I added a new commit to make the nixos-nixpkgs module work the same way - passing on the original arguments instead of a fully elaborated system.

[roberth]

stdenv is a derivation, and == on derivations is limited to outPath.
So this would be equivalent and avoid confusion:

Suggested change

	isIdempotent = set: discardEvaluationErrors (pkgs.${set}.stdenv == pkgs.${set}.${set}.stdenv);
	isIdempotent = set: discardEvaluationErrors (pkgs.${set}.stdenv.outPath == pkgs.${set}.${set}.stdenv.outPath);

However, I think you do want to check more:

Suggested change

	isIdempotent = set: discardEvaluationErrors (pkgs.${set}.stdenv == pkgs.${set}.${set}.stdenv);
	isIdempotent = set: discardEvaluationErrors (pkgs.${set}.stdenv.outPath == pkgs.${set}.${set}.stdenv.outPath
	&& pkgs.${set}.stdenv.hostPlatform == pkgs.${set}.${set}.stdenv.hostPlatform
	&& pkgs.${set}.stdenv.buildPlatform == pkgs.${set}.${set}.stdenv.buildPlatform
	);

[wolfgangwalther]

Hm.. according to the test failures I got during development, I assumed the outPath changes when host or build platform change. Is that not correct?

I tried adding the hostPlatform & buildPlatform checks. Instead of ==, I had to use lib.systems.equals. I didn't get any test failures this way. This doesn't mean I don't need those changes, but do you have any example where a change to hostPlatform / buildPlatform would not affect stdenv's outPath?

[wolfgangwalther]

This seems to support the idea that those platforms should affect stdenv's outpath:

nixpkgs/pkgs/stdenv/generic/default.nix

Line 167 in 48d5c5f

inherit buildPlatform hostPlatform targetPlatform;

[wolfgangwalther]

For now, I made the .outPath comparison explicit, but didn't add the platform checks, yet.

Edit: I reverted the change to use outPath, because nixfmt then requires it to be split across two lines awkwardly. This'd be a net-negative for readability.

[roberth]

Extensive use of this makes the test less effective.

[wolfgangwalther]

Without those, I have errors like this quickly:

error: x86_64 Darwin package set can only be used on Darwin systems.

Those are the errors thrown in pkgs/top-level/stage.nix. Any idea how I could target those more specifically - without replicating the logic around those into the tests?

[wolfgangwalther]

I went through the code again and I think discardEvaluationErrors does not necessarily make the test less effective. In fact it makes it more specific. The calls to this function are placed only around conditionals, so the evaluation errors can only come from evaluating those package sets.

These tests are not meant to test the evaluation of each package set, but to test the composability of them, i.e. they are supposed to test the code in stage.nix, not further downstream. If stdenv for a certain package set is broken, this is not the place to test it, imho.