libjulia 1.5: link LibOSXUnwind dynamically

[fingolfin]

WIP trying to fix the issue on macOS reported here by @barche: on macOS, we get this error:

dyld: lazy symbol binding failed: Symbol not found: __Unwind_Resume
  Referenced from: /Users/user/tmp/clean_julia_depot_cxx/artifacts/1417b691b9d1117225089c656ede8940fbf2eba8/lib/libtypes.dylib
  Expected in: /Applications/Julia-1.5.app/Contents/Resources/julia/bin/../lib/libjulia.dylib

dyld: Symbol not found: __Unwind_Resume
  Referenced from: /Users/user/tmp/clean_julia_depot_cxx/artifacts/1417b691b9d1117225089c656ede8940fbf2eba8/lib/libtypes.dylib
  Expected in: /Applications/Julia-1.5.app/Contents/Resources/julia/bin/../lib/libjulia.dylib

This is because libosxunwind is linked statically but it really needs to be linked dynamically.

[fingolfin]

@benlorenz gave pointed out the crucial bits; here is my summary: apparently before Julia 1.5.1 (so including Julia 1.5.0), libosxunwind was always linked statically. Julia 1.5.1 and later link it dynamically. For that, LibOSXUnwind_jll had to be updated from 0.0.5 to 0.0.6, because in 0.0.5 there was only a fake libosxunwind.dylib; trying to link against that produces an error.

However, @staticfloat rebuilt it in #2038 to add aarch64-apple-darwin support. Except this requires Julia 1.6; yet the version of the JLL was not changed, so this produced 0.0.6+1 which was merged in #2038 and set a compat entry for version 0.0.6 of the JLL, restricting it to Julia >= 1.6!

This would have broken building Julia 1.5, but Julia actually locks the build version of the JLLs it uses, so it kept using 0.0.6+0. Unfortunately, nothing else has the luxury of ignoring the registry and forcing a specific build version, so now we can't build libjulia_jll because we can't install a version of LibOSXUnwind_jll 0.0.6:

ERROR: LoadError: Unsatisfiable requirements detected for package LibOSXUnwind_jll [a83860b7]:
 LibOSXUnwind_jll [a83860b7] log:
 ├─possible versions are: 0.0.5-0.0.6 or uninstalled
 ├─restricted to versions 0.0.6 by an explicit requirement, leaving only versions 0.0.6
 └─restricted by julia compatibility requirements to versions: 0.0.5 or uninstalled — no versions left

I think to fix this, at the very least the julia compat bound of 1.6 for LibOSXUnwind_jll needs to be removed the the general registry. Actually, I think it should be replaced to say "1.5", as we don't want LibOSXUnwind_jll 0.0.6 in Julia 1.3/1.4.

I think the aarch64 binary only causes a warning in Julia 1.5 (not a hard error), but if I am wrong about that, also 0.0.6+1 should be yanked, or better a new 0.0.6+2 be released (so that it still uses JLLWrappers). The aarch64 support could come in a new 0.0.7 release.

[fingolfin]

Hrm:

ERROR: LoadError: Unsatisfiable requirements detected for package LibOSXUnwind_jll [a83860b7]:
 LibOSXUnwind_jll [a83860b7] log:
 ├─possible versions are: 0.0.5-0.0.6 or uninstalled
 ├─restricted to versions 0.0.6 by an explicit requirement, leaving only versions 0.0.6
 └─restricted by julia compatibility requirements to versions: 0.0.5 or uninstalled — no versions left

That's weird -- locally I can install 0.0.6 just fine in both Julia 1.5.3 and 1.6. Why would it fail like that? Could it be looking at an outdated version of the registry?!

[fingolfin]

This still doesn't build here, but it does build using BB on my own server, where it successfully picks up LibOSXUnwind_jll v0.0.6+2.

[fingolfin]

Works now thanks to a hint by @benlorenz that I need to adjust the julia_version from 1.5 (= 1.5.0) to at least 1.5.1. I now changed it to match the exact Julia version we are building for. Not sure if this has drawbacks, though -- @staticfloat ?

[staticfloat]

Why do you need to change the julia_version tag? That shouldn't have any impact on anything. Indeed, the default comparison strategy for julia_version only looks at major and minor for precisely this reason; that julia_version should be used only to tell the difference between v1.5 and v1.6 since the binary compatibility shouldn't change between minor versions.

[fingolfin]

I had to change it so that it would pick up the correct LibOSXUnwind_jll

[fingolfin]

@staticfloat to elaborate: Julia >= 1.5.1 wants to dynamically link against libosxunwind.dylib but that requires LibOSXUnwind_jll 0.0.6 (as 0.0.5 only has a stub dylib that does not actually work) -- specifically, 0.0.6+0 is hardcoded in the Julia build system. But there is also 0.0.6+1 which added an aarch64 variant and a julia_compat >= 1.6 entry and as such is not usable with Julia 1.5, resulting in the JLL refusing to build.

We've since fixed LibOSXUnwind_jll by (a) yanking 0.0.6+1 and (b) adding a 0.0.6+2 which removes the aarch64 binary again, and changes the julia_compat to 1.5.1; the idea being that the aarch64 variant can go into LibOSXUnwind_jll 0.0.7.

However, that still didn't work, because julia_version = 1.5 apparently is treated as "1.5.0" for package resolution purposes, not "latest in the 1.5.x" series.

That leaves the following options (at least that's the ones I can think of), not mutually exclusive:

1. changes libjulia_jll to use julia_version = 1.5.1 or later (that's what this PR does; we use 1.5.3 simply because that's most convenient, as we already have that version "around")
2. change LibOSXUnwind_jll again and change the julia_compat to 1.5.0 or even outright drop it (I guess it's not really needed?) - see PR LibOSXUnwind: drop julia_compat #2199
3. change the way julia_version is handled so that 1.5 is not treated as "must be compatible with 1.5.0" but rather something else (but what? not sure if this option is a sensible option at all).

Oh, and I guess for the aarch64 effort we may need LibOSXUnwind_jll 0.0.7 with aarch64 supported added back in (and then change Julia master to use that?) -- see PR #2200 for a start on that.

[staticfloat]

Ahh, I see the problem. We take the VersionNumber(platform["julia_version"]) tag and then feed that in as julia_version to Pkg.Context(). Because Pkg is going to use that as a fixed version of Julia, we end up in this strange position.

I think it's "okay" to specify "1.5.3" as the julia_version here, but let me reiterate that the purpose of the julia_version platform tag is to allow for a single JLL release to support multiple Julia versions.

In general, artifact platform tags should be used when something about the host determines which binary you want installed, not whether a binary should be installed or not. So the julia_version platform tag is not for enforcing compatibility bounds; that's what julia_compat is for. The julia_version tag is for allowing a single JLL release to support Julia v1.5, v1.6 and v1.7 all in one go.

[barche]

The julia_version tag is for allowing a single JLL release to support Julia v1.5, v1.6 and v1.7 all in one go

This will only work from Julia 1.6 onwards, right? If so, I think it's OK to merge this, as it will allow us to rebuild the libcxxwrap-julia binaries.

[fingolfin]

Yeah, I am aware that julia_version is not intended for enforcing compat bounds. And I do not want to use it for that; rather, I am (or rather: we have been ;-) ) abusing it (yes, I know it's abuse) in libjulia_jll as a trick to ensure we get the correct JLLs. This is abuse bu OK here because we never intend for this to be installed on a regular user machine; it is only for use in BinaryBuilder recipes.

Which is why I think it ought to be OK, here.

Of course in the future, we will want to correctly use julia_version for things like libcxxwrap_julia_jll so that once we require Julia >= 1.6, we can drop the whole mess with the "every releases requires multiple versions, one for each Julia 1.x we support". But we are not yet there.

[staticfloat]

Yep, it all sounds good. So are we agreed on the path forward?

[barche]

OK for me!

[fingolfin]

@staticfloat sounds good to me, but I can only provide the parts, not merge them :-)

[fingolfin]

Oh and to make it explicit: I think "the plan" is to merge this PR as is, and then perhaps also merge PRs #2199 and #2200 (but those are independent from this PR here, really)