JDK 21 toolchain

[sgammon]

Summary

This PR updates dependencies and build process for Pkl to bring GraalVM and Truffle support up to their latest versions. Today, Pkl builds against Java 17+, and requires Java 17+ to run.

After merging this PR, Pkl would build against Java 21, test against Java 21, and impose no change to lib consumers (i.e. still only requiring Java 17+ to run or transitively compile against Pkl).

Test downstream here (updated), via Github Actions. On that fork PR, the change set is expressed in full, with fixups.

PR Tree

• chore: upgrade build tooling → jdk21 pkl-project-commons#6

Rationale

• Gradle Toolchains: Uses Gradle's toolchains feature to provision and use different versions of the JDK.

• JDK 21 Toolchain: Builds (and tests) occur under JDK 21, with bytecode targeting JVM 17. Build bytecode adopts JVM 21 to accelerate build tooling with no change to lib consumers.

• Truffle SVM Dependency: Certain superclasses used by Pkl (notably, AbstractTruffleException and TruffleFeature) have moved to the new org.graalvm.nativeimage:truffle-runtime-svm coordinate.

• Multi-JDK testing: Adds stronger guarantees for backward compatibility with regard to Pkl and JVM execution (i.e. java -jar ... with a fat JAR). Tests against versions 18-23 with test suites and java -jar ... executions.

• Flags and JSR305 compat for kotlinc: Applies -Xjsr305=strict support for Pkl's own annotations, org.pkl.core.util.Nullable and org.pkl.core.util.Nonnull. Java parameter names are retained in Kotlin bytecode, enabling smoother Java/Kotlin interop and debugging.

• Kotlin ABI Check: Checks Kotlin public library symbols for drift. Run apiCheck to make sure a change does not break downstream Java/Kotlin library consumers; run apiDump to re-seal public API lockfiles.

Known Issues

• Failing Tests (pkl-gradle)
• Exports on JVM17: Running against newer Truffle on JVM17 may need --add-exports=...

Pre-merge Checklist

• Refresh Gradle lockfiles and review

Changelog

• feat: support for jvm21+ toolchain
• feat: support for gradle toolchains
• feat: pass -PnativeArch=native to build with -march=native
• test: multi-jdk testing support
• feat: jsr305 compat for kotlinc
• feat: upgrade to latest truffle api, dsl
• feat: kotlin abi checks, initial dump
• fix: remove mrjar classes at >jvm17 from fatjars
• fix: use jdk21 to run the tests (needed for Unsafe.ensureInitialized)
• fix: truffle svm dependency is required after graalvm 24.0.0
• fix: warnings for gvm flag usage, renamed truffle svm macro
• fix: build with --add-modules=jdk.unsupported where needed
• fix: don't use gu tool for modern graalvm versions
• chore: buildinfo changes for JVM targets, toolchains
• chore: bump graalvm/truffle libs → 24.1.2
• chore: update circleci jobs for multi-jdk testing

[bioball]

This is great, thanks, Sam!

I will do a review of this by next week.

FYI: Pkl today supports Java 21. The range enabled here is Java 22+ (updated your PR title).

[StefMa]

@sgammon feel free to extract the toolchain support into another PR. Indeed, my PR is quited dated. I will close it...

[sgammon]

@StefMa roger, no worries :) I am a well known offender in that regard

[bioball]

Thanks for this!

Also, general note: we'll need to set up our CI to use Java 21 to compile (for various compileJava tasks), and ideally to test with all of JDK 17/21/22+. I actually think toolchains probably makes sense for this, so, I'd be okay with the toolchain changes coming back into this PR.

With toolchains, maybe the version of Java used for tasks.test can come from a build flag.

[sgammon]

Initial support for multi-JDK testing. Runs testsuites as well as pkl-cli execution tests across JDK 18-23 (with JDK22 and JDK23 as experimental targets, so failures do not block the build).

This change is isolated to one commit so that it can be dropped or included here easily.

[sgammon]

Rebased against main to pick up #900

[sgammon]

Multi-vendor JDK testing is off by default, but supported, in cases where checking compat across vendors is desirable. It adds test tasks to the check matrix if activated.

By default, multi-JDK tests are run against Azul Zulu for speed. With multi-vendor testing on, they are additionally run with GraalVM CE and Oracle JDKs.

[sgammon]

after #900, the new compilerOptions { ... } Kotlin Gradle Plugin syntax can be used and the old kotlinOptions DSL is deprecated.

• javaParameters: Allows Java reflection downstream on method parameters, and improves debugging by embedding parameter names within Kotlin bytecode

• allWarningsAsErrors: Considers Kotlin compiler warnings as errors. Enabling this had no effect, as the Pkl project already essentially enforces this by convention.

• -Xjsr305=@...:strict: Instructs kotlinc to consider Pkl's nullability annotations as authoritative. Java/Kotlin interop will respect these annotations such that Java types behave as T? if annotated with @Nullable, and T if annotated with @Nonnull. Enabling this had no negative impact.

[sgammon]

Certain methods of AbstractDeque produce warnings, but they are just used here in tests, and the warnings involved describe missed optimizations. So I have suppressed them.

[sgammon]

I've added Kotlin ABI checking so that the Kotlin version change and JDK 21 change can be tested (w.r.t. library use) before release.