Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI should run Standard Library Tests in both JVM and Native modes #12123

Open
4e6 opened this issue Jan 23, 2025 · 0 comments
Open

CI should run Standard Library Tests in both JVM and Native modes #12123

4e6 opened this issue Jan 23, 2025 · 0 comments
Assignees
Labels
-build-script Category: build script -ci

Comments

@4e6
Copy link
Contributor

4e6 commented Jan 23, 2025

Description

The engine-checks.yml runs Engine, JVM Tests, and Standard Library Tests jobs with the following steps.

  • Engine
    • sbt buildEngineDistribution <benchmarks>/compile */buildNativeImage enso/verifyGeneratedPackage
    • ENSO_BENCHMARK_TEST_DRY_RUN=True bin/enso --jvm --run test/Benchmarks
    • bin/enso --run test/<tests> __NON_EXISTING_TEST__
    • bin/enso --jvm --run test/<tests> __NON_EXISTING_TEST__
  • JVM Tests
    • sbt buildEngineDistributionNoIndex
    • sbt test
  • Standard Library Tests
    • sbt buildEngineDistribution
    • bin/enso --run test/<tests>

The Standard Library Tests job only runs the tests in JVM mode (assuming they use bin/bash shell script and/or bin/enso.bat file as launchers).

TODO

  • Use ENSO_LAUNCHER support as provided by:
  • Set ENSO_LAUNCHER=native,test in Standard Library Tests jobs
  • Duplicate the Standard LibraryTests jobs to form:
    • Standard Library Tests (JVM)
    • Standard Library Tests (native)
    • Add appropriate bin/enso --jvm --run test<tests> targets to the Standard Library Tests (JVM) job.
    • Skip test/*_Tests that aren't ready for Standard Library Tests (native) yet
    • with Run test/Table_Tests in native image mode #12006
    • we shall be ready to run test/Base_Tests, test/Geo_Tests, test/Table_Tests, test/Image_Tests
    • we know that test/Microsoft_Tests/, test/Snowflake_Tests, test/Tableau_Tests do not run
  • Verify the right execution mode is used:
    • Standard Library Tests (JVM) shall execute all tests
    • Standard Library Tests (native) doesn't execute JavaScript and Python tests
    • Standard Library Tests (JVM) can load any class file dynamically
    • Standard Library Tests (native) contains list of ahead-of-time builtin Java classes and cannot load any new classes dynamically

Efficiency

  • All jobs run buildEngineDistribution step.
  • It is reasonable to move it in a separate step and re-use the built-distribution directory.
  • Set ENSO_LAUNCHER=native,test in such a separate step
  • See Storing and sharing data from a workflow
@4e6 4e6 added -build-script Category: build script -ci labels Jan 23, 2025
@4e6 4e6 self-assigned this Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
-build-script Category: build script -ci
Projects
Status: New
Development

No branches or pull requests

1 participant