Unable to add and compile "elixir-nx/ortex" as a dependency

[RudolfVonKrugstein]

Hi,

I am trying to add elixir-nx/ortex to my rust project.
Ortex uses the rust library ort via rustler, and that seems to lead to problems during compilation.
Here is a minimal example how I set it up:

gleam new ortex_test
cd ortex_test
gleam add ortex
gleam build

Then I get his error:

== Compilation error in file lib/ortex/native.ex ==
** (UndefinedFunctionError) function Jason.decode!/1 is undefined (module Jason is not available). Make sure the module name is correct and has been specified in full (or that an alias has been defined)
    Jason.decode!("{\"packages\":[{\"name\":\"adler\",\"version\":\"1.0.2\",\"id\":\"registry+https://github.com/rust-lang/crates.io-index#adler@1.0.2\",\"license\":\"0BSD OR MIT OR Apache-2.0\",\"license_file\":null,\"description\":\"A simple clean-room implementation of the Adler-32 checksum\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"dependencies\":[{\"name\":\"compiler_builtins\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"req\":\"^0.1.2\",\"kind\":null,\"rename\":null,\"optional\":true,\"uses_default_features\":true,\"features\":[],\"target\":null,\"registry\":null},{\"name\":\"rustc-std-workspace-core\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"req\":\"^1.0.0\",\"kind\":null,\"rename\":\"core\",\"optional\":true,\"uses_default_features\":true,\"features\":[],\"target\":null,\"registry\":null},{\"name\":\"criterion\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"req\":\"^0.3.2\",\"kind\":\"dev\",\"rename\":null,\"optional\":false,\"uses_default_features\":true,\"features\":[],\"target\":null,\"registry\":null}],\"targets\":[{\"kind\":[\"lib\"],\"crate_types\":[\"lib\"],\"name\":\"adler\",\"src_path\":\"/home/nathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/adler-1.0.2/src/lib.rs\",\"edition\":\"2015\",\"doc\":true,\"doctest\":true,\"test\":true},{\"kind\":[\"bench\"],\"crate_types\":[\"bin\"],\"name\":\"bench\",\"src_path\":\"/home/nathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/adler-1.0.2/benches/bench.rs\",\"edition\":\"2015\",\"doc\":false,\"doctest\":false,\"test\":false}],\"features\":{\"compiler_builtins\":[\"dep:compiler_builtins\"],\"core\":[\"dep:core\"],\"default\":[\"std\"],\"rustc-dep-of-std\":[\"core\",\"compiler_builtins\"],\"std\":[]},\"manifest_path\":\"/home/nathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/adler-1.0.2/Cargo.toml\",\"metadata\":{\"docs\":{\"rs\":{\"rustdoc-args\":[\"--cfg=docsrs\"]}},\"release\":{\"no-dev-version\":true,\"pre-release-commit-message\":\"Release {{version}}\",\"tag-message\":\"{{version}}\",\"pre-release-replacements\":[{\"file\":\"CHANGELOG.md\",\"replace\":\"## Unreleased\\n\\nNo changes.\\n\\n## [{{version}} - {{date}}](https://github.com/jonas-schievink/adler/releases/tag/v{{version}})\\n\",\"search\":\"## Unreleased\\n\"},{\"file\":\"README.md\",\"replace\":\"adler = \\\"{{version}}\\\"\",\"search\":\"adler = \\\"[a-z0-9\\\\\\\\.-]+\\\"\"},{\"file\":\"src/lib.rs\",\"replace\":\"https://docs.rs/adler/{{version}}\",\"search\":\"https://docs.rs/adler/[a-z0-9\\\\.-]+\"}]}},\"publish\":null,\"authors\":[\"Jonas Schievink <jonasschievink@gmail.com>\"],\"categories\":[\"algorithms\"],\"keywords\":[\"checksum\",\"integrity\",\"hash\",\"adler32\",\"zlib\"],\"readme\":\"README.md\",\"repository\":\"https://github.com/jonas-schievink/adler.git\",\"homepage\":null,\"documentation\":\"https://docs.rs/adler/\",\"edition\":\"2015\",\"links\":null,\"default_run\":null,\"rust_version\":null},{\"name\":\"aho-corasick\",\"version\":\"0.7.20\",\"id\":\"registry+https://github.com/rust-lang/crates.io-index#aho-corasick@0.7.20\",\"license\":\"Unlicense OR MIT\",\"license_file\":null,\"description\":\"Fast multiple substring searching.\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"dependencies\":[{\"name\":\"memchr\",\"source\":\"registry+https://github.com/rust-lang/crates.io-index\",\"req\":\"^2.4.0\",\"kind\":null,\"rename\":null,\"optional\":false,\"uses_default_features\":false,\"features\":[],\"target\":null,\"registry\":null}],\"targets\":[{\"kind\":[\"lib\"],\"crate_types\":[\"lib\"],\"name\":\"aho_corasick\",\"src_path\":\"/home/nathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aho-corasick-0.7.20/src/lib.rs\",\"edition\":\"2018\",\"doc\":true,\"doctest\":true,\"test\":true}],\"features\":{\"default\":[\"std\"],\"std\":[\"memchr/std\"]},\"manifest_path\":\"/home/nathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aho-corasick-0.7.20/Cargo.toml\",\"metadata\":null,\"publish\":null,\"authors\":[\"Andrew Gallant <jamslam@gmail.com>\"],\"categories\":[\"text-processing\"],\"keywords\":[\"string\",\"search\",\"text\",\"aho\",\"multi\"],\"readme\":\"README.md\",\"repository\":\"https://github.com/BurntSushi/aho-corasick\",\"homepage\":\"https://github.com/BurntSushi/aho-corasick\",\"documentation\":null,\"edition\":\"2018\",\"links\":null,\"default_run\":null,\"rust_version\":null},{\"name\":\"autocfg\",\"version\":\"1.1.0\",\"id\":\"registry+https://github.com/rust-lang/crates.i" <> ...)
    (rustler 0.35.1) lib/rustler/compiler/config.ex:63: Rustler.Compiler.Config.build/1
    (rustler 0.35.1) lib/rustler/compiler.ex:8: Rustler.Compiler.compile_crate/3
    lib/ortex/native.ex:11: (module)
error: Shell command failure

There was a problem when running the shell command `mix`.

(When I create an elixir project with mix, add ortex and compile it, it works).

So the error seems to suggest, that when ortex is compiled, jason is not available also it is a dependency of ortex.

Ortex seems to compile its rustler dependecy from this file: https://github.com/elixir-nx/ortex/blob/main/lib/ortex/native.ex
(I have no experience with rustler, just pointing it out as it might be relevant).

I have no Idea why this is happening?
Any help and what I can do or how I should investigate?

Thank you!

[RudolfVonKrugstein]

One other thing (maybe related, maybe not). After I tried this, and than i.E. want to add jason (maybe that helps) I get:

> gleam add jason
  Resolving versions
error: Corrupt manifest.toml

The `manifest.toml` file is corrupt.
Hint: Please run `gleam update` to fix it.

That might be an unrelated bug, I dont know.

[lpil]

Hello! This package is using a very large amount of metaprogramming so I cannot determine what is happening. Jason has been compiled and the bytecode for it has been given to mix.

TRACE command_exec program="elixir" args="-pa /Users/louis/Desktop/thingy/build/dev/erlang/*/ebin -S mix compile --no-deps-check --no-load-deps --no-protocol-consolidation" env=[("MIX_BUILD_PATH", "/Users/louis/Desktop/thingy/build/packages/ortex/_build/prod"), ("MIX_ENV", "prod"), ("MIX_QUIET", "1"), ("TERM", "dumb")] cwd=Some("/Users/louis/Desktop/thingy/build/packages/ortex")

[RudolfVonKrugstein]

Hey, thanks for looking into it! I will try to use the rustler library from ortex directly writing my own bindings.