Add Nix configuration files

[wch]

This PR adds a Nix configuration file. The basic idea is that it provides a declarative specification of an environment, so that it is easy to create a reproducible build/development environment.

Prerequisites:

• Install Nix (more on that below)

Once you have Nix installed on your system, all you need to do to use the build environment is the following:

nix develop

The first time you run this, it will take a while because it will download a number of packages -- the ones specified in the flake.nix file.

Then you can build as usual, with:

./configure && make

You can exit the development environment with a Ctrl-D or exit.

The flake.nix file specifies what goes into the development environment; the flake.lock file is automatically generated, and ensures that others will get the exact same versions of dependencies. (They are somewhat analogous to package.json and package-lock.json files)

---

Normally when you run nix develop, it will make the specified packages available overlaid on top of the normal stuff you have available in your path.

For example, the flake.nix says to include the curl package. When you are in the development environment, and you type which curl, it will show that the nix copy of curl will be used, instead of the system copy (which is in /usr/bin/curl).

$ which curl
/nix/store/x7km8sc29zkz3p4d44si597ll6yxzyns-curl-8.4.0-bin/bin/curl

This works because Nix modifies the PATH and puts the development environment directories in front of the existing PATH.

However, for programs that aren't specified by the flake.nix, it will fall through to the usual path. For example, flake.nix does not specify rsync, but it can find the system copy from the PATH:

$ which rsync
/usr/bin/rsync

If you want to try a purer development environment, use -i or --ignore-environment. This causes it to ignore/reset all external environment settings. If you do that, it will not find rsync, because /usr/bin won't be in the path.

$ nix develop -i
(nix:nix-shell-env) MBP16:webr winston$ rsync
bash: rsync: command not found

Using nix develop -i is useful for testing if flake.nix truly captures all the dependencies needed to build webR.

---

To install Nix, I used the installer from Determinate Systems. Instructions are here: https://zero-to-nix.com/start/install

The installation is a one-liner:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

---

In the future, we should be able to build in CI using Nix. I've only built on my Mac, and hopefully the Nix config file would also work for Linux.

We could also publish to nixpkgs and flakehub so that other Nix users could have webR available just by adding a webr entry to their Nix config files.

[wch]

I made a few more changes:

• The version of emscripten is locked to 3.1.45 (version found via nixhub).
• The emscripten cache dir now also has the version number as part of its name.
• The startup script prints out the cache dir every time it runs. This is because I got into trouble from having a stale cache. Putting the message right there will remind us that an old cache can cause problems.

I also found out that there's no way to find the top-level directory, where flake.nix lives. I think this will be fixed in the future, but what that means for now is that if you run nix develop from a subdir, the it will create the emscripten cache directory in that subdir. (That's another reason to print out the cache dir at startup, so that there's some indication that the cache dir is in a weird place.)

[wch]

I made some more changes. Notably, the most recent commit does the following:

• Builds the flang-webr package from a nix flake at https://github.com/wch/flang-webr. This involved copying files from tools/flang/ to that repo, with some small modifications, and adding a flake.nix file.
• Turns webr itself into a nix package.

To build, run the following:

nix build github:wch/webr/use-flang-flake --print-build-logs

(This will automatically pull in and build the flang-webr package.)

If you want to run a shell with both of them present, run:

nix shell github:wch/flang-webr/main github:wch/webr/use-flang-flake

You can see both of them in the path if you run echo $PATH. (Note that the flang and webr binaries aren’t actually in the path — they’re in a subdir). Another way of finding their directories is to run nix path-info. Then you can poke around inside those directories and see what files are there.

❯ nix path-info github:wch/webr/use-flang-flake            
/nix/store/7gbiq6avmwyldp767ca3vmbddzlpa435-webr

❯ ls /nix/store/7gbiq6avmwyldp767ca3vmbddzlpa435-webr
dist/

❯ ls /nix/store/7gbiq6avmwyldp767ca3vmbddzlpa435-webr/dist/
index.html                 repl.css                   repl.mjs                   webr-serviceworker.js      webr-worker.js.map
libRblas.so*               repl.css.map               repl.mjs.map               webr-serviceworker.js.map  webr.mjs
libRlapack.so*             repl.html                  vfs/                       webr-worker.js             webr.mjs.map

[georgestagg]

This involved copying files from tools/flang/ to that repo, with some small modifications, and adding a flake.nix file.

Are the modifications you made general, as in could they be merged back into the webR project repo?

I ask because I'd prefer not to have duplication of build scripts, if possible. For example, the Dockerfile in r-wasm/webr-flang clones the main webR repo, rather than duplicating the LLVM flang build scripts there. I feel like it would be better if we could merge your changes to tools/flang back here.

I guess that would only make sense if nix can be told to use a flake.nix file from a subdirectory of a GitHub repo, so that we can tell it to build flang from tools/flang/flake.nix in github:r-wasm/webr/main.

[wch]

Are the modifications you made general, as in could they be merged back into the webR project repo?

I originally started by adding a flang-webr build target in this repo, but then I realized that any changes to this repo would cause flang to rebuild. At least, I think that's what will happen. It's possible there's a way around this, but I don't know Nix well enough to figure it out.

A couple of possibilities I can think of, some better than others:

• Have the flake.nix for webr point to a specific commit of this repository for building flang. It would have to be updated whenever the flang build stuff changes. This would be weird, but it might work.
• Bring in the flang-webr git repo as a git submodule.
• Whenever changes happen here in the tools/flang/ directory, have the flang-webr git repo copy those changes. This could be semi or fully automated.

[wch]

By the way, the only change I made to the flang stuff was this one commit:
wch/flang-webr@b5db98f

Previously, the emfc script was created from emfc.in at build time, and it hard-coded an absolute path. If you were then to move the location of the generated binaries, emfc would no longer work. Nix does just that, and so it didn't work.

I changed it to get the path dynamically, which also meant that the emfc.in -> emfc build step was no longer necessary.

I think that would probably be a good change to make to the original sources in this repository as well.

[georgestagg]

I originally started by adding a flang-webr build target in this repo, but then I realized that any changes to this repo would cause flang to rebuild. At least, I think that's what will happen.

Okay, fair enough. I think we should combine r-wasm/webr-flang and wch/flang-webr into a single repo can build both the Docker container and/or the Nix flake, sharing the build scripts. In the future that repo could then upload the resulting Docker image and the Nix flake to the GH container registry and cachix (or whatever) through CI.

In fact, I'd even like to combine that repo with our LLVM source. That way something like r-wasm/llvm-project would be the authoritative source of truth for our LLVM patches, the Docker container, and the Nix flake. We'd then be able to remove the tools/flang subdirectory in this repo and make it a git submodule pointing to r-wasm/llvm-project instead.

I'm taking a look at LLVM 17.0.5 to see how difficult it would be to rebase our WebAssembly changes onto the latest release, and if it looks doable then experiment with merging in your wch/flang-webr Nix setup and my Dockerfile. If it does not look doable, we'll stick with your wch/flang-webr repo.

---

I changed it to get the path dynamically, which also meant that the emfc.in -> emfc build step was no longer necessary.

Nice! Yes, we should definitely backport that back here.

[georgestagg]

LLVM Flang for WebAssembly is now available as a Nix package at https://github.com/r-wasm/flang-wasm.

Following up on that, I've pushed a bunch of commits here so that this PR can build webR as a Nix package as part of GitHub Actions CI. A short list of changes:

• Reworked how webR builds LLVM flang, with the flang-wasm checked out as a submodule.
• Updated webR's configure and Makefiles to work with the updated LLVM flang v17.0.5
• Minor further tweaks to flake.nix for building webR on Linux.
• Modified the GHA workflows in .github so that webR is built in CI using Nix, as well as Docker.
• Tweak make clean to not delete a local LLVM build. Instead this can be done using make distclean.

TODO: Allow webR to build with sandbox = true in the Nix config. Probably this can be done in a future PR.

[georgestagg]

Updated to nixos-23.11. The separate input entry for Emscripten has been removed.