Need help resolving node-gyp issue for new package (scrypted)

[omares]

Issue description

I am in the process of creating a package for Scrypted which would also fulfill #220494.

Scrypted uses a Node/npm environment, and I have managed to configure most of it. You can view the progress of the package derivation at my current testing branch not based on nixpkgs.

Scrypted depends on @scrypted/node-pty, which uses node-gyp. When building my derivation, I encounter the following error related to node-gyp:

...
scrypted> npm error code 1
scrypted> npm error path /build/source/server/node_modules/@scrypted/node-pty
scrypted> npm error command failed
scrypted> npm error command sh -c prebuild-install || node-gyp rebuild
scrypted> npm error prebuild-install warn install getaddrinfo EAI_AGAIN github.com
scrypted> npm error gyp info it worked if it ends with ok
scrypted> npm error gyp info using node-gyp@10.1.0
scrypted> npm error gyp info using node@20.18.1 | linux | x64
scrypted> npm error gyp info find Python using Python version 3.12.8 found at "/nix/store/c9m6yd8fg1flz2j5r4bif1ib5j20a0cy-python3-3.12.8/bin/python3"
scrypted> npm error gyp WARN read config.gypi ENOENT: no such file or directory, open '/nix/store/qwrzxkqmsspprgq9s5j8bcr45lkign4k-nodejs-22.11.0-source/include/node/config.gypi'
scrypted> npm error gyp info spawn /nix/store/c9m6yd8fg1flz2j5r4bif1ib5j20a0cy-python3-3.12.8/bin/python3
scrypted> npm error gyp info spawn args [
scrypted> npm error gyp info spawn args '/nix/store/j7dx1n6m5axf9r2bvly580x2ixx546wq-nodejs-20.18.1/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
scrypted> npm error gyp info spawn args 'binding.gyp',
scrypted> npm error gyp info spawn args '-f',
scrypted> npm error gyp info spawn args 'make',
scrypted> npm error gyp info spawn args '-I',
scrypted> npm error gyp info spawn args '/build/source/server/node_modules/@scrypted/node-pty/build/config.gypi',
scrypted> npm error gyp info spawn args '-I',
scrypted> npm error gyp info spawn args '/nix/store/j7dx1n6m5axf9r2bvly580x2ixx546wq-nodejs-20.18.1/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
scrypted> npm error gyp info spawn args '-I',
scrypted> npm error gyp info spawn args '/nix/store/qwrzxkqmsspprgq9s5j8bcr45lkign4k-nodejs-22.11.0-source/common.gypi',
scrypted> npm error gyp info spawn args '-Dlibrary=shared_library',
scrypted> npm error gyp info spawn args '-Dvisibility=default',
scrypted> npm error gyp info spawn args '-Dnode_root_dir=/nix/store/qwrzxkqmsspprgq9s5j8bcr45lkign4k-nodejs-22.11.0-source',
scrypted> npm error gyp info spawn args '-Dnode_gyp_dir=/nix/store/j7dx1n6m5axf9r2bvly580x2ixx546wq-nodejs-20.18.1/lib/node_modules/npm/node_modules/node-gyp',
scrypted> npm error gyp info spawn args '-Dnode_lib_file=/nix/store/qwrzxkqmsspprgq9s5j8bcr45lkign4k-nodejs-22.11.0-source/$(Configuration)/node.lib',
scrypted> npm error gyp info spawn args '-Dmodule_root_dir=/build/source/server/node_modules/@scrypted/node-pty',
scrypted> npm error gyp info spawn args '-Dnode_engine=v8',
scrypted> npm error gyp info spawn args '--depth=.',
scrypted> npm error gyp info spawn args '--no-parallel',
scrypted> npm error gyp info spawn args '--generator-output',
scrypted> npm error gyp info spawn args 'build',
scrypted> npm error gyp info spawn args '-Goutput_dir=.'
scrypted> npm error gyp info spawn args ]
...

The full log output can be found in this gist: https://gist.github.com/omares/3ccb8ff65dc455849dabd11849f51d6a.

Based on my research, I theorize that the node-gyp dependency of @scrypted/node-pty attempts to download header files, which fails due to the enabled sandboxing mode. I tried applying various "patches" I found during my research to resolve the issue, but without success. As information is scarce, and I am still learning, I hope to receive support through this issue request.

Thank you!

Steps to reproduce

1. Checkout https://github.com/omares/nix-homelab/tree/add-scrypted
2. Run nix build .#scrypted --system x86_64-linux -L --show-trace

Technical details

My host machine is a aarch64-darwin system, although my package is built on a x86_64-linux remote builder.

Attached the nix-info output of said builder

root@build-01:~/ > nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.63, NixOS, 25.05 (Warbler), 25.05pre-git`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.24.10`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[aos]

I would also post this on https://discourse.nixos.org (or at least link to it). You'll likely find more help there.

[omares]

i did so, https://discourse.nixos.org/t/need-help-resolving-node-gyp-issue-node-addon-api-not-available/58618, still no luck so far.

[Jacob-P-Busby]

I'm having similar issues trying to create a package for frappe books, an electron accounting app. It appears the intended build behaviour is to fetch some header files from the internet, but it can't due to the nix build environment.

This is the error message I'm given:

  • rebuilding native dependency  name=better-sqlite3 version=9.3.0
  ⨯ cannot execute  cause=exit status 1
                    out=$ prebuild-install || node-gyp rebuild --release
    
    info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
    
                    errorOut=warning You don't appear to have an internet connection. Try the --offline flag to use the cache for registry queries.
    prebuild-install warn install getaddrinfo EAI_AGAIN github.com
    gyp info it worked if it ends with ok
    gyp info using node-gyp@10.2.0
    gyp info using node@22.11.0 | linux | x64
    gyp info find Python using Python version 3.12.8 found at "/nix/store/c9m6yd8fg1flz2j5r4bif1ib5j20a0cy-python3-3.12.8/bin/python3"
    gyp http GET https://electronjs.org/headers/v22.3.27/node-v22.3.27-headers.tar.gz
    gyp http fetch GET https://electronjs.org/headers/v22.3.27/node-v22.3.27-headers.tar.gz attempt 1 failed with EAI_AGAIN
    gyp WARN install got an error, rolling back install
    gyp ERR! configure error 
    gyp ERR! stack FetchError: request to https://electronjs.org/headers/v22.3.27/node-v22.3.27-headers.tar.gz failed, reason: getaddrinfo EAI_AGAIN electronjs.org
    gyp ERR! stack at ClientRequest.<anonymous> (/nix/store/fkyp1bm5gll9adnfcj92snyym524mdrj-nodejs-22.11.0/lib/node_modules/npm/node_modules/node-gyp/node_modules/minipass-fetch/lib/index.js:130:14)
    gyp ERR! stack at ClientRequest.emit (node:events:518:28)
    gyp ERR! stack at emitErrorEvent (node:_http_client:103:11)
    gyp ERR! stack at _destroy (node:_http_client:886:9)
    gyp ERR! stack at onSocketNT (node:_http_client:906:5)
    gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:91:21)
    gyp ERR! System Linux 6.12.8
    gyp ERR! command "/nix/store/fkyp1bm5gll9adnfcj92snyym524mdrj-nodejs-22.11.0/bin/node" "/nix/store/fkyp1bm5gll9adnfcj92snyym524mdrj-nodejs-22.11.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--release"
    gyp ERR! cwd /build/source/node_modules/better-sqlite3
    gyp ERR! node -v v22.11.0
    gyp ERR! node-gyp -v v10.2.0
    gyp ERR! not ok 

I found a GitHub issue on the node-gyp repo here but I'm unsure how that would be implemented in a nix-build environment where the header file and hash may change.

Additionally, the header version seems to differ from the actual nodejs version used, so I'm really confused on where to go from here

[aos]

You can download things from the internet, but you'll have to specify the output hash after you download your inputs. This is known as a fixed-output derivation (this is how our source fetchers like Github, etc. work). Here is a good intro to it: https://bmcgee.ie/posts/2023/02/nix-what-are-fixed-output-derivations-and-why-use-them/

Here's an example recently where Caddy needs to download and build plugins:

nixpkgs/pkgs/by-name/ca/caddy/plugins.nix

Lines 79 to 81 in 90a1f92

	outputHashMode = "recursive";
	outputHash = hash;
	outputHashAlgo = "sha256";

Here's the manual reference for outputHash: https://nix.dev/manual/nix/2.24/language/advanced-attributes#adv-attr-outputHash