Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nixos/nebula: add CAP_NET_BIND_SERVICE when lighthouse node serves DNS #353665

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

nixos/nebula: add CAP_NET_BIND_SERVICE when lighthouse node serves DNS #353665

wants to merge 2 commits into from
+59 −8

Conversation

siriobalmelli
Copy link
Contributor

Serving DNS fails in the absence of CAP_NET_BIND_SERVICE.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Nov 4, 2024
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Nov 5, 2024
@siriobalmelli siriobalmelli force-pushed the sb/nebula-dns branch 2 times, most recently from 4fdd4f0 to 51d1018 Compare November 5, 2024 17:03
@siriobalmelli siriobalmelli marked this pull request as draft November 5, 2024 17:04
@siriobalmelli siriobalmelli marked this pull request as ready for review November 6, 2024 06:54
@siriobalmelli
Copy link
Contributor Author

@numinit I would appreciate your review on this 🙏

numinit
numinit suggested changes Nov 17, 2024
View reviewed changes
nixos/modules/services/networking/nebula.nix Outdated
@@ -210,6 +210,11 @@ in
''
settings
);
capabilities = concatStringsSep " " ([
"CAP_NET_ADMIN"
] ++ (optionals (settings.lighthouse.serve_dns or false) [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition should probably be:

let
  nebulaPort = settings.listen.port;
  dnsPort = if (settings.lighthouse.serve_dns or false) then settings.lighthouse.dns.port or -1 else -1;
in nebulaPort > 0 && nebulaPort < 1024 || dnsPort > 0 && dnsPort < 1024;

Do you mind adding services.nebula.networks.<name>.dns.{enable,port,host} as well? The port should likely default to 5353 so we don't need the extra capability.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should still have this condition. If any port is < 1024 and > 0 we need CAP_NET_ADMIN.

@numinit
Copy link
Contributor

numinit commented Nov 17, 2024

May be worth updating the NixOS test too.

@siriobalmelli siriobalmelli marked this pull request as draft November 19, 2024 11:00
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 10, 2024
@siriobalmelli siriobalmelli force-pushed the sb/nebula-dns branch 3 times, most recently from 2ceb113 to f77e832 Compare December 21, 2024 23:32
@numinit
Copy link
Contributor

numinit commented Dec 21, 2024

Hey, thanks for taking this btw! Let me know if you need any help getting it across the finish line.

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 22, 2024
@siriobalmelli siriobalmelli marked this pull request as ready for review January 6, 2025 09:55
@siriobalmelli
Copy link
Contributor Author

Thanks :)

Got it done and working well on my own nebula network.

A bit stumped about how to extend the tests to cover this however, any help there is appreciated.

@siriobalmelli siriobalmelli force-pushed the sb/nebula-dns branch 2 times, most recently from 73d2f99 to d09287b Compare January 13, 2025 07:10
numinit
numinit suggested changes Jan 13, 2025
View reviewed changes
nixos/modules/services/networking/nebula.nix Outdated
@@ -231,6 +249,10 @@ in
''
settings
);
capabilities = lib.concatStringsSep " " (
(lib.optional (!settings.tun.disabled) "CAP_NET_ADMIN")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can get rid of the parens around (lib.optional ..) - ++ is higher precedence than the function call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, TIL

nixos/modules/services/networking/nebula.nix Outdated

lighthouse.dns.port = lib.mkOption {
type = lib.types.nullOr lib.types.port;
default = 53;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe default this to something like 5353

@numinit
Copy link
Contributor

numinit commented Jan 13, 2025

For the tests, maybe make a dig query to the other hosts in the network that we test. It's mostly optional though.

@siriobalmelli siriobalmelli marked this pull request as draft January 13, 2025 08:58
@siriobalmelli siriobalmelli force-pushed the sb/nebula-dns branch 4 times, most recently from 74887ed to 34707a9 Compare January 13, 2025 12:53
@siriobalmelli siriobalmelli requested a review from numinit January 13, 2025 12:53
@siriobalmelli siriobalmelli marked this pull request as ready for review January 13, 2025 12:53
@siriobalmelli siriobalmelli force-pushed the sb/nebula-dns branch 8 times, most recently from 6edb421 to 40fa11c Compare January 15, 2025 09:04
numinit
numinit suggested changes Jan 20, 2025
View reviewed changes
nixos/modules/services/networking/nebula.nix Outdated

lighthouse.dns.host = lib.mkOption {
type = lib.types.str;
default = "0.0.0.0";
Copy link
Contributor

@numinit numinit Jan 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better default is possibly localhost. This uses dial under the hood so it should work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree localhost is better as a "secure defaults" choice, so the default doesn't listen on public interfaces. The description may want a callout, though, or have an example value of some nebula address like "10.0.0.5".

@numinit
Copy link
Contributor

numinit commented Jan 20, 2025

NixOS tests all pass, looks good except for the above unless @Jaculabilis has any more changes. Thanks for the contribution!

numinit
numinit reviewed Jan 20, 2025
View reviewed changes
nixos/tests/nebula.nix Outdated
@@ -338,6 +349,8 @@ import ./make-test-python.nix (
# allowAny can ping the lighthouse, but not allowFromLighthouse because of its inbound firewall
allowAny.succeed("ping -c3 10.0.100.1")
allowAny.fail("ping -c3 10.0.100.3")
# allowAny can also resolve DNS on lighthouse
allowAny.succeed("dig @10.0.100.1 allowToLighthouse")
Copy link
Contributor

@numinit numinit Jan 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dig returns 0 if there's a NXDOMAIN. Recommend dig @10.0.100.1 allowToLighthouse | grep -E 'allowToLighthouse\.\s+[0-9]+\s+IN\s+A\s+10\.0\.100\.4' (at least, I think that's how it will respond)

@siriobalmelli
Copy link
Contributor Author

Respun with requested changes. Unable to run nixos tests at the moment to verify DNS grep, apologies.

@numinit
Copy link
Contributor

numinit commented Jan 20, 2025

Indeed it works :-)

allowAny: must succeed: dig @10.0.100.1 allowToLighthouse | tee /dev/stderr | grep -E 'allowToLighthouse\.\s+[0-9]+\s+IN\s+A\s+10\.0\.100\.4'
allowAny # 
allowAny # ; <<>> DiG 9.18.28 <<>> @10.0.100.1 allowToLighthouse
allowAny # ; (1 server found)
allowAny # ;; global options: +cmd
allowAny # ;; Got answer:
allowAny # ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41232
allowAny # ;; flags: qr rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
allowAny # ;; WARNING: recursion requested but not available
allowAny # 
allowAny # ;; QUESTION SECTION:
allowAny # ;allowToLighthouse.          IN      A
allowAny # 
allowAny # ;; ANSWER SECTION:
allowAny # allowToLighthouse.   3600    IN      A       10.0.100.4
allowAny # 
allowAny # ;; Query time: 3 msec
allowAny # ;; SERVER: 10.0.100.1#53(10.0.100.1) (UDP)
allowAny # ;; WHEN: Mon Jan 20 07:12:50 UTC 2025
allowAny # ;; MSG SIZE  rcvd: 68
allowAny #

@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Jan 20, 2025
@numinit numinit mentioned this pull request Jan 25, 2025
Open
@numinit
Copy link
Contributor

numinit commented Jan 25, 2025

This looks good and should be fine to merge.

@WillPower3309 WillPower3309 mentioned this pull request Jan 25, 2025
Closed
11 tasks
@numinit numinit mentioned this pull request Jan 25, 2025
Closed
12 tasks
@siriobalmelli
nixos/nebula: add DNS options
610b36e 
Add the options:

- lighthouse.serve_dns
- lighthouse.dns.host
- lighthouse.dns.port

Improve systemd capabilities handling:

- do not give CAP_NET_ADMIN when tunnel interface is disabled
- give CAP_NET_BIND_SERVICE when DNS is enabled

Add self as maintainer: I'm using Nebula on NixOS in prod.

Signed-off-by: Sirio Balmelli <sirio@b-ad.ch>
@siriobalmelli
nixos/nebula: add DNS tests
7a24b07 
Signed-off-by: Sirio Balmelli <sirio@b-ad.ch>
@wegank wegank removed the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Jaculabilis Jaculabilis Jaculabilis left review comments

@numinit numinit numinit approved these changes

Assignees
No one assigned
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@siriobalmelli @numinit @Jaculabilis @wegank