Private + Public Netbird Reverse-Proxy in same Cluster fail DNS resolution

Describe the problem

For 3 month I’m trying to set up public + private Netbird Reverse-Proxies in a Netbird Reverse-Proxy-Cluster but can’t get DNS resolution to the backends working.

  1. Host “vm1560”
    1. Debian 13 vServer in data-center
    2. Public static IPv4 + IPv6 address
    3. Netbird Management-Server with Netbird Reverse-Proxy (single stack install)
    4. Some services shall be private inside Netbird. Some services shall be public.
    5. NB_PROXY_PRIVATE=true
    6. Netbird client on the host itself for maintenance (SSH, exit node usage later)
  2. Host “docker”
    1. Debian 13 Proxmox VE VM in LAN
    2. Public dynamic IPv6 address + Public dynamic IPv4 address with NAT
    3. All Docker services in a common Docker network “docker1” including Netbird Reverse-Proxy + Netbird Client
    4. NB_PROXY_PRIVATE=false
    5. Netbird client on the host itself for maintenance (SSH)

In Netbird Management-Server I added a Network “Docker1” with a ressource for each Domain “*.docker1”, Docker IPv4 subnet, Docker IPv6 subnet. Resolving the docker1-FQDNs fails because the network ressource seems to have a priority of -100 in the DNS resolution order compared with the Netbird Primary Nameserver (Netbird Primary Forwarder would be the technically correct term). So I added a Netbird Nameserver (aka forwarder) matching domain “docker1”. I tried “127.0.0.11” (internal Docker network resolver) or the IP-address of the Netbird client in the “docker1” Docker-network as Netbird Nameserver (=resolver) IP adress. Both failed to resolve.

I also added Netbird Network Ressources per docker1 backend FQDN. No improvement. With “direct_upstream=1” the LAN Netbird Reverse-Proxy can resolve the docker1 backend-services because of it’s direct docker1 network access, but the vm1560 Netbird Reverse-Proxy can’t, of course.

For split-brain DNS I have also set up a Netbird DNS zone with the same domain name as for the Netbird Reverse-Proxy to overwrite the Netbird Management and Netbird Reverse-Proxy IP-address in the LAN via a group “Internal” not served to vm1560.

Expected behavior

Netbird Reverse-Proxies shall be able to resolve and route docker1 hostnames of backend.

Are you using NetBird Cloud?

self-hosted

NetBird version

0.70.0 - 0.75.1

Is any other VPN software installed?

No. Netbird shall replace all of them.

Debug output

To help us resolve the problem, please attach the following anonymized status output

renne@aspire:~$ netbird status -dA
Peers detail:
proxy-d9kr4p3r5ksc73bj3chg-3-51.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.3.51
NetBird IPv6: 2001:db8:ffff::
Public key: Aoec7O74jOo7L7bblvWA/Snxs3+wGZGTPIan4/rgclw=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/prflx
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.64:56781
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 47 seconds ago
Last WireGuard handshake: 1 minute, 3 seconds ago
Transfer status (received/sent) 6.8 KiB/53.5 KiB
Quantum resistance: false
Networks: -
Latency: 734.941µs

homeassistant.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.22.70
NetBird IPv6: 2001:db8:ffff::1
Public key: 5i7RyOimsX7rCKEQtFLXGdJ6YCcidKQJbxaMQQjY/hI=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

volta.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.61.221
NetBird IPv6: 2001:db8:ffff::2
Public key: +cx9mf4HmfIxTaGMPJRpf/FoC+8bSSwOIt9No7N4ymk=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

router-docker1.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.101.175
NetBird IPv6: 2001:db8:ffff::3
Public key: oKpFvTnznPHJpYvkoEbICfPRDc6c2Ru8XuGGWI8tJ3M=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/prflx
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.64:51821
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 47 seconds ago
Last WireGuard handshake: 1 minute, 23 seconds ago
Transfer status (received/sent) 4.6 MiB/2.2 MiB
Quantum resistance: false
Networks: 198.51.100.0/24, 2001:db8:ffff::4/64, anon-NHhjA.domain, anon-PN4q0.domain, anon-WM0J1.domain, anon-jR9s2.domain, anon-lXOBq.domain, anon-m9vUd.domain, anon-nQnbk.domain, anon-nvYfq.domain, anon-r64Sm.domain, anon-s7aXv.domain, anon-zzCgE.domain
Latency: 1.53931ms

blackwell.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.102.167
NetBird IPv6: 2001:db8:ffff::5
Public key: hHVC3pzzR0cxWCKg0EhVmUKmk5NxDCspYwEqOhb+DEk=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

vm1560.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.129.207
NetBird IPv6: 2001:db8:ffff::6
Public key: yaGpzK7iNWgRqXjmJhfBSVB2kljX/TdR6c/7RGBcPk0=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/host
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/198.51.100.1:51821
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 47 seconds ago
Last WireGuard handshake: 2 minutes, 50 seconds ago
Transfer status (received/sent) 2.6 KiB/4.6 KiB
Quantum resistance: false
Networks: -
Latency: 11.617801ms

home-router-1.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.133.237
NetBird IPv6: 2001:db8:ffff::7
Public key: AEDZCTJzq5TY5mH2IGnKoRdgxZMrymVaQ/nYIW2AFXc=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

pbs1.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.149.83
NetBird IPv6: 2001:db8:ffff::8
Public key: wEiSH+xMJFEKWOUXqyQVYL/9/9L1P2NkC2T2xxrQBSk=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/prflx
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.52:51820
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: 1 minute, 59 seconds ago
Transfer status (received/sent) 6.9 KiB/1.9 KiB
Quantum resistance: false
Networks: -
Latency: 1.544931ms

haos-whm.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.157.22
NetBird IPv6: 2001:db8:ffff::9
Public key: zzxff7otexIXcoSNJzC7j0yvLtHsC2GRZZ21OBvLqT0=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/prflx
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.30:51820
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: 2 minutes, 50 seconds ago
Transfer status (received/sent) 2.6 KiB/4.6 KiB
Quantum resistance: false
Networks: -
Latency: 3.14323ms

pve1.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.160.1
NetBird IPv6: 2001:db8:ffff::a
Public key: cKj6hpojptNKGQU3zEhXmN3yVDWnC9ZSVIj7EEdolHw=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/host
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.3:51820
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: 1 minute, 59 seconds ago
Transfer status (received/sent) 1.9 KiB/6.9 KiB
Quantum resistance: false
Networks: -
Latency: 556.218µs

docker.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.195.31
NetBird IPv6: 2001:db8:ffff::b
Public key: wvBwRU8FFIHxoeJ4ikNpTF/HpSaNbIFB/esj2MvyjGA=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/host
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.64:51820
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 47 seconds ago
Last WireGuard handshake: 2 minutes, 50 seconds ago
Transfer status (received/sent) 2.6 KiB/4.6 KiB
Quantum resistance: false
Networks: -
Latency: 663.771µs

hermes.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.223.134
NetBird IPv6: 2001:db8:ffff::c
Public key: BK02IJ6049Us53iRR7Szno45r6pki6DZKSfspi2hphg=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

pve2.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.226.67
NetBird IPv6: 2001:db8:ffff::d
Public key: J/6h/4r3iu6k1MYWuA3TuUvwlsCP3OnknPMA672/rx0=
Status: Connected
-- detail --
Connection type: P2P
ICE candidate (Local/Remote): host/host
ICE candidate endpoints (Local/Remote): 192.168.178.36:51820/192.168.178.2:51820
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: 1 minute, 52 seconds ago
Transfer status (received/sent) 2.1 KiB/6.9 KiB
Quantum resistance: false
Networks: -
Latency: 899.355µs

tegu.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.239.103
NetBird IPv6: 2001:db8:ffff::e
Public key: 9I+Tl7qnaAbsBDehZXwuJRjKxawXXe+uVH34GJf8q3w=
Status: Connected
-- detail --
Connection type: Relayed
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 26 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/740 B
Quantum resistance: false
Networks: -
Latency: 0s

mra9.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.245.92
NetBird IPv6: 2001:db8:ffff::f
Public key: efxpAVUW6x/3c8eC68BQIvdqSf2dOs98pNW/9qACoHQ=
Status: Connecting
-- detail --
Connection type: -
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address:
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: -
Transfer status (received/sent) 0 B/0 B
Quantum resistance: false
Networks: -
Latency: 0s

proxy-d9kr4tbr5ksc73bj3gug-248-219.nb.anon-ZjSxZ.domain:
NetBird IP: 100.106.248.219
NetBird IPv6: 2001:db8:ffff::10
Public key: NcWI4DIgN7qgHvGYxLMdtR0zanLJNXj42eutVQy3xSE=
Status: Connected
-- detail --
Connection type: Relayed
ICE candidate (Local/Remote): -/-
ICE candidate endpoints (Local/Remote): -/-
Relay server address: rels://netbird.anon-ZjSxZ.domain:443
Last connection update: 43 minutes, 48 seconds ago
Last WireGuard handshake: 3 seconds ago
Transfer status (received/sent) 4.8 KiB/2.8 KiB
Quantum resistance: false
Networks: -
Latency: 0s

Events:
[INFO] SYSTEM (83579ea9-c318-42eb-ba6b-2a2219fde5cd)
Message: daemon config changed (source=startup)
Time: 43 minutes, 50 seconds ago
Metadata: source: startup, type: config_changed
[INFO] SYSTEM (be7c6a17-d7d9-4461-ab18-8f4527f7dcdc)
Message: Network map updated
Time: 43 minutes, 48 seconds ago
OS: linux/amd64
Daemon version: 0.75.1
CLI version: 0.75.1
Profile: default
Management: Connected to https://netbird.anon-ZjSxZ.domain:443
Signal: Connected to https://netbird.anon-ZjSxZ.domain:443
Relays:
[stun:netbird.anon-ZjSxZ.domain:3478] is Available
[rels://netbird.anon-ZjSxZ.domain:443] is Available via ws
Nameservers:
[100.106.101.175:53] for [docker1] is Available
[2001:db8:ffff::12:53, 2001:db8:ffff::13:53, 198.51.100.2:53] for [.] is Available
[2001:db8:ffff::14:53, 2001:db8:ffff::15:53] for [anon-Amryk.domain] is Available
FQDN: aspire.nb.anon-ZjSxZ.domain
NetBird IP: 100.106.48.141/16
NetBird IPv6: 2001:db8:ffff::11/64
Interface type: Kernel
Wireguard port: 51820
Quantum resistance: false
Lazy connection: false
SSH Server: Enabled
Networks: -
Peers count: 10/16 Connected
renne@aspire:~$

Create and upload a debug bundle, and share the returned file key:

2996f1131e813c87c3cd282fbe2d7ff174c0c5987c5d3e8b4b63438686b90a63/4aa38f79-ffd6-4e04-a23e-1ed5aecbdf70

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings

Detailed discussion at Catch-all NameServerGroup shadows fallback handler on embedded proxy and routing peers · netbirdio/netbird · Discussion #6994 · GitHub