Environment
- NetBird Client Version: 0.78.1 (Reproducible on Linux, macOS, and iOS)
- NetBird Server Version: 0.78.1 (Self-hosted, combined management/signal container)
- Client OS: Ubuntu 24.04 LTS (amd64), macOS (latest), iOS (latest)
- Deployment: Self-hosted behind an external Nginx reverse proxy (HTTP/2 + gRPC fully verified and functional)
Describe the Problem
Across multiple platforms (Ubuntu, macOS, and iOS), the NetBird client successfully authenticates via the Setup Key flow and receives a 200 OK response from the /management.ManagementService/Login gRPC endpoint.
However, immediately after the successful login, the client daemon hangs for exactly ~50 seconds and then fails with:
Error: daemon up failed: call service up method: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Subsequently, netbird status -d reports:
Management: Connected to https://<domain>:443
Signal: Disconnected
Relays:
Peers count: 0/0 Connected
To Reproduce
- Deploy a self-hosted NetBird server (v0.78.1) behind a standard Nginx reverse proxy.
- On a fresh client (Ubuntu, macOS, or iOS), ensure all local state is cleared.
- Attempt to connect using a valid Setup Key:
sudo netbird up --management-url https://<domain> --setup-key <key> - Observe the CLI hang for ~50 seconds before returning the
DeadlineExceedederror.
Forensic Evidence & Root Cause Analysis
We have performed exhaustive debugging, including packet captures and strace analysis. The evidence definitively points to a client-side deadlock triggered by an invalid MTU value provided during the WireGuard kernel module initialization test.
1. The Smoking Gun: strace Analysis
Running the daemon under strace -f reveals the exact moment of failure:
sendto(20, [{nlmsg_len=80, nlmsg_type=RTM_NEWLINK, ...},
{ifi_family=AF_UNSPEC, ...},
[[{nla_len=17, nla_type=IFLA_IFNAME}, "mustnotexist"],
[{nla_len=8, nla_type=IFLA_MTU}, 4294967295],
[{nla_len=20, nla_type=IFLA_LINKINFO}, [{nla_len=13, nla_type=IFLA_INFO_KIND}, "wireguard"...]]]], 80, ...) = 80
recvfrom(20, [{nlmsg_len=100, nlmsg_type=NLMSG_ERROR, ...}, {error=-EINVAL, msg=[...]}], ...) = 100
- The client attempts to create a dummy WireGuard interface named
"mustnotexist"to test kernel support. - It requests an MTU of
4294967295(which is0xFFFFFFFF, or-1as a signed 32-bit integer). - The Linux kernel correctly rejects this absurd value with
error=-EINVAL. - The Deadlock: Instead of gracefully falling back to userspace or using a default MTU, the Go runtime enters a panic/deadlock loop. The
straceshows the main thread aggressively spammingtgkill(..., SIGURG)to all child threads until the process is manually terminated viaCtrl+C(SIGINT).
- Nginx Access Logs Confirm the Missing
SyncCall
The Nginx logs show a perfect authentication cycle, but the crucialSyncstream is never established:
"POST /management.ManagementService/GetServerKey HTTP/2.0" 200 61
"POST /management.ManagementService/Login HTTP/2.0" 200 366
# ... 50 seconds of silence (client is deadlocked locally) ...
"POST /management.ManagementService/GetServerKey HTTP/2.0" 200 61 <-- Client retries loop
"POST /management.ManagementService/Login HTTP/2.0" 200 366
- Cross-Platform Correlation
The fact that this identical behavior occurs on macOS and iOS strongly suggests that the NetBird server is sending a malformed, missing, or null MTU value in the configuration payload. The v0.78.1 client appears to misinterpret this missing value as-1(casting it to the unsigned4294967295), which triggers the kernel rejection and subsequent Go runtime deadlock on Linux, and likely a similar silent failure in the Network Extension on Apple platforms.
Workaround / Mitigation
Forcing the client to use the userspace WireGuard engine completely bypasses the kernel module test, preventing the -EINVAL rejection and the subsequent deadlock.
On Linux, the following command allows the client to connect successfully and establish the Signal stream:
sudo NB_WG_ENGINE=userspace netbird up --management-url https://<domain> --setup-key <key>
Note: Setting Environment="NB_WG_ENGINE=userspace" in the systemd service file provides a permanent workaround.
Request for Maintainers
- Could you investigate why the server might be sending an invalid or missing MTU value in the
Login/Syncresponse payload? - Why does the v0.78.1 client fail to gracefully handle an
-EINVALresponse from the kernel during themustnotexisttest, resulting in a hard Go runtime deadlock instead of falling back to a safe default or userspace? - Is there a known regression in how MTU defaults are handled in the
config.yamlor database schema in recent versions?
(Attached: Full strace log demonstrating the SIGURG deadlock loop following the -EINVAL MTU rejection).