it might be an easy setup, but can someone give guidance on my specific goal?
I want proxmox on VLAN A to be technically “offline” all the time. Therefore I blocked all traffic on internal firewall, fixed IPs, no DHCP etc. On this Vlan Proxmox is utilizing a trunk port to distribute VMs to my router (Unifi zone based firewall).
The goal: Proxmox should have Internet access to be managed normal, but should never be directly exposed to Internet (hybrid air gap?).
I am self hosting Netbird on a Hetzner Server. An dedicated VM with Ununtu in another VLAN B should be the routing peer for “offline” proxmox and other VLAN machines. Internet traffic should run through my own ISP (VLAN B if possible) and not Hetzner - at least mainly (because of limited included traffic on Hetzner).
I am writing here, asking for help because I dont want to try something and “destroy” my 100% clean/offline Proxmox setup, with quite a few VLANs and VMs, LXCs.
If any information is needed in addition, please let me know and I will provide it. Thank you. Milo
This is beyond the scope of that I know, at the best gues I’d say you could make a network and have NetBird route the traffic. But considering there is no IP address. At the best case senario, you could limit the firewall to only allow connections from a NB server that connects throught BirdNet and have it proxy those connections as it would normally. Only allowing people that route through NB to connect. But that’s just a rough idea. Air gaps like this are complicated and outside my scope
Thank you. Now I feel better when a pro doesn’t have a solution right away. Internal I have an IP local set on Proxmox and also on all VMs etc. But the overlay network and routing functions would enable a “new/separate” Network when I see that correctly.
So I saw people install Netbird on Proxmox directly, to at least not expose proxmox directly to Internet? But mainly I want proxmox to get internet access without exposing it and not just connect to it from outside.
I would still try to find a solution to achieve a jump box function. I am just not that deep into this to do it myself.
I think it might be a great addition to create a guide for this since it would help to have an solution for a lot of homelab people (I guess).
@Xeravax can I ask you 2 things? Or better wording: Can you answer these 2 or 1 of those questions please?
Forgive me, but I am trying for weeks now to make it work but there might be better/easier ways to do it, maybe - so
A) what would one nerd do to achieve best possible security for proxmox when a full offline node is not practical in an homelab
B) do you maybe know someone who might be able to help. E.g. From the Netbird Team created webinar, there I saw a video for Kubernetes connections with Netbird and that seemed to be pretty close to this usecase, but alone I am not able to connect the dots it seems. In the future I want to build a very secure system for business and my company, but to make it work I wanted to build it in private first.
yes that is the goal to use it normal, preventing mirror solutions to get updates (only)
in my case a trunk port holds VLAN A with proxmox and is completely offline via firewall (management VLAN A), but other VMs on different VLANs are online, running on this exact node and proxmox (used NFS share to install VMs and LXCs)
with this I would be able to explicitly open up single ports for specific traffic in firewall for the proxmox alone (that was my goal to have that option if needed)
what is your opinion on this topic. Is it worth it for extra security to invest more time, or would you just install Netbird on Proxmox directly and bring it online?
I tried so far:
-using several tools and app internally and externally on Hetzner Ubuntu VPS to create a jump box style setup (stopped at port forwarding since I do want to avoid this)
-Wireguard wg0 or tun0 creation to VLAN B Ubuntu and the using squid or similar to forward explicit traffic to proxmox in VLAN A (stopped there as well because of complicated setup and hard to do for average people like me)
Your opinion would mean a lot to me, since I am stuck, but willing to wait in case you see a chance to solve this - as a security feature for a lot of homelab people that did not even know they are exposing a lot running proxmox, docker, vms, homeassistant etc - without thinking about it
there more you know, and I know nothing and it is crazy what you can see going on - going into more and more detail
I just wanted to give it a try because Netbird seems to be the best option I found so far. very close to what I would consider a protected, non exposed Proxmox (VMs can always be reinstalled with a snapshot, if something went wrong)
Hello @Milo, as the node has internet access, I would say just installing NetBird there would be enough, and would probably work fine without opening inbound ports. On our cloud service, you would need to ensure that these ports are open for outbound: FAQ - NetBird Docs
With this, you will have Proxmox accessible via NetBird, and you can control who can access it via our dashboard.
node is online but proxmox itself is not and I dont want it to be „normal“ way in this management VLAN A. Now the question: So I can leave proxmox completely blocked and install proxmox directly on proxmox via NFS share - setting an exit node on same node but on other VLAN B and proxmox will be online through peer to peer connection then? I assume I have to set routes then? Or do I have to take additional steps? Can the exit node be on other VLAN or does it have to be on same VLAN internally?
Assuming your network has the VLAN’s to prevent one subnet from going into the other, you may have to yes. Otherwise, if you have it open enough internally to do that you should be able to connect without doing that. There is a chance that you’re going to have to setup a route on the Proxmox machine itself to make sure it works. But that is just a guess from my end.
update: tried a lot but nothing worked.
I now installed via Brandons Guide an LXC for Netbird WITHIN same Vlan as proxmox. Proxmox nic is handling all vlans and all vms and IPs with dhcp off.
Proxmox is in native VLAN port and distributes traffic via vlan tag as intended for all VMs that are on other VLANs.
Netbird is running as LXC as Network. Policies are set. connection works.
I also set same LXC as exit node with the goal to route all internet traffic from Proxmox (offline, airgapped, firwall full block). I call it Jumpbird
2 things are in the way now:
Ip Tables are not distributed in LXC (due to rights and unpriviledged LXC I assume)
Proxmox needs routing setup (but without harming vlan distribution and VMs - only internet routing external)
Brandon had additional lines for tun0 creation. Is there a chance to get automatic IP table distribution working same way with correct commands and rights for that?
lxc.cap.keep: net_admin
this prevents LCX from starting up when I tried it…
What would be the Proxmox configuration to use exit node Netbird masked tunnel for Internet of Proxmox? Proxmox should never be directly “visible” to Internet.
I hope someone can help. Thanks. Milo
Internal I have an IP local set on Proxmox and also on all VMs etc. But the overlay network and routing functions would enable a “new/separate” Network when I see that correctly.
Or better wording: Can you answer these 2 or 1 of those questions please? 