After I decided to have my mac mini (running Catalina) to play the role of home server (with 9 Watts idle) a VPN solution to make the LAN reachable from the “outside” had to be added. The mac mini replaced a Rasp-Pi with home assistent and Wireguard in docker (5 Watts idle). Installing the Wireguard App on Catalina is tricky since the latest version (1.0.16) does not support Catalina anymore. Luckily I installed the 1.0.15 version of the App from the Appstore some time ago and I could download it again (the cloud symbol) on Catalina (the Appstore informs you that you will download a previous version compatible with your system). Alternatively install with brew. The config is not that much different between the two, both implement Wireguard after all.
Wireguard setup instructions can be found on the web everywhere. Unfortunately they often end when a “point-to-point” VPN is up and running. So you have a separate net with 2 addresses, one on the client device and one on the mac. Together with fixed IP, dynamic DNS setup and port forwarding on the router, the client device now communicates with the mac.
But communication with other devices on the LAN will fail, let alone internet access…
Assume a VPN network 10.10.10.0/24 with 10.10.10.2 on the client and 10.10.10.1 on the mac (VPN server) and a LAN network 192.168.1.0/24 with router/gateway at 192.168.1.1 and VPN server (the mac mini) at 192.168.1.101 connected via ethernet.
When the VPN is active, the mac has two active interfaces, a virtual utun (for example utun4) interface with 10.10.10.1 connected to Wireguard and an ethernet interface en0 with 192.168.1.101 connected to the LAN. Since the MacOS IP-Stack connects (binds) to all interfaces by default, the mac mini is reachable via the VPN.
To get out onto the LAN via the VPN, forwarding from local interface utun4 to local interface en0 is needed. On a router, this is a basic functionality and enabled by default. On MacOS it is enabled with:
sudo sysctl -w net.inet.ip.forwarding=1
Note that such a configuration is not persistent anymore with Catalina. You need to set it at startup using a plist file otherwise it’s gone after a reboot. Example instructions can be found here.
Now the mac mini on 192.168.1.101 should be reachable, but not my router/gateway at 192.168.1.1. Well technically it is reachable, a ping to 192.168.1.1 from the client will be forwarded to the 192.168.1.0 network and will reach 192.168.1.1 without problems. But that’s only half of the story. IP has little memory and return traffic has to determine the route all over again (remember where IP networks came from). This return route can be completely different from the initial route.
So on the router, the ping answer must be returned to sender. The addresses are reversed, destination now becomes 10.10.10.2 and sender 192.168.1.1. The poor router is asked how to get to the 10.10.10.0 network, and.. it doesn’t know how. It only knows it’s direct connected networks: the LAN and the WAN/Internet/Outside World. Since 10.10.10.0 is a private network (see RFC1918) the WAN remains off limits, the packet is dropped and my ping remains unanswered.
The simplest solution is to tell the router at 192.168.1.1 that network 10.10.10.0 can be reached via 192.168.1.101, the mac mini. Unfortunately not all SOHO routers support adding this information, which is called a static route. My ageing 7490 Fritzbox supports it, and after adding the route, my ping got answered.
Pinging any other device on the LAN now also succeeds, why? Well, the ‘to’ route is the same, the ‘from’ route not. Assume a ping to 192.168.1.20, my beaglebone, linux based streaming server. This box does not know the 10.10.10.0 network, so return traffic is send to 192.168.1.1, the last resort destination (default gateway). Now we’re at the 192.168.1.1 router again where we just added the static route. So in this scenario, return traffic takes a different route.
Would this impact performance? With my LAN topology, the (physically) shortest route from 192.168.1.20, the Beaglebone, to 192.168.1.101, the mac mini, is via the router at 192.168.1.1. The difference is however, that now the Fritzbox as router performs “routing” which is, on such devices, normally done in software.
What about internet access via the VPN? Well since the router is reachable, the “internet” is as well, since it’s directly connected to the router.
But what if you cannot configure a static route on your router? From the above it should be clear that then 10.10.10.x addresses on the LAN have to go, meaning these addresses have to be translated into 192.168.1.x ones. So network address translation (NAT) is required on the VPN server, i.e. the mac mini. Unfortunately, here the disadvantage of MacOS comes into play. On a linux box, you would configure the NAT with iptables, not so on the mac. MacOS has the option of internet sharing which is basically NAT, and you can even select the Wireguard interface, but MacOS sets a network for you. Matching that with the Wireguard configuration could work…
PS: GRE (RFC2784) can be used to add layer 2 communication using gretap interfaces and bridging. This is described here for linux systems.