@ -135,9 +135,9 @@ See https://github.com/pirate/wireguard-docs for example code and documentation
# Intro
Whether living behind the Great Wall of China or just trying to form a network between your servers, Wireguard is a great option and serves as a "lego block" for building networks (much in the same way that ZFS is a lego block for building filesystems).
Whether living behind the Great Wall of China or just trying to form a network between your servers, WireGuard is a great option and serves as a "lego block" for building networks (much in the same way that ZFS is a lego block for building filesystems).
## Wireguard Overview
## WireGuard Overview
- minimal config, low tunable surface area and sane defaults
- minimal key management work needed, just 1 public & 1 private key per host
@ -146,16 +146,16 @@ Whether living behind the Great Wall of China or just trying to form a network b
- ability to some traffic or all traffic to/through arbitrary hosts on the VPN LAN
- modern encryption, secure by default with forward secrecy & resilience to downgrade atttacks
- modern encryption, secure by default with forward secrecy & resilience to downgrade attacks
- ideally support for any type of Level 2 and control traffic, e.g. ARP/DHCP/ICMP (or ideally raw ethernet frames), not just TCP/HTTP
- ability to join the VPN from Ubuntu, FreeBSD, iOS, macOS, Windows, Android (via open-source apps or natively)
- ability to join the VPN from Ubuntu, FreeBSD, iOS, MacOS, Windows, Android (via open-source apps or natively)
- supports both running on the host routing traffic for docker or running in a docker container routing for the host
**Things wireguard does not do:**
**Things WireGuard does not do:**
- form a self-healing mesh network where nodes automatically gossip with neighbors
- break through double NATs with a signalling server (WebRTC-style)
- handle automatically distributing & revoking keys through a cetral authority
- handle automatically distributing & revoking keys through a central authority
- allow sending raw layer-2 ethernet frames (it's at the IP layer)
But you can write your own solutions for these problems using WireGuard under the hood (like [AltheaNet](https://althea.net/)).
@ -163,7 +163,7 @@ But you can write your own solutions for these problems using WireGuard under th
## List of Other VPN Solutions
- [WireGuard](https://www.wireguard.com/)
- [IPSec (IKEv2)](https://github.com/jawj/IKEv2-setup)/strongSwan: in my exprience, there was lots of brittle config that was different for each OS, the NAT busting setup is very manual and involves updating the central server and starting all the others in the correct order, it wasn't great at becoming stable again after network downtime, had to be manually restarted often. your mileage may vary.
- [IPSec (IKEv2)](https://github.com/jawj/IKEv2-setup)/strongSwan: in my experience, there was lots of brittle config that was different for each OS, the NAT busting setup is very manual and involves updating the central server and starting all the others in the correct order, it wasn't great at becoming stable again after network downtime, had to be manually restarted often. your mileage may vary.
- [OpenVPN](https://openvpn.net/vpn-server-resources/site-to-site-routing-explained-in-detail/): can work over UDP or be disguised as HTTPS traffic over TCP
- StealthVPN: haven't tried it, should I?
- [DsVPN](https://github.com/jedisct1/dsvpn): I think it does TCP-over-TCP which usually doesn't end well...
@ -196,7 +196,7 @@ But you can write your own solutions for these problems using WireGuard under th
### Example Strings
These are demo hostnames, domain names, ip addresses, and ranges used in the documentation and example configs.
These are demo hostnames, domain names, IP addresses, and ranges used in the documentation and example configs.
Replace them with your preferred values when doing your own setup.
- Example domain: `example-vpn.dev` can be replaced with any publicly accessible domain you control
@ -224,16 +224,16 @@ A publicly reachable peer/node that serves as a fallback to relay traffic for ot
### Subnet
A group of IPs separate from the public internet, e.g. 192.0.2.1-255 or 192.168.1.1/24. Generally behind a NAT provided by a router, e.g. in office internet LAN or a home WiFi network.
A group of IPs separate from the public internet, e.g. 192.0.2.1-255 or 192.168.1.1/24. Generally behind a NAT provided by a router, e.g. in office internet LAN or a home Wi-Fi network.
### CIDR Notation
A way of defining a subnet and its size with a "mask", a smaller mask = more address bits usable by the subnet & more IPs in the range. Most common ones:
+ `192.0.2.1/32` (a single ip address, `192.0.2.1`) netmask = `255.255.255.255`
+ `192.0.2.1/24` (255 ips from `192.0.2.1`-`255`) netmask = ` 255.255.255.0`
To people just getting started `192.0.2.1/32` may seem like a weird and confusing way to refer to a single IP. This design is nice though because it allows peers to expose multiple IPs if needed without needing multiple notations. Just know that anywhere you see something like `192.0.2.3/32`, it really just means `192.0.2.3`.
### NAT
A subnet with private IPs provided by a router standing in front of them doing Network Address Translation, individual nodes are not publicly accessible from the internet, instead the router keeps track of outgoing connections and forwards responses to the correct internal ip (e.g. standard office networks, home wifi networks, free public wifi networks, etc)
A subnet with private IPs provided by a router standing in front of them doing Network Address Translation, individual nodes are not publicly accessible from the internet, instead the router keeps track of outgoing connections and forwards responses to the correct internal IP (e.g. standard office networks, home Wi-Fi networks, free public Wi-Fi networks, etc)
### Public Endpoint
The publicly accessible address:port for a node, e.g. `123.124.125.126:1234` or `some.domain.tld:1234` (must be accessible via the public internet, generally can't be a private ip like `192.0.2.1` or `192.168.1.1` unless it's directly accessible using that address by other peers on the same subnet).
The publicly accessible address:port for a node, e.g. `123.124.125.126:1234` or `some.domain.tld:1234` (must be accessible via the public internet, generally can't be a private IP like `192.0.2.1` or `192.168.1.1` unless it's directly accessible using that address by other peers on the same subnet).
### Private key
@ -285,9 +285,9 @@ More complex topologies are definitely achievable, but these are the basic routi
- **Node behind local NAT to public node**
When 1 of the 2 parties is behind remote NAT (e.g. when a laptop behind NAT connects to `public-server2`), define the publicly accessible node with a hardcoded `Endpoint` and the NAT-ed node without. The connection will be opened from NAT client -> public client, then traffic will route directly between them in both directions as long as the connection is kept alive by outgoing `PersistentKeepalive` pings from the NAT-ed client.
- **Node behind local NAT to node behind remote NAT (via relay)**
Most of the time when both parties are behind NATs, the NATs do srcport randomization making direct connections infeasible, so they will both have to open a connection to `public-server1`, and traffic will forward through the intermediary bounce server as long as the connections are kept alive.
Most of the time when both parties are behind NATs, the NATs do source port randomization making direct connections infeasible, so they will both have to open a connection to `public-server1`, and traffic will forward through the intermediary bounce server as long as the connections are kept alive.
- **Node behind local NAT to node behind remote NAT (via UDP NAT hole-punching)**
While sometimes possible, it's generally infeasible to do direct NAT-to-NAT connections on modern networks, because most NAT routers are quite strict about randomizing the srcport, making it impossible to coordinate an open port for both sides ahead of time. Instead, a signaling server (STUN) must be used that stands in the middle and communicates which random srcports are assigned to the other side. Both clients make an initial connection to the public signaling server, then it records the random srcports and sends them back to the clients. This is how WebRTC works in modern P2P web apps. Even with a signalling server and known srcports for both ends, sometimes direct connections are not possible because the NAT routers are strict about only accepting traffic from the original destination address (the signalling server), and will require a new random srcport to be opened to accept traffic from other IPs (e.g. the other client attempting to use the originally communicated srcport). This is especially true for "carrier-grade NATs" like cellular networks and some enterprise networks, which are designed specifically to prevent this sort of hole-punching connection. See the full section below on [**NAT to NAT Connections**](#NAT-to-NAT-Connections) for more information.
While sometimes possible, it's generally infeasible to do direct NAT-to-NAT connections on modern networks, because most NAT routers are quite strict about randomizing the source port, making it impossible to coordinate an open port for both sides ahead of time. Instead, a signaling server (STUN) must be used that stands in the middle and communicates which random source ports are assigned to the other side. Both clients make an initial connection to the public signaling server, then it records the random source ports and sends them back to the clients. This is how WebRTC works in modern P2P web apps. Even with a signalling server and known source ports for both ends, sometimes direct connections are not possible because the NAT routers are strict about only accepting traffic from the original destination address (the signalling server), and will require a new random source port to be opened to accept traffic from other IPs (e.g. the other client attempting to use the originally communicated source port). This is especially true for "carrier-grade NATs" like cellular networks and some enterprise networks, which are designed specifically to prevent this sort of hole-punching connection. See the full section below on [**NAT to NAT Connections**](#NAT-to-NAT-Connections) for more information.
More specific (also usually more direct) routes provided by other peers will take precedence when available, otherwise traffic will fall back to the least specific route and use the `192.0.2.1/24` catchall to forward traffic to the bounce server, where it will in turn be routed by the relay server's system routing table (`net.ipv4.ip_forward = 1`) back down the VPN to the specific peer that's accepting routes for that traffic. WireGuard does not automatically find the fastest route or attempt to form direct connections between peers if not already defined, it just goes from the most specific route in `[Peers]` to least specific.
@ -351,9 +351,9 @@ Further reading:
### How WireGuard Manages Keys
Authentication in both directions is achieved with a simple public/private keypair for each peer. Each peer generates these keys during the setup phase, and shares only the public key with other peers.
Authentication in both directions is achieved with a simple public/private keypair for each peer. Each peer generates these keys during the setup phase, and shares only the public key with other peers.
No other certificates or preshared keys are needed beyond the public/private keys for each node.
No other certificates or pre-shared keys are needed beyond the public/private keys for each node.
Key generation, distribution, and revocation can be handled in larger deployments using a separate service like Ansible or Kubernetes Secrets.
@ -381,7 +381,7 @@ Most of the time however, every peer should have its own pubic/private keypair s
## Usage
### Quickstart
### QuickStart
Overview of the general process:
@ -395,7 +395,7 @@ Overview of the general process:
- `[Peer]` Create a peer section for each public peer not behind a NAT, make sure to specify a CIDR range for the entire VPN subnet when defining the remote peer acting as the bounce server `AllowedIPs = 192.0.2.1/24`. Make sure to specify individual IPs for remote peers that don't relay traffic and only act as simple clients `AllowedIPs = 192.0.2.3/32`.
5. Start WireGuard on the main relay server with `wg-quick up /full/path/to/wg0.conf`
6. Start WireGuard on all the client peers with `wg-quick up /full/path/to/wg0.conf`
7. Traffic is routed from peer to peer using most specific route first over the WireGuard interface, e.g. `ping 192.0.2.3` checks for a direct route to a peer with `AllowedIPs = 192.0.2.3/32` first, then falls back to a relay server that's accepting ips in the whole subnet
7. Traffic is routed from peer to peer using most specific route first over the WireGuard interface, e.g. `ping 192.0.2.3` checks for a direct route to a peer with `AllowedIPs = 192.0.2.3/32` first, then falls back to a relay server that's accepting IPs in the whole subnet
# check that remote NAT-ed peers are available via VPN
ping 192.0.2.3
# check that NAT-ed peers in your local lan are available via VPN
# check that NAT-ed peers in your local LAN are available via VPN
ping 192.0.2.4
```
@ -580,7 +580,7 @@ WireGuard config is in [INI syntax](https://en.wikipedia.org/wiki/INI_file), def
The config path is specified as an argument when running any `wg-quick` command, e.g:
`wg-quick up /etc/wireguard/wg0.conf` (always specify the full, absolute path)
The config file name must be in the format `${name of the new wireguard interface}.conf`. WireGuard interface names are typically prefixed with `wg` and numbered starting at `0`, but you can use any name that matches the regex `^[a-zA-Z0-9_=+.-]{1,15}$`.
The config file name must be in the format `${name of the new WireGuard interface}.conf`. WireGuard interface names are typically prefixed with `wg` and numbered starting at `0`, but you can use any name that matches the regex `^[a-zA-Z0-9_=+.-]{1,15}$`.
Config files can opt to use the limited set of `wg` config options, or the more extended `wg-quick` options, depending on what command is preferred to start WireGuard. These docs recommend sticking to `wg-quick` as it provides a more powerful and user-friendly config experience.
@ -721,7 +721,7 @@ This option can be specified multiple times, with commands executed in the order
@ -782,11 +782,11 @@ This option can appear multiple times, as with <a href="#PreUp">PreUp</a>
### `[Peer]`
Defines the VPN settings for a remote peer capable of routing traffic for one or more addresses (itself and/or other peers). Peers can be either a public bounce server that relays traffic to other peers, or a directly accessible client via lan/internet that is not behind a NAT and only routes traffic for itself.
Defines the VPN settings for a remote peer capable of routing traffic for one or more addresses (itself and/or other peers). Peers can be either a public bounce server that relays traffic to other peers, or a directly accessible client via LAN/internet that is not behind a NAT and only routes traffic for itself.
All clients must be defined as peers on the public bounce server. Simple clients that only route traffic for themselves, only need to define peers for the public relay, and any other nodes directly accessible. Nodes that are behind separate NATs should _not_ be defined as peers outside of the public server config, as no direct route is available between separate NATs. Instead, nodes behind NATs should only define the public relay servers and other public clients as their peers, and should specify `AllowedIPs = 192.0.2.1/24` on the public server that accept routes and bounce traffic for the VPN subnet to the remote NAT-ed peers.
In summary, all nodes must be defined on the main bounce server. On client servers, only peers that are directly accessible from a node should be defined as peers of that node, any peers that must be relayed by a bounce sherver should be left out and will be handled by the relay server's catchall route.
In summary, all nodes must be defined on the main bounce server. On client servers, only peers that are directly accessible from a node should be defined as peers of that node, any peers that must be relayed by a bounce server should be left out and will be handled by the relay server's catchall route.
In the configuration outlined in the docs below, a single server `public-server1` acts as the relay bounce server for a mix of publicly accessible and NAT-ed clients, and peers are configured on each node accordingly:
@ -796,13 +796,13 @@ In the configuration outlined in the docs below, a single server `public-server1
- **in `public-server2` `wg0.conf` (simple public client)**
WireGuard can sometimes natively make connections between two clients behind NATs without the need for a public relay server, but in most cases this is not possible. NAT-to-NAT connections are only possible if at least one host has a stable, publicly-accessible IP address:port pair that can be hardcoded ahead of time, whether thats using a FQDN updated with Dynamic DNS, or a static public IP with a non-randomized NAT port opened by outgoing packets, anything works as long as all peers can communicate it beforehand and it doesn't change once the connection is initiated.
WireGuard can sometimes natively make connections between two clients behind NATs without the need for a public relay server, but in most cases this is not possible. NAT-to-NAT connections are only possible if at least one host has a stable, publicly-accessible IP address:port pair that can be hardcoded ahead of time, whether that's using a FQDN updated with Dynamic DNS, or a static public IP with a non-randomized NAT port opened by outgoing packets, anything works as long as all peers can communicate it beforehand and it doesn't change once the connection is initiated.
A known port and address need to be configured ahead of time because Wireguard doesn't have a signalling layer or public STUN servers that can be used to search for other hosts dynamically. WebRTC is an example of a protocol that can dynamically configure a connection between two NATs, but it does this by using an out-of-band signaling server to detect the ip:port combo of each host. WireGuard doesn't have this, so it only works with a hardcoded `Endpoint` + `ListenPort` (and `PersistentKeepalive` so it doesn't drop after inactivity).
A known port and address need to be configured ahead of time because WireGuard doesn't have a signalling layer or public STUN servers that can be used to search for other hosts dynamically. WebRTC is an example of a protocol that can dynamically configure a connection between two NATs, but it does this by using an out-of-band signaling server to detect the IP:port combo of each host. WireGuard doesn't have this, so it only works with a hardcoded `Endpoint` + `ListenPort` (and `PersistentKeepalive` so it doesn't drop after inactivity).
#### Requirements for NAT-to-NAT setups
- At least one peer has to have to have a hardcoded, directly-accessible `Endpoint` defined. If they're both behind NATs without stable IP addresses, then you'll need to use Dynamic DNS or another solution to have a stable, publicly accessibly domain/IP for at least one peer
- At least one peer has to have a hardcoded UDP `ListenPort` defined, and it's NAT router must not do UDP source port randomization, otherwise return packets will be sent to the hardocded `ListenPort` and dropped by the router, instead of using the random port assigned by the NAT on the outgoing packet
- At least one peer has to have a hardcoded UDP `ListenPort` defined, and it's NAT router must not do UDP source port randomization, otherwise return packets will be sent to the hardcoded `ListenPort` and dropped by the router, instead of using the random port assigned by the NAT on the outgoing packet
- All NAT'ed peers must have `PersistentKeepalive` enabled on all other peers, so that they continually send outgoing pings to keep connections persisted in their NAT's routing table
#### The hole-punching connection process
@ -963,13 +963,13 @@ A known port and address need to be configured ahead of time because Wireguard d
This process of sending an initial packet that gets rejected, then using the fact that the router has now created a forwarding rule to accept responses is called "UDP hole-punching".
When you send a UDP packet out, the router (usually) creates a temporary rule mapping your source address and port to the destination address and port, and vice versa. UDP packets returning from the destination address and port (and no other) are passed through to the original source address and port (and no other). This is how most UDP applications function behind NATs (e.g. Bittorent, Skype, etc). This rule will timeout after some minutes of inactivity, so the client behind the NAT must send regular outgoing packets to keep it open (see `PersistentKeepalive`).
When you send a UDP packet out, the router (usually) creates a temporary rule mapping your source address and port to the destination address and port, and vice versa. UDP packets returning from the destination address and port (and no other) are passed through to the original source address and port (and no other). This is how most UDP applications function behind NATs (e.g. BitTorrent, Skype, etc). This rule will timeout after some minutes of inactivity, so the client behind the NAT must send regular outgoing packets to keep it open (see `PersistentKeepalive`).
Getting this to work when both end-points are behind NATs or firewalls requires that both end-points send packets to each-other at about the same time. This means that both sides need to know each-other's public IP addresses and port numbers ahead of time, in WireGuard's case this is achieved by hard-coding pre-defined ports for both sides in `wg0.conf`.
#### Drawbacks and limitations
As of 2019, many of the old hole-punching methods used that used to work are no longer effective. One example was a novel method pioneered by [pwnat](https://github.com/samyk/pwnat) that faked an ICMP Time Exceeded response from outside the NAT to get a packet back through to a NAT'ed peer, thereby leaking its own srcport. Hardcoding UDP ports and public IPs for both sides of a NAT-to-NAT connection (as described above) still works on a small percentage of networks. Generally the more "enterprisey" a network is, the less likely you'll be able to hole puch public UDP ports (commercial public wifi and cell data NATs often don't work for example).
As of 2019, many of the old hole-punching methods used that used to work are no longer effective. One example was a novel method pioneered by [pwnat](https://github.com/samyk/pwnat) that faked an ICMP Time Exceeded response from outside the NAT to get a packet back through to a NAT'ed peer, thereby leaking its own source port. Hardcoding UDP ports and public IPs for both sides of a NAT-to-NAT connection (as described above) still works on a small percentage of networks. Generally the more "enterprisey" a network is, the less likely you'll be able to hole punch public UDP ports (commercial public Wi-Fi and cell data NATs often don't work for example).
##### Source port randomization
@ -977,7 +977,7 @@ NAT-to-NAT connections are not possible if all endpoints are behind NAT's with s
##### Using a signaling server
NAT-to-NAT connections from behind NATs with strict source-port randomization is possible, you just need a singaling server to tell each side the other's ip:port tuple. Here are a few implementations that achieve this with WireGuard:
NAT-to-NAT connections from behind NATs with strict source-port randomization is possible, you just need a signaling server to tell each side the other's IP:port tuple. Here are a few implementations that achieve this with WireGuard:
@ -1120,7 +1120,7 @@ You can read in a file as the `PrivateKey` by doing something like:
### Containerization
WireGuard can be run in Docker with varying degrees of ease. In the simplest case, `--privileged` and `--cap-add=all` args can be added to the docker commands to enable the loading of the kernel module.
WireGuard can be run in Docker with varying degrees of ease. In the simplest case, `--privileged` and `--cap-add=all` arguments can be added to the docker commands to enable the loading of the kernel module.
Setups can get somewhat complex and are highly dependent on what you're trying to achieve. You can have WireGuard itself run in a container and expose a network interface to the host, or you can have WireGuard running on the host exposing an interface to specific containers.
@ -1247,7 +1247,7 @@ Setups can get somewhat complex and are highly dependent on what you're trying t
- https://news.ycombinator.com/item?id=17659983
- https://news.ycombinator.com/item?id=17846387
For more detailed instructions, see the [Quickstart](#Quickstart) guide and API reference above. You can also download the complete example setup here: https://github.com/pirate/wireguard-example.
For more detailed instructions, see the [QuickStart](#QuickStart) guide and API reference above. You can also download the complete example setup here: https://github.com/pirate/wireguard-example.
Nick Sweeting
4 years ago
committed by
GitHub