Rick van Rein

Fri 26 June 2020


Networking #2: Reliable Peering over 6bed4

For client-server networking, NAT traversal is a solved problem. For peer-to-peer networks it is not possible to do in general, but the potential of these networks in the liberation of users from "central" services is quite big. The 6bed4 tunnel allows applications to be designed as peer-to-peer IPv6 applications with only a fallback (to your own tunnel server) if necessary.

This article is part of a series on networking.

There are a few tunnels for IPv6, and most are difficult to use. The 6bed4 tunnel is relatively easy to use; it installs as a bit of software and behaves somewhat like a VPN client, though its main purpose is routing over IPv6.

Applications can assume native IPv6 routing (but may impose any form of firewalling they like) and specifically that they can reach ports without NAT getting in the way as with IPv4. If any protocol flourishes on IPv6 it is going to be peer-to-peer protocols. The omnipresence of IPv4 however, lets designs of peer-to-peer protocols use that "for the time being" and lands them in the terrible problems of NAT traversal. For which there is never a perfect solution. By being generic, 6bed4 can solve these problems once, and have all the applications benefit.

The design of 6bed4 attempts direct connections between peers, but hides the details in the tunnel, which runs over UDP/IPv4 and does all the things that are otherwise loaded onto the designer of a peer-to-peer protocol. The places where 6bed4 can be used are:

  • As a tunnel serving a single host; this runs as a 6bed4peer process that connects to a 6bed4router that runs on a server; this could be your server or the default one, run at by SURFnet.
  • As a tunnel on a router, serving a LAN; this also runs a 6bed4peer, but can offer a range of 65534 IPv6 addresses through DHCPv6 software (like DNSmasq) to a local network. The 6bed4router is around 35 kB and 6bed4peer is about 50 kB; they only depend on a C library.
  • As part of an application; we have this as a currently open issue but we are closing in on it fast. The protocol is quite open so nobody depends on our software anyway.

How Addresses Look in 6bed4

The traffic routes in 6bed4 depend on prefixes. There are three kinds of interest:

  • fc64::/16 is a local range, completed to fc64:<netid>:<ipv4>::/64 where the <netid> can be used to distinguish network segments and <ipv4> is the address of the 6bed4router serving it.
  • TBD1::/32 is intended to be globally routable, but we need to formally file for such a prefix assignment. The intention is to make it recognisable that the address is a 6bed4-routed address. The range is completed to TBD1:<ipv4>::/64 with <ipv4> referencing the address of the 6bed4router, as for the previous form.
  • Native /64 prefixes can be used. These do not convey an IPv4 address, so they are considered local to the 6bed4router address.

The former two kinds of address are supportive of direct peering connections. All forms have the peer's UDP port and IPv4 address in the lower part of the address, but not all prefixes let us recognise that this information is there. The fc64::/16 prefix will be assumed to have this meaning on a 6bed4 network; the TBD1::/32 prefix will be assumed to have this meaning everywhere on the Internet and the native /64 prefixes can only be assumed to have this meaning on the 6bed4 network, and only if it is a prefix under which the peer configured its own address.

How Routing Works in 6bed4

The destination address determines where traffic is forwarded. In 6bed4, the source address must match the sending peer; it is vital that the UDP port and IPv4 address of a sender match the lower half of its source IPv6 address, to avoid tampering with traffic to make it look like it's coming from somewhere else.

Routing Diagram for 6bed4

The green parts of this diagram form "your" network, the yellow node is another user of "your" upstream 6bed4router and the red parts are considered remote parts, either residing under another 6bed4router or none at all. The thick lines are the 6bed4peer and 6bed4router that you have setup for yourself, the rest comes for free.

Note that 6bed4peer processes may connect directly, at least if the destination prefix is recognised by the sender as one whose UDP port and IPv4 address can be derived from the lower half of the IPv6 address. It is this direct peering that is so beneficial. Your 6bed4peer process works with the other side to make reliable direct connections when it can, but goes through the 6bed4router if it cannot do so reliability.

Your 6bed4router may recognise its own /64 prefix, regardless of the form it takes, and relay the traffic to the 6bed4peer that uses that same 6bed4router service. It is the task of each 6bed4peer to always keep a link open to its 6bed4router, so this is a reliable fallback route in any case.

Something else your 6bed4router may do is pass the traffic to another 6bed4router for delivery to a 6bed4peer that keeps a reliable link open to it. This is called trapezium routing, and its use is to keep everything in the 6bed4 network and allow for bypasses to be made. At present, the only considered bypass is direct peering. This kind of routing is only possible when your 6bed4router can know the the prefix is a 6bed4 prefix, with the right information for delivery to a 6bed4peer in the lower half. This is why native traffic is not delivered in this manner.

Native traffic however, can be delivered if your 6bed4router offers it. This may involve specific routes or a default route, which just looks like ::/0 in IPv6. Delivery can be to local interfaces sitting behind the 6bed4router, or it may pass traffic on to external routers that care for the delivery.

As a special case, the IPv6 address of the 6bed4router may be connected to backend addresses and ports. In this case, a service at that address is contacted. It is even possible to map to another address, which is actually a form of NAT and awkward in IPv6; this hack is called masquerading and may disappear from future versions.

Return Traffic over 6bed4

When you send something, you normally want a response. This is why the UDP port and IPv4 address of your 6bed4peer occur in the lower half of its assigned IPv6 address; through this, the 6bed4router can always reach you and, if they are lucky, so can other 6bed4peer nodes. In relation to non-6bed4 nodes, the concern is mostly if traffic can route back to the 6bed4router.

The first case is also the most difficult; return traffic to fc64::/16 sources is only possible for local nodes, because the addresses are locally defined only. It is a concern when setting up the 6bed4router process; read the man page!

The second case is possible as soon as we have the TBD1::/32 prefix assigned to 6bed4, because it allows the traffic to be passed to any 6bed4router for the first 32 bits, and then the traffic is relayed to the IPv4 address that follows after the prefix. Note how this is a matter of routing rules, but otherwise already encompassed in this diagram.

Finally, if your 6bed4peer was setup with a native prefix, as is the case for our default router as provided by SURFnet on, your 6bed4router will receive the return traffic through normal IPv6 routing and relay it based on the knowledge it happens to have about the lower part of the address.

Note that all this is derived from the addresses themselves. There is no need to keep state. This means that there is no reason to log anything, especially because complaints about abuse of an IPv6 address can be translated into complaints of an IPv4 address and a UDP port.

Reliable Peering in 6bed4

A protocol similar to 6bed4 has been tried before, by the name of TEREDO. It is built into Windows machines, and is the cause why Windows users tend to switch off IPv6. The tunnel is flawed, because it makes assumptions about its NAT traversal possibilities. Most peer-to-peer systems suffer from the same fate, and occassionally missroute traffic as a result. You may have had single-sided media connections and this is where it comes from: assumptions based on classification of NAT traversal.

Instead of this inductive appraoch, 6bed4 is deductive. It simply tries if NAT traversal works, and chooses to go through the 6bed4router if this is not possible.

When communicating with a potential direct peer, your 6bed4peer will send empty Probe messages to the direct address. It will not do this for every packet you send, but try a few times and slow down the pace when no response comes in. It never fully gives up, so when the remote peer comes online later it will be discovered.

When traffic returns from the remote peer, it can set a "Seen" flag to indicate that traffic got through. This may have been a Probe or actual traffic. The "Seen" flag is sent on all return traffic for 2 seconds after it was seen, and it permits 27 seconds of direct peering after reception. This assumes that NAT holes stay open for at least 30 seconds, a common assumption but one that can be changed on the remote peer if its NAT works differently; this impacts the "Seen" timing.

This process happens independently in both directions. The returned attempt is usually even simpler, because your 6bed4peer already sent a Probe. And it will try again after 1 second and after 2 seconds, so under normal protocols this quickly leads to bidirectional direct peering. As long as your NAT allows direct peering at the same IPv4 address and UDP port, then 6bed4 should discover this rather quickly.

There is a method to guaranteer that your NAT supports direct peering. This is done with port forwarding. As described before, a 6bed4router can be run in a router box, and this combines perfectly with port forwarding. If the standard port for 6bed4 is used, this even means that the IPv4 address in the top half fc64:<netid>:<ipv4>::/64 or TBD1:<ipv4>::/64 can be the public IPv4 address of this router. This may explain why trapezium routing can be beneficial.

Traffic Classes in 6bed4

The IPv6 header "Traffic Class" is used to give a clue about routing preferences for a single packet. Applications can use this to request a bulk and eventually reliable delivery, or fast but possibly dropped treatment.

We use the Traffic Class to distinguish whether the traffic may, must, or must not pass through your 6bed4router. This being the only reliable route, excluding it reduces the chances of contact, but you may prefer that for some applications.

We define the following Traffic Classes:

  • proper-peering(0) is the default treatment, going through the 6bed4router for reliability, but passing to a direct peer when this is a reliable alternative.
  • prohibited-peering(1) is the mode where all traffic flows through your 6bed4router.
  • presumptious-peering(2) tries for a few seconds to setup direct peering, but when this fails it falls back on your 6bed4router.
  • persistent-peering(3) goes even further, and will never route through your 6bed4router; it would rather drop traffic and send errors about it.

These choices are made for individual packets. This means that different kinds of traffic can be treated differently (SIP versus RTP versus MSRP for example). You might even try out one approach and switch to an alternative if you have to.

The relation to a given remote peer is managed while traffic flows, so the state of reliability is known when a later packet hits, and any delays of a few seconds learning time are shared. You might pass a few seconds of meaningless traffic and then add a secret stream as soon as it is safe.

The "Seen" flag is passed in this field too. You cannot set it in the application and should send its value as 0 for future extensibility, but you can learn its value from received traffic. This can help you make decisions about routing policies with this peer, or whether to turn on the green light that indicates safe mode, whether to start a secret flow, and so on.

Go Top