❝Beating BGP is harder than we thought❞

▼ In a paper for the HotNets'19, seven researchers admit that "beating BGP is harder than we thought". (Discovered through Aaron '0x88cc' Glenn.) The researchers looked at techniques used by big content delivery networks, including Google, Microsoft and Facebook, to deliver content to users as quickly as possible. This varies from using DNS redirects to PoPs (points of presence) close to the user, using BGP anycast to route requests to a PoP closeby and keeping data within the CDN's network as long as possible ("late exit" or "colt potato" routing).

Turns out, all this extra effort only manages to beat BGP as deployed on the public internet a small fraction of the time. So it's probably not really worth the effort. Also interesting: when BGP is worse, that's usually consistent over relatively long timescales, and when things deteriorate over one path, they tend to also get worse over alternative paths.

That seems strange, as the authors observe that "BGP, the Internet’s inter-domain routing protocol, is oblivious to performance and performance changes." However, BGP isn't deployed in a vacuum. People either install capacity where BGP is going to use it, or they tune BGP parameters to use capacity that's installed.

So having automated traffic management doesn't seem to help much—or does it? Maybe all paths deteriorate together because the automated traffic management solutions do their job and distribute the traffic equally over the available paths, keeping performance the same.

However, there are some cases when one of the options (regular BGP, anycast, late exit, DNS redirect) performs a lot poorer than the alternative(s). So it's probably more important to focus on avoiding really bad paths rather than trying to pick really good ones.

Also interesting: apparently ISPs rarely include the EDNS client subnet option, which DNS redirect really needs to work well. However, the Google and OpenDNS/Cisco public DNS services seem to do support it (but not CloudFlare), so if you're experiencing poor performance towards a CDN, you may want to try using the Google or OpenDNS DNS servers.

Download the paper here.

Permalink - posted 2019-12-30

Valid address space, bogons and martians

▼ There are some advantages to filtering out packets with invalid addresses in them. That would be a packet with a private source or destination address, for instance. Those never have any business traveling across the internet. (Not to be confused with BCP 38 filtering.) For instance, there have been instances where spammers grab an unused prefix, start announcing it in BGP, do a spam run and then drop the prefix. When packets with private addresses enter your network, bad things may happen if you use those addresses yourself. And these invalid "martian" packets are just an annoyance, using up traffic and generating log entries.

The term martian refers to packets that seem to come from Mars, as they can't come from any place here on earth. The BGP announcements for invalid address ranges are often called "bogons". I'm not sure if everyone makes that distinction, but I'll use martian for any address or address block that has a specific purpose, and that purpose means it shouldn't appear in the global routing table. IANA maintains registries of the IPv4 special purpose addresses and the IPv6 special purpose addresses. Note that some addresses are special purpose, but also globally reachable/routable. This is made explicit on the lists.

Bogons include the martians, but also any address space that is currently unused. These are the IPv4 and IPv6 full bogons lists that Team Cymru distributes. The regular, non-full bogons list has only the martians.

If you're considering filtering martians and/or bogons, it's important to realize that these lists change over time. However, martian lists are quite stable, so if you fail to update filters based on the martian lists at regular intervals, that's unlikely to result in problems. However, the full bogons lists changes daily, so it's absolutely crucial to have a solid update mechanism in place if you decide to do full bogon filtering. And note that for IPv4, the full bogons list is quite short as we're pretty much out of IPv4 space, while for IPv6 it's very long as we've barely put a dent into the IPv6 address space.

Last but not least, it's important to realize what part of the IPv4 and IPv6 address space is set aside for global unicast (= regular) use. The IPv4 address space consists of five classes:

• Class A: 0.0.0.0/1 (0.0.0.0/8 - 127.0.0.0/8), global unicast
• Class B: 128.0.0.0/2 (128.0.0.0/16 - 191.255.0.0/16), global unicast
• Class C: 192.0.0.0/3 (192.0.0.0/24 - 223.255.255.0/24), global unicast
• Class D: 224.0.0.0/4, multicast
• Class E: 240.0.0.0/4, reserved for future use

The bogons and martian lists will filter out classes D and E.

With IPv4 class E space we had the unfortunate situation that because many systems considered 240.0.0.0/4 "invalid", it was impossible to repurpose it as regular address space when we started running out of IPv4 address space. So currently, people are paying good money to obtain IPv4 address space because the RIRs don't have any anymore, but 268 million IPv4 addresses remain unused. That's $5 billion worth of IPv4 at $20 per address!

For IPv6, the situation is a bit more complex. Currently, only 2000::/3 is actually used for global unicast. So any valid (non-multicast) packet you'll see traveling across the internet starts with 2xxx: or 3xxx:. However, to avoid repeating the IPv4 class E debacle, the standards documents explicitly designate all IPv6 address space that isn't set aside for a specific different use as global unicast. From RFC 4291:

• Unspecified: ::/128
• Loopback: ::1/128
• Multicast: FF00::/8
• Link-Local unicast: FE80::/10
• Global Unicast: (everything else)

The bogons and martian lists will filter out all the non-global unicast space.

Permalink - posted 2019-11-28