The effectiveness of AS path prepending

▼ In a recent blog post The Effectiveness of AS Path Prepending (1) Russ White asks:

Just about everyone prepends AS’ to shift inbound traffic from one provider to another—but does this really work?

(AS path prepending means making the network path as BGP sees it longer to make a path less attractive so traffic will flow over another, shorter path.)

That's an interesting question, as I've been telling people for a long time that it often works too well. A single prepend can flip more than half your traffic from one link to another. In part because of that, Rolf Winter and I worked on a new Origin Preference Attribute that would provide a finer-grained tool for BGP traffic engineering. (Slides, old Internet-Draft, SAC'12 publication.)

But the answer that Russ cites from research into AS path prepending makes sense:

We observe that the effectiveness of prepending can strongly depend on the location (for around 20% of cases, ASPP has moved no targets, while for another 20% , it moved almost all targets).

The the fact that different networks get such different results when performing AS path prepending can be explained by looking at the type of network and the type of connectivity they have. Suppose a small network X has connections to big ISPs A and B. A and B have direct peering relations with most of the other large ISPs. So to the rest of the world, the two paths to X are the same length:

A X = 2 hops
B X is 2 hops

When X now prepends towards A, it's A X X = 3 hops vs B X = 2 hops so most traffic will now flow through the B X path. Prepending was extremely effective.

But what if instead of X doing prepending, A prepends. Now for traffic to X, it's still:

A A X = 3 hops
B X = 2 hops

But other networks that directly connect to A and have traffic for A itself, it doesn't matter if they see A = 1 hop or A A = 2 hops: the traffic needs to go to A, so the path length isn't considered. Even when network C is a customer of B and C sees a direct path and a path over B:

A A A = 3 hops
B A = 2 hops

C will use the direct path, even though it's longer, because the rule is: paths to customers are prioritized over paths to peers and paths to peers over paths to transit providers.

So: will AS path prepending be effective? As with most things in life, it depends.

Permalink - posted 2021-05-13

Zevencijferige telefoonnummers in Rotterdam, Den Haag, Amsterdam

▼ Ik kwam laatst een oude kaart van Den Haag tegen en ik vroeg me af van wanneer die was. Er stond geen jaartal op. Wel viel me op dat de telefoonnummers die erop stonden maar zes cijfers hadden, dus van voor dat alle Haagse telefoonnummers er een 3 bij kregen. Maar wanneer was dat? Het lukte mij niet deze info terug te vinden via Wikipedia of Google.

Ik herinner me dat in Amsterdam er een tijd zowel zes- als zevencijferige telefoonnummers gebruikt werden. En iemand schreef zelfs dat dit vanaf rond 1972 was. Via de Fidonet nodelist kon ik achterhalen dat de overschakeling van zes- naar zevencijferig tussen oktober 1990 en januari 1991 gebeurd is.

Voor zover ik me kan herinneren had Rotterdam al een hele tijd zevencijferige nummers die met een 4 begonnen voor wij in Den Haag de 3 kregen. Uit de nodelist blijkt dat in januari 1990 Rotterdam de 4 al had.

En voor Den Haag zien we de overgang van zes naar zeven cijfers tussen augustus 1989 en januari 1990.

Op 10 oktober 1995 volgde met Operatie Decibel de rest van Nederland, met ofwel een driecijferig netnummer en zeven cijfers voor het abonneenummer, ofwel een viercijferig netnummer en een zescijferig abonneenummer.

Sindsdien is er in Nederland voor zover ik weet niks meer veranderd aan de telefoonnummers, maar uit onderzoek voor twee los van elkaar staande zaken viel me wel op hoe veel en vaak telefoonnummers in verschillende delen van de wereld veranderen.

Permalink - posted 2021-05-13

Hunting down the stuck BGP routes

▼ Ben Cox (Benjojo) has an interesting post about stuck BGP routes and a flaw in many BGP implementations where they hang when their neighbor stops accepting data over TCP: Hunting down the stuck BGP routes

A stuck BGP route means that a prefix was advertised at some point, and then it's withdrawn but the withdrawal somehow gets lost somewhere, so part of the internet still sees the withdrawn route.

There doesn't seem to be an obvious way to get these prefixes unstuck, although I would try advertising them again and withdrawing them again. The AS in the middle where the prefixes are stuck should probably clear (reset) BGP sessions until the problem is fixed, although I would be tempted to just reboot the whole router just to be safe.

The hanging issue happens when a router gets behind on processing incoming BGP updates. At some point its TCP receive buffer fills up, so it sets its TCP window size to 0. Then the buffer on the sending BGP neighbor starts to fill up with updates and keepalives. When that buffer is full, the sending BGP process can't write data to the TCP socket anymore... and bad things start to happen.

Ben feels that the BGP spec needs to be updated to address this issue. However, this is not a matter of interoperation that needs to be standardized, but simply an implementation issue that each vendor can fix on their own, in my opinion.

Permalink - posted 2021-04-22

Getting the "Music" miniplayer back to actual mini

▼ I just had a moderate-size scare: I typed something into iTunes Music, and the miniplayer now showed playback controls:

That's not what I want. I want it to show the album artwork. That's how I've used iTunes and now Music for many years. If I need to interact with the controls, those will then come up when I place my mouse pointer over the miniplayer window. In the view menu, there is no option to get back to my usual setup.

But it's pretty easy to do this if you know how: just drag the bottom corner of the miniplayer window up until the window is roughly rectangular, and after releasing the mouse button:

This is actually quite a good way to change the layout of the miniplayer. (You can also make the window even smaller or even larger to get different results.) However, it's also extremely un-discoverable.

A free tip, from me to you.

Permalink - posted 2021-04-13

VLANs on the Mikrotik hAP ac³

▼ For my BGP lab/training setups I have a bunch of routers, real or virtual, that each need several IP addresses to talk to other routers. VLANs are the perfect way to keep all of this manageable: with VLANs, it's possible to have separate IP interfaces, but still just use a single Ethernet port to hook everything up. Simple unmanaged switches simply forward the packets to the right port without looking at the VLAN header, so no need to configure the network.

Until I got my Mikrotik hAP ac³ this week...

In its default configuration it will do what unmanaged switches do: ignore the VLAN header. In addition, you can add VLAN interfaces to the Mikrotik and give those an IP address. I even got IPv6 prefix delegation to work. For this, set up the DHCPv6 client to request a prefix. Then fill in the name of a pool. But don't create that pool yourself—that won't work. The pool will be created automatically. Then configured an IPv6 on a VLAN interface that I created and specified that the pool should be used for that. "Advertise" is enabled by default, which means that the Mikrotik will send out router advertisements and machines that have the VLAN configured will configure an IPv6 address for themselves on that VLAN interface.

Eventually I want to set up a guest network with its own VLAN and IPv4 NAT. For now I just set up IP addresses on the VLAN interfaces so the Mikrotik will be able to talk to my virtual BGP routers on these VLANs. That initially went well.

However, I also wanted have these BGP VMs talk to an old Cisco 2503 router, which doesn't support VLANs. So I wanted to set up an access port. The idea was that the Cisco would be part of VLAN 203 without the need for it to insert a VLAN TAG itself. On the other ports, VLAN 203 (and a few more) get a VLAN header. However, not exactly like a trunk port where all packets have a VLAN header: regular traffic to/from the internet goes out on these same ports untagged.

Mixed tagged/untagged ports turned out to be unsupported by the Atheros 8327 chip that powers the switch ports. What you can do is have one or more access ports and one or more trunk ports. This is already somewhat complex. The access ports need to be set up with a default VLAN. (Under "switch" and then "ports".) Makes sense. In the documentation, it says to use add-if-missing will add a VLAN header if there isn't one. Makes sense. However, it turns out that you need to do this on the outgoing port, i.e., not the access port itself, but the trunk port(s) that are part of the same VLAN. However, then (with a slight delay?) untagged traffic from the default VLAN won't work anymore.

The good news is that even though the hardware can't have mixed ports, the software can. So remove those settings above, and go to the bridge section of the configuration instead. There, select your bridge instance and turn on VLAN filtering. After that, you can go to the VLAN tab an add one or more VLANs there, where you can specify which ports should be tagged and untagged for that VLAN. So now we can have access ports and mixed ports. I do believe performance will be lower because VLAN traffic will need to be handled in software. Also not the PVID setting, which is the default VLAN untagged traffic. (1 out of the box.)

Unfortunately, the Mikrotik's IP addresses on the VLAN interface I tested with were now unreachable, and I couldn't figure out any settings to fix that. So after I was done with the Cisco 2503, I unchecked the "VLAN filtering" in the bridge settings... and the Mikrotek's IP addresses on VLAN interfaces came back to life.

Permalink - posted 2021-04-04

No joke: running BGP on a $100 home router / Wi-Fi access point

▼ For some time, I've been hearing about Mikrotek routers, which couple being quite capable with being affordable. But I never got my hands on one. I'm now in the process of upgrading my home network, and learned about the Mikrotik hAP ac³. The ac³ defies easy classification, but I think it's mostly a home router and/or Wi-Fi access point. I paid € 95 and shipping, and I believe it's available in the US for about $100.

I was somewhat disappointed to learn that "5 gigabit ports" doesn't mean ports that are capable of 5 gigabit, but 5 ports that just ordinary 1 Gbps Ethernet. Initially it seemed the box didn't support IPv6, but it turns you have to enable that under "packages" and then reboot. (Not shutdown.)

However, I wasn't disappointed to learn that the ac³ supports RIP, OSPF and BGP, both for IPv4 and IPv6. So I made a configuration on the ac³ that fits with the lab network that I use for my training courses. Initially it didn't seem to work, but then after a reboot of the ac³ the BGP sessions came up. Pretty cool!

The hardest part was setting up VLAN IP interfaces. For some reason I needed to use the command line to create the VLANs:

/interface vlan add interface=bridge name=vlan1401 vlan-id=1401
/interface ethernet switch vlan add ports=ether3,switch1-cpu independent-learning=no switch=switch1 vlan-id=1401


After that, I could use the web interface to add IP addresses. I also set up BGP through the web interface, but the command line export of that looks like this:

> /routing bgp export
# apr/01/2021 20:50:37 by RouterOS 6.48.1
# software id = C3HH-K6NW
#
# model = RBD53iG-5HacD2HnD
# serial number = E7290D96999E
/routing bgp instance
set default disabled=yes
add as=65082 client-to-client-reflection=no name=Router82 router-id=203.0.113.82
/routing bgp network
add network=10.0.0.0/8 synchronize=no
/routing bgp peer
add address-families=ip,ipv6 hold-time=1m30s instance=Router82 name=ISP30 out-filter=out-as-filter
remote-address=192.0.2.21 remote-as=65030
add instance=Router82 name=ISP40 remote-address=192.0.2.41 remote-as=65040


Initially it seems the Mikrotik won't show you any BGP info, but with a bit more digging you'll get there:

Way back in 1996 the global BGP table was 30,000 prefixes and that just about fit in the 16 MB that was the maximum a Cisco 2501 router would take. The ac³ has 128 MB storage (using less than half of that) and 256 MB RAM, which is probably enough for 200k - 500k prefixes but not a full table. And of course the ac³ is not an obvious choice for such a task in the first place.

Permalink - posted 2021-04-01