Overview

Normally on a peering exchange, all connected parties will establish bilateral peering relationships with each other member port connected to the exchange. As the number of connected parties increases, it becomes increasingly difficult to manage peering relationships with members of the exchange. A typical peering exchange full-mesh eBGP configuration might look something similar to the diagram on the left hand side.

Full mesh peering
Route server peering

The full-mesh BGP session relationship scenario requires that each BGP speaker configure and manage BGP sessions to every other BGP speaker on the exchange. In this example, a full-mesh setup requires 7 BGP sessions per member router, and this increases every time a new member connects to the exchange.

However, by using a route server for all peering relationships, the number of BGP sessions per router stays at two: one for each route server. Clearly this is a more sustainable way of maintaining IXP peering relationships with a large number of participants.

Should I use this service?

The Denver IX route server cluster is aimed at:

  • small to medium sized members of the exchange who don’t have the time or resources to aggressively manage their peering relationships
  • larger members of the exchange who have an open peering policy, but where it may not be worth their while managing peering relationships with smaller members of the exchange.

The general advice is, if you do not have any good reason not to use the route server cluster, you should probably use it.

The service is designed to be reliable. It operates on two physical servers, on different network segments. The service is available on both IPv4 and IPv6. Each server runs a separate routing daemon per protocol. Should a single BGP server die for some unlikely reason, no other BGP server is likely to be affected. If one of the physical servers becomes unavailable, the second server will continue to provide BGP connectivity.

Denver IX has also implemented inbound prefix filtering on its route-server cluster. This uses both RPKI and internet routing registry data from the various IRR databases to allow connected members announce only the address prefixes which they have registered publicly. If your prefix has a valid RPKI ROA, it will pass. If the RPKI ROA check result is unknown (you have not set up a ROA), we fall back to the IRRDB test. Invalid ROAs are rejected.

Denver IX uses BIRD (Bird Internet Routing Demon) running on Debian 9 LTS for its route server cluster. BIRD is widely used at internet exchanges for route server clusters, and has been found to be reliable in production.

How do I use this service?

If enabled in the portal, the route servers are setup to accept BGP connections from your router. Once this has been done, you need to configure a BGP peering session to the correct IP address and protocol. Peering with the route collector is mandatory and used for network analysis only. The addresses are as follows:

Fabric Route Collector 01 Route Server 01 Route Server 02
LAN01, IPv4 149.112.18.4 149.112.18.2 149.112.18.3
LAN01, IPv6 2001:504:109::4 2001:504:109::2 2001:504:109::3
LAN01 ASN AS399557 AS399557 AS399557

We recommend consulting your manufacturer's manual for the exact BGP setup on your router/switch as this can vary. Remember to setup sane import and export policies that make sense for your use case.

Note that the route server system depends on information in either RPKI (setup with your RIR, typically ARIN) or the various IRR databases (typically the ARIN or RADB IRR database). If you have not published either a valid ROAs or correct route:/route6: objects in this database, your prefix announcements will be ignored by the route server and your peers will not route their traffic to you via the exchange. This is checked by Denver IX Operations during provisioning and guidance and assistance is provided as required.

Community based prefix filtering

The Denver IX route server system also provides well known communities to allow members to control the distribution of their prefixes. These communities are defined as follows:

Description Community
Prevent announcement of a prefix to a peer 0:peer-as
Announce a route to a certain peer 64599:peer-as
Prevent announcement of a prefix to all peers 0:64599
Blackhole a route 64599:666

Example: to instruct the route server to distribute a particular prefix only to AS64111 and AS64222, the prefix should be tagged with communities: 0:64599, 64599:64111 and 64599:64222.

Alternatively, to announce a prefix to all Denver IX members, excluding AS64333, the prefix should be tagged with community 0:64333.

Denver IX’s route server clusters support BGP large community prefix distribution control on all peering fabrics, using the following policy:

Description Community
Prevent announcement of a prefix to a peer 399557:0:peer-as
Announce a route to a certain peer 399557:1:peer-as
Prevent announcement of a prefix to all peers 399557:0:0
Announce a route to all peers 399557:1:0

Denver IX’s route server clusters also support BGP large community AS path prepending control on all peering fabrics, using the following policy:

Description Community
Prepend to peer AS once 399557:101:peer-as
Prepend to peer AS twice 399557:102:peer-as
Prepend to peer AS three times 399557:103:peer-as

If your router supports large communities, you should use extended communities over standard 16-bit communities as a large number of Denver IX members (including Denver IX itself) now have a 32-bit ASN. You should not mix standard 16-bit communities and large communities – please choose one or the other, we highly recommend using extended communities.

Looking Glasses

You can find Denver IX’s looking glasses for all route server (and route collector) instances here: https://www.denverix.com/technical/looking-glass.

If the route server is filtering a prefix, it will show in the looking glass with a warning symbol in the route listings under State/PfxRcd. You can click on Details next to this to see why a prefix is filtered.

Filtering Policy

Denver IX’s Route Server filtering policy is summarized below:

  1. Drop small prefixes – longer than /24 for ipv4 and longer than /48 for ipv6.
  2. Drop all well-known martians and bogons.
  3. Ensure that there is at least 1 ASN and less than 64 ASNs in the AS path.
  4. Ensure that the peer AS is the same as the first AS in the AS path.
  5. Drop any prefix where the next-hop IP address is not the same as the peer IP address. This prevents prefix hijacking.
  6. Drop any prefix with a transit network ASN in the AS path.
  7. Ensure that origin AS is in set of ASNs from the client’s IRRDB AS-SET.
  8. If the prefix is evaluated as RPKI valid, accept.
  9. If the prefix is evaluated as RPKI invalid, drop.
  10. If the prefix is evaluated as RPKI unknown, revert to standard IRRDB prefix filtering.

RFC1997 Pass-Through

RFC1997 defines some well-known communities including NO_EXPORT. Denver IX’s route servers do not interpret these well-known communities but passes them through.