Subject: Re: [RAM] Number of DFZ routers

From: [email-masked]
Date: Mon, 28 May 2007 04:51:33 EDT

I think, these are excellent questions of common interest and would   
appreciate if knowledgeable folks communicated the asked information  on-line.
 
Heiner
 
In einer eMail vom 28.05.2007 00:36:19 Westeuropäische Normalzeit schreibt  
[email-masked]:

Can  anyone provide an estimate for the number of routers in
the  DFZ?

Transit routers  (DFZ)

Multihomed border  routers  (DFZ)

Singlehomed border routers

Can anyone  suggest the range of number of interfaces these
routers have?  For  instance, what is the largest number of
interfaces on an operational  router?

What would be a typical range of interface numbers  for
transit routers?

In addition to direct forwarding of Internet  traffic, and
handling the communications paths between the  routers
themselves, do transit routers often perform other  functions,
such as handling MPLS?

I would also be curious about  typical transit router models and
interface speeds.

Please respond  off-list if this is too off-topic.

-  Robin

_______________________________________________
RAM  mailing  list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: "Chris L. Morrow" <[email-masked]>
Date: Mon, 28 May 2007 15:41:52 +0000 (GMT)

> Singlehomed border routers

These are likely difficult to guess correctly... so I won't try, sorry.

> interfaces on an operational  router?

I think that the current 'best' number for this is likely the cisco IDB
maximum which is 2048? (idb = interface database) So, at a maximum a
router could have 2k interfaces (logical and physical).

I suspect that there are DSL platforms that scale beyond this, but they
may not live fully in the DFZ, the same probably applies to DOCSIS/UBR
sorts of devices.

> What would be a typical range of interface numbers  for

Range being speed? or number?

> themselves, do transit routers often perform other  functions,

This probably depends on the network in question... There are several
large providers that advertise use of MPLS in their core networks. The use
of MPLS could be purely for traffic engineering reasons or for services,
its not clear without asking the provider in question I suspect. Example
traceroutes may tell more:

 6  204.255.169.2 (204.255.169.2)  29 ms  16 ms  16 ms
 7  tbr1.wswdc.ip.att.net (12.123.8.114) [MPLS: Label 32690 Exp 0]  36 ms
48 ms  39 ms
 8  tbr1.sl9mo.ip.att.net (12.122.10.30) [MPLS: Label 32490 Exp 0]  34 ms
36 ms  37 ms

for instance...

> I would also be curious about  typical transit router models and

I suspect that most 'transit' routers in larger networks are the higher
end cisco/juniper devices supporting interface speeds of upwards of
2.5gbps.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 28 May 2007 18:01:40 +0200

> may not live fully in the DFZ, the same probably applies to DOCSIS/UBR

http://www.cisco.com/warp/public/63/idb_limit.html

---

From: "Chris L. Morrow" <[email-masked]>
Date: Mon, 28 May 2007 16:36:35 +0000 (GMT)

> http://www.cisco.com/warp/public/63/idb_limit.html

 sweet! so essentially 4k, not 2k. Thanks eliot.

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 28 May 2007 09:50:31 -0700

>  sweet! so essentially 4k, not 2k. Thanks eliot.

If you look at the tables on the Web page Eliot cited, it lists idb  
limits on a per-platform/per-software basis - some have more, some less.

<http://www.cisco.com/warp/public/63/idb_limit.html#idb_limits>

------------------------------------------------------------------------
Roland Dobbins <[email-masked]> // 408.527.6376 voice

You may not be interested in strategy, but strategy is interested in  
you.

                       -- Leon Trotsky

---

From: Simon Leinen <[email-masked]>
Date: Tue, 29 May 2007 23:43:40 +0200

Eliot Lear writes:
> Chris L. Morrow wrote:
>> I think that the current 'best' number for this is likely the cisco
>> IDB maximum which is 2048? (idb = interface database) So, at a
>> maximum a router could have 2k interfaces (logical and physical).

>> I suspect that there are DSL platforms that scale beyond this,
>> [...]

> http://www.cisco.com/warp/public/63/idb_limit.html

I think Robin's question was about *operational* routers, i.e.,
configuration of routers in actual use in the Internet, not about

>>> interfaces on an operational  router?

We're not a Tier-1, just a backbone for the academic community in a
small country (Switzerland), but since nobody so far gave *any*
numbers from an operational network, here are a few from our network.
Let's hope that this encourages others.

We have 40 backbone/peering/customer access routers.  I'm counting all
non-lab routers running (i)BGP in our backbone AS.

The number of non-tunnel, non-loopback, non-shutdown interfaces on
these routers varies between 3 and 38.  Most of the interfaces are to
point-to-point GigE and 10GE links (and three residual POS links),
with a few to multipoint GigEs (Internet exchange points or customer
DMZ's).  This is an ordered list of #s of non-trivial interfaces for
all our backbone routers:

3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 6 6 6 6 6 7 7 7 9 10 10 15
16 16 17 18 19 23 38

Our network is probably atypical in many respects - we have quite a
few routers with just three interfaces, which is sort-of the minimum
configuration where a router can make any sense at all.  We also have
no lower-speed access links (a single Fast Ethernet, all the rest are
GigE and 10GE).  We also don't run full BGP for IPv4 - 35'000 routes
is sufficient for us, thank you.
-- 
Simon.
SWITCH (AS559)

The following script may help produce numbers if you have Cisco
(-like) routers and use something RANCID:

#!/usr/local/bin/perl -w
##
## Grovel through RANCID directory, parse Cisco configuration files
## there, and compute simple statistics for backbone routers.
##
## In particular, this will print, for each non-test router with a BGP
## router configured in our backbone AS, the number of non-loopback,
## non-tunnel interfaces that aren't administratively shut down.
##

use strict;
my $config_dir = "/usr/local/rancid/backbone/configs";

chdir $config_dir or die "Cannot chdir to $config_dir: $!";

## Backbone AS - SPECIFIC TO SWITCH
my $my_as = 559;

my %interfaces;

foreach my $config (<*>) {
    next unless -f $config;
    open CONFIG, $config or die "open $config: $!";
    my $have_bgp = 0;
    my $hostname;
    my $in_interface = 0;
    while (<CONFIG>) {
	if (/^hostname (.*)$/) {
	    $hostname = lc $1;
	    ## Skip test routers - SPECIFIC TO SWITCH
	    last if $hostname =~ /^swi(6T1|6netCE1|(EZ|LS)T(32|72|76))$/i;
	} elsif (/^interface (.*)/) {
	    die "No hostname in $config"
		unless defined $hostname;
	    my $interface = $1;
	    next if $interface =~ /^(Loopback|Tunnel)/;
	    ++$interfaces{$hostname};
	    $in_interface = 1;
	} elsif ($in_interface) {
	    if (/^[^ ]/) {
		$in_interface = 0;
	    } elsif (/^ shutdown$/) {
		--$interfaces{$hostname};
	    }
	} elsif (/^router bgp (\d+)$/ and $1 eq $my_as) {
	    ++$have_bgp;
	}
    }
    close CONFIG or die "close $config: $!";
    delete $interfaces{$hostname} unless $have_bgp;

}

foreach my $hostname (sort { $interfaces{$a} <=> $interfaces{$b} }
		      keys %interfaces) {
    printf "%5d %s\n", $interfaces{$hostname}, $hostname;
}
1;
_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: "Chris L. Morrow" <[email-masked]>
Date: Tue, 29 May 2007 21:47:34 +0000 (GMT)

> hardware/software limits:

there was a reason I quoted the idb limits... on a SP network more ports
(logical and physical) you can stick in a rack the better you are. SP's
are always driving for more ports in less face-plate-space. The IDB limit
is probably a good start for today, and tomorrow I'd expect SP's to drive
that higher...

---

From: Simon Leinen <[email-masked]>
Date: Wed, 30 May 2007 00:11:38 +0200

> there was a reason I quoted the idb limits... on a SP network more
> ports (logical and physical) you can stick in a rack the better you
> are. SP's are always driving for more ports in less
> face-plate-space. The IDB limit is probably a good start for today,
> and tomorrow I'd expect SP's to drive that higher...

Sure, we all LIKE to have large routers with many ports, preferably
each generating LOT$ of revenue.  (And hopefully even *our* tiny
routers will by upgraded over time to break Cisco's IDB limits :-)

But it's slightly unnerving to me that nobody comes up with actual
numbers, even rough numbers, of what they actually have in the field.
Individual SPs may hide behind "company secrets", but maybe one could
run guesstimates from vendor and service provider yearly reports.

So Tony Li tells us a CRS-1 could support a million DS3 interfaces.
But how many DS3 interfaces - or any kind of interfaces - were sold
for CRS-1s? Let's try to put some real-world data into the discussion.
What I'm hearing so far reminds me too much of "traffic doubles every
100 days!".  Not good for basing rational decisions on if you ask me.
-- 
Simon.

---

From: "Chris L. Morrow" <[email-masked]>
Date: Wed, 30 May 2007 00:43:46 +0000 (GMT)

> routers will by upgraded over time to break Cisco's IDB limits :-)

I didn't say we'd like to have, I said that the IDB limit is there and
being pushed for a reason. It's not uncommon to have oc-12->ds1
channelized interfaces today at, 330+ ds1's per interface. It'd be not
uncommon to have 3-4 of these on a single chassis with 2 uplink
interfaces. That's 1200 interfaces on a single device.

> 100 days!".  Not good for basing rational decisions on if you ask me.

Also keep in mind that mutli-chassis systems exist now, I'm not sure
(perhaps Tony can say?) if the IDB limits scale with multi-chassis or stay
constant for a single 'system'... So, more interfaces still per 'system'.
There are multi-chassis systems in existence today in the field even.