Subject: Re: A tunneling proposal

From: xxvaf <[email-masked]>
Date: Mon, 16 Jul 2001 17:14:35 PDT

> Is there a known limit to the number of routes in the global routing
> table? I know bad things started to happen at about 4000 and 10000, but
> obviously those problems have been solved. Routers run just fine with 100,000
> routes at present, and unless I'm mistaken, the most common types of router
> have CPUs and memory that are well below what most of us have in our desk top
> PCs.

The veracity of "Routers run just fine with 100,000 routes..." depends on
how you define "fine".

Modern routers don't fall over with 100,000 routes in them. But initial table
load and BGP convergence times when paths change are both a lot longer than
many would like.

> On top of that, each route takes a LOT of memory: 240 bytes for the routing 
> table and for each peer route in the BGP table in a Cisco.
> 
> If we can both increase memory and decrease the route/BGP table entry size by
> an order of magnitude, we should be able to run with a global routing table
> of 10,000,000 routes. That's about the entire IPv4 space as individual /24's.

The size of individual routing state entries in modern routers has been the
subject of a great deal of optimization over the years. Don't expect to see
it improved by an order of magnitude or even by a factor of two. 

Memory size is not the principal issue; memory speed and routing table update
bandwidth are.

> I think it's worth it to look at this, because with CIDR it is pretty much
> impossible to efficiently route traffic: many locations are hidden behind
> aggregates. Someone in Chicago has to choose between routing traffic for the
> aggregate to the west coast or the east coast, and in either case some
> traffic will have gone in the wrong direction.

If global routing state size resumes a hyper-exponential growth pattern that
exceeds Moore's law, the problem will get worse faster than CPUs increase
in speed. Eliminating CIDR will guarentee hyper-exponential growth - for all
the talk about how CIDR has "failed" in that growth is continuing, the growth
rate would be far, far worse without it - aggregation has successfully
"hidden" at least an order of magnitude of growth.

CIDR-style aggregation is the only technology that has been shown to contain
routing state growth to a manageable level.

Look back at old archives of the IETF and other lists to read some of Noel
Chiappa's and others' writings on the mathematics of network topology and
addressing. If the "addresses" used by routing don't follow the underlying
network topology, excessive state is introduced. For multihoming to really
work, it needs to use topologically-significant addressing. That suggests
that the "multi" in "multihoming" also implies multiple addresses, with
something like SCTP to handle them intelligently.

	--Vince

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 17 Jul 2001 11:21:55 +0200 (West-Europa (zomertijd))

> The veracity of "Routers run just fine with 100,000 routes..." depends on

A few weeks ago, someone posted a survey on the list. Nobody answered "we
can't handle the huge number or routers" on it.

> load and BGP convergence times when paths change are both a lot longer than

Ok, but is that enough reason to throw out the current way of multihoming?

> it improved by an order of magnitude or even by a factor of two. 

Really? It seems to me that this has not changed since I first started
running BGP in 1995.

Optimizations have been done under the current CIDR-paradigm, where every
route is unique and carries a lot of additional information (such as
communities) with it. If we optimize for very many nearly identical routes
the results could be very different.

> Memory size is not the principal issue; memory speed and routing table update

You have a point on the table updates. A solution for that could be to only
allow information about routes going down to be immediately forwarded, since
this is presumably an operation that can be optimized not to be very
expensive. New route announcements could be slowed down.

I don't believe memory speed is a reason to keep the routing table at its
current size: doing a binary search on 10,000,000 rather than 100,000 items
only takes 24 steps rather than 17. If this is a problem for 10,000,000
routes, 100,000 routes is too much as well.

> > aggregates.

> in speed.

Do we have any inidication that the global routing table is growing faster
than the Moore's Law rate?

> Eliminating CIDR will guarentee hyper-exponential growth - for all

I'm not saying "CIDR is dead, everybody stop aggregating". But CIDR has
reached the limits of its capabilities. Just look at the routing table vs the
number of AS numbers: for every assigned ASN there are five routes. This has
very little to do with the evil multihomers, since they have little reason to
announce more than a single route.

> something like SCTP to handle them intelligently.

1. Forget SCTP: if we want this, we should build it into TCP and not change
   transport layers to a protocol that has just one desirable feature and
   forget 20 years of experience with the most successful protocol in
   history.

2. The current way of multihoming works much better for the multihomed
   network, what is their incentive to go with multiple addresses?

Iljitsch van Beijnum

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 17 Jul 2001 09:05:34 -0400

At 05:21 17/07/01, Iljitsch van Beijnum wrote:
>> The veracity of "Routers run just fine with 100,000 routes..." 
>> depends on how you define "fine".
>

NOTE:
        Probability of a given person answering a survey on the NANOG list 
is inversely proportional to the probability that the person works
for a major IP carrier and has at least one clue about routing.

>> Modern routers don't fall over with 100,000 routes in them. 
>> But initial table load and BGP convergence times when paths change 
>> are both a lot longer than many would like.
>

        Yes.

>>> On top of that, each route takes a LOT of memory: 240 bytes 
>>> for the routing table and for each peer route in the BGP table 
>>> in a Cisco.
>
>> The size of individual routing state entries in modern routers 
>> has been the subject of a great deal of optimization over the years. 
>> Don't expect to see it improved by an order of magnitude or even 
>> by a factor of two. 
>
>Really? 

        Yes.

>> Memory size is not the principal issue; memory speed and 
>> routing table update bandwidth are.
>

Immediately forwarding information about routes going down
increases the load on the overall routing system due to route flaps.

Dealing with this system loading issue caused by route flaps
is precisely why BGP has dampening.

>routes, 100,000 routes is too much as well.

Some think that 100K prefixes is already uncomfortably large 
for the algorithms currently being used.  Others think that
that the critical point is more nearly 200K prefixes.  Folks
mileage varies somewhat on where the critical point lies
with the current algorithms.  Note my phrasing.  I talk about
"algorithms" rather than "implementations" quite deliberately.

>Do we have any inidication that the global routing table is 
>growing faster than the Moore's Law rate?

As I recall, tli has made precisely this claim in the past.  
I'm strongly inclined to just take tli's word on such a point.

>> the talk about how CIDR has "failed" in that growth is continuing, 
>> the growth rate would be far, far worse without it - aggregation 
>> has successfully "hidden" at least an order of magnitude of growth.
>
>reached the limits of its capabilities. Just look at the routing table 
>vs the number of AS numbers: for every assigned ASN there are five routes.

However, a typical AS is not originating 5 routes.  One needs to be
fairly careful in playing with statistics in this space.

>   forget 20 years of experience with the most successful protocol in

In the past, this sort of change to TCP was precisely forbidden
procedurally.  I agree the community should seriously examine ways
in which the clues from SCTP multihoming might be injected into
the UDP and TCP protocols.  I don't understand if examining such
changes would be within scope for this WG or would belong elsewhere.

>   network, what is their incentive to go with multiple addresses?

Economics is a major constraint on the potential solution space.

Ran
[email-masked]

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 17 Jul 2001 17:16:47 +0200 (West-Europa (zomertijd))

> for a major IP carrier and has at least one clue about routing.

I don't know about the carrier size, but from what I saw these people seemed
to have some clue at least.

>         Yes.

Well, I disagree. Slow initial sync and convergence times are bad, but
outlawing current multihoming is very bad as well. And I'm certaintly not
convinced this is well-optimized. Not much is.

>         Yes.

So how does this relate to my experience that the amount of memory used by
BGP table entries is the same as six years ago?

> increases the load on the overall routing system due to route flaps.

Going down just once isn't really flapping. Flapping is when routes keep
coming up and going down. Going down shouldn't have to be an expensive
operation if you leave the information in place but just mark it as
"invalid". Coming up means inserting information, which is a more expensive
operation. If we slow this down, the routes can't go back down again. When
going down means staying down for a while, people will start to pay more
attention to unnecessary flapping.

> is precisely why BGP has dampening.

Unfortunately, a major source of flapping seems to be unaddressed: when an
AS has a route coming in from a lot of different directions, a lot of iBGP
updates will start flying around when this route disappears. I wouldn't be
surprised if each of those iBGP state changes also caused an eBGP flap.

> for the algorithms currently being used.

Which algorithms? Forwarding or route processing?

> Others think that

I've read about tests with 2 million routes on some boxes.

Note that CIDR causes route lookup algorithms to be less efficient: in stead
of just finding a single match, the algorithm has to eveluate all possible
prefix lengths. Unless there is some brilliant algorithm for this I'm unaware
of, this makes the lookup about 1.5 - 2 times as expensive.

> I'm strongly inclined to just take tli's word on such a point.

Well, have a look at http://www.cisco.com/warp/public/759/ipj_4-1/ipj_4-1_bgp.html

It seems the growth between the beginning of 1999 and the beginning of 2001
was a little less than a factor two, so we're a little below Moore, but not a
lot. (But enough to reach 1 billion routes in 2027 rather than 2021.) It
would be interesting to know how much of this growth is from multihoming.
Since the absolute number of routes grows much faster than the absolute
number of ASNs, most of the growth has to be from poor aggregation, probably
due to the address conservation efforts of the regional registries.

> Economics is a major constraint on the potential solution space.

I'm not even talking about economics, but just basic functionality and human
nature: how are you going to convince someone stop doing something that works
for them to help some greater good, that doesn't show obvious signs of
needing help? "Global warming? Let me turn up the airco..."

If there really is a limit above which global routing breaks down, we should
implement some policies to prevent the number of routes from reaching this
limit. This means forcing existing ASes to aggregate, and limiting the growth
in the number of ASes. That way, SCTP-like solutions become more attractive.

But I think we can:

1. optimize protocols
2. optimize implementations
3. look for alternative ways to aggregate, for instance on geography

Iljitsch van Beijnum

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Jul 2001 18:31:35 +0300 (EEST)

> in the number of ASes. That way, SCTP-like solutions become more attractive.

How about "all the specifics you'd want to annouce are being filtered by
other parties in the Internet so your multihoming only works for 1% of
Internet users" ?

Could be rather effective if people stick to these prefix length
filters...

-- 
Pekka Savola                 "Tell me of difficulties surmounted,
Netcore Oy                   not those you stumble over and fall"
Systems. Networks. Security.  -- Robert Jordan: A Crown of Swords

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Tue, 17 Jul 2001 08:41:20 -0700 (PDT)

> in the number of ASes. That way, SCTP-like solutions become more attractive.

Currently, multihomers pay each their ISPs extra to carry non-aggregatable
routes. This means if the ISPs encounter problems with the
additional routes, they are directly going to charge the sites
more. This way, one can impose artificial constraints on how
many sites multihome, but this is exactly what we are trying to avoid.

As for backpatcing TCP to work with multihoming, this would
not work with the current apps, and socket API if done the SCTP
way---SCTP requires the apps to *explicitly* give a list of addresses
to use on a connection. To backpatch TCP to do the same would mean
requiring the TCP layer itself to figure out what additional addresses
it can use *without* the app knowledge. This means that the protocol
stack can get it wrong if there is address translation going on
for getting provider independence, etc...
Ultimately, it will basically look like GxSE...

-ramki

---

From: Tony Li <[email-masked]>
Date: Tue, 17 Jul 2001 13:30:03 -0700

| > >> Modern routers don't fall over with 100,000 routes in them.  But
 | > >> initial table load and BGP convergence times when paths change are
 | > >> both a lot longer than many would like.
 | 
 | > >Ok, but is that enough reason to throw out the current way of
 | > >multihoming?
 | 
 | >         Yes.
 | 
 | Well, I disagree. Slow initial sync and convergence times are bad, but
 | outlawing current multihoming is very bad as well. And I'm certaintly
 | not convinced this is well-optimized. Not much is.

You are welcome to disagree.  However, the facts of the matter are very
clear.  If the Internet is to grow, it will continue to add 'sites' to it,
almost indefinitely.  Some percentage of those sites will want to
multihome.  Under the current multihoming architecture, those sites will
appear in the DFZ routing table.  If the growth rate of those sites exceeds
Moore's law, then it will be become exceedingly expensive to support the
routing subsystem.

It is vital to the life of the net that the routing subsystem continue to
scale well.  The price of this scalability is the discipline to adhere to
a scalable architecture.

Are you willing to pay that price?

 | > >> The size of individual routing state entries in modern routers has
 | > >> been the subject of a great deal of optimization over the years.
 | > >> Don't expect to see it improved by an order of magnitude or even by
 | > >> a factor of two.
 | 
 | > >Really?
 | 
 | >         Yes.
 | 
 | So how does this relate to my experience that the amount of memory used
 | by BGP table entries is the same as six years ago?

Your experience does not match mine.

 | > >Do we have any inidication that the global routing table is growing
 | > >faster than the Moore's Law rate?
 | 
 | > As I recall, tli has made precisely this claim in the past.  I'm
 | > strongly inclined to just take tli's word on such a point.

To be 100% accurate: I made the point that the rate of growth is increasing
and that if left unchecked, this implies that the growth will someday
exceed Moore's law.

 | I'm not even talking about economics, but just basic functionality and
 | human nature: how are you going to convince someone stop doing something
 | that works for them to help some greater good, that doesn't show obvious
 | signs of needing help? "Global warming? Let me turn up the airco..."

We're well aware that we have a tragedy of the commons scenario.  The first
step has to be to convince you that the IPv6 routing architecture needs to
be something scalable.

 | If there really is a limit above which global routing breaks down, we
 | should implement some policies to prevent the number of routes from
 | reaching this limit. This means forcing existing ASes to aggregate, and
 | limiting the growth in the number of ASes. That way, SCTP-like solutions
 | become more attractive.
 | 
 | But I think we can:
 | 
 | 1. optimize protocols

This has been in progress for 8 years.

 | 2. optimize implementations

This has also been in progress for 8 years.

 | 3. look for alternative ways to aggregate, for instance on geography

Local geographic aggregation (i.e. metro aggregation) has issues about
forced connectivity that we are not able to regulate.  Global geographic
aggregation (i.e., at the continental scale) has been struck down as a bad
idea. 

If you ignore history, you're doomed to repeat it.  Just like this
conversation.

Tony

---

From: "Jon (Taz) Mischo" <[email-masked]>
Date: Tue, 17 Jul 2001 20:32:24 -0500 (CDT)

>    forget 20 years of experience with the most successful protocol in

Wow...let's not use something based on 20 years of experience that already
exists, let's re-invent the wheel instead.

>    network, what is their incentive to go with multiple addresses?

The current way?  The incentive is seamless failover.

-Taz

-- 
        "Be liberal in what you accept,
      and conservative in what you send."
--Jon Postel (1943-1998) RFC 1122, October 1989

---

From: Geoff Huston <[email-masked]>
Date: Wed, 18 Jul 2001 10:48:07 +1000

> > in the number of ASes. That way, SCTP-like solutions become more 

It would be good if we could say "this is the number of routes in the 
global table which will cause the integrity of the global routing system to 
collapse". Unfortunately we know of no such hard limit. We suspect that 
somewhere between where we are (a little over 100,000) and 4 billion route 
entries (every /32 address fragment is announced globally) there is such a 
limit. Our concern is that we will not know what such a threshold is until 
a number of As's cross it.

>many sites multihome, but this is exactly what we are trying to avoid.

Do you have some evidence of such an arrangement? While I've heard of such 
charging for routing entry schemes in the past I've yet to see it become 
commercially viable in a heavily contested IP transit market. Your comment 
suggests that such a scheme has become commercially viable in some IP 
transit markets, and it wouldbe interesting to learn more about such a 
development.

>Ultimately, it will basically look like GxSE...

The problem is that multi-homing may be a number of AS's away. i.e. the 
local AS may single-home to an upstream AS which in turn may multi-home to 
a number of AS's via peering connections, multiple upstreams, etc. How then 
does the local TCP session  understand the situation if the upstream AS is 
doing multi-homing via dynamic translation of the routing goop part of the 
end point's address?

regards,

   Geoff

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Tue, 17 Jul 2001 21:47:25 -0700 (PDT)

> doing multi-homing via dynamic translation of the routing goop part of the

In SCTP, it is the job of the app to be feed the transport with a list of
correct globally routable destination addresses, and be prepared to
receive error notifications after an address fails. This is different
than the GxSE's behavior of making the destination advertise the addresses
where it can be reached.

As for GxSE's behavior in the multi-level multihoming scenario,
I think Paul is probably the best person to
answer this, as I am not certain how the site border router knows to
signal the upstream provider's router to attach the address-list extension
header.

As far as tunneling is concerned, although not strictly related to
multihoming per se, one can use address translation (or goop rewriting) to
get provider independence. In this multi-level multihomed case, this
translation would work as follows.

Consider the following concrete scenario, to ease explaining things a
bit:
                                 D
                                 |
                           +-----+-----+
                           |  Internet |
                           +---+----+--+
                              |      |
                             |        ||
                            |           |
                        //-+\\   L5  //--+-\\
                       |  ISP ******* ISP2   |
                       |  1   |      * --+-//
                        \\-+/|     **   |
                             |   **L2   | L3
                          L4 | **     /++
                       +--+---*      |    \
                       |    R2|+-----+R1  |<--Multihomed Enterprise E
                       |ISP3  |       \-+/
                       +------+  L1     |
                                        S

Assume that E multihomes via ISP3 and ISP2, whule ISP3 itself multihomes
via ISP1 and ISP2. Further, assume that both ISP3 and E use PI addresses,
and border exit routers R2 and R1 respectively rewrite the source
addresses on outgoing packets, and destination addresses on incoming
packets. Destination D will correctly see a globally routable address for
source S even after the packet uses R1-L1-R2-L4-ISP1-(Internet)-D, while
reverse packets correctly reach S with the PI address set by E. Thus, S
does not need to know a globally routable address as part of the ISP3
prefix space (it may still have a globally routable address from ISP2
prefix space to use for traffic via S-L3-ISP2-(Internet)).

If ISP2 has problems, and links L3 become unusable, R1 tries to setup a
tunnel to ISP2 using a globally routable address in ISP2 via ISP3 so that
existing connections can survive. Note that ISP3 does not have to setup a
tunnel himself---when R1 requests a tunnel setup, since L2 is assumed to
have become useless, he has no other way than use L4-ISP1-ISP2 as the path
on which to forward the tunnel setup request as well as further tunnel
traffic. ISP2 knows the globally routable address of R1 because ISP3 would
have translated it to become a globally routable one, and now ISP2 would
use L5-ISP1-L4-L1 as the tunnel path for traffic that was previously
using D-Internet-ISP2-L3-E-S. Thus, nothing short of a complete outage
within ISP2 should hamper traffic flow.

The sticky question seems to be whether it is acceptable for R1 to request
ISP3 to trigger a global BGP announcement for the site's address prefix
drawn from ISP2 so that traffic can be forwarded via
(Internet)-ISP1-L4-ISP3-L1 directly instead of a tunnel. I don't know what
bad things happen in this case.

Of course, the unworked out part in the above is how R1 knows a valid
address in ISP2 that can be used as the other tunnel endpoint.

thanks,
ramki

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Tue, 17 Jul 2001 21:55:57 -0700 (PDT)

> If ISP2 has problems, and links L3 become unusable, R1 tries to setup a

in my previous mail should read:

If ISP2 has problems, and links *L2 and* L3 become unusable, R1 tries to
setup a (etc.)

-ramki

---

From: "R.P. Aditya" <[email-masked]>
Date: Tue, 17 Jul 2001 22:10:05 -0700

The crux of the problem with the tunnel approach, and/or the conditional
announcement approach is not technical, it is that it requires a level of
inter-provider cooperation that experience has shown is not likely to exist.
Requiring it will likely engender yet more resistance to migrating to v6.

Furthermore, it is more complex for the customer and providers. I can easily
envision scenarios where one provider will refuse to create a tunnel to
certain other providers based on politics or even internal lack of capacity.

Customers will be forced to multihome only with providers who have agreed to
work together; providers will have to trust customers to send them automatic
messages to make what amount to TE decisions within their network. Just
getting providers to accept well-known communities (saying nothing of getting
them to tell you) is painful now.

Adi

---

From: Tony Li <[email-masked]>
Date: Tue, 17 Jul 2001 22:37:38 -0700

| In SCTP, it is the job of the app to be feed the transport with a list of
 | correct globally routable destination addresses, and be prepared to
 | receive error notifications after an address fails. 

I don't mean to be a pedantic layerist, but it seems to me that the network
and transport layers probably should keep the multihoming problems out of
the hands of the applications.

Tony

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Tue, 17 Jul 2001 22:55:43 -0700 (PDT)

For most applications, what you say is true. But given that SCTP was first
devised for SS7 signaling where operator alerts, log generation, etc. may
be required, I can sympathize with this design...

-ramki

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Tue, 17 Jul 2001 23:02:24 -0700 (PDT)

Also, I do know that, currently, large server farms have about 11
connections to the Internet that run at roughly 90% of the capacity, so
that they can gracefully tolerate a link failure. Whether these 11 links
feed into just one or many ISPs is something that I am not very certain.

regards,
ramki

---

From: "Jon (Taz) Mischo" <[email-masked]>
Date: Wed, 18 Jul 2001 10:58:05 -0500 (CDT)

> be required, I can sympathize with this design...

Also, it doesn't HAVE to be that way.  You *CAN* bind to all the addresses
the machine has.

-Taz

---

From: "Jon (Taz) Mischo" <[email-masked]>
Date: Wed, 18 Jul 2001 11:09:30 -0500 (CDT)

On Wed, 18 Jul 2001, Paul Francis wrote:

> The signalling is done by the end host...the end host attaches an extension
> that is basically a placeholder for the prefix-list that the site border
> router fills in.

> I hadn't considered the multi-level case.  My first thought is that the site
> border router should be configured to know of the multiple upstream
> prefixes, although this does open up the problem of having to reconfigure
> all of the site border routers when the mom&pop ISP decides to
> change/add/delete its upstream ISP...

This is required anyway, so it's not a big deal.  Besides, as I said
before, if you propagate some of this information into routing, such
changes can be less painful.

-Taz

---

From: "Paul Francis" <[email-masked]>
Date: Wed, 18 Jul 2001 08:33:14 -0700

PF

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Wed, 18 Jul 2001 09:24:00 -0700 (PDT)

On Wed, 18 Jul 2001, Jon (Taz) Mischo wrote:

> > > Furthermore, it is more complex for the customer and providers. I
can
> > easily
> > Internal lack of capacity seems a valid technical reason; this should
be
> > addressed in the SLA between the provider and the subscriber. Note,
> > however, that we
> > are not asking one provider to carry traffic, tunneled or otherwise,
that
> > *exceeds* the subscriber connectivity bandwidth. The amount of traffic
> > would in no (significant) way be altered if the multihomed subscriber
> > became singlehomed, and received/sent all the traffic along the
original
> > line.
>
> Yes you are.  You are asking a provider to take all traffic destined for
a
> specific egress point and re-route it through their network to a peering
> point.  You are *DOUBLING*  the traffic they are carrying for that
> subscriber.

I am sorry, but I may be missing something here---true, the traffic is
doubled, but how to deal with the doubled traffic should be a decision
between the provider and the subscriber according to the SLA. Increased
load could cause higher packet loss, greater latency, etc. Indeed, if the
provider were using fair queuing, this would what happen, without
affecting any other customers of the provider.

Like I said before, how is this case different than the one where the
subscriber cancels the service from the other provider, due to which the
load on the first provider would be increased anyway? Remember that we
are not even requiring the first provider to participate in the tunnel
setup...

> > > work together;
> > This is a difficult question for me to answer. But what kinds of
> > arrangements are drawn up for carrying multicast traffic, where these
> > kinds of political problems are potentially more severe?
>
> Multicast traffic is a whole other ball of wax.
>
> > Also, I am not sure if this wg should outlaw any proposals based on
> > political reasons alone...If so, we probably should have the rough
> > guidelines rewritten down in the requirements document.
>
> This isn't based on political reasons, but feasibility reasons, and we
> don't outlaw, we just decide what we predict will be best.

Well, there is an easy, although socially questionable, answer to
that---use IPSec on the tunnel so that the provider can not find out
anything about the payload...

Once again, I don't see how doing this, while suboptimal and probably
unfair, is going to violate the contract between the provider and the
subscriber. All the provider would see is a surge in the traffic for the
subscriber, and he will probably use the usual tools (fair queuing,
increased latency and drop rate, etc. ) to deal with them...

thanks,
ramki