Subject: Re: A new draft on inter-domain level resource management

From: Ping Pan <[email-masked]>
Date: Mon, 17 Jan 2000 18:03:47 -0500

Curtis Villamizar wrote:
> 
> 
> You should be clear in your discussions of BGRP what problem it is you
> are trying to solve.  BGRP is trying to solve the scalability problems
> that arrise when using MPLS for microflows.  This is what is commonly
> referred to as QoS routing.
> 

Hi, Curtis,

BGRP is not a routing protocol. It uses the information provided by BGP4
(specifically, AS_PATH and NEXT_HOP attributes) to set up reservation
trunks in the backbone. First, it involves only with BGP border routers.
Second, it further reduces the number of "flows" a border router has to
maintain.

I think QoS routing is to periodically advertise link/resource
information throughout the network, which is not really what we are
targeting here. Sorry about the confusion.

> [ aside:
> 
> This is why I have insisted that we not confuse the term "QoS routing"
> with "traffic engineering".  The two have similarities, but some very
> significant differences.  Traffic engineeering as it is currently
> practiced by ISPs is not concerned with microflows (even very large
> ones) and in a stable network would be expected to generate zero
> setups despite huge numbers (could be hundreds of thousands or
> millions) of microflows appearing and going away per second.
> 

For traffic engineering, there could be many long-lived "flows" that
established by the ISP's. The cost of setting them up in terms of
processing CPU cycles and storage space may not be a big deal, but the
cost of maintain these "flows" may be a big problem for the routers.
Soft-state is a good idea to allow the routers to adjust to route
changes. But the side-effect here is that soft-state requires the
routers to maintain a copy of all the states. If there are thousands or
millions of microflows that need to refresh, I doubt the routers can
sustain the CPU cost.

> For traffic engineered backbones, the total number of LSPs is bounded
> by the number of LSRs in the network and the number of classes of
> service offerred.  Most providers estimate the total number of LSPs
> that will be needed in their network to be on the order of 10^3 or
> 10^4.  The number going through any one LSR should be even smaller.
> The setup rate should normally be zero or very close to zero except in
> response to network failure.

Actually, as being pointed out in RFC2430 (PASTE by Tony Li and Yokov
Rekhter), the number of trunks (or LSP's, or reservations) is N * (N-1)
* C, where C is the number of service classes, N is the number of
MPLS-speaking routers. This is the N**2 problem we are talking about
here. In case of intra-domain, N is the number of border routers at
backbone edge, and is in the range of ~100. To setup a full-meshed
network to support traffic engineering, the ISP needs indeed 10^3 or
10^4 trunks. The current RSVP/LDP should have no problem for
intra-domain.

But to setup such trunks over multiple domains, we will have problem
here. As pointed out in our draft, we have thousands of AS's and routes
over the network. To provide a similar level of traffic engineering
functionality in multi-domain environment would require millions of
trunks. We proposed a new protocol here to provide trunk setup mechanism
for inter-domain only.

When routers use BGRP to setup inter-domain trunks, the border routers
will interface with intra-domain signaling protocols to set up internal
trunks. BGRP is hopping between the border routers to trigger LSP
setups.

> 
> Some of those estimating higher than 10^4 LSPs tend to be interested
> in voice and counting on doing one LSP per T1 voice trunk (not even
> aggregating to trunk group) rather than engineering IP networks.
> Those who believe that MPLS traffic engineering should be used solely
> to provide more efficient routing through provider backbones may
> consider this broken by design (though it may be essential to some
> VoMPLS approaches, not that it would be the first time something
> broken by design was proposed within or outside the IETF).
> 

IMHO, I hope the ISP's consider traffic in terms of service classes and
bandwidth only (as being defined in DiffServ), not in terms of
application. The end users at network edge can choose any service class
to carry their voice depending on how much they are willing to pay. 

>  ... end of aside]
> 
> 
> A question for the MPLS WG is "is this a problem the MPLS WG is trying
> to solve?"  Perhaps the MPLS deployment WG in the OPs area can help
> provide an answer.
> 

I have the same question. That's why I send the note to multiple lists.
;-)

> Curtis

Regards,

--
Ping Pan	http://www.cs.columbia.edu/~pingpan

---

From: Jon Crowcroft <[email-masked]>
Date: Tue, 18 Jan 2000 11:14:23 +0000

In message <[email-masked]>, Ping Pan typed:

hmm - interesting - thouhg i am still not sure why you can't work on
an evolutionary path with RSVP rather than a whole new protocol 

but alternatively,
what i'd like is to do a joint re-design of inter-domain routing, and
inter-domain signaling rather than just pile more and more things into
BGP and new signaling protocols...one possibility:

 to use NIMROD architecture 
http://ana-3.lcs.mit.edu/~jnc/nimrod/docs.html
 to exchange two types of thing:

1/ tag a map for a region (aka core, or trunk area) with current traffic
load advertisement (what one wishes to export as effective available
capacity -the map based approach alows you to scale state and size of
exchanges...the use of IP or AS numbers or MPLS labels as the means of
description of flows or aggregations is, imho, doomed to fail...

2/ tag requests for class based trunk adaptive sized DELTAs in
capacity requests from routers at the edge
of such a region - delta's allow you to scale the message rate, as
well as the state - as the rate of edge device requests increases
(iptel call setup or teardown rate goes up), you just modify the
deltas to anticipate the desired trunk sizes...rsvp can be modified to
do this relatively easily (documents on aggregate reservations only
need modest changes in identifying who is requesting what of whom...)

some discussion from noel chiappa and others might ensue (pnni
too...?)

 cheers

   jon

---

From: Ping Pan <[email-masked]>
Date: Tue, 18 Jan 2000 08:08:02 -0500

> an evolutionary path with RSVP rather than a whole new protocol

We've tried very hard to address the scaling problem with RSVP in the
past year or so. Lou Berger, George Swallow, Lixia (and her students)
and us have tried to enhance RSVP soft-state properties in various
drafts. At this point, I believe RSVP (and its LSP extension) will have
no problem whatsoever to handle intra-domain requirements. But RSVP does
have the N**2 problem. When you look at the data we had gathered in
Section-2 of our draft, any signaling protocol with N**2 property would
be difficult work in inter-domain environment.

One of the issues in RSVP is the notion of "route-pinning" at PATH
processing time. This forces the routers to remember every reservation
sender. At RESV processing time, the routers need to map the RESV to the
previous PATH. This means that the routers need to maintain N**2 states.
I think this is designed to prevent reservation looping problem. (BTW,
Bob Braden told me a couple of years ago that during the RSVP design,
people had thought about not storing PATH at routers, but...)

In BGRP, we still use the two-pass reservation mechanism. But the PROBE
messages (equivalent to RSVP PATH) gather the route information, and the
receivers use the gathered routing data to construct sink-trees. At
reservation time, the routers need only to remember sink-tree
information, which is O(N).

Later,

--
Ping Pan		htpp://www.cs.columbia.edu/~pingpan

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 19 Jan 0 8:37:47 JST

Ping Pan;

> be difficult work in inter-domain environment.

As I have pointed out in RSVP-related WGs and several research papers,
RSVP has no scalability problem from the begining.

So, I agree with you that it is very hard (impossible) to solve
your problem, because the problem does not exist.

> previous PATH.

No. As I pointed it out in some I-D and the INET'97 paper, the solution
is to route RESV first and let PATH follows the route.

Note that route stabilization is easy that route-pinning is not
necessary at all.

There are simple reasons why BGP like policy can not be applicable to
resource reservation. But it seems to me that the topic is to hard
for you to understand.

							Masataka Ohta

---

From: Ping Pan <[email-masked]>
Date: Wed, 19 Jan 2000 10:46:03 -0500

> RSVP has no scalability problem from the begining.

Huh? Please provide the pointers to your draft.

> your problem, because the problem does not exist.

Man. This is deep! 

> resource reservation. But it seems to me that the topic is to hard

In the past few years, sir, you have been insulting many people in
public without much technical input. It's not the topic too hard for me
to understand, rather it's your attitude that is too hard for me to
understand. Please provide meaningful technical input NICELY.

Thank you!

- Ping

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 20 Jan 0 6:00:35 JST

Ping;

> Huh? Please provide the pointers to your draft.

That could have been a fair request if you had provided the pointers
to papers or something like that describing the (non-existing)
scalability problem in detail.

So far, even though there are a lot of proposals to try to get
around the problem in vain, I have never seen anyone who had
done so, which is another evidence that there is no problem.

Instead, I wrote drafts and published papers and pointers are given.

> understand. Please provide meaningful technical input NICELY.

That's your problem.

							Masataka Ohta

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 19 Jan 2000 18:14:33 -0500

In message <[email-masked]>, Masataka Ohta wr
> scalability problem in detail.

I'm familiar with the scalability problems that Ping is referring to.
The problem of too much state and too high a setup rate in connection
oriented protocols is well known.  For example, RFC1932 (published in
1996) states:

   ATM offers QoS and traffic management capabilities that are well
   suited for certain types of services.  It may be advantageous to
   use separate ATM VC for such services.  Other IP services such as
   DNS, are ill suited for connection oriented delivery, due to their
   normal very short duration (typically one packet in each
   direction).  Short duration transactions, even many using TCP, may
   also be poorly suited for a connection oriented model due to setup
   and state overhead.

Most people who have been active in the IETF routing and transport
areas are very much aware that there are scaling problems associated
with providing end-to-end connections (scaling withn regard to
connection state and setup rate) or even virtual paths at too fine a
granularity.

The question I raised was whether we want to try to solve the problem
of scaling virtual paths at a fine granularity.  The implication was
that we might be better off taking a fundamentally different approach
which doesn't pose these scaling problems.  For example, currently
using plain old IP forwarding at boundaries between traffic engineered
MPLS networks avoids the scaling problems.  In the past I've suggested
that we pass only the traffic classification in the MPLS stack when
crossing these boundaries and do an IP lookup, combining the lookup
and classification to get traffic on the tunnel in the MPLS domain
receiving the packet.

Pings work does have merit and it does seem clear (to me at least)
that BGP will be involved in any solution to the problem of
interdomain QoS.  I'm not sure it is the right approach.

Maybe you don't understand the problem but Ping does and I do.  I'm
just questioning whether we can cleverly avoid it rather than try to
solve it.  It is very clear that you haven't looked at the web pages
and drafts that Ping referenced (or couldn't understand them).

> Instead, I wrote drafts and published papers and pointers are given.

I didn't see any URLs.  Given your past history (readers may recall
RFC1932 was discussed in the IP over ATM WG at about the last time
Ohta was insulting people who didn't agree with his "Conventional IP
over ATM") I won't be going out of my way to look for them.

Please provide the URLs and provide some technical comments of
substance or shut up.  Please stop insulting people.

Curtis

ps- diffserv dropped at Brian's request.  Any other WG chairs care to
declare this out of scope?

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 20 Jan 0 9:38:20 JST

Curtis;

Thank you for your attempt to provide some pointer for the first time
in the long history of misunderstanding.

However, your attempt does not affect the fact that Ping did not gave
pointers to his problem in his mail.

> I'm familiar with the scalability problems that Ping is referring to.

Worse, you misunderstand the problem, which should be attributed
to Ping who did not give any pointers.

>    also be poorly suited for a connection oriented model due to setup

We are working on resource reserved communication, which is expected
to have long (if you think 3 minutes of telephony call long) duration.

Ping mentioned multicast, which means DNS and TCP are inappropriate
examples, routing of which needs considerable amount of time and
control packets regardless of how resource is reserved.

> connection state and setup rate) or even virtual paths at too fine a

Thank you again for demonstrating the confusion caused by the
lack of pointers to the problem.

> Maybe you don't understand the problem but Ping does and I do.  I'm

You should, like me, better give minimal technical reasoning hopefully
founded by published research papers, if you want to say someone is
insulting himself by expressing silly opinions.

But, first of all, you should ask Ping what his problem is.

							Masataka Ohta

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 19 Jan 2000 20:57:56 -0500

In message <[email-masked]>, Masataka Ohta wr
> pointers to his problem in his mail.

Section 2 of the internet-draft cited by Ping is entitled "Problem
Definition".  I did not agree that the problem needed to be solved,
but the statement of the problem seemed very clear to me.

The fact remains that Ping gave two URLs.  One was to the draft itself
and the other was:

  http://www.cs.columbia.edu/~pingpan/projects/bgrp.html
  
And you gave none.

> to Ping who did not give any pointers.

What I described matches the section in Ping's draft.  See below.

> lack of pointers to the problem.

The description of the problem was in the draft that Ping suggested we
consider.  You may be confused because you didn't read the cited draft
or web pages before sending your insults to the mailing list.

Curtis

ps- I will not be publicly responding to any further mail from Ohta on
this.  I express my appologies to the 3 WGs that remain on the Cc for
my contribution to the noise so far.

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 20 Jan 0 11:06:38 JST

Curtis;

> And you gave none.

I gave a lot several times including this time.

> What I described matches the section in Ping's draft.

And does contradict with Ping's mail. So?

> or web pages before sending your insults to the mailing list.

It seems to me that you are saying people should read my paper
just because I cited it in my mail.

Thank you very much, but it is better to just ignore the entire thread.

> my contribution to the noise so far.

Don't mind.

That's not your but Ping's fault.

							Masataka Ohta

---

From: Ping Pan <[email-masked]>
Date: Wed, 19 Jan 2000 23:22:15 -0500

> > > pointers to his problem in his mail.

I did give the pointer to my draft. In the draft, I gave 27 references.
What is your problem here?

> Thank you very much, but it is better to just ignore the entire thread.

Oh, boy! You're one of kind... I wish you do ignore the entire thread
and leave us alone. 

> That's not your but Ping's fault.

Excuse me? 
 
Once again, regardless of the noise, I do wish people to read about our
work at http://www.cs.columbia.edu/~pingpan/projects/bgrp.html

Here is the situation: we have investigated the scaling problem in
reservation protocols, proposed a solution, and wanted to hear sensible
ideas from all of you to improve the work. At the same time, we are very
busy to iron out some of the protocol details before the actual
implementation. I have no desire and have no time to waste on some
personal attacks from Ohta-son. From now on, I will not response to
Ohta's mails (surely he will have more to come, unfortunately.)

Sorry about all this!

- Ping

---

From: Ping Pan <[email-masked]>
Date: Fri, 21 Jan 2000 12:45:36 -0500

Curtis Villamizar wrote:
> which doesn't pose these scaling problems. 

Curtis:

Good question. The way I look at this is the following: end users should
have the option to ask networks for resource reservation, and the ISP's
should have the ability to setup and maintain big bandwidth "pipes"
among each other.

End-user level reservation should be designed to handle applications
such as telephony, TV, inter-active games, and mobile user
communication. This type of reservation has short duration (less than a
day) and small bandwidth usage (at most 1Mbps), but the networks need to
respond the reservation setup requests very quickly (within a few
seconds). (Note, I don't quite believe WEB traffic and pre-stored
multimedia traffic request end-user reservations. I normally have no
problem to hear music from RealPlayer from my office LAN. ;-))

Network-level reservations are used for an entirely different set of
applications: business-to-business communication, Internet server access
(such as Blockbuster on-line), data exchange among WEB proxy servers.
The requirements for network-level reservation are thus different from
end-user reservation: longer duration (weeks if not months), larger
bandwidth, and more tolerant recovery time (probably in terms of
minutes).

I believe this is a good model for the future Internet. The potential
for this type of network will  make the 90's like minor-league games.
But given the number of end users, networks and ISP's, and their growth
rate, we have to have a hierarchical architecture for resource
reservation. At end-user level, RSVP and YESSIR (another pet project
that I am working on ;-)) will be good enough. Our proposed work, BGRP,
is targeted to setup and maintain Big-Fat-Reservations (BFR) among
ISP's. At network edge, it's up to the access routers to classify small
end user reservations into the BFR pipes. Between ISP's, BGRP uses the
bi-lateral agreement information to determine how/where to lay the
pipes.

> interdomain QoS.  I'm not sure it is the right approach.

Using BGP is to borrow the domain infrastructure for the hierarchical
model. I don't think we should define another way to characterize large
networks. (Let's not re-invent OSI addressing model for the Internet.
;-)) So yes, let's use BGP for the support of inter-domain QoS/CoS.

The mechanism that we used in BGRP comes from the dozens of solutions
that we could think of. For example, the reason for sending PROBE
messages prior to reservation is due to the fact that the reservation
senders really don't know the exact inter-domain routing path. This is
because BGP is not driven by some fixed shorest-path algorithm, rather
it's based on the local routing policies at border routers....

We love the hear more suggestions on better mechanism to achieve
sink-tree aggregation for inter-domain traffic.

Thanks alot for everything!

My 2 cents,

---

From: Ping Pan <[email-masked]>
Date: Mon, 17 Jan 2000 18:03:47 -0500

Hi, Curtis,

Regards,

_______________________________________________
diffserv mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/diffserv
Archive: http://www-nrg.ee.lbl.gov/diff-serv-arch/

---

From: Ping Pan <[email-masked]>
Date: Tue, 18 Jan 2000 08:08:02 -0500

Later,

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 19 Jan 0 8:37:47 JST

Ping Pan;

                                                        Masataka Ohta

---

From: Jon Crowcroft <[email-masked]>
Date: Tue, 18 Jan 2000 11:14:23 +0000

but alternatively,

 cheers

   jon

---

From: Ping Pan <[email-masked]>
Date: Wed, 19 Jan 2000 10:46:03 -0500

Man. This is deep! 

Thank you!

- Ping