Subject: RE: prime time MPLS/fiber

From: Shahram Davari <[email-masked]>
Date: Wed, 5 Jan 2000 10:06:54 -0800

Hi,

MPLS at the access can also reduce the classification burden of the edge
routers and make it distributed.

-Shahram

> -----Original Message-----
> From: Grenville Armitage [mailto:[email-masked]]
> Sent: Wednesday, January 05, 2000 1:05 PM
> To: Jeremy Lawrence
> Cc: [email-masked]; [email-masked]
> Subject: Re: prime time MPLS/fiber
> 
> 
> 
> 
> Jeremy Lawrence wrote:
> 	[..]
> 
> > At the edge (between the home and the first SP router), I don't
> > think MPLS makes sense. MPLS may well make sense in the network
> > core, for the reasons just discussed, but the access just needs
> > to be IP-over-anything. IP/x/fiber, where x is an appropriate
> > data link layer (PPP, fast ethernet, gigabit ethernet, etc.),
> > could certainly be used.
> 
> In principle MPLS might be useful in the access networks for the
> same reasons ATM is sometimes used in access networks - a 'connection'
> abstraction allows things like multiple provider selection over
> single physical links. Each LSP can be a virtual access link.
> 
> cheers,
> gja
>

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Wed, 5 Jan 2000 13:31:59 -0600

Agreed. Moving classification and other MPLS-based services (VPN et al) to
the customer premise would reduce the burden on the network edge and enhance
the ability to offer new services. However, connecting this thread to the
"Reordering and MPLS?" thread, isn't some kind of ordered delivery guarantee
required to create true connection-oriented CPE-based services?

Robert

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]]
Sent: Wednesday, January 05, 2000 12:07 PM
To: 'Grenville Armitage'; Jeremy Lawrence
Cc: [email-masked]; [email-masked]
Subject: RE: prime time MPLS/fiber

Hi,

-Shahram

---

From: Jeremy Lawrence <[email-masked]>
Date: Thu, 06 Jan 2000 10:26:10 +1100

Robert,

At 13:31 01/05/2000 -0600, Johnson, Robert (Broadwing) wrote:
[...]
>However, connecting this thread to the

If the services are suitable for connection over a packet network,
then it can be strongly argued (as Noel has done recently) that the
services must be able to deal gracefully with occasional packet
mis-ordering. Are there services proposed for connection over MPLS
which do not meet this requirement?

Regards,

Jeremy Lawrence

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Wed, 5 Jan 2000 18:43:23 -0600

Jeremy,

>which do not meet this requirement?

I apologize for not clarifying, but I was implying the inclusion of those
services that are not suitable for use over a pure, connectionless packet
network. There has been some discussion of using MPLS to provide
connection-oriented services, some of which are not designed to survive the
rigors of a connectionless packet network. If MPLS were to be utilized as a
transport protocol for customer access, it would need to be capable of
supporting these types of services. In other words, it needs to guarantee
ordered packet delivery or it should be not be used for these services.

Robert

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Wed, 5 Jan 2000 21:17:22 -0500

> From: "Johnson, Robert (Broadwing)" <[email-masked]>

    > I was implying the inclusion of those services that are not suitable
    > for use over a pure, connectionless packet network. There has been some
    > discussion of using MPLS to provide connection-oriented services, some
    > of which are not designed to survive the rigors of a connectionless
    > packet network. ... it needs to guarantee ordered packet delivery or it
    > should be not be used for these services.

How do these services deal with lost packets? (C.f. my point to Bora Akyol
about how the effects of lost packets and reordered packets can look almost
identical, at the receiver.) Or are you saying that these services won't work
in networks which can lose packets? If so, are you now proposing that MPLS
guaranteee to never drop packets?

And if these services can somehow deal with lost packets (e.g. by simply
ignoring them, the way packet voice does), why can't they just discard
out-of-order packets - if a service can handle lost packets, it can handle
discarded packets, right?

	Noel

---

From: Jeremy Lawrence <[email-masked]>
Date: Thu, 06 Jan 2000 14:25:26 +1100

Robert,

>ordered packet delivery or it should be not be used for these services.

What services? Unless I've missed something, no-one has stated
a problem for which it was argued that the best solution is
modifying MPLS to ensure in-order packet delivery.

If the problem is "carrying voice over MPLS", then guaranteed
in-order delivery clearly isn't required. A suitable transport
in a voice codec can simply discard out-of-sequence packets,
and any vaguely adequate codec can gracefully deal with an
occasional dropped packet. 

If the problem is "using my equipment with an extremely poor/old
voice codec over an MPLS network", then the best solution seems
to be:
- Update the equipment to use an adequate codec (packetized
   64kb/s PCM with an appropriate transport could be adequate, for
   example)
- In the meantime, use ATM circuit emulation. All ATM-LSRs which
   I know about can simultaneously support circuit emulation
   services over PVCs, on the same links and switches that
   constitute the MPLS network backbone.

If the problem is something else, then what is it?

Regards,

Jeremy Lawrence

---

From: "David Allan" <[email-masked]>
Date: Mon, 10 Jan 2000 08:30:40 -0600

Juha, Jeremy:

It strikes me that a default route has utility whereby the the provider
router has to perform classification on best effort upstream traffic.
Otherwise the number of "downstream on demand" label requests would just
about double the amount of traffic. Where I would see MPLS to the prem
having utility would be for specific QoS or CoS LSPs to be dynamically set
up where the default path and latency (due to extra classification step)
would not be adequate. At least for multimedia, pushing classification into
the customer gear for voice and other "lots of little packet" services would
have advantages. It strikes me that it does not scale to the general traffic
case.

So I don't want to ask for /32s, I want a /0 best effort, and then ask for
/32s or whatever when I need somthing extra or somthing non-IP in the
payload.

Cheers
Dave

> From:	Juha Heinanen [SMTP:[email-masked]]
> Sent:	Monday, January 10, 2000 7:26 AM
> To:	Jeremy Lawrence
> Jeremy Lawrence writes:
>  > It is might possible to change LDP to remove this assumption, but
>  > this would involve a lot of work and debate, on both LDP itself,
>  > and on the architectural issues. LDP is currently designed to
>  > work in close conjunction with a routing protocol.
> 
> ok, in that case an lsr that wishes to terminate label requests to
> itself, needs to advertise its ip address(es) as /32 routes.  i assume
> that an actual routing protocol is not needed, but that the advertisement
> can be staticly configured.
> 
> regarding the architectural issues, if a network is only interested to
> run ldp in downstream on demand/ordered control mode, then it is not
> clear to me what the architectural and prototol problems would be that
> would prevent an lsr from forwarding a label request based on longest
> match.
> 
> -- juha

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Mon, 10 Jan 2000 09:29:07 -0600

Jeremy,

I would refer you to the short discussion thread on Private Line over MPLS.
I am also looking into Voice over MPLS. Both of these applications require
additional services which MPLS does not provide today (i.e. guaranteed
ordered packet delivery). Therefore, I suppose that we are talking about
additional adaptation layers to provide the facilities necessary to deliver
these services over an MPLS infrastructure. Without going into detail, I
think that you can see the problems that might occur if this isn't made
abundantly clear up-front.

Robert Johnson

-----Original Message-----
From: Jeremy Lawrence [mailto:[email-masked]]
Sent: Wednesday, January 05, 2000 9:25 PM
To: Johnson, Robert (Broadwing)
Cc: [email-masked]; [email-masked]
Subject: RE: prime time MPLS/fiber

Robert,

Regards,

Jeremy Lawrence

---

From: Jeremy Lawrence <[email-masked]>
Date: Tue, 11 Jan 2000 10:18:03 +1100

David,

At 08:30 01/10/2000 -0600, David Allan wrote:
[...]

>So I don't want to ask for /32s, I want a /0 best effort, and then ask for /32s or whatever when I need somthing extra or somthing non-IP in the payload.

By the argument in previous emails, if:
1. the CPE asks for a label for 0.0.0.0/0, and
2. there is no routing protocol or similar means of ensuring
    consistent routing information between the CPE and operator LSR, and
3. the operator LSR knows more specific routes in addition to 0.0.0.0/0

then label forwarding would run correctly if and only if the operator
LSR returned labels for all routes it knows, in response to the
request for a binding for 0.0.0.0/0.

Otherwise, the CPE will always pretend that the longest prefix match
for an address a.b.c.d is 0.0.0.0/0, when often there will
be a better match, say a.b.0.0/16. This would mean that the operator
LSR often would not correctly forward traffic from the CPE.

(LDP as it stands is the wrong tool for this job, as it assumes
the reverse of point #2.)

Regards,

Jeremy Lawrence

---

From: Jeremy Lawrence <[email-masked]>
Date: Tue, 11 Jan 2000 10:27:54 +1100

Robert,

At 09:29 01/10/2000 -0600, Johnson, Robert (Broadwing) wrote:
>ordered packet delivery). 

That's certainly not true for carrying voice over MPLS. Per the
discussions last week, virtually any packet voice system can deal
with small amounts of packet misordering (by discarding the
out-of-order packets).

>Therefore, I suppose that we are talking about

Private Lines are another issue, and yes, there would need to be
some mechanism to ensure packet ordering. However I wonder if we're
looking at the right problem here. What, exactly, is the problem you
are trying to solve? (I suspect that it may be "carrying private lines
and MPLS on the same equipment", which is subtly different to
"carrying private lines over MPLS".)

Regards,

Jeremy Lawrence

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Mon, 10 Jan 2000 19:54:34 -0500

> Both of these applications require additional services which MPLS does
    > not provide today (i.e. guaranteed ordered packet delivery).

I'm still curious to see what your answer is to a question I put previously
(although perhaps somehow my mail to the MPLS list didn't get to you):

    How do these services deal with lost packets? ... Or are you saying
    that these services won't work in networks which can lose packets? If
    so, are you now proposing that MPLS guaranteee to never drop packets?

    And if these services can somehow deal with lost packets (e.g. by
    simply ignoring them, the way packet voice does), why can't they just
    discard out-of-order packets - if a service can handle lost packets, it
    can handle discarded packets, right?

Do I take it from your phrase above ("guaranteed ordered packet delivery")
that you do in fact propose that MPLS networks guarantee to never lose
packets?

    > I suppose that we are talking about additional adaptation layers to
    > provide the facilities necessary to deliver these services over an MPLS
    > infrastructure.

So, if I correctly understand you, you're now proposing that we provide this
reliable, ordered service by means of an AL above MPLS. In that case, my query
is whether you propose that this AL be something provided in the end-nodes, or
by the switches in the middle - i.e., fundamentally, whether a packet lost in
the network is replicated from a preceding switch, or whether the ultimate
originator has to resend the packet.

	Noel

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Mon, 10 Jan 2000 22:59:50 -0600

>with small amounts of packet misordering (by discarding the

I suppose that depends on the definition of VoMPLS. If VoMPLS were simply
VoIP within MPLS, it should still be referred to as VoIP. However, if the IP
layer is removed and the RTP/UDP layers are retained, then they would act as
the adaptation layer for voice services over the MPLS transport layer.
However, I am not currently up-to-date on the activities in the VoMPLS area,
so I apologize if my comments are out-of-order.

>"carrying private lines over MPLS".)

Agreed. Private line and MPLS traffic carried over the same switch is quite
different from using MPLS as the transport protocol to carry the private
line traffic. However, as I understand it, there are those with the
intention of using MPLS in the same manner that ATM is used in carrier
networks today to provide CES. However, MPLS does not provide all of the
facilities to support CES that ATM does.

In short, I do not think that MPLS is anymore of a convergence layer than
other proposed convergence layers that have gone before it. It will require
modifications to support some of the services that are being delivered today
over networks using ATM or physical leased facilities.

Robert Johnson

---

From: Jeremy Lawrence <[email-masked]>
Date: Tue, 11 Jan 2000 17:04:46 +1100

Robert,

>facilities to support CES that ATM does.

This begs the question "why add them?". MPLS was invented to do some
things well that nothing else did well, but CES wasn't one of those
things. ATM does CES well. If it is easy to modify MPLS to do CES
well, it might be justifiable to modify it. Otherwise, why not just
use ATM to do CES?

Jeremy

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Tue, 11 Jan 2000 07:16:47 -0600

Jeremy,

>well, it might be justifiable to modify it. Otherwise, why not just

That has been my argument to those who suggest that MPLS will completely
eliminate the applications for ATM in short order. However, by not
clarifying that MPLS will not support all applications that ATM supports,
perhaps we are inviting such discussion.

Robert Johnson

---

From: "David Allan" <[email-masked]>
Date: Tue, 11 Jan 2000 11:54:43 -0600

Jeremy:

	<snip>

I understand what you are saying, my suggestion is that MPLS transport be
extended to the CPE but classification be distributed between the CPE and
the operator LSR, hence traffic on 0.0.0.0/0 best effort cannot be directly
forwarded without re-classification by a box with more authoritative
knowledge of the routes. There may well be a better match known to the
operator LSR, but I think the correct statement is that the operator LSR
will not optimally forward traffic. It is capable of correctly forwarding
it. This to me is a trade off as for short flows either:

i)  all this information needs to be a priori distributed to the CPE which
adds to CPE complexity, OR 
ii) the CPE needs to request an explicit binding,  which to me seems to
defeat the point, adding additional transactions to simplify relatively few
forwarding steps, OR 
iii) the CPE just forwards the packet to the provider LSR.

For long flows, or those with specific characteristics, then requesting a
specific binding to push classification to the edge and perform proper MPLS
layer forwarding at the provider LSR makes sense. It addresses all my
concerns.

cheers
Dave

---

From: Jeremy Lawrence <[email-masked]>
Date: Wed, 12 Jan 2000 10:30:55 +1100

David,

At 11:54 01/11/2000 -0600, David Allan wrote:
[...]
>I understand what you are saying, my suggestion is that MPLS transport be extended to the CPE but classification be distributed between the CPE and the operator LSR, hence traffic on 0.0.0.0/0 best effort cannot be directly forwarded without re-classification by a box with more authoritative knowledge of the routes.

Well, you could deal with this by the method that Juha suggests,
which is to forward the traffic on an LSP established to w.x.y.z/32,
which is the (say) loopback address of the operator LSR, instead of
0.0.0.0/0? That way, the operator LSR would examine the IP packets
that come out the LSP, and forward them appropriately. (It may be
better, though, to simply forward unlabelled IP packets to the
operator LSR.)

Regards,

Jeremy Lawrence

---

From: "Johnson, Robert (Broadwing)" <[email-masked]>
Date: Tue, 11 Jan 2000 21:00:07 -0600

Noel,

I apologize for not answering your question sooner and, fundamentally, I
agree with your argument. A core LSR should not be responsible for operating
an adaptation layer to support services that cannot deal with the loss or
reordering of packets. However, I disagree with the focus of your argument,
which is on packet-based services. If a service is not packet-based, but is
being packetized to ride an MPLS network, then adaptation layers must be
defined to provide those facilities. Either that, or the idea that
non-packet-based services can be adapted to an MPLS network should be
dismissed out-of-hand. I am compliant with either approach.

Robert Johnson

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Thu, 13 Jan 2000 14:02:22 -0500

> A core LSR should not be responsible for operating an adaptation layer
    > to support services that cannot deal with the loss or reordering of
    > packets.

Concur. Too much extra complexity to be worth it.

    > I disagree with the focus of your argument, which is on packet-based
    > services. If a service is not packet-based, but is being packetized to
    > ride an MPLS network, then adaptation layers must be defined to provide
    > those facilities.

I don't necessarily disagree with that. (One can think of TCP as an adaption
layer that synthesizes a reliable stream facility out of an unreliable packet
service.) The two places I might branch out are:

- Whether, if one needs a reliable service, it makes sense to define a
separate MPLS-only facility to do that. If you need reliability, and it's
going to be basically end-end (even if it's provided by an access box, and not
the actual end user), you're going to need sequence numbers, time-outs,
retransmits, etc - and if you've got all that, why not just use TCP, or some
similar internetwork service? (Long analysis of pros and cons of
MPLS-specific, versus IP-based approachs elided - not enough energy to type it
all in.)

- The problem with doing reliability on a basically end-end basis is it might
conflict with timing requirements. E.g. if you have a real-time application,
in which timely delivery of data is important, you might not be able to stand
an end-end roundtrip to pick up missing data. And using a long playback buffer
to hide such delay variance might run afoul of overall delay budgets. It's at
that point that doing reliability in the switches in the middle (i.e. a
packet lost in the network is replicated from a preceding switch) starts to
look better.

However, there are sometimes ways around this, e.g. depending on what %-age of
overall traffic this application is, and how critical it is that you really,
really get the data. For example, (and this is really simplistic) I might send
all data three times, using a sliding window: e.g. packet 1 contains data
blocks D, E, F; packet 2 contains E, F, G; packet 3 contains F, G, H. A much
smaller playback buffer will allow you to smooth out the data stream over any
missing packets, and it takes 3 losses in a row to lose a data block
completely. If this traffic is a small %-age of the overall traffic, such a
scheme is probably more cost-effective overall than requiring per-switch
packet reliability (by local replication).

Depending on the exact details of the application, other schemes (different
in detail, but similar in spirit) might work for them.

    > Either that, or the idea that non-packet-based services can be adapted
    > to an MPLS network should be dismissed out-of-hand.

I think that's excessive. As I indicated, TCP is not (to the user) a "packet
based service". However, the exact requirements of such services need to be
quantified before it can be ascertained whether it can be supported on an
MPLS fabric.

I would argue, though, that anything you can support on basic MPLS should be
supportable on IP+QoS, and in fact it would be preferable to do the latter.
(Why limit the scope of such a new service to MPLS fabric, when it could
*potentially* operate over the whole Internet, if transmission fabric with the
right data-carrying attributes are available?)

	Noel