Subject: RE: Reordering and MPLS?

From: "J. Noel Chiappa" <[email-masked]>
Date: Wed, 5 Jan 2000 13:13:05 -0500

> From: Bora Akyol <[email-masked]>

    > all MPLS/IP router vendors have a responsibility to guarantee ordered
    > arrival packets belonging to the same flow

There's a big difference between "guarantee to never do <X>" and "do <X>
very, very rarely". From this next comment I gather you meant the second,
though?

    > I expect that all MPLS equipment out there will do their best to keep
    > packets in order.

This is entirely reasonable.

    > I would suggest that we add this to the MPLS architecture document.

Sure, as long as it's "do their best", not "guarantee".

Of course, if you leave out a specific numberic target, then you run the risk
of some smart-aleck saying "oh, our box does reorder one packet in ten, but
we are doing our best". However, as pointed out on the IETF list in another
thread, the IETF doesn't get into saying whether implementations meet a spec
anyway, so even if the spec did say "don't reorder more than .001% of
packets", it wouldn't do that much good. (And the argument about what number
to put in could and possibly would go on forever! :-)

	Noel

---

From: Eric Gray <[email-masked]>
Date: Wed, 05 Jan 2000 13:33:28 -0500

Noel,

	Then the wording to be added would be something
like:

   "For packets transported on a single LSP, and receiving
    the same level of service, in-order delivery SHOULD be
    preserved."

Correct?

---

From: Greg Waters <[email-masked]>
Date: Wed, 05 Jan 2000 18:27:06 -0500

At 01:33 PM 1/5/00 -0500, you wrote:
>Noel,
>	Then the wording to be added would be something like:

No, LSRs are obliged to limit reordering of IP traffic in the same
manner as IP routers, until someone presents a reason to reduce
reordering in LSRs relative to high-end IP routers.

This was discussed long ago (early 1999) on the MPLS list, and I agree
with the original discussion (which I don't keep handy; sorry).

The reason is that very fast LSRs engage in traffic engineering
methods that include load-balancing, analogous to equal-cost
multi-path routing in very fast IP routers.  What is important for
IPv4 and IPv6 is to "almost always" deliver packets in order
within a 6-tuple IPv4 or IPv6 flow (DiffServ class, IP source,
IP dest, protocol, source port, dest port).

For simplicity, core-network switches need not exploit (protocol,
source port, dest port) variation for extra load balancing.  The
most useful invariants of an "IP forwarding flow" are
(DiffServ class, IP source, IP dest).

Thanks to DWDM, load balancing a 100-Gbit/sec trunk LSP across
dozens of lambdas is an operationally important feature in the near
future.  Fortunately for this discussion, high-end routers have
always done some of that to IPv4 due to ECMP, so letting LSRs
reorder within a fat LSP (for a given class) is equally reasonable.

When an LSP is not configured to carry pure IPv4 (and/or IPv6)
traffic, or other traffic with widely-known flow identifiers,
then that LSP should not suffer reordering within a DiffServ
class.  Therefore, non-IP LSPs will typically be confined to one
lambda of capacity.

"Almost always" means that reordering is to be expected during
re-routing, until otherwise stated.  Changing an LSP's route at
high frequency is an unsociable act, absent a mechanism to keep
each flow's packets in order.

What MPLS specs could say is that reordering in an LSP should be
limited in the same manner as for the network protocol(s) carried
within the LSP, as appropriate.  Hypothetically, if a fat LSP
carried IPv6/FrameRelay/MPLS, the reordering expectations of
IPv6 and/or FrameRelay might apply to the LSP.

MPLS specs could stipulate that a mode of LSP mid-point "should"
be supported which from a forwarding perspective has unidentifiable
flows.  If supported, that mode of LSP should keep all packets in
order for a given Exp-coded service class.  --gw