Subject: Re: [lisp] Flow label redux [Re: IPv6 UDP checksum issue]

From: [email-masked] (Noel Chiappa)
Date: Fri, 7 Aug 2009 15:31:11 -0400 (EDT)

> From: Francis Dupont <[email-masked]>

    > the O UDP checksum proposal obsoletes all the today deployed nodes
    > which check them (so all hosts I know and perhaps a lot of routers too)

OK, so what are the other options for encapsulating a packet in a IPv6
packet?

I'm told by some people that UDP-Lite isn't a standard yet? Or is it? (It
seems to have a protocol number issued?) Does UDP-Lite work through NAT
boxes? (LISP has a mobile-node mode, which we would like to see work through
NAT boxes, so any proposed alternative solution has to work through NAT boxes
too.)

Other ideas?

Also, hosts aren't an issue - and in fact it's a _feature_ that hosts will
discard any LISP packets they see (for having bad checksums).

    > From: Shane Amante <[email-masked]>

    > this is the first "iteration" of LISP, there may be 'natural' points in
    > the LISP evolution path ... where it makes sense to consider evolving
    > the encapsulation

Excellent point. We have indeed already considered deploying other
encapsulations, e.g. for use over MPLS.

    > From: Iljitsch van Beijnum <[email-masked]>

    > there isn't that much IPv6 traffic in the first place ...
    > So the lack of a fine-grained optimal solution to the load balancing
    > issue is not a problem in practice.

Is it your prediction for the near (i.e. 2-3 year) future that this level of
traffic will continue to be the case, operationally?

	Noel

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 7 Aug 2009 22:05:35 +0200

On 7 aug 2009, at 21:31, Noel Chiappa wrote:

> seems to have a protocol number issued?)

http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml

Protocol 136, RFC 3828.

> Does UDP-Lite work through NAT

For IPv6?

> traffic will continue to be the case, operationally?

The amount of traffic where lack of load balancing granularity beyond  
the 3-tuple starts to hurt is in the order of a quarter of the link  
speed (could be 25% of 100 Mbps to 40 Gbps). So if we can now fill in  
the number of users that are encapsulated together between an ITR and  
an ETR (could be hundreds to hundreds of thousands) and the amount of  
traffic that is IPv6 (currently 0.2% at AMS-IX, perhaps 10% in the  
near future or even 50% or more) then we can see how big the problem  
really is.

It's not impossible to arrive at numbers where the load balancing  
starts being problematic with only the 3-tuple but to get there within  
a year or three (or even five) you have to assume radical IPv6 and  
LISP uptake coupled with either massive ITRs/ETRs or miniscule links.

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 7 Aug 2009 21:22:51 -0400

On Fri, Aug 7, 2009 at 8:22 PM, Havard Eidnes<[email-masked]> wrote:
>>     > the O UDP checksum proposal obsoletes all the today deployed nodes
>>     > which check them (so all hosts I know and perhaps a lot of routers too)
>>
> Um, surely, routers are not specified to validate layer-4
> checksums for transit traffic?!?
>
> Let's look at this another way?  As I understood it, UDP 0 would
> be used by LISP encapsulating/decapsulating devices.
>
> If some random (non-LISP encap/decap) host by mistake received a
> 0 UDP packet, it would be dropped, which should do no harm.

warning this discssion morphed from a 'what should ipv6/ipv4
translators do..' discssion. While a non-lisp node receiving a LISP
udp/0 packet dropping it seems fine to me, a translator dropping a
udp/0|null-sum packet instead of translating it properly or telling
the source-system: "oops, something bad happened" is unacceptable (in
my mind).

> In practical terms, only LISP encap/decap devices would need to
> be modified to accept 0 UDP packets under some specific rule /
> circumstance, as an exception to the general rule.

See above comment. I really think that in the wider context the
udp-checksum decision for ipv6 needs to be revisited. There are more
than a few cases where the current state of inconsistency needs to be
trued up.

-chris

---

From: Havard Eidnes <[email-masked]>
Date: Sat, 08 Aug 2009 02:22:04 +0200 (CEST)

The only thing which would prevent this would be one of
conformance to the letter of the original spec, which apparently
bans 0 UDP checksums.

So what's so bad about that?

Regards,

- Håvard

---

From: Gorry Fairhurst <[email-masked]>
Date: Sat, 08 Aug 2009 07:36:39 +0100

> NAT boxes, so any proposed alternative solution has to work through NAT boxes
<SNIP>

UDP-Lite spec: RFC 3828.
UDP-Lite MIB:  RFC 5097.
Unicast UDP Usage Guidelines (for UDP and UDP-Lite): RFC 5405.

UDP-Lite has been part of the Linux kernel since version 2.6.20.

Gorry

---

From: Brian Haberman <[email-masked]>
Date: Sat, 08 Aug 2009 10:33:36 -0400

> the source-system: "oops, something bad happened" is unacceptable (in

This statement about morphed discussions is absolutely correct.  The 
discussion on the ipv6 mailing list should be about the proposals on the 
table for modifying the UDP checksum rules.  Such a change impacts 
IPv4/IPv6 translators, LISP, and AMT.

Please keep all other discussions (LISP behavior, translation 
techniques, etc.) on their respective lists.

Thanks,
Brian

---

From: Francis Dupont <[email-masked]>
Date: Mon, 10 Aug 2009 16:24:28 +0200

=> UDP-lite or a direct encapsulation (IMHO LISP should ask for
a (IPv6) protocol number).

   I'm told by some people that UDP-Lite isn't a standard yet? Or is it?

=> RFC 3828, protocol 136.

   Does UDP-Lite work through NAT boxes?

=> I don't know but there is no deployed IPv6 NATs.
   
Regards

[email-masked]

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 10 Aug 2009 10:08:40 -0700

> checksums for transit traffic?!?

Right, or have to build pseudo-headers in a contiguous buffer with the  
*entire* data packet to calculate such a UDP checksum.

> be used by LISP encapsulating/decapsulating devices.

Right. But on the ETR, it is spec'ed to "ignore" checksums because  
there are NATs that rewrite UDP checksum fields regardless if the NAT  
receives the packet with a zero or non-zero UDP checksum.

> 0 UDP packet, it would be dropped, which should do no harm.

As would a router (acting as a hsot) that is not doing LISP.

> circumstance, as an exception to the general rule.

Right (modulo what I wrote above about ignoring UDP checksums on the  
decapsulator).

> conformance to the letter of the original spec, which apparently

Right.

> So what's so bad about that?

Good question.

Dino

>
> Regards,
>
> - Håvard
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 10 Aug 2009 13:11:25 -0400

Actually, looking at this as a LISP problem is probalby misleading.

This issue applies to any intermediate device based, high capacity, 
tunnel mechanism.
1) High capacity tunnel mechanisms are going to be concerned to 
implement in hardware, where the UDP checksum may be difficult
2) High capacity tunnels are the very ones whose traffic could and 
should make use of ECMP / LAG operation.

Starting from there, the line of argument then proceeds to look at what 
works.
What works in the real world IPv6 deployments is UDP with 0 checksum. 
That enables ECMP / LAG without excessive effort / overhead.
Note that UDP-lite is not an answer in terms of what works now.  The 
vast majority of deployed provider-edge (PE) and provider-core (P) 
devices will not look at the port numbers in the UDP-lite header because 
they do not recognize the protocol.

The next question is whether this could change.  I am still collecting 
information on this.  Product seems to fall into three categories:
1) Software forwarders.  Obviously, these can change
2) Microcoded packet processors - These can probably change which field 
is looked at, within limits
3) Hardware mechanisms - these really can not change their handling, as 
they are in the field.

Preliminary data indicates that there are a non-trivial number of 
hardware devices already out there and handling IPv6.
There are a noticeable number of core devices that could be changed in 
small ways (microcoded or software), for example to use the flow-label 
instead of some other piece of information in the packet.

We can look at the hardware and complain, or we can look at it and thank 
the vendor for acting promptly on RFCs.
We can not reasonably expect the hardware to change in any meaningful 
time frame.

I am still colelcting implementation data, so I may be misled by what I 
have gotten.  But it seems to me that we are unfortunately stuck with 
UDP with 0 checksums if we want high capacity tunnels to be able to make 
use of ECMP / LAG in the moderate time period (out through at least 5 
years, maybe longer.)

Yours,
Joel

---

From: Ross Callon <[email-masked]>
Date: Mon, 10 Aug 2009 14:20:02 -0400

I agree with Joel that this problem is broader than just LISP. This is an issue with any device that is encapsulating quite a bit of traffic (that represents many smaller flows that could be split) in a tunnel over IPv6. It would (IMHO) serve us well to do this right. 

I also agree with Joel that there is a wide range of existing equipment deployed, and at least some of the existing high speed (core) IPv6 routers could include the flow label in the hash, at the cost of a microcode software update, and probably removing something else from the hash. 

There isn't all that much IPv6 traffic right now (some please correct me if this is wrong), and the ramp-up speed seems relatively slow. Thus to me it seems that the time frame that it would take to change router hardware is long, but no longer than the time frame that it is likely before we find ourselves with enough IPv6 traffic that we actually need to do a good job of ECMP splitting of traffic going over an IPv6 tunnel. Thus I think that it would be okay to do this by putting a hash of higher layer flow information into the flow field, if we think that is the right way to do it. 

Ross 
(speaking as an individual contributor with my own personal opinion only). 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: 10 August 2009 13:11
To: Havard Eidnes
Cc: [email-masked]; [email-masked]
Subject: Re: [lisp] Flow label redux

Yours,
Joel

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 10 Aug 2009 14:41:09 -0400

On Mon, Aug 10, 2009 at 2:20 PM, Ross Callon<[email-masked]> wrote:

> There isn't all that much IPv6 traffic right now (some please correct me if this is wrong), and the ramp-up speed seems relatively

'much global ipv6 traffic'. There are places with more ipv6 traffic,
where LAG/ECMP is important. These 'internet protocols' get used on
'not the internet' networks as well.  Additionally, across links not
gig in size, but still lag/ecmp-aware.

> slow. Thus to me it seems that the time frame that it would take to change router hardware is long, but no longer than the time
> frame that it is likely before we find ourselves with enough IPv6 traffic that we actually need to do a good job of ECMP splitting of

'enough global/public ipv6 traffic' - I would caution that there are
places where LAG/ECMP is necessary today, or will be in the very near
future, that are not publicly visible.

> traffic going over an IPv6 tunnel. Thus I think that it would be okay to do this by putting a hash of higher layer flow information into
> the flow field, if we think that is the right way to do it.

-chris

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 10 Aug 2009 22:50:10 -0400

Hi Noel,

> OK, so what are the other options for encapsulating a packet in a IPv6

We could just do an IP-in-IP encapsulation, so the headers would be  
IPv6-LISP-IP(v4 or v6), but that does not address the LAG/ECMP concern.
> NAT boxes, so any proposed alternative solution has to work through  

UDP-Lite is a proposed standard, RFC 3828.
> discard any LISP packets they see (for having bad checksums).

Unless the host implements LISP.  See my earlier message on this  
subject...  Do we think it will be possible to disable IPv6 UDP  
checksums everywhere that LISP needs to run?  On many of today's host  
systems, sending a UDP packet with a zero checksum is simply treated  
as an indication that the NIC card should perform the checksum.

Margaret

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 10 Aug 2009 23:02:12 -0400

>> NAT boxes, so any proposed alternative solution has to work through  

Sorry I didn't reply to this in the earlier message...

(1) There isn't enough NAT deployment for IPv6 (yet) to know how IPv6  
NAT boxes will behave.  It is possible to implement at v6 NAT that  
doesn't touch the transport headers at all.

(2) I've heard statements about LISP working through NATs, but I can't  
figure out how that would work...  Could you explain to me how LISP  
would work using IPv4 encapsulation behind an IPv4 NAT?   Are you  
expecting that there will be LISP ALGs to convert the inner headers as  
they are transmitted through the NAT in the outbound direction?   
Otherwise, it seems like EIDs from private address space (such as net  
10) will get injected into other networks, where they will be  
meaningless.

Margaret

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 10 Aug 2009 23:08:55 -0400

Hi Havard,

> 0 UDP packet, it would be dropped, which should do no harm.

So, you are assuming that LISP is the _only_ protocol that will choose  
to specify zero UDP checksums?  And that LISP will be implemented  
solely on systems that implement LISP and no other protocols?

Margaret

---

From: Sam Hartman <[email-masked]>
Date: Tue, 11 Aug 2009 10:09:25 -0400

>>>>> "Noel" == Noel Chiappa <[email-masked]> writes:

    >> From: Francis Dupont <[email-masked]> the O UDP
    >> checksum proposal obsoletes all the today deployed nodes which
    >> check them (so all hosts I know and perhaps a lot of routers
    >> too)

    Noel> OK, so what are the other options for encapsulating a packet
    Noel> in a IPv6 packet?

    Noel> I'm told by some people that UDP-Lite isn't a standard yet?
    Noel> Or is it? (It seems to have a protocol number issued?) Does
    Noel> UDP-Lite work through NAT boxes? (LISP has a mobile-node
    Noel> mode, which we would like to see work through NAT boxes, so
    Noel> any proposed alternative solution has to work through NAT
    Noel> boxes too.)

We have not reached a consensus that LISP needs to work through NATs.
I'll take your message as a statement in favor of that and a personal
opinion that they need to.

I have not seen a lot of disagreement of this assumption, nor do I
want to start that discussion now.  I'm just trying to keep the
mailing list history accurate to make my job as a chair easier when I
have to turn this discussion into a conclusion.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 19 Aug 2009 12:01:30 +0200

> opinion that they need to.

Please put me down in the "that's insane" column.