Subject: Re: [v6ops] new draft: draft-generic-v6ops-tunmtu

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 4 Jun 2012 14:57:20 -0700

FYI, the tunnel MTU discussion seems to have spilled over
onto the nanog list, where some of the same issue points
are being discussed.

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 12 Jun 2012 15:50:54 -0700

Hi,

This document has now been updated to address issues
raised on the list and to expand on the roles of
inner fragmentation before encapsulation and outer
fragmentation after encapsulation. The answer is
that inner is needed in certain use cases, outer
is needed in certain others and both are needed in
still yet others.

The latest draft is now available here:

https://datatracker.ietf.org/doc/draft-generic-v6ops-tunmtu/

Can we talk about this during the v6ops session at
the Vancouver meeting?

Thanks - Fred
[email-masked]

---

From: Fred Baker <[email-masked]>
Date: Mon, 18 Jun 2012 04:19:41 -0700

> Can we talk about this during the v6ops session at

I have seen comments on-list from Daniel Roesen, Mark Andrews, and Washam Fan. In each case, they have commented on the draft without saying whether they think it is a valuable working group output, which I take quite literally - I don't know their opinion.

Working group, is this draft something you would like to discuss?

> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: Wuyts Carl <[email-masked]>
Date: Mon, 18 Jun 2012 13:51:56 +0200

Hi Fred,

I thought the target of the v6ops was to avoid these kind of things, and promote proper IPv6 packet handling.  After all, it is promoted from the start, that IPv6 brings you some extra things, like fixed header size, no more fragmentation (only at host), a MANDATORY PMTU (while for v4 it was only added AFTER the protocol was there), ....  Now I read in this draft to start doing some fragmentation with inner/outer, whatever, headers etc ??

First of all, the "all links are 1500" statement is correct for Ethernet-based networks, but as I'm working for a company dealing with DSL, this is not always the case, so don't take it for granted.  Moreover, CPE will be heavily involved in this (6rd etc).
Secondly, if ICMPv6 PTB messages gets filtered out (by some wrongly configured firewall or any other device) then it should be fixed where it goes wrong, not work-around it.  You're replacing a wrong filter with some complex mechanism of fragmentation.  Why is this filter in place ?
Third, I see the definition of an "atomic" packet, but that's quite some assumption to be made for IPv6 packets.

Just for the record.  Our CPE can do fragmentation, so that's not an issue, but we'd like to avoid this becoming normal mode of operation, to avoid lots of potential issues with it.  Moreover, what's next ? 

regs
Carl Wuyts

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Fred Baker
Sent: maandag 18 juni 2012 13:20
To: [email-masked] WG
Cc: [email-masked]
Subject: Re: [v6ops] new draft: draft-generic-v6ops-tunmtu

On Jun 12, 2012, at 3:50 PM, Templin, Fred L wrote:

> Hi,
> 
> This document has now been updated to address issues raised on the 
> list and to expand on the roles of inner fragmentation before 
> encapsulation and outer fragmentation after encapsulation. The answer 
> is that inner is needed in certain use cases, outer is needed in 
> certain others and both are needed in still yet others.
> 
> Can we talk about this during the v6ops session at the Vancouver 
> meeting?

---

From: Fred Baker <[email-masked]>
Date: Mon, 18 Jun 2012 05:11:52 -0700

On Jun 18, 2012, at 4:51 AM, Wuyts Carl wrote:

> First of all, the "all links are 1500" statement is correct for Ethernet-based networks, but as I'm working for a company dealing with DSL, this is not always the case, so don't take it for granted. 

Personal opinion...

"All links are 1500 bytes" is a silly statement. There are days that I think that some engineers only look as far as the interfaces on their laptops to figure out what is interesting. There are some, for example, that seem to think IEEE 802.15.4 has value; 802.15.4g has a 2047 byte MTU; 802.15.4 uses 127. Something on the order of 9K bytes is pretty common in optical networks (e.g. gigabit speed including Ethernet), but is not a hard limit, and in the Ethernet case isn't in the IEEE specification and so may or may not be supported. Ask yourself, for example, how a 1500 byte Ethernet frame is encapsulated in MPLS and placed on another Ethernet...

What might be a useful survey would be for someone to figure out what interface types are currently in use and what their MTUs are.

---

From: Fred Baker <[email-masked]>
Date: Mon, 18 Jun 2012 05:18:24 -0700

On Jun 18, 2012, at 4:51 AM, Wuyts Carl wrote:

> I thought the target of the v6ops was to avoid these kind of things, and promote proper IPv6 packet handling.  After all, it is promoted from the start, that IPv6 brings you some extra things, like fixed header size, no more fragmentation (only at host), a MANDATORY PMTU (while for v4 it was only added AFTER the protocol was there), ...

On the mandatory PMTU...

Yes, it is mandatory to implement. Those operator guys - on alternate tuesdays I think some of them are either completely clueless or are actively trying to make things more complex. Example, for the IPv6 Day a year ago, I ran around looking at an amazing number of web sites. One that I couldn't read was Juniper's. My IPv6 is a Hurricane Electric tunnel, which I am quite sure is correctly sending ICMPv6 packet-too-large messages, as I can read lots of other web pages, but at that time I suspect that Juniper might have been dropping incoming ICMPv6 messages. I know at least one large company configured all of its equipment to never send a packet larger than 1280 bytes, which I would consider a sad eventual outcome.

It would be Really Nice if it were "mandatory to actually make work".

Or, alternatively, if end systems would implement RFC 4821 or something like it so we didn't have to.

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 18 Jun 2012 07:54:57 -0700

On Jun 18, 2012 5:18 AM, "Fred Baker" <[email-masked]> wrote:
> > I thought the target of the v6ops was to avoid these kind of things,
and promote proper IPv6 packet handling.  After all, it is promoted from
the start, that IPv6 brings you some extra things, like fixed header size,
no more fragmentation (only at host), a MANDATORY PMTU (while for v4 it was
only added AFTER the protocol was there), ...
>
> On the mandatory PMTU...
>
> Yes, it is mandatory to implement. Those operator guys - on alternate
tuesdays I think some of them are either completely clueless or are
actively trying to make things more complex. Example, for the IPv6 Day a
year ago, I ran around looking at an amazing number of web sites. One that
I couldn't read was Juniper's. My IPv6 is a Hurricane Electric tunnel,
which I am quite sure is correctly sending ICMPv6 packet-too-large
messages, as I can read lots of other web pages, but at that time I suspect
that Juniper might have been dropping incoming ICMPv6 messages. I know at
least one large company configured all of its equipment to never send a
packet larger than 1280 bytes, which I would consider a sad eventual
outcome.
> Or, alternatively, if end systems would implement RFC 4821 or something
like it so we didn't have to.

Afaik, failure to do pmtu is fairly quick and obvious breakage.

So blocking ptb icmpv6 is not an operational practice, it is an
implementation misstep that is resolved relatively quickly in the real
world.

It is not a systemic problem that needs to be solved by the ietf, it is
just part of the ipv6 learning curve.

CB

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jun 2012 09:05:03 -0700

Hi,

I touched this draft up for a -04 version to add a new
section on "tunnel fragmentation". No other changes were
made to the document. I don't expect any further revisions
ahead of the IETF, but should any occur I would announce
them here. In any case, the document can be tracked in
the datatracker at:

The text of the new section appears below:

Thanks - Fred

---
6.  Tunnel Fragmentation

   A third and less commonly known method of fragmentation is called
   "tunnel fragmentation".  Tunnel fragmentation occurs following any
   inner fragmentation but before any outer fragmentation.  Tunnel
   fragmentation requires separate packet Identification and
   segmentation control bits in a mid-layer of encapsulation that is
   added between the inner and outer IP headers.  As for outer
   fragmentation, the tunnel egress is responsible for reassembly.

   Tunnel fragmentation can be particularly useful for tunnels over
   IPv4, since the mid-layer encapsulation can include an extended
   Identification field that avoids the identification wrapping issues
   seen for IPv4 fragmentation [RFC4963].  Furthermore, when tunnel
   fragmentation is used the tunnel ingress has assurance that the
   egress can reassemble up to (1500 + HLEN) since both the ingress and
   egress are required to implement the scheme.  An example of tunnel
   fragmentation appears in SEAL [I-D.templin-intarea-seal].
---

> Sent: Monday, June 18, 2012 4:20 AM
> To: [email-masked] WG
> I have seen comments on-list from Daniel Roesen, Mark Andrews, and Washam
> Fan. In each case, they have commented on the draft without saying whether
> they think it is a valuable working group output, which I take quite
> literally - I don't know their opinion.
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 18 Jun 2012 09:23:04 -0700

Hi Carl,

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Wuyts Carl
> Sent: Monday, June 18, 2012 4:52 AM
> To: Fred Baker; [email-masked] WG
> I thought the target of the v6ops was to avoid these kind of things, and
> promote proper IPv6 packet handling.  After all, it is promoted from the
> start, that IPv6 brings you some extra things, like fixed header size, no
> more fragmentation (only at host), a MANDATORY PMTU (while for v4 it was
> only added AFTER the protocol was there), ....  Now I read in this draft
> to start doing some fragmentation with inner/outer, whatever, headers etc
> ??

IPv6 fixes fragmentation and reassembly by virtue of the
extended Identification field. I think that legislating IPv6
fragmentation out of the network was a well-intentioned mistake
that was heavily influenced by earlier works including the 1987
"Fragmentation Considered Harmful". That work was published a
long time ago and made valid points, but many of the assumptions
and issues it documented no longer apply.
 
> First of all, the "all links are 1500" statement is correct for Ethernet-
> based networks, but as I'm working for a company dealing with DSL, this is
> not always the case, so don't take it for granted.  Moreover, CPE will be
> heavily involved in this (6rd etc).
> Secondly, if ICMPv6 PTB messages gets filtered out (by some wrongly
> configured firewall or any other device) then it should be fixed where it
> goes wrong, not work-around it.  You're replacing a wrong filter with some
> complex mechanism of fragmentation.  Why is this filter in place ?

We're not trying to say that all links are 1500 - only that
1500 is the minimum path MTU that most hosts expect to see
unless they connect directly to a sub-1500 link. What we
would like to get to is support for MTU diversity so that
whatever link MTUs occur in the path can be naturally
accommodated. 
 
> Just for the record.  Our CPE can do fragmentation, so that's not an
> issue, but we'd like to avoid this becoming normal mode of operation, to
> avoid lots of potential issues with it.  Moreover, what's next ?

I don't think anything is next necessarily, other than
migration to links with larger MTUs. The choices for the
CPE are to either hope that the network delivers the
necessary PTB messages or to take matters into its own
hands and perform a little bit of fragmentation.

Thanks - Fred

---

From: Fred Baker <[email-masked]>
Date: Mon, 18 Jun 2012 09:28:38 -0700

> Afaik, failure to do pmtu is fairly quick and obvious breakage.

It is, but it may not be visible to the culprit. The fact that *I* block ICMPv6 as a swath means that *you* don't get PMTU. *I* don't see any breakage unless I am looking for it.

> So blocking ptb icmpv6 is not an operational practice, it is an implementation misstep that is resolved relatively quickly in the real world.
> 
> It is not a systemic problem that needs to be solved by the ietf, it is just part of the ipv6 learning curve.

I agree, but then... 

PMTU works the same way in IPv4, and in fact was developed for IPv4. ICMP is commonly blocked in IPv4, sending or receiving, and the practice is copied to IPv6. That said, I asked Google about "security practice ICMP" and got back links making recommendations on both sides - "Block ICMP type <> because it is used in attacks", "block outbound ICMP", and "permit PMTU messages". I would about bet that as soon as one steps away from people likely to be on this list, you'll get to people that say "the right solution is to block anything you can't say why you need - and *I* don't implement any tunnels..."

On the other hand, I can't say I think many of those people actually read RFCs, so I don't know what providing an RFC does for them.

---

From: Washam Fan <[email-masked]>
Date: Tue, 19 Jun 2012 13:48:17 +0800

Hi Fred,

We've discussed before in this link
http://www.ietf.org/mail-archive/web/v6ops/current/msg13206.html
I've read the section 4, -04 version. I noticed only clause 1) bullet
1) trigger PTB messages. Should it not better to let clause 1) bullet
2 and clause 2) also trigger PTB messages expecting PMTU works and
subsequent arriving packets just fit in the tunnel MTU?

Thanks,
washam

2012/5/23 Templin, Fred L <[email-masked]>:
> Please review and comment on 'draft-generic-v6ops-tunmtu':
>
> This new document takes the radical approach of proposing
> IPv6 router fragmentation at the tunnel ingress so that
> the tunnel MTU can be jacked up to 1500 bytes.
>
> The document notes issues that arise when the IPv6 router
> performs fragmentation - most notably, the selection of
> the Identification value to place in the (router-inserted)
> fragmentation header.
>
> Please review and comment on this thread. Remember that
> this approach is proposing something new and different
> and should be discussed further on the list.
>
> Thanks - Fred
>> [email-masked]
>> Sent: Wednesday, May 23, 2012 5:45 AM
>> To: [email-masked]
>> Subject: [v6ops] new draft: draft-generic-v6ops-tunmtu
>>
>>
>> A new draft has been posted, at http://tools.ietf.org/html/draft-generic-
>> v6ops-tunmtu. Please take a look at it and comment.

---

From: Washam Fan <[email-masked]>
Date: Tue, 19 Jun 2012 13:54:10 +0800

Hi Fred,

> and issues it documented no longer apply.

I think this issue is the only concern for me on this draft. the draft
explicitly says it *ignore* the requirement that router can't fragment
ipv6 packets enroute. If we would've had an agreement that this
requirement was a mistake, should we first update rfc2460 and then
persuit this draft?

Thanks,
washam

>> Fred Baker

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jun 2012 09:15:58 -0700

Hi Washam,

Thanks for the continued discussion, and see below:

> From: Washam Fan [mailto:[email-masked]]
> Sent: Monday, June 18, 2012 10:48 PM
> To: Templin, Fred L
> Cc: [email-masked]; [email-masked]; draft-generic-v6ops-
> [email-masked]
> 1) trigger PTB messages.

Yes, that is correct. But, the tunnel ingress has no way
of assuring that the PTB will be delivered all the way
back to the original source.

> Should it not better to let clause 1) bullet

Clause 1) bullet 2 is about fragmentable packets, and
is not a case where a PTB should be returned. The clause
ensures that the fragmentable packets get through by
fragmenting them to a still smaller size so that they
will not encounter a restricting link on the way to the
final destination (where they will be reassembled).

Clause 2) is the controversial portion of the proposal.
The goal is to ensure that 1500 and smaller packets get
through to the final destination even if a small amount
of fragmentation is necessary and without returning a
PTB (which again cannot be assured to be delivered to
the original source). In that case, it is the tunnel
ingress and not the original source that supplies the
Identification value.

This is all to meet the expectation of source hosts
that 1500 or smaller packets will be delivered to the
destination, but there are no such assurances for 1501
or larger packets. For those, the source host should
use RFC4821. It might be a good idea for me to add
some words to this effect in the next document version.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jun 2012 09:23:10 -0700

Hi Washam,

> Sent: Monday, June 18, 2012 10:54 PM
> To: Templin, Fred L
> Cc: Wuyts Carl; Fred Baker; [email-masked] WG; draft-generic-v6ops-
> [email-masked]
> and

That is indeed an interesting point. I believe there
were at least two factors that led to the omission of
network fragmentation in IPv6:

1) that including a 32-bit Identification value and
   16-bit fragmentation control field on each packet
   would add excessive overhead to the already-large
   IPv6 header

2) that the IPv6 header was already aligned on a nice
   quad-word boundary, and so adding one more field
   to the header would necessitate adding at least 8
   more bytes, making the header 48bytes instead of 40.

I don't know, but I suspect the 40byte IPv6 header is
burned into far too much silicon at this point to
consider retro-fitting to a 48byte header. But, maybe
others have ideas on this.

Thanks - Fred

> >> Third, I see the definition of an "atomic" packet, but that's quite
> some
> >> assumption to be made for IPv6 packets.
> >

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Jun 2012 20:50:36 +0200

Hi,

> others have ideas on this.

Incompatible changes to the packet format are right out - if you do that,
what little IPv6 deployment we have achieved so far is all dead.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 19 Jun 2012 12:26:30 -0700

> From: Gert Doering [mailto:[email-masked]]
> Sent: Tuesday, June 19, 2012 11:51 AM
> To: Templin, Fred L
> Cc: Washam Fan; [email-masked]; [email-masked]
> WG

I tend to agree, and a quick check shows that in-the-network
fragmentation was left out of the design from the very earliest
days going back to the SIPP proposal in the 1993/94 timeframe.
If anyone has references to earlier works and decision points
on this issue I'd be interested to see them.

Thanks - Fred

---

From: Washam Fan <[email-masked]>
Date: Wed, 20 Jun 2012 11:02:35 +0800

Hi Fred,

> final destination (where they will be reassembled).

OK. I see.

> the original source).

Right. If a PTB is returned, it is probably blocked or ignored, but it
probably also be seen by the original source. In the latter case, the
subsequent packages from the original source would fit in 1280 and
tunnel ingress no need to fragment afterward.

Thanks,
washam

> In that case, it is the tunnel

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 20 Jun 2012 09:44:49 -0700

Hi Washam,

> Sent: Tuesday, June 19, 2012 8:03 PM
> To: Templin, Fred L

OK.

> tunnel ingress no need to fragment afterward.

OK - so, the tunnel ingress should both send the packet
*and* also return a PTB? That's a very interesting idea
and sounds OK on the surface. But, the source host upon
receiving the PTB is going to reduce its packet sizes
to 1280 even if the tunnel is configured over all-9KB
links, i.e., the Clause 1 case would never be met.

There is also another consideration with clauses 2 and 3.
If there will be nested encapsulations on the path between
the tunnel ingress and egress then even a 1280 packet may
be too big to enter another tunnel ingress on the path. So
we should leave extra room for additional encapsulations
and change clauses 2 and 3 to:

     2) if the packet is between 1025 - 1500:
        - break the packet into 2 pieces (where each piece
          is a random length between 500-1000 bytes) and
          admit each piece into the tunnel.

     3) if the packet is 1024 or less:
        - admit the packet into the tunnel

That said, we could use an adaptation of your idea and
change clause 2 to:

     2) if the packet is between 1025 - 1500:
        - break the packet into 2 pieces (where each piece
          is a random length between 500-1000 bytes), admit
          each piece into the tunnel and return a PTB
          message with MTU=1024.

Then, when the source host receives the PTB, it will
honor the final paragraph of RFC2460, Section 5 and
begin inserting a fragment header in the packets it
sends. The tunnel ingress can then use the Id value
in the fragment header when performing fragmentation.

What do you think?

Fred

> >> >> From: [email-masked] [mailto:[email-masked]] On
> Behalf
> >> Of
> >> >> [email-masked]

---

From: Washam Fan <[email-masked]>
Date: Thu, 21 Jun 2012 13:03:43 +0800

Hi Fred,

I think it might be better to discuss this issue by introducing tunnel
MTU. See inline for details.

> links, i.e., the Clause 1 case would never be met.

Why not we return a PTB >= tunnel MTU instead of 1280? if the tunnel
is over all 9kB links, all tunnelled packets would be fit in the
tunnel without fragmentation, because now the tunnel MTU is 9kB minus
the encapsulation header (which is very smaller compared to 9kB)

In a case where ipv6 packets transmitted over ipv4 network, the tunnel
MTU would be 1480 = 1500 - 20 (the lengh of ipv4 header without
options). Then
1. any packets whose length exceeds 1500, drop the packets and return PTB>=1480
2. any packets whose length is between 1480 and 1500, fragment the
packets and return PTB>=1480
3. any packets whose length is under 1480, transmit without fragments

> and change clauses 2 and 3 to:

if netsted encapsulations case is considered, the tunnel MTU would be
1500 - 20 - additional costs

>        - admit the packet into the tunnel

In the above case, 1025 would be tunnel MTU (which is under 1280)

> in the fragment header when performing fragmentation.

Yes. if the tunnel MTU is under 1280,  the above statement would be valid.

> What do you think?

What do you think ?

Thanks,
washam