Subject: Re: [v6ops] new draft: draft-generic-v6ops-tunmtu

From: Joe Touch <[email-masked]>
Date: Wed, 20 Jun 2012 16:07:13 -0700

> Please review and comment on 'draft-generic-v6ops-tunmtu':
>
> https://datatracker.ietf.org/doc/draft-generic-v6ops-tunmtu/
>
> This new document takes the radical approach of proposing
> IPv6 router fragmentation at the tunnel ingress so that
> the tunnel MTU can be jacked up to 1500 bytes.
>
> The document notes issues that arise when the IPv6 router
> performs fragmentation - most notably, the selection of
> the Identification value to place in the (router-inserted)
> fragmentation header.

Hi, all,

This draft was recently brought to my attention in a discussion on the 
main IETF list. This doc's premise is badly flawed:

Even assuming the pending draft-ietf-intarea-ipv4-id-update-05, it is 
impossible to overwrite the DF and ID of inner IPv4 datagrams in a way 
that makes fragmentation usefully safe. Consider:

	A. non-atomic datagram fragments that enter the ingress
	
	B. non-atomic datagrams that have not yet been fragmented,
	but that this ingress needs to fragment

	C. non-atomic datagram fragments that reach the destination
	but do not use the tunnel at all

The IDs that the ingress overwrites CANNOT use IDs that are used by any 
of (A), (B), or (C). (B) can be avoided by overwriting those IDs too. 
(A) requires the ingress keep track of the IDs seen and avoid using them 
for 2MSL. IDs used by (C) aren't seen at all - and thus cannot be avoided.

Keep in mind that (C) could also be the result of one of the proposed 
tunnels from this draft on a different path in the network.

So (A) and (B) make this complex, and (C) makes it impossible. I.e., the 
result of inserting your own IDs into someone else's packets is that you 
could corrupt reassembly at the destination, and you can't know when 
that will happen.

----

I have stated before many times that tunnels must clean up the mess they 
create. The only viable path for a tunnel that receives a DF=1 datagram 
is to generate fragments in the outer header, and reassemble them at the 
egress.

Joe

---

From: Washam Fan <[email-masked]>
Date: Thu, 21 Jun 2012 13:42:50 +0800

Hi Joe,

2012/6/21 Joe Touch <[email-masked]>:
> This draft was recently brought to my attention in a discussion on the main
> IETF list. This doc's premise is badly flawed:
>
> impossible to overwrite the DF and ID of inner IPv4 datagrams in a way that
> makes fragmentation usefully safe. Consider:
>
> The IDs that the ingress overwrites CANNOT use IDs that are used by any of
> (A), (B), or (C). (B) can be avoided by overwriting those IDs too. (A)
> requires the ingress keep track of the IDs seen and avoid using them for
> 2MSL. IDs used by (C) aren't seen at all - and thus cannot be avoided.
>
> could corrupt reassembly at the destination, and you can't know when that
> will happen.
>

the last 2nd para, section 4 admits the exist of the collision, do you
think the randomness assignment can not suffiently address this issue?

> create. The only viable path for a tunnel that receives a DF=1 datagram is
> to generate fragments in the outer header, and reassemble them at the
> egress.
>

Right. This way eliminate the fragment ID collision concern. But  this
draft is to avoid reassemble at egress.

Thanks,
washam

>
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jun 2012 08:04:53 -0700

Hi, Washam,

> think the randomness assignment can not suffiently address this issue?

It might for IPv6, but the ID space for IPv4 is far too small to avoid such collisions.

The draft says:

   Specifically, the tunnel ingress must ensure that there will be no IP
   fragments alive in the system with duplicate Identification values.
   Since [RFC2460] specifies that the maximum time a node may retain an
   incomplete fragmented packet is 60 seconds, this means that the
   tunnel ingress must not allow the Identification values to be
   repeated within this timeframe.  The tunnel ingress can therefore
   calculate a maximum data rate for admission of fragmented packets
   into the tunnel.

The tunnel ingress cannot know the fragments that are alive in the system that don't use the ingress.

FWIW, there are other problems:
- admitting fragments with IDs that are recently used (within 60 seconds)
at a rate of 11 Mbps, regardless of how random IDs are generated, the IDs *must* repeat, which means that *all* incoming fragments must be discarded.

- the 60 second rule for the receiver is used in this paper as if there were one such timer per ID forever. consider that fragments could arrive at the receiver separated by up to 2MSL (let's assume the MSL is reasonable); if three such segments arrive spaced at t=0seconds and t=119seconds, then the receiver is susceptible to collisions for 179 seconds.

Overall, the ingress cannot rewrite IDs faster than a rate of 4Mbps ONLY IF it is the *ONLY* generator of IDs on the path - i.e., only if the source never generates non-atomic packets AND there are no other tunnels doing what this draft proposes.

The probability of collision is proportional to the rate of nonatomic packets of all entities that rewrite IDs on a path. That cannot be known. It is unsafe and should not be suggested.

> draft is to avoid reassemble at egress.

That's a common goal, but always creates problems when it ignores the DF bit. That bit can be set because of a negotiation between the source and destination using higher level information (e.g., HTTP request context), sometimes to take into account limitations on reassembly limitations at the receiver (where reassembly may be supported but might consume memory or CPU resources, such as for cellphones).

See section 8 of http://tools.ietf.org/html/draft-ietf-intarea-tunnels-00 (granted, I'm overdue to update that...)

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 21 Jun 2012 10:00:05 -0700

Hi Joe,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Joe Touch
> Sent: Wednesday, June 20, 2012 4:07 PM
> To: [email-masked]
> Subject: Re: [v6ops] new draft: draft-generic-v6ops-tunmtu
> 

I can see the point about IPv4. In that case, perhaps the
document should recommend against inner fragmentation and
jump straight to section 5 on Outer Fragmentation. This
would be consistent with RFC6333.

> is to generate fragments in the outer header, and reassemble them at the

With IPv6, I believe it should be OK for the router to
perform inner fragmentation if the packet already has
a fragment header - in other words, treat the packet as
non-atomic the same as for IPv4 with DF=0. If the packet
does not already have a fragment header then insert one,
have the tunnel ingress supply an Identification value
(random? sequential?), and also send a PTB message back
to the original source telling it to begin inserting
fragment headers. The draft doesn't currently say this,
but will fix that in the next version.

Fred
[email-masked]

---

From: Joe Touch <[email-masked]>
Date: Thu, 21 Jun 2012 15:59:55 -0700

Hi, Fred,

On 6/21/2012 10:00 AM, Templin, Fred L wrote:
...
> would be consistent with RFC6333.

That's what I would suggest.

> non-atomic the same as for IPv4 with DF=0.

I disagree. IPv6 establishes that the source - and only the source - 
determines fragment boundaries. There are various reasons to retain 
control over that.

I haven't seen a general recommendation to relax that constraint. Absent 
such a general update to RFC 2460, I think that this doc should stay 
within existing requirements or be very clear about relaxing them in 
general. There's simply no reason to treat this tunnel case as distinct.

 > If the packet

Inserting the header has the same problem as ignoring DF=1. It defeats 
all path MTU mechanisms downstream of the decsion. Yes, the ID space is 
larger and potential for collisions is lower, but I don't like the idea 
of a tunnel making this gamble - there are consequences to the 
endpoints, but none to the tunnel.

Sending a PTB is fine for both IPv4 and IPv6. Fragmenting the outer is 
fine in both cases, and clean. Anything else requires a general revision 
to RFC791/RFC1122, RFC2460, and probably both PMTU docs. I don't see the 
reason for *this* tunnel mechanism as a justification for any of that.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Jun 2012 09:23:32 -0700

Hi Joe,

> From: Joe Touch [mailto:[email-masked]]
> Sent: Thursday, June 21, 2012 4:00 PM
> To: Templin, Fred L
> Cc: [email-masked]

I spun the draft again to take out inner fragmentation for
IPv4 as the inner payload. For IPv4-over-IPv4 tunnels, however,
even outer fragmentation at high data rates is dangerous and
cannot be recommended without rate limiting. Does this match
your understanding?

For IPv4-over-IPv6 tunnels, outer fragmentation is feasible
since host-based fragmentation is fixed in IPv6. However,
existing tunnel protocols have not specified the minimum MRU
that the tunnel egress must configure, so the ingress has no
way of knowing whether the egress can reassemble a 1500 byte
inner packet plus the size of the encapsulation headers. That
means that the ingress would have to set a (1500-HLEN) MTU
at the maximum, and can lead to PMTUD problems when the
ingress needs to drop a packet that is larger than this
MTU but no larger than 1500.  

> general. There's simply no reason to treat this tunnel case as distinct.

I think we should relax the RFC2460 requirement that routers
not (re)fragment IPv6 packets that already include a fragment
header. Otherwise, we can run into problems when there are
multiple tunnel nestings between the original source and
final destination, i.e., even if the source takes care in
setting conservative fragment boundaries. To relax this
would take an update to RFC2460, Section 4.5 to drop the
following statement:

   "(Note: unlike IPv4, fragmentation in IPv6 is performed
    only by source nodes, not by routers along a packet's
    delivery path -- see section 5.)"

> endpoints, but none to the tunnel.

If the tunnel sets a conservative rate limit for admitting
inner packets for which it has added a fragmentation header,
the incidence of collisions will be minimized, and the source
is given incentive to begin including fragment headers in the
packets it sends. Namely, inner packets that already include
a fragment header do not need to be rate limited.

> fine in both cases, and clean.

Outer fragmentation for tunnels over IPv4 is subject to a
too-slow rate limit. Outer fragmentation for tunnels over
IPv6 do not have this limitation, but there is no assurance
that the tunnel egress can reassemble a 1500 byte inner
packet.

A third alternative called "tunnel fragmentation" is
available if the tunneling protocol inserts a mid-layer
encapsulation with a suitably long Identification field.
That is what SEAL does, but it requires both the ingress
and egress to be aware of SEAL.

> Anything else requires a general revision

No changes to RFC791, RFC1122 and the PMTU docs. The only
update would be to RFC2460 to re-allow router fragmentation
and to encourage hosts to include fragment headers with
the packets they send.

It should be clear by now that I believe the abolishment
of router fragmentation in IPv6 was a mistake that seems
to be interfering with tunnels. It would be very nice to
find a way to fix this by only having to visit the tunnel
ingress (and not both the ingress and egress), and inner
fragmentation is the only way to do that unless we are
willing to pull in SEAL.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 22 Jun 2012 10:05:17 -0700

Hi Fred,

...
> cannot be recommended without rate limiting. Does this match

Yes.

> inner packet plus the size of the encapsulation headers.

Agreed. IPv4 reassembly is assumed up to 576 bytes, but not above that.

> That

Strictly, it has to set to the min of the MTU of IPv4 and IPv6 - which 
is 576 as per above.

However, if you're not willing to make this bet on behalf of the egress 
- which you can specify herein - why are you expecting that the ultimate 
endpoint has the capability to reassemble 1500-byte IPv4 packets either?

> header.

That should not happen as a side issue of this doc. That should be a 
standalone proposal.

There was a lot of debate around this as I recall; I'll be surprised if 
it passes, FWIW.

 > Otherwise, we can run into problems when there are
> setting conservative fragment boundaries.

Multiple nestings always cause problems, but I don't think it's useful 
to sacrifice robustness for efficiency. Multiple nestings always work if 
the tunnels frag/reassemble - it's inefficient, but robust. All other 
solutions are not, AFAICT.

> To relax this

Agreed - that's the standalone proposal I refer to above.

> the incidence of collisions will be minimized,

Collision probability is determined by the combination of what you do 
and what other sources do - since you can't know the latter, you can't 
know the total.

> and the source
> packets it sends.

The source has no idea what you're doing in this downstream tunnel. You 
can try to send an ICMP upstream, but those are often blocked. So you 
should not assume the source will adjust its behavior.

 > Namely, inner packets that already include

Agreed.

> that the tunnel egress can reassemble a 1500 byte inner

Yes. But there's similarly no assurance that the destination can either, 
as noted above. If you have to make this assumption, make it inside your 
system.

> and egress to be aware of SEAL.

Yup. And that's the best solution for IPv4.

> and to encourage hosts to include fragment headers with

See above; as a separate proposal.

> to be interfering with tunnels.

I think this is the wrong conclusion. IMO, tunnels always are 
implemented in pairs - ingress and egress - and the entire system needs 
to be defined and implemented.

My conclusion is that doing tricks at the ingress to avoid work at the 
egress is inappropriate, because it *always* pushes that work to the 
final destination. IMO, the ingress shouldn't place any bets that the 
egress can't pay out.

> It would be very nice to

I don't see that as a bad thing.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 22 Jun 2012 10:17:06 -0700

One PS -

IMO, fragmenting something that says "don't fragment" is a bad idea.

IPv4 does this with DF=1, we agree.

IMO, IPv6 does this by omitting the frag header.

Mucking with either one destroys PMTU determination.

If you want to update IPv6 to allow downstream fragmentation, you are 
still doing something dangerous when you add a frag header when it 
wasn't there. Regardless of the potential for frag ID collisions (which 
is admittedly low for IPv6), you're still expecting

	a) that the destination can reassemble the components
	instead of the egress

	b) that there's no interference with PMTU algs

I would suggest that - at best - you might want to see how changing IPv6 
to allow downstream fragmentation is received, but that inserting a new 
frag header where one doesn't exist should still be avoided.

Joe

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 22 Jun 2012 21:43:22 +0000

Coming back to this as a meta-issue.

v6ops is about operational considerations and procedures, but not protocols; disputing RFC 2460, aka redesigning IPv6, seems like a protocol issue.

The reason to not do inner fragmentation, if memory serves, has to do with the behavior of fragmentation in the network and its effect on communications. For example, suppose you and I are in 9K clean networks (so the TCP MSS starts out as 9K), my link to the public network has an MTU of 1500, and somewhere en route to you there is another link with an MTU of 1400. When I send a 9K packet, it will become six 1500 byte packets with a small caboose that picks up the size of five IP headers (IPv4 or IPv6), and what you will receive is six 1400 byte packets interspersed with six 100+IP byte packets, followed by the original caboose. What if the fragmenting router's queue, at the time of fragmentation, was one packet short of the needed capacity? Maybe the retransmission follows a different path and is fragmented differently, resulting in funny overlaps whose handling isn't very well specified. There's nothing *incorrect* about a stream of 13 packets of various sizes being reassembled, but integrating retransmissions gets messy. IIRC, they just wanted to clean that up.

Which brings me to the following consideration.

If we're talking about having one tunnel endpoint put a message into a tunnel datagram and then fragment it, and have the other tunnel endpoint reassemble the original and forward it, we are talking about an operational procedure that requires support in a router, but which I can correlate with section 5 of RFC 2460.

One thing I would invite is discussion of operational experience with RFC 4821. Wouldn't it be nice if the endpoint actually chose an MSS based on what actually worked (shades of Happy Eyeballs), rather than depending on error messages that network operators routinely filter out?

If we're talking about changing the recommendation of RFC 2460 regarding who does fragmentation, that sounds like an IPv6 protocol change, and I'd like to refer that to 6MAN.

Does that make sense?

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Jun 2012 15:27:58 -0700

Hi Joe,

> Sent: Friday, June 22, 2012 10:05 AM
> To: Templin, Fred L
> Cc: [email-masked]

OK.

> Agreed. IPv4 reassembly is assumed up to 576 bytes, but not above that.

OK, so IPv4 routers are only required to reassemble up to
576 bytes and are not required to reassemble any more - is
that right? So, tunnel protocols that specified router
reassembly without specifying a tunnel egress router's
maximum reassembly unit (MRU) are only guaranteed 576?

> Strictly, it has to set to the min of the MTU of IPv4 and IPv6 - which

Well, minMRU for IPv6 is 1500 while minMRU for IPv4 is
only 576. So, an IPv6 tunnel egress should be able to
reassemble inner packets up to (1500-40) bytes while
an IPv4 tunnel egress only needs to reassemble inner
packets up to (576-20) bytes in length - right?

> endpoint has the capability to reassemble 1500-byte IPv4 packets either?

Hosts are expected to reassemble as much as their
connected network interfaces, which is 1500 in the
vast majority of cases. Unless by "ultimate endpoint"
you mean that that could be another tunnel egress
router instead of a host - in which case we are
back to 576, right?

> That should not happen as a side issue of this doc. That should be a

Maybe we can peel this out of this document and go
standalone - maybe in a different wg?

> There was a lot of debate around this as I recall; I'll be surprised if

We will see...

> solutions are not, AFAICT.

Certainly having the tunnels drop packets and return
PTBs is a brittle solution.

> Agreed - that's the standalone proposal I refer to above.

OK.

> and what other sources do - since you can't know the latter, you can't

OK, I'll have to admit the point. The tunnel ingress can
only control what it does and not what other network
elements might do. 

> should not assume the source will adjust its behavior.

OK - but we can still

> > a fragment header do not need to be rate limited.

OK.
 
> as noted above. If you have to make this assumption, make it inside your

OK, but there needs to be some requirements set for tunnel
egress MRUs so that a 1500 inner packet can be reassembled
> Yup. And that's the best solution for IPv4.

Good. This is the place where we can set the requirement
for the tunnel egress MRU.

> See above; as a separate proposal.

OK.

> final destination. IMO, the ingress shouldn't place any bets that the

OK.
 
> I don't see that as a bad thing.

Pulling in SEAL, you mean? If so, I agree and I'll make sure
that SEAL is ready to go.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Jun 2012 15:35:30 -0700

Hi Fred,

> From: Fred Baker (fred) [mailto:[email-masked]]
> Sent: Friday, June 22, 2012 2:43 PM
> To: Templin, Fred L; [email-masked] WG
> v6ops is about operational considerations and procedures, but not
> protocols; disputing RFC 2460, aka redesigning IPv6, seems like a protocol
> issue.

OK - we can take this up outside of v6ops if necessary.

> The reason to not do inner fragmentation, if memory serves, has to do with
> the behavior of fragmentation in the network and its effect on
> communications. For example, suppose you and I are in 9K clean networks
> (so the TCP MSS starts out as 9K), my link to the public network has an
> MTU of 1500, and somewhere en route to you there is another link with an
> MTU of 1400. When I send a 9K packet, it will become six 1500 byte packets
> with a small caboose that picks up the size of five IP headers (IPv4 or
> IPv6), and what you will receive is six 1400 byte packets interspersed
> with six 100+IP byte packets, followed by the original caboose. What if
> the fragmenting router's queue, at the time of fragmentation, was one
> packet short of the needed capacity? Maybe the retransmission follows a
> different path and is fragmented differently, resulting in funny overlaps
> whose handling isn't very well specified. There's nothing *incorrect*
> about a stream of 13 packets of various sizes being reassembled, but
> integrating retransmissions gets messy. IIRC, they just wanted to clean
> that up.

I think you could probably extend this to even worse cases
if you said that the 1400 link was followed by a 1380 link,
which was followed by a 1350 link, etc. The caboose-trimming
would become pathological very quickly. So, why not require
that fragmenting routers make all fragments to be roughly
the same size (instead of roughly MTU-sized) so they will
fit through smaller links further down the line?

> If we're talking about having one tunnel endpoint put a message into a
> tunnel datagram and then fragment it, and have the other tunnel endpoint
> reassemble the original and forward it, we are talking about an
> operational procedure that requires support in a router, but which I can
> correlate with section 5 of RFC 2460.

Right.

> One thing I would invite is discussion of operational experience with RFC
> 4821. Wouldn't it be nice if the endpoint actually chose an MSS based on
> what actually worked (shades of Happy Eyeballs), rather than depending on
> error messages that network operators routinely filter out?

ICMP-based PTB feedback is nice to have if you can get
it. But, you can't always count on getting the PTBs,
hence the need for RFC4821.
 
> If we're talking about changing the recommendation of RFC 2460 regarding
> who does fragmentation, that sounds like an IPv6 protocol change, and I'd
> like to refer that to 6MAN.
> 
> Does that make sense?

Sure. If we decide to make a recommendation for RFC2460
we can take that up in 6MAN. But, that would be a separate
proposal.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Fri, 22 Jun 2012 16:23:51 -0700

Hi, Fred,

...
> that right?

Correct - that's as per RFCs 791 and 1122. Search for 576 and you'll see 
the relevant text.

> So, tunnel protocols that specified router

Correct.

> packets up to (576-20) bytes in length - right?

An tunnel egress needs to be able to reassemble IPv6 inner packets that 
sum up to 1500 bytes, or IPv4 inner packets that sum up to 576 bytes.

In both cases that includes the IP headers and their options. Note that 
this means the egress needs to reassemble (outer) IP packets that are 
larger than the minimum in both cases (476 or 1500 + the outer + any 
other signatures or shim layers).

> vast majority of cases.

That's probably statistically correct, but the only requirement is 476 
for IPv4 interfaces.

> Unless by "ultimate endpoint"

I meant the destination of the packet that arrives at the ingress, i.e., 
the "final destination".

As per above, though, tunnel egresses always need to have higher 
reassembly bounds than "final destination" hosts on the network.

Jumping ahead
...
> PTBs is a brittle solution.

Agreed, but that need not happen if frag/reassembly is supported. In 
that case PTB is advisory, not critical to successful transmission.

Jumping again
...
>> should not assume the source will adjust its behavior.

This seems incomplete. Yes, you can still send the ICMPs and hope they 
get through, but you probably want to assume they won't.

Jumping one last time
...

Yes.

Joe

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Sat, 23 Jun 2012 00:32:05 +0000

> fit through smaller links further down the line?

          OOO  M     M  GGG
         O   O MM   MM G   G
         O   O M M M M G
         O   O M  M  M G  GG
         O   O M     M G   G
         O   O M     M G   G
          OOO  M     M  GGG

If you want to go there, there are some 20-year-old discussions I need for you to look at. Options proposed include at least
  - someone (a router, a host) fragmenting traffic should make all chunks the same size
  - the first N-1 chunks should be approximately MTU-sized and the last a caboose
  - the last N-1 chunks should be approximately MTU-sized and the first a caboose

There have been passionate discussions, each approach having proponents with believed-by-them-to-be-good arguments for their position. And I'm pretty sure there were other positions.

The one thing I'll take serious exception to is a proposal that a router reassemble packets and then re-fragment some other way in the normal course of doing business.

Excellent fodder for an IRTF draft. Go for it, with my blessing. I, personally, won't be there to keep track of the angels and pinheads...

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Sat, 23 Jun 2012 06:32:22 +0000

> Agreed. IPv4 reassembly is assumed up to 576 bytes, but not above that.

</chair>

That may be the letter of the law. Practically speaking, both IPv4 and IPv6 routers have to be able to deal with the messages that hosts exchange with them, and there is no obvious limitation on that. I think it's fair to assume that if there is an MTU somewhere in the network of a given amount, some host will take the opportunity to send a message that size, and be very confused if whatever it's talking with (at the IP layer, it doesn't know whether the peer is a router or a host) can't reassemble a message that it sent and (however it happened) got fragmented.

For today's interfaces, I would say that boils down to a *market* requirement (not a legal value, just something that affects, oh, purchase orders, among other things) to reassemble a 9K packet.

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 25 Jun 2012 09:20:50 -0700

Hi Fred,

> Sent: Friday, June 22, 2012 5:32 PM
> To: Templin, Fred L
> Cc: [email-masked] WG
> If you want to go there, there are some 20-year-old discussions I need for
> you to look at. Options proposed include at least

I guess you are talking about p.45 of RFC1812? I've looked
at that many times and have always wondered about the option
to lead with a caboose as the first fragment. As a receiver,
I'd like to be able to look at the first fragment as a way
of measuring the path MTU, but I don't get that option when
the first fragment is tiny. 

> There have been passionate discussions, each approach having proponents
> with believed-by-them-to-be-good arguments for their position. And I'm
> pretty sure there were other positions.
> 
> The one thing I'll take serious exception to is a proposal that a router
> reassemble packets and then re-fragment some other way in the normal
> course of doing business.

Per RFC1812, I agree for packets that are just passing
through the router, but the router has to reassemble packets
that are explicitly addressed to itself. RFC1122 also says
that the router only needs to reassemble 576 at minimum
so that is all that the tunnel near end can assume if the
tunnel far end is (or might be) a router.

> Excellent fodder for an IRTF draft. Go for it, with my blessing. I,
> personally, won't be there to keep track of the angels and pinheads...

I guess you didn't like the departure into fragmentation
technique discussions. We don't have to go there, so we
can back off from that line of discussion if you find it
offensive.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 25 Jun 2012 09:54:06 -0700

...
> the first fragment is tiny.

Since there is no requirement that the first fragment be "as large as 
possible", it is inappropriate to infer anything from its size anyway.

> that are explicitly addressed to itself.

I don't think there's any disagreement there. There are two kinds of 
packets "router boxes" process:
	forwarded packets (when the box acts like a router)
	packets originating/terminating at the router (when the box acts
		like a host)

Note that tunnel ingress/egress processing falls into the latter 
category for the outer packet, and the former for the inner packet.

> RFC1122 also says

RFC1122 applies only to hosts (and routers when they act as a host, 
e.g., as a source or destination of packets). And yes, 576 is the 
largest reassembled MTU for IPv4 (the largest MTU, e.g., that you can 
assume for fragments, is 64).

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 25 Jun 2012 11:18:16 -0700

Hi Joe,

> Sent: Monday, June 25, 2012 9:54 AM
> To: Templin, Fred L
> Cc: Fred Baker (fred); [email-masked] WG
> >>    - someone (a router, a host) fragmenting traffic should make all
> chunks
> >> the same size
> >>    - the first N-1 chunks should be approximately MTU-sized and the
> last a
> >> caboose
> >>    - the last N-1 chunks should be approximately MTU-sized and the
> first a
> >> caboose
> >

Maybe or maybe not. If the first fragment is significantly
larger than the minimum link MTU isn't it fair to say that
it is likely to approximate the largest unfragmented packet
that can traverse the tunnel? The only risk then is in
under-estimating the tunnel MTU (and not over-estimating).

An historical anecdote - Fred B. may know more about this.
In the mid-1980's companies such as Digital Equipment
Corporation produced 1st generation Ethernet gear that
could not always keep up with wire-speed. I know of at
least one of these 1st generation NICs that would reset
itself if it dropped a packet, which happened quite
frequently if the card received a burst of fragments
from a large fragmented datagram. Was this the genesis
of the RFC1918 text that says:

  "Work with slow machines leads us to believe that if it is
   necessary to fragment messages, sending the small IP fragment
   first maximizes the chance of a host with a slow interface of
   receiving all the fragments."

??
  
> category for the outer packet, and the former for the inner packet.

Right. And, the philosophy I'm suggesting is that the
router kick the can down the road as far as possible
toward the inner packet destination as long as it is
done within the specified limits for inner fragmentation
and reassembly. When that can't be done, then the router
at the other end of the tunnel has to do the reassembly.

> RFC1122 applies only to hosts (and routers when they act as a host,

Right; router acting as host is the case of interest.

> assume for fragments, is 64).

Did you mean 68?

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 25 Jun 2012 11:29:23 -0700

Hi Fred,

...
> that can traverse the tunnel?

No; there are no requirements on how packets are fragmented, so making 
any assumptions based on any subset of fragments is not being "liberal 
in what you accept".

> The only risk then is in

Technically that's true - any fragment you see must have gotten through, 
so you can assume that the E2E path supports at least that size fragment 
(for the given endpoints; for a tunnel that's the ingress/egress if 
we're talking about outer fragmentation).

However, I'm not sure that "at least this big" helps all that much for 
any given fragment. You'd need to see a stream over time to draw any 
reasonable conclusions.

> of the RFC1918 text that says:

I think you mean RFC 1812 ;-)

That RFC includes a discussion of various rationales for how to 
fragment, and basically recommends only that routers don't unnecessarily 
create more fragments than necessary.

>     receiving all the fragments."

I would expect the excerpt above refers to systems where the need to 
create a new entry for a new fragment interferes with receive 
processing, so a small first fragment helps in that case. It does assume 
that there's no persistent reordering, and assumes a somewhat broken 
receiver anyway though, IMO.

> at the other end of the tunnel has to do the reassembly.

Routers shouldn't kick problems down the road towards the destination. 
They should fragment inner packets ONLY when they need to, and that 
necessity doesn't exist for tunnels.

...
>> assume for fragments, is 64).

Yeah - typed too quick. 68.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 25 Jun 2012 11:39:51 -0700

> >Was this the genesis

That's about the 10th time I've mis-typed that today...
no associations btw the two documents intended! :^}

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 25 Jun 2012 13:25:19 -0700

Hi again Joe,

> receiver anyway though, IMO.

Actually, I have some real-world experience behind this.
In 1986, I was asked to look into a problem where a DEC
VAXStation with a DELQA NIC was performing very poorly as
an NFS file server. We found that the DELQA actually reset
itself if it ran out of receive ring buffers, which is
exactly what was happening when enough fragmented
IPv4/UDP/NFS packets arrived in bursts. There was literally
nothing to be done for it - the hardware was deemed broken,
and the VAXStation/DELQA configuration was disqualified
as an NFS server platform.

Right around the same time, a DEC colleague was off writing
"Fragmentation Considered Harmful", which (IMO) is where the
whole fragmentation/reassembly/MTU debate went sideways. I
was never consulted on that writing, but my input would have
been that the protocol shouldn't be changed to accommodate
broken hardware. We are still living with the "harmful"
stigma today, but we need to get beyond that if we're
going to accommodate tunnels.

Fred

---

From: Joe Touch <[email-masked]>
Date: Mon, 25 Jun 2012 13:39:06 -0700

Hi, Fred,

> as an NFS server platform.

Sure, it's not surprising...

> going to accommodate tunnels.

Yup - you note too it seems like not being able to setup the fragment 
reception at line rate is an implementation failure, not necessarily 
something we should modify protocols to address.

Joe

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Mon, 25 Jun 2012 21:23:59 +0000

>   receiving all the fragments."

It may have been. I was the editor, a long time after the fact of that text being written. I can tell you that there were a list of NIC issues in the 1980's; Novell could only receive packets if there was a certain gap between them, Cisco AGS with a SEEQ chip had a gap requirement, and so on. In 2012, I wouldn't read a lot into those comments.