Subject: [tcpm] MSS & TCP Options

From: Danny McPherson <[email-masked]>
Date: Fri, 16 Jun 2006 11:10:02 -0600

Folks,
Given the discussions regarding TCP authentication and "Options"
as of late, and a problem we've seen manifest itself under certain
conditions "in the field", I'm hoping someone here might be able to
help me clarify something.  And please be patient, I'm far from
being a TCP expert.

In a nutshell, I think my question can be summed up like this (with
a much more verbose set of observations trailing):

Is there a _definitive source for how the Eff.snd.MSS value should
be calculated when TCP Options are employed, and precisely what
should be contained in an advertised MSS wrt TCP Options?

It seems to me an ambiguity arises from the fact that some
deployed implementations subtract the size of the TCP options in
the MSS value they advertise, AND they implicitly expect the
senders to do the same (e.g., as specified in RFC 2385 and RFC
2581).  Therefore they do NOT dynamically calculate the
Eff.snd.MSS to include options when sending a TCP segment.

Other implementations (e.g., Linux, OpenBSD, NetBSD)  do NOT
include TCP options in the advertised MSS, but rather, calculate
them on a per-segment basis as specified in RFC 1122.

The latter of these two methods seems more appropriate, given
that some options are per-segment (e.g., SACK & Window Scale)
and therefore the receiver has no way of knowing which options
it's peer will be sending.

Not only does RFC 2385 (TCP MD5 Signature Option) and
RFC 2581 suffer from this, so to do the two TCP authentication
I-Ds being discussed.

Section 12.4 of this I-D appears to have copied the relevant text
directly from RFC 2385, and therefore is equally susceptible:

http://www.ietf.org/internet-drafts/draft-bonica-tcp-auth-04.txt

And section 4.4 - 4.6 of this I-D has no mention of the calculations
at all.  I'm a bit surprised but perhaps this would lead someone back
to an RFC 1122 model, rather than diverging from that as RFC
2385 and RFC 2581 have done:

http://tools.ietf.org/wg/tcpm/draft-touch-tcpm-tcp-simple-auth-00.txt

Finally, one other point worth mentioning is that if the end stations
don't agree on how to calculate the Eff.snd.MSS then PMTU doesn't
really fix it, and therefore transactions between TCP senders that
expect the sender to include TCP option size in the advertised MSS
(and therefore don't calculate dynamically into the Eff.snd.MSS)
value can still be affected.

Unless I'm missing something there's a dire need here for a single
definitive source on this, and all the TCP Options work should
reference it as such.

---
So, some more details...

RFC 1122; 4.2.2.6 talks about this a bit:

             The maximum size of a segment that TCP really sends, the
             "effective send MSS," MUST be the smaller of the send MSS
             (which reflects the available reassembly buffer size at the
             remote host) and the largest size permitted by the IP  
layer:

                Eff.snd.MSS =
                   min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize

            where:

             *    SendMSS is the MSS value received from the remote  
host,
                  or the default 536 if no MSS option is received.

             *    MMS_S is the maximum size for a transport-layer  
message
                  that TCP may send.

             *    TCPhdrsize is the size of the TCP header; this is
                  normally 20, but may be larger if TCP options are  
to be
                  sent.

             *    IPoptionsize is the size of any IP options that TCP
                  will pass to the IP layer with the current message.

I think the description in RFC 1122 is itself a little misleading.  The
first paragraph says "the smaller of the send MSS and the largest size
permitted by the IP layer", however, the formula provided clearly
shows that the TCP header size must be subtracted from the send
MSS to calculate the effective send MSS.

For example: (ipmtu=1500 - sizeof tcp - sizeof ip) = 1460

  1460+20 - (sizeof(tcp) +sizeof(md5 option)) = 1480 - 40 = 1440

But both RFC 2581 and 2385 disagree with the above - and some
comments from Thomas Narten and Bob Braden many moons ago
found in the thread here:

http://tcp-impl.grc.nasa.gov/list/archive/1961.html

"Likewise, a receiver has no way of knowing what options its peer will
be sending (i.e., they could vary on a segment-by-segment basis), so
it should not consider options when calculating the MSS value it
advertises." -- Thomas Narten"

After passing this question by Thomas Narten, he suggested he
today might reword that to something like this (with which I agree):

"A TCP receiver (at the TCP layer) needs to advertise how large a
segment it is able to accept. A peer TCP MUST NOT send a segment
(data + header + options) larger than this, because the receiver
presumably doesn't have enough buffer space to actually hold the
received packet. That would be very bad..."

However, more from Thomas and the previously mentioned thread:

"But, my read of the thread suggests its not 100% clear what the
advertised MSS should contain. e.g,
http://tcp-impl.grc.nasa.gov/list/archive/1962.html says:

> So the recommended answer is: 65532 - 20 - 20 bytes should be
> sent in an MSS option.
>

implying the MSS is not the segment size, but the segment size
excluding the fixed TCP header. (That would be a fine definition too,
but it is (obviously) critically important that everyone agree on what
the definition is.)"

---
Another reference point is from RFC2525, ``Known TCP
Implementation Problems'', where problem 2.18 ("Options missing
from TCP MSS calculation") says this:

       When a TCP determines how much data to send per packet, it
       calculates a segment size based on the MTU of the path.  It must
       then subtract from that MTU the size of the IP and TCP headers in
       the packet.  If IP options and TCP options are not taken into
       account correctly in this calculation, the resulting segment size
       may be too large.

But actual calculation of the Eff.snd.mss isn't discussed.

So, with that, comments, clues and pointers welcome...  Thanks!

-danny

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Fri, 16 Jun 2006 11:13:19 -0700

> should be contained in an advertised MSS wrt TCP Options?
...
> http://tools.ietf.org/wg/tcpm/draft-touch-tcpm-tcp-simple-auth-00.txt

That's correct - no mention. The reason is that there is no reason to
treat this option uniquely from other options regarding MSS. I agree it
would be useful to remind readers of that.

I don't agree that RFC 3285 overrides RFC 1122 on this point - it too
just presents a reminder. RFC 2581 presents the MSS values differently,
and I haven't verified if it overrides RFC 1122 on this point. 1122,
IMO, states:

- Eff.snd.mss is the size of the DATA that can be sent in a TCP segment
	so needs to subtract the headers and their options

- the advertised MSS is the size of the entire packet that can be
received and parsed by the receiver, and so includes all headers).

Its equation:

Eff.snd.MSS = min(SendMSS-20, MMS_S) - TCPhdrsize - IPoptionsize

is equivalent to saying:

Eff.snd.MSS = min(SendMSS, MMS_S+20) - TCPhdrsize - IPhdrsize

where IPhrdsize = IP header (i.e., 20) and IP options

SendMSS thus expresses the space used to receive the entire packet into,
and includes headers, and it'the other side's MMS_R

MMS_S is the largest IP payload assuming a fixed header of 20 bytes -
that's why the 20 is added before subtracting out space for the options
and other headers.

----

this seems to me to be why there's a difference between the 'send once'
of MSS_R during connection setup (the buffer for the segment doensn't
change size) vs. the per-segment calculation of eff.snd.MSS (it
determines how much data goes in THIS segment).

Joe

---

From: Danny McPherson <[email-masked]>
Date: Fri, 16 Jun 2006 17:37:18 -0600

Thanks for the response Joe, comments inline...

- -danny

On Jun 16, 2006, at 12:13 PM, Joe Touch wrote:
> just presents a reminder.

A reminder from what?  RFC 1122 is unclear about how to derive the
"advertised MSS" value, and the only other relevant text would be
RFC 879 before that, which states:

              The MSS counts only data octets in the segment, it does  
not
              count the TCP header or the IP header.

and even it says nothing about what to do with TCP/IP options.

Section 4.3 of RFC 2385 states:

    As with other options that are added to every segment, the size of
    the MD5 option must be factored into the MSS offered to the other
    side during connection negotiation.

RFC 2385 specifies that the TCP MD5 Signature Option is factored
into the advertised MSS value - "as with other options that are added"
- - what options and where is the formula for this defined?

> RFC 2581 presents the MSS values differently,

In particular, Section 2 definition of RMSS:

    RECEIVER MAXIMUM SEGMENT SIZE (RMSS):  The RMSS
    is the size of the largest segment the receiver is willing to  
accept.
    This is the value specified in the MSS option sent by the receiver
    during connection startup.  Or, if the MSS option is not used, 536
    bytes [Bra89].  The size does not include the TCP/IP headers
    and options.

RFC 2581 says TCP/IP header sizes and options are NOT in the
MSS value advertised to the peer.

> 	so needs to subtract the headers and their options

Yes, I agree with this, but Eff.snd.MSS is not my concern IF we're
consistent with how to calculate the advertised MSS.  Because we're
not some implementations factor option sizes while others aren't, and
that's my concern.

> received and parsed by the receiver, and so includes all headers).

But where did you obtain this specific equation from as it relates
to calculating "advertised MSS"?

> Eff.snd.MSS = min(SendMSS-20, MMS_S) - TCPhdrsize - IPoptionsize

And assuming this was taken from RFC 1122, it's actually:

  Eff.snd.MSS = min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize

However, this is talking about the Eff.snd.MSS, not the "Advertised  
MSS",
which are very different.  RFC 1122 doesn't clearly define how to
calculate the "advertised MSS", which is where my concern arises.  I was
looking for a definitive answer or citation about which behavior is  
correct.

> how much data goes in THIS segment).

But the problem that arises is this...

One host interprets it as RFC 2385 specifies where it assumes the  
size of
TCP/IP options were factored into the MSS received from the peer.  As  
such,
it never subtracts those option values when calculating the Eff.snd.MSS
for that peer.

The other host assumes that the remote host will factor the TCP/IP  
options
values when calculating it's Eff.snd.MSS.  As such, when options are
employed there's a possibly that segments received from the remote host
will exceed it's MMS_R.

We're seeing this in a real deployment on two popular stacks, so  
there is
clearly confusion.

I was looking for a pointer that defines which behavior is correct for
calculating the advertised MSS value.

Thanks again for the response!

- -danny

---

From: Joe Touch <[email-masked]>
Date: Fri, 16 Jun 2006 17:01:15 -0700

Danny McPherson wrote:
> A reminder from what?

That this option, like other options, affects MSS calculations. Failure
to take that into account has consequences.

...
>    is the size of the largest segment the receiver is willing to accept.

What it should be reporting is the size of the buffer into which it can
accept a segment.  2581 implies that data and headers are in different
locations, and that only data has limited space. That seems an oversight
to me.

> not some implementations factor option sizes while others aren't, and

Understood.

>  Eff.snd.MSS = min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize

Yes.

> However, this is talking about the Eff.snd.MSS, not the "Advertised MSS",

I agree; this is about the send side decision, which includes the headers.

> One host interprets it as RFC 2385 specifies where it assumes the size of
> TCP/IP options were factored into the MSS received from the peer.  As such,
> The other host assumes that the remote host will factor the TCP/IP options
> We're seeing this in a real deployment on two popular stacks, so there is
> clearly confusion.

This appears to be caused by 2385, but may be a result of assuming that
data and headers are in separate areas.

Joe

---

From: Danny McPherson <[email-masked]>
Date: Mon, 19 Jun 2006 14:23:55 -0600

On Jun 19, 2006, at 2:16 PM, Kevin Lahey wrote:

> On Fri, 16 Jun 2006 11:10:02 -0600
> Danny McPherson <[email-masked]> wrote:
>
> Section 2.18 of RFC2525 discusses exactly this issue, not that I'd
> call that a definitive answer.[*]

Yep, I think I cited this in an email in an earlier thread.

> If we're to take TCP options into account when calculating the MSS
> option, what do we do about negotiated options like timestamps?   
> Should
> we not take them into account when sending the first SYN, but take  
> them
> into account when acknowledging a SYN (when we know that both sides
> support them)? It seems that way lies madness.

I agree, hence my opinion is that options should NOT be included in
the advertised MSS and should always be calculated per-segment
when sending (Eff.snd.MSS), and both sending and receiving TCP
MUST be aware of this.

Perhaps an I-D specifying this behavior more explicitly is in order?

-danny

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Jun 2006 13:54:16 -0700

Danny McPherson wrote:
>> option, what do we do about negotiated options like timestamps?  Should
>> we not take them into account when sending the first SYN, but take them
> the advertised MSS

IMO, the receiver should state what it's capable of receiving; this
means the limit of its space is explicit, and let the sender decide how
to make each transmitted segment fit...

> and should always be calculated per-segment
> when sending (Eff.snd.MSS), 

Agreed.

> and both sending and receiving TCP
> MUST be aware of this.

Agreed, though I come to an overall different conclusion:

	- advertised MSS should include all options and headers
	- it is the sender's responsibility to fit what it sends
	to that advertised limit

That fits the 'be conservative in what you send, liberal in what you
receive' rule. Liberal receiver = don't pick what's header and what's
body, just state the buffer limit. Conservative sender = your job to fit
to the receiver's limits.

The receiver should, of course, truncate received segments at its
advertised MSS to avoid buffer overruns.

> Perhaps an I-D specifying this behavior more explicitly is in order?

I'm not sure this needs a stand-alone I-D, or if we should collect such
mods in a pending list somewhere and publish them when over some
time/length threshold.

FWIW, this was also revisited on the E2E list in 2005:
http://mailman.postel.org/pipermail/end2end-interest/2005-April/004920.html

Joe

---

From: Danny McPherson <[email-masked]>
Date: Mon, 19 Jun 2006 15:07:02 -0600

> to make each transmitted segment fit...

I'm fine with that as well, though with one slight concern,
see below.

> 	- it is the sender's responsibility to fit what it sends

Hrmm, so I guess at no point would the sender ever add an
option to a segment that it wasn't aware of at connection setup
time then, is that correct?

> to the receiver's limits.

Sure, though I think I could spin this to accommodate either
model.

> advertised MSS to avoid buffer overruns.

Agreed, though with mismatched MTUs or the like it may lead
to silent drops at some hop between the sender and receiver,
no?  Also, is it difficult to DoS harden such a model, or is this
common practice today?

> mods in a pending list somewhere and publish them when over some

I'm happy with that, I just want it definitively documented and
referencable some place.  Where would such a collection of mods
be contained today?

> http://mailman.postel.org/pipermail/end2end-interest/2005-April/ 

Completely missed this thread when looking before asking, thanks for
the pointer.

- -danny

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Jun 2006 14:47:19 -0700

> time then, is that correct?

I don't see why that's a conclusion from the above. If the sender makes
the decision of what options to send on a per-segment basis, it must fit
to the advertised limit on a per-segment basis.

The limit of what the receiver can buffer is the only thing here that
doesn't change after connection setup; I don't see why it should, and it
would be hard to coordinate a change if it did.

...
> no? 

They already do (PMTU failures).

> Also, is it difficult to DoS harden such a model, or is this
> common practice today?

I'm not sure what you mean - can you explain?

> referencable some place.  Where would such a collection of mods

RFC 2525 is the closest thing, but it focused on implementation issues,
not spec issues.

I'm proposing that TCPM might keep a list of such mods...

Joe

---

From: David Borman <[email-masked]>
Date: Mon, 19 Jun 2006 18:13:41 -0500

*Sigh*.  This topic keeps coming up, and it's really not that  
difficult.  The MSS value that is put into the TCP MSS option does  
*not* include space for either TCP or IP options.  You only adjust  
for the fixed IP and TPC headers.  When sending a packet, you  
calculate Eff.snd.MSS, and in that calculation you do subtract off  
for the TCP and IP options that you know are going into the packet!

			-David Borman

Here's my message from the last time this came up:

> Subject: 	Re: [e2e] Question on MTU
> Date: 	April 21, 2005 5:13:22 PM CDT
> To: 	  [email-masked]
> Cc: 	  [email-masked]
>
> Joe,
>
> The "effective send MSS" takes into account options, but the MSS value
> put into the TCP MSS option should not.  In section 4.2.2.6 of RFC
> 1122:
>              The MSS value to be sent in an MSS option must be less  
> than
>              or equal to:
>
>                 MMS_R - 20
>
>              where MMS_R is the maximum size for a transport-layer
>              message that can be received (and reassembled).  TCP  
> obtains
>              MMS_R and MMS_S from the IP layer; see the generic call
>              GET_MAXSIZES in Section 3.4.
>
> And in section 3.3.2:
>
>           There MUST be a mechanism by which the transport layer can
>           learn MMS_R, the maximum message size that can be  
> received and
>           reassembled in an IP datagram (see GET_MAXSIZES calls in
>           Section 3.4).  If EMTU_R is not indefinite, then the  
> value of
>           MMS_R is given by:
>
>              MMS_R = EMTU_R - 20
>
>           since 20 is the minimum size of an IP header.
>
> The receiver can't reliably predict what IP or TCP options the sender
> is going to put into the packets, so it doesn't include them in the  
> MSS
> option.  The sender then does take those options into account when
> calculating the "effective send MSS", because it knows exactly what
> options are going into the packet.
>
>                         -David Borman
...

---

From: Danny McPherson <[email-masked]>
Date: Mon, 19 Jun 2006 17:43:35 -0600

On Jun 19, 2006, at 5:13 PM, David Borman wrote:

> difficult.

I agree, I've now seen three archived instances of this precise
discussion, yet no authoritative source for implementers, and
I've seen problems as a result in a live deployment.

>   The MSS value that is put into the TCP MSS option does *not*
> include space for either TCP or IP options.  You only adjust for the
> fixed IP and TPC headers.  When sending a packet, you calculate
> Eff.snd.MSS, and in that calculation you do subtract off for the TCP
> and IP options that you know are going into the packet!

So this is what my opinion of what should occur is, but I'm also
OK with what Joe recommended, so long as folks agree and it's
documented as such.

The problem is that an ambiguity exists, albeit one revisited in
various forums over many years, and it should be resolved as to
avoid future implementation problems.

So, your recommendations about how to best avoid revisiting
this topic?

-danny

---

From: Ethan Blanton <[email-masked]>
Date: Mon, 19 Jun 2006 19:59:27 -0400

> OK with what Joe recommended, so long as folks agree and it's

In my opinion, this is the only reasonable way to define it.  When a
sender puts the MSS option on a SYN packet, it is indicating to the
*receiver* the maximum size segment it can accept -- and it has no
idea what options the *receiver* may wish to put on the TCP header.
For this reason, subtracting the options the sender expects to put on
its own TCP headers is useless at best -- particularly because, at SYN
time, the sender doesn't even necessarily know what options it will
wind up negotiating!

> So, your recommendations about how to best avoid revisiting

I'm not sure how the question comes up in the first place; it seems
clear enough to me that there's really only One Way to Do It.

Ethan

-- 
The laws that forbid the carrying of arms are laws [that have no remedy
for evils].  They disarm only those who are neither inclined nor
determined to commit crimes.
		-- Cesare Beccaria, "On Crimes and Punishments", 1764
_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Danny McPherson <[email-masked]>
Date: Mon, 19 Jun 2006 18:02:40 -0600

> would be hard to coordinate a change if it did.

Like I said, so long as the two implementations agree we're fine.
However, currently an ambiguity exists where implementations don't
agree on how to calculate the advertised MSS (and subsequently,
Eff.snd.MSS) and this presents an issue.

With deployed implementations we also need to be concerned
with backwards compatibility and therefore assuming the remote
TCP will calculate options into Eff.snd.MSS in addition to the
received MSS value may be a stretch.

Finally, while I agree that if "advertised MSS" was used to convey
absolute TCP reassembly buffer space it wouldn't be an issue, but
there are implementations out there today in which this clearly is
not the case.  Per segment options should be considered in
whatever text we generate.

> They already do (PMTU failures).

If the sender and receiver don't agree on semantics of this then
problems can occur regardless of PMTU, as I mentioned in the first
message I sent about this.

> I'm not sure what you mean - can you explain?

WRT your comment "The receiver should, of course, truncate
received segments at its advertised MSS to avoid buffer
overruns" I was simply asking if by "truncate" you're suggesting
that if a TCP segment exceeds  available reassembly buffer
space the receiving system should do what to correct/indicate
the error?  Is this a common practice in implementations today?

> RFC 2525 is the closest thing, but it focused on implementation  

And it's "done", which means we can't really do anything new
with it..

> I'm proposing that TCPM might keep a list of such mods...

I thought in the IETF Internet-Drafts for the correct means to
convey such a thing - is there a better mechanism with which
I'm unaware?  As per David's email, I'd prefer not to revisit this
again in n months and would very much like the two
implementations with which I'm most concerned at the moment
(per a real problem because of an ambiguity in the specs)
to have some referencable point at which future specifications
and implementation may comply.

Thanks for your response...

- -danny

---

From: Danny McPherson <[email-masked]>
Date: Mon, 19 Jun 2006 18:08:06 -0600

> clear enough to me that there's really only One Way to Do It.

Heh..

With my indulgence into the issue over the past few weeks, I'm not
terribly surprised that I could name three popular implementations
that do it one way and two that do it another.

As such, what may seem intuitive enough to one person may seem
entirely different or just implementation issues to another.  In  
addition,
the fact that there are several RFCs that muddy the water further by
suggestive or misleading text at best, I'm not at all surprised such an
issue continues to arise.

- -danny

---

From: David Borman <[email-masked]>
Date: Mon, 19 Jun 2006 21:22:46 -0500

Danny,

On Jun 19, 2006, at 7:02 PM, Danny McPherson wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>

How to calculate Eff.snd.MSS is very clear in RFC 1122.  On page 85,  
it says:

                Eff.snd.MSS =

The "SendMSS" value is the value received in the TCP MSS option.  It  
has been reduced by the fixed IP and TCP header sizes, i.e. 40  
bytes.  In the above equation, TCPhdrsize is the full size of the TCP  
header including any options.  So the "+20" puts back the length of  
the fixed TCP header so that it isn't subtracted out twice.  The  
above equation could also be:
		Eff.snd.MSS =
		    min(SendMSS, MMS_S - 20) - TCPoptionsize - IPoptionsize
which results in the same answer.

When calculating Eff.snd.MSS, you have to account for both the TCP  
options and the IP options.  You don't get to assume that the value  
received in the TCP MSS option has already been adjusted for them.

> received MSS value may be a stretch.

It is not a stretch, it is a requirement of RFC 1122.  Any TCP  
implementation that does not take into account both IP and TCP  
options when calculating Eff.snd.MSS is broken.  RFC 1122 has been  
out since 1989.

If the bottom line goal in interoperability, then the only way you  
get into trouble in this area is if a TCP implementation assumes that  
the received TCP MSS option has been adjusted for options and it  
hasn't.  In this case, packets that are too large will be generated  
and won't get through.

Conversely, if a broken TCP has adjusted down the value in the TCP  
MSS option that it sends to account for TCP options, and the other  
side also correctly adjusts down the Eff.snd.MSS to account for TCP  
options, then the packets that get sent will be smaller than what  
could really be sent, but they will still go through and  
interoperability will succeed.

So, the only way to guarantee interoperability in the packets you  
send is to always assume that the TCP MSS option only reflects the  
fixed IP & TCP headers, and you have to take into account both IP and  
TCP options when calculating Eff.snd.MSS, just as required by RFC 1122.

I've also re-read section 2.18 of RFC 2525, and I don't see anything  
in there that contradicts RFC 1122 w.r.t how Eff.snd.MSS is  
calculated, or how to calculate the value to put into the TCP MSS  
option.

			-David Borman

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 20 Jun 2006 13:02:46 +0800

> clear enough to me that there's really only One Way to Do It.

Maybe you will find the following interesting...

http://blogs.sun.com/roller/page/kcpoon/20050614

-- 

						K. Poon.
						[email-masked]

---

From: David Borman <[email-masked]>
Date: Tue, 20 Jun 2006 09:18:49 -0500

Interesting, yes.  But I don't agree with it.  The basic idea is "If  
I receive an MSS value of X in a SYN packet, I will not send an MSS  
value greater than X in my SYN,ACK".  But the whole thing is premised  
upon a broken middle box that is manipulating down the MSS value in  
the SYN packet (due to its insertion of PPPoE headers) and not  
modifying the MSS in the returning SYN,ACK packet, and also not  
generating ICMP Fragmentation Needed messages.

The middle box is broken.  While putting in short term fixes in the  
hosts helps the hosts to work around the broken middle boxes, in the  
long run that is a disservice to the network as a whole, as it  
removes pressure on the vendor of the broken middle box to fix the  
middle box.

The introduction of broken systems into the network should glaringly  
point out the problems of the broken systems, so that they will get  
noticed and fixed as soon as possible.  The network as a whole  
shouldn't have to work around every piece of broken hardware that is  
added into the network.

			-David Borman

---

From: Joe Touch <[email-masked]>
Date: Tue, 20 Jun 2006 08:19:11 -0700

David Borman wrote:
> The middle box is broken.
...

Or what the middle box wants do to is "broken" ;-)

> noticed and fixed as soon as possible.  The network as a whole shouldn't
> have to work around every piece of broken hardware that is added into
> the network.

Indeed.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 20 Jun 2006 08:20:44 -0700

Danny McPherson wrote:
>> *Sigh*.  This topic keeps coming up, and it's really not that difficult.
> 

I'm just reiterating David's summary from the E2E list. I agree with
him; this issue isn't one. Like many protocol issues, it's resolved
completely if you dig into the docs sufficiently - I see no reason why
this issue is more obscure than any other.

What you really have is a _known_ implementation error that is already
described by RFC2525. What further needs to be said?

Joe

---

From: Danny McPherson <[email-masked]>
Date: Tue, 20 Jun 2006 11:12:09 -0600

- -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> this issue is more obscure than any other.

Really?  This is surprising considering you and David disagree on
what the right solution is (i.e., David says calculations should be
per segment via Eff.snd.MSS and therefore advertised MSS should
NOT include options, which is what I believe should occur as well;
you say that advertised MSS should include options and therefore
the Eff.snd.MSS should assume options were included in
Eff.snd.MSS).

If this is so obvious, why do you disagree with David?

> described by RFC2525. What further needs to be said?

RFC 2525 does NOT define how to calculate advertised MSS.  Section
2.18's title and focus of this item in the document are even a bit  
misleading,
considering it's actually talking about "Options missing from  
Effective send
MSS calculation" when that's the piece that's clear in RFC 1122, it's  
the
calculation for deriving advertised MSS that introduces this ambiguity,
and "Options missing from TCP MSS calculation" is simply a symptom
of that.

I'd be quite content had it stated, in the "How to fix" section, that  
the
*advertised MSS* calculation MUST NOT factor options, as they're to be
calculated on a per segment basis by the sender - as this is really  
where
the issue arises.

Eff.snd.MSS is seemingly sufficiently defined in RFC 1122, but as such,
it implicitly requires that a sender NOT include options in the  
advertised
MSS.  RFC 2581 introduces some new definitions to muddy the water
more.  RFC 2385 completely diverges from this stating "As with other
options that are added to every segment, the size of the MD5 option
must be factored into the MSS offered to the other side during
connection negotiation."

David again says "Sigh."  Then provides a mailing list reference and
an Appendix A pointer from RFC 1323bis (a 3 year old expired I-D).

I'm confused as to why the two of you are adamant that this isn't an
issue when you yourselves disagree, there's apparently no explicit
definitive source about calculation of advertised MSS, we're seeing
a problem from this in deployments in the field, many of the options
drafts themselves are clearly contradictory and the only references
you provide are an ID that expired 3 years ago and mailing list
archives and suggest that's sufficient?

In David's words, sigh.  If there were something more clear and
authoritative the issue wouldn't keep resurfacing.

- - -danny

- -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEmCxgVbIg27AGOxoRAsLMAJ0T9/O/zJnj0cFrEJbxMNQoGCxq6ACfQiu+
ARNP+NTIzgXY1ehMndz45MA=
=IpZT
- -----END PGP SIGNATURE-----

---

From: Joe Touch <[email-masked]>
Date: Tue, 20 Jun 2006 10:34:10 -0700

> the Eff.snd.MSS should assume options were included in

I said that:

I also said later in that same message that:

This was intended to indicate that the advertised MSS includes the space
used for all options and headers - which is why the advertiser should
NOT subtract that stuff out, but the user of that info (the side sending
segments based on that advertised MSS) would be doing the fitting - as
the next line states:

> If this is so obvious, why do you disagree with David?

I don't think we disagree at all.

> considering it's actually talking about "Options missing from Effective
> send
> MSS calculation" when that's the piece that's clear in RFC 1122, it's the
> I'd be quite content had it stated, in the "How to fix" section, that the
> calculated on a per segment basis by the sender - as this is really where
> the issue arises.

RFC2525 wasn't intended to imply that there were errors in specification
regarding section 2.18; the error is one of implementation.

> it implicitly requires that a sender NOT include options in the advertised

So far, you're highlighting a possible error in 2385. That can be fixed
sufficiently in any of the proposed revisions by addressing that section
directly - and saying that "as with other options, this is NOT
subtracted out of the advertised MSS". I.e., 2385 has the right premise
(as with other options), but the wrong recommendation.

2581 defines its own terms, which has its own problems.

> authoritative the issue wouldn't keep resurfacing.

That's an assumption I do not agree with; we see the same issues
resurfacing all over the place. More documents don't always help the
situation.

Joe

---

From: David Borman <[email-masked]>
Date: Tue, 20 Jun 2006 12:50:07 -0500

> 	- advertised MSS should include all options and headers

I think this is what is causing confusion in the current discussion.   
The TCP MSS option is the total amount of available data, less the  
fixed IP and TCP headers; i.e. the amount of TCP payload that can be  
received if you have no IP or TCP options (or any other headers).

> 	- it is the sender's responsibility to fit what it sends

Yes, but with the caveat that the value that was received in the TCP  
MSS option has already been adjusted down to reflect the size of the  
fixed IP and TCP headers, and that has to be taken into account when  
calculating Eff.snd.MSS, as described in RFC 1122.

			-David Borman

---

From: Danny McPherson <[email-masked]>
Date: Tue, 20 Jun 2006 12:36:43 -0600

On Jun 20, 2006, at 11:50 AM, David Borman wrote:

> I think this is what is causing confusion in the current  
> discussion.  The
> TCP MSS option is the total amount of available data, less the  
> fixed IP
> and TCP headers; i.e. the amount of TCP payload that can be received
> if you have no IP or TCP options (or any other headers).

Yes, I believe this is where the confusion arises as well.

> calculating Eff.snd.MSS, as described in RFC 1122.

Yes, I agree.  And with the caveat that the IP and TCP Options
were not calculated into the advertised MSS conveyed by the remote
host, and therefore, should be considered as such when calculating
Eff.snd.MSS.  This is what isn't explicit in the current text.

-danny

---

From: David Borman <[email-masked]>
Date: Wed, 21 Jun 2006 09:28:29 -0500

On Jun 20, 2006, at 1:36 PM, Danny McPherson wrote:
...
> Eff.snd.MSS.  This is what isn't explicit in the current text.

Huh?  Section 4.2.2.6 of RFC 1122 is very explicit about how to  
calculate Eff.snd.MSS, there is no ambiguity there.  You adjust down  
to account for any IP and TCP options.

The value to put into the TCP MSS option is:
	MMS_R - 20 (section 4.2.2.6)
	MMS_R = EMTU_R - 20 (section 3.3.2)
So, that's EMTU_R - 40; 20 bytes for the fixed TCP header, 20 bytes  
for the fixed IP header.

And that isn't explicit enough?

			-David Borman

---

From: Joe Touch <[email-masked]>
Date: Wed, 21 Jun 2006 08:26:09 -0700

David Borman wrote:
> calculate Eff.snd.MSS, there is no ambiguity there.  You adjust down to
> account for any IP and TCP options.
> 
> So, that's EMTU_R - 40; 20 bytes for the fixed TCP header, 20 bytes for
> the fixed IP header.
> 

Right - it doesn't matter whether the other side erroneously reduces
advertised MSS by the size of options it expects; if this side
implements 1122 as required, the transfer might be a little less
efficient but it won't break.

Joe

---

From: Danny McPherson <[email-masked]>
Date: Wed, 21 Jun 2006 10:41:51 -0600

On Jun 21, 2006, at 8:28 AM, David Borman wrote:
> calculate
> Eff.snd.MSS, there is no ambiguity there.  You adjust down to  
> account for
> any IP and TCP options.
>
> for

I feel like we're going in circles here, perhaps my wording was a
bit loose.  What I meant was that the confusion surrounding this
whole issue is simply because:

o there are implementations that calculate options into
    the advertised MSS (because of inconsistencies that
    documents like RFC 2385 introduce)

o and implicitly, as a result, implementations that expect the
    remote host to include options in the MSS it advertised

o and therefore, said implementation does not comply with
    RFC 1122s Eff.snd.MSS algorithm by factoring options on
    a per-segment basis when sending

o and if the two TCPs don't agree on this (which is what we're
    experiencing) then there's some probability you run into
    brokenness.

If the two TCPs agree on Eff.snd.MSS calculation but not the
advertised MSS calculation then you likely just get an inefficiency
where segments could be a bit larger - but this is still an issue.

However, what we've seen is that implementations which presume
options were included in the MSS advertised from the remote
system tend to not (re)calculate options on a per segment basis
when sending, which is perhaps something that seems intuitive
to an implementer.

My position is that because Eff.snd.MSS says you calculate
options when sending on a per segment basis then options
MUST NOT be conveyed in the advertised MSS - but such an
implicit requirement in RFC 1122 for advertised MSS calculations
and inconsistencies introduced by RFC 2385 and the like, the
issue keeps resurfacing.

The fact that even you and Joe seem to disagree on this is
indicative of a problem, the fact that it keeps resurfacing indicates
there's at least some ambiguity, and the fact that I'm seeing three
widely deployed implementations that do it one way and two that
do it another, well, is an existence proof.

-danny

---

From: Joe Touch <[email-masked]>
Date: Wed, 21 Jun 2006 10:08:33 -0700

Danny McPherson wrote:
>> Huh?  Section 4.2.2.6 of RFC 1122 is very explicit about how to calculate
>> Eff.snd.MSS, there is no ambiguity there.  You adjust down to account for
>> any IP and TCP options.
>>

This was addressed in RFC2525 - which came 8 months later. 2385 is
already somewhat in flux; we can address than in one of the updates
(i.e., correct the erroneous text).

The rest seems to be based solely on 2385. Let's just fix that (which we
have three proposals for anyway)...

Joe