Subject: Re: [tcpm] poll for consensus on Middlebox Discover draft as a TCPM WG item

From: "Mahdavi, Jamshid" <[email-masked]>
Date: Tue, 22 Dec 2009 15:48:53 -0800

WARNING: contains banned part

Subject: Re: poll for consensus on Middlebox Discover draft as a TCPM WG item
From: "Mahdavi, Jamshid" <[email-masked]>
To: <[email-masked]>
Date: Tue, 22 Dec 2009 15:48:53 -0800

I'd just like to echo some of Andrew's points.  We have a group of authors with a history of working with the IETF directly or with IETF standards extensively, and when we came together to start working on this we came to it with the mindset of wanting to do the right thing.

I think we have heard some very supportive statements from some people on the list.  In my interpretation, we have heard a couple of key things:

- There is a need for an in-band discovery mechanism.  Several people have said this, and we also know from practical experience that there is such a need in our products and in the products of other people working in this space.

- We don't want people camping out on unallocated option numbers for this.

We have also heard a couple of key objections:

- The use of this option will consume too much space in the TCP header.  With regards to this point, my observation would be that options are, by definition, "optional".  Especially in the case of middleboxes, it is up to those devices to decide what would be the best set of options to choose for their particular application.  In our experience, space has not been a problem, and if it becomes one we would have to make such trade-offs.

- I believe there may be some question as to whether vendors would adopt this if a standard were created.  We can only speak for ourselves of course, but we would certainly intend to adopt it.  Speaking in general terms, customers like it when vendors follow standards, so I can't see why other vendors would not also follow suit if a standard is provided.

- Finally, I think there was also some question early on about the actual usefulness of the mechanism we've proposed.  If we take this on as a WG item this is something we'd be happy to get back to discussing.

I might be putting some words into Joe's mouth here, but it sounded like you would be ok with adopting the general problem of middlebox discovery as a WG item, but not necessarily starting from the solution that we have proposed.  I think that if we actually need to go back to starting at square one we could do that, although you'll understand why we'd prefer not to.

It sounds like in either event we should probably plan for some type of in-person discussions at the next IETF.  The March IETF will be pretty convenient for us and I think we could commit to some of us being present to take the next steps.

--J

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Dec 2009 16:22:18 -0800

Mahdavi, Jamshid wrote:
...
> I might be putting some words into Joe's mouth here, but it sounded
> like you would be ok with adopting the general problem of middlebox
> discovery as a WG item, but not necessarily starting from the solution
> that we have proposed. I think that if we actually need to go back to
> starting at square one we could do that, although you'll understand why
> we'd prefer not to.

I want to start with understanding the problem and why it has to be
solved with TCP options in devices that aren't supposed to be modifying
a TCP header (as opposed to IP options in those packets).

If this were to move forward, we'd need to understand how many current
Internet standards are updated by allowing intermediate devices to
modify TCP headers.

I don't see a huge problem if these issues are addressed using the
current solution, e.g., moving forward as suggested by Wes. However, if
these cannot be sufficiently addressed, I hope we can accept a solution
that is compatible with the architecture (whether modified or completely
different - I'm not suggesting I know which is the case, though) - not
just the one that happens to be deployed.

A key question is whether this qualifies as a minor modification of TCP,
or whether this ought to be in the TSVWG. IMO, adding an option is
minor, but changing the standards to allow middleboxes to modify options
is major.

Joe

---

From: "Mahdavi, Jamshid" <[email-masked]>
Date: Tue, 22 Dec 2009 16:44:30 -0800

As it happens, we don't modify options at all.  What our boxes do is terminate one TCP connection (essentially masquerading as the server), and then create a new connection to the server (with whatever new options we choose) and optionally masquerade as the client on that connection (commonly known as ip-spoofing).

Not all middleboxes work this way, of course.  NATs are an obvious example where connections are not terminated.  Boxes like that may (I believe) modify options, perhaps by removing ones they don't know about.  We see this sort of thing from time to time.  Even outside of what we are doing for discovery, we also rely on options for things like TCP large windows support so it is something we watch for.

So, in my view, our draft does not propose to either allow or disallow "modification" of TCP options.  Instead, we define a new option which may be used by devices that originate connections (either clients or proxies), or by devices which are already in the business of twiddling with existing connections.

--J

-----Original Message-----
From: Joe Touch [mailto:[email-masked]]
Sent: Tue 12/22/2009 4:22 PM
To: Mahdavi, Jamshid
Cc: [email-masked]
Subject: Re: [tcpm] poll for consensus on Middlebox Discover draft as a TCPM WG item

Mahdavi, Jamshid wrote:
...

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Dec 2009 16:52:33 -0800

Mahdavi, Jamshid wrote:
> As it happens, we don't modify options at all. What our boxes do is
> terminate one TCP connection (essentially masquerading as the server),
> and then create a new connection to the server (with whatever new
> options we choose) and optionally masquerade as the client on that
> connection (commonly known as ip-spoofing).

Masquerading is not standards track. You're not only asking for an
experimental RFC, you're asking us to allow masquerading.

...
> So, in my view, our draft does not propose to either allow or 
> disallow "modification" of TCP options. Instead, we define a new
> option which may be used by devices that originate connections
> (either clients or proxies), or by devices which are already in the
> business of twiddling with existing connections.

You're defining an option that is only useful by a box that violates
existing standards - whether they already do so or not, that causes a
problem AFAICT for TCPM to take this on as a WG item.

Joe

---

From: "Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]" <[email-masked]>
Date: Tue, 22 Dec 2009 21:34:58 -0600

There are already several RFCs that discuss the dangers of various middlebox behaviors like splitting TCP connections and pretending to be other hosts, and discourage those behaviors.  We don't really have the ability to decide whether to "allow" these behaviors or not; they're already widespread despite our expressed concerns and the lack of any formal standards encouraging them.  The situation goes from bad to worse when they start poaching options numbers and other assigned values though, without even leaving a paper trail for the community to be aware of what's going on.  IANA's tables become meaningless if big parts of the community stop paying attention to them and deploy new values without regard to the maintenance (or authority) of the registries.

Though we can't make some of the naughty things middleboxes do go away, I'd think we could at least define this option for them as a sandbox to play within rather than continue to watch them pollute the whole space of options numbers with landmines that will only be painfully discovered in the future.  We'd clearly have the ability to say something like: "Please see references XXX and YYY as to why certain types of middleboxes are short-sighted optimization attempts and harmful to the Internet.  If you choose to ignore that advice and do these things anyways and also decide to use TCP options for some portion of your functionality, as many in-the-wild devices are presently, then please at least do not poach unallocated TCP option numbers which are supposed to be under IANA's control.  Please use option number Z and the accompanying option format defined in this document, which includes a user-specified portion flexible to your needs."

I'd think that's along the lines of what's being proposed.  It's about (1) raising awareness that poaching assigned values is yet another harmful behavior of middleboxes to be avoided (I'd never heard of it, at least ... ignorance was bliss), and (2) showing the vendors that there's a very easy way they can avoid the problem.

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Joe Touch [[email-masked]]
Sent: Tuesday, December 22, 2009 7:52 PM
To: Mahdavi, Jamshid
Cc: [email-masked]
Subject: Re: [tcpm] poll for consensus on Middlebox Discover draft as a TCPM WG item

...

Joe

> minor, but changing the standards to allow middleboxes to modify options
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Dec 2009 22:56:03 -0800

Hi, Wes,

You make a convincing argument.

Since they already aren't following the standards, I see no reason to
limit them to the existing 0-255 range. I propose option #256 for these
mechanisms.

Joe

---

From: Lars Eggert <[email-masked]>
Date: Wed, 23 Dec 2009 13:12:56 +0200

Hi,

> Though we can't make some of the naughty things middleboxes do go away, I'd think we could at least define this option for them as a sandbox to play within rather than continue to watch them pollut…

speaking as an individual, this to me sounds like a reasonable document for the WG to produce.

Lars

---

From: "Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]" <[email-masked]>
Date: Wed, 23 Dec 2009 14:57:39 -0600

I wasn't suggesting that you put it into your document, I was just
mentioning that I think that some similar context-setting makes
sense to add if there's a working group product on the topic,
because as Joe pointed out, there's otherwise potential for some
confusion.  There are probably much better ways to phrase the actual
words :).

There are two things going on that I was seeing as distinct (though
related):

(1) Typical middlebox behaviors (connection-splitting, etc.)
(2) TCP option number poaching

I think we would be making a statement about (2), and leaving
advise on (1) to the several existing statements?

--
Wes Eddy
MTI Systems

>From: Andrew Knutsen [mailto:[email-masked]]
>Sent: Wednesday, December 23, 2009 3:14 PM
>To: Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]
>Cc: Joe Touch; Mahdavi, Jamshid; [email-masked]
>    Are you suggesting we add the text you mention to the draft?  I
>think we could come up with some similar (if less biased ;-) text.
>However we need the values of XXX and YYY, preferably with chapter and
>verse, if its going to be relevant.
>
>    Please don't get us wrong; Blue Coat does believe in
>interoperability (at the appropriate point), and we do have a long-term
>goal of helping bring standards to today's Internet. But we do have to
>start where we're at...
>
>Andrew
>
>	There are already several RFCs that discuss the dangers of various
>middlebox behaviors like splitting TCP connections and pretending to be
>other hosts, and discourage those behaviors.  We don't really have the
>ability to decide whether to "allow" these behaviors or not; they're
>already widespread despite our expressed concerns and the lack of any
>formal standards encouraging them.  The situation goes from bad to worse
>when they start poaching options numbers and other assigned values
>though, without even leaving a paper trail for the community to be aware
>of what's going on.  IANA's tables become meaningless if big parts of
>the community stop paying attention to them and deploy new values
>without regard to the maintenance (or authority) of the registries.
>
>	Though we can't make some of the naughty things middleboxes do go
>away, I'd think we could at least define this option for them as a
>sandbox to play within rather than continue to watch them pollute the
>whole space of options numbers with landmines that will only be
>painfully discovered in the future.  We'd clearly have the ability to
>say something like: "Please see references XXX and YYY as to why certain
>types of middleboxes are short-sighted optimization attempts and harmful
>to the Internet.  If you choose to ignore that advice and do these
>things anyways and also decide to use TCP options for some portion of
>your functionality, as many in-the-wild devices are presently, then
>please at least do not poach unallocated TCP option numbers which are
>supposed to be under IANA's control.  Please use option number Z and the
>accompanying option format defined in this document, which includes a
>user-specified portion flexible to your needs."
>
>	I'd think that's along the lines of what's being proposed.  It's
>about (1) raising awareness that poaching assigned values is yet another
>harmful behavior of middleboxes to be avoided (I'd never heard of it, at
>least ... ignorance was bliss), and (2) showing the vendors that there's
>a very easy way they can avoid the problem.
>
>
>
>	-----BEGIN PGP SIGNED MESSAGE-----
>	Hash: SHA1
>
>
>
>	Mahdavi, Jamshid wrote:
>
>
>	-----BEGIN PGP SIGNATURE-----
>	Version: GnuPG v1.4.9 (MingW32)
>
>	iEYEARECAAYFAksxadEACgkQE5f5cImnZrutFACcD4ZggKX7+wXmRfHDEqXp4mmu
>	NfkAoPD2L9FBOV6RUn4UBrDRm4hxC1di
>	=3xz3
>	-----END PGP SIGNATURE-----

---

From: "Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]" <[email-masked]>
Date: Wed, 23 Dec 2009 15:04:56 -0600

Just to clarify, by "existing statements", I mean documents like
RFC 3234 and 3135:
http://www.rfc-editor.org/rfc/rfc3234.txt
http://www.rfc-editor.org/rfc/rfc3135.txt
which already have positions on the middlebox behaviors.  They
don't say anything about sneaking options away from IANA though.

--
Wes Eddy
MTI Systems

>From: Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]
>Sent: Wednesday, December 23, 2009 3:58 PM
>To: 'Andrew Knutsen'
>Subject: RE: [tcpm] poll for consensus on Middlebox Discover draft as a
>TCPM WG item
>

---

From: "Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]" <[email-masked]>
Date: Wed, 23 Dec 2009 15:31:59 -0600

>From: Joe Touch [mailto:[email-masked]]
>Sent: Wednesday, December 23, 2009 4:10 PM
>
>Nobody "sneaks" things from IANA, AFAICT.

There's an existence proof that several companies have been for years.

>They use numbers that have not been assigned by IANA. IANA lists cases
>it is aware of as "inappropriate use". That should be taken by the
>public as a signal of whose products to avoid, IMO.

Yet IANA has no inherent means to detect inappropriate use, and
no IT department checks IANA's registries as part of their process
of selecting products and vendors.  This is a completely after-
the-fact way to document numbers inappropriately used, but it's
neither a deterrent or a path to friendlier behavior.

If nobody has done so already, I'll at least report the numbers
we've been told are in-use in the wild to IANA so they don't get
allocated into a collision at some future time.  The real problem
remains though, in that next year's product line(s) may use
different numbers, and playing keep-up won't be possible.

--
Wes Eddy
MTI Systems

---

From: Andrew Knutsen <[email-masked]>
Date: Wed, 23 Dec 2009 14:10:25 -0800

Please let us know what we can do in the draft to alleviate your 
concerns.

    Thanks for the RFC references.  The draft already references 3234; I 
can add a similar reference for 3135. A quick reading of the latter 
gives me the impression it takes the opposite side of the argument from 
some parts of 3234 though: it focuses more on how middleboxes are 
useful, rather than potential issues.

    On looking through 3234 again, I noticed section 3 refers to some 
other WGs: MIDCOM,  "a working group with focus on the architectural 
framework and the requirements for the protocol between a requesting 
device and a middlebox and the architectural framework for the interface 
between a middlebox and a policy entity"; and WEBI (Web Intermediaries), 
"a working group that addresses specific issues in the world wide web 
infrastructure (as identified by the WREC working group), by providing 
generic mechanisms which are useful in several application domains 
(e.g., proxies, content delivery surrogates)."

    Perhaps the higher level discussions on how the Internet should 
evolve in the next century (to create a standard middlebox architecture, 
or make them unnecessary) should take place in these groups, if they 
still exist. I can add text to this effect to the draft if desired.

Andrew

Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP] wrote:
>> 			discovery as a WG item, but not necessarily starting
>>     
> >from the solution
>

---

From: Andrew Knutsen <[email-masked]>
Date: Wed, 23 Dec 2009 12:13:48 -0800

Andrew