Subject: [tcpm] WGLC on draft-ietf-tcpm-persist

From: Wesley Eddy <[email-masked]>
Date: Thu, 13 Jan 2011 22:38:36 -0500

Based on the discussion of section 7 (programming considerations), the 
document on clarifying the persist condition has been updated by the 
authors.

The document can be found at:
https://datatracker.ietf.org/doc/draft-ietf-tcpm-persist/

We think this is ready for working group last call now, so with this 
email, I'd like to start a two-week WGLC on the document to last until 
1/28.  Please send your comments to the TCPM list by then.

Thanks in advance.

--
Wes Eddy
MTI Systems

---

From: Wesley Eddy <[email-masked]>
Date: Sun, 30 Jan 2011 23:27:06 -0500

> 1/28. Please send your comments to the TCPM list by then.

Since this last call has concluded, I'm planning to do the PROTO writeup 
and send up the document to the IESG late this week.  Since this 
document is fairly short, includes relatively simple technical material, 
and was well-discussed earlier in its history, I wasn't concerned by the 
lack of WGLC feedback, but will note it in the writeup.

If there's any disagreement on this, or someone wants more time to 
review, please shout in the next couple days, before I submit the 
document to the IESG for publication.

--
Wes Eddy
MTI Systems

---

From: "Agarwal, Anil" <[email-masked]>
Date: Mon, 31 Jan 2011 08:12:17 -0500

Wes,

In https://datatracker.ietf.org/doc/draft-ietf-tcpm-persist, the
"Programming Considerations" section seems to describe two different
techniques for TCP-to-application interaction for persist timeout
handling. 
  a) TCP sends a timeout indication to the application which then 
     closes the connection; 
  b) TCP itself terminates the connection. 

1. The two techniques are written using very different description
styles -
   would be beneficial to use one consistent style.

2. A sentence clarifying that these are two separate suggested
techniques
   would also be useful.

3. Would be useful to clarify that TCP will close the connection and
reclaim
   resources if a close is performed by the application for a connection
in
   "persist" state, even if there is unsent data in the transmit buffer.
   Does it? 
   If it does, approximately how long will it (should it) take before
   the connection is terminated and resources are reclaimed?

Anil Agarwal
ViaSat Inc.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Wesley Eddy
Sent: Sunday, January 30, 2011 11:27 PM
To: [email-masked]
Subject: Re: [tcpm] WGLC on draft-ietf-tcpm-persist

--
Wes Eddy
MTI Systems
_______________________________________________
tcpm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Mon, 31 Jan 2011 08:02:01 -0800

Anil,
     You probably missed the 3+ years of on/off discussion that went on
with this short draft. The wordings are so chosen in a such a way to
honor the RFC 1122 (or 793) spirits that TCP by itself cannot abort the
connection and it is ok for TCP to abort the connection if the
application or system (OS) has indicated so. So, in both cases where you
mention below we simply suggest that as long as the basic premise is
followed (i.e, TCP by itself cannot abort the connection), we are fine
with any method an implementation uses. The programming consideration
section is there to simply illustrate the concept.
    Pl see inline for specific answers :-

> Agarwal, Anil
> Sent: Monday, January 31, 2011 5:12 AM
> To: Wesley Eddy; [email-masked]

The base of these "2 techniques" is that the application has requested
to close the connection if some conditions are met using a an API as
mentioned in the previous paragraph. 

I'
> 2. A sentence clarifying that these are two separate suggested

Well, we really don't want to emphasize a whole lot, there were some
folks who felt that there is no need for having a programming
considerations section at all. Your suggestion seems to be supporting
that and add more text, but we have had so much debate (unfortunately)
on this topic and I would be reluctant to add more text unless there is
a WG consensus. 
>    the connection is terminated and resources are reclaimed?

An application close would be handled the same way as it is today. We
haven't changed any of those semantics. We can add a line if there is a
confusion.

Thanks,
-Anantha

---

From: Wesley Eddy <[email-masked]>
Date: Mon, 31 Jan 2011 11:50:06 -0500

> 2. A sentence clarifying that these are two separate suggested

I think these are good points and it's helpful to have a fresh set of 
eyes on the document.  On looking at that section again, I agree that it 
isn't particularly clear that there are 2 different techniques that the 
API supports.  Since this should be possible to remedy with just a 
couple sentences, I think it should be done.

--
Wes Eddy
MTI Systems

---

From: Mahesh Jethanandani <[email-masked]>
Date: Mon, 31 Jan 2011 09:36:46 -0800

On 1/31/2011 8:50 AM, Wesley Eddy wrote:
> couple sentences, I think it should be 
> done.<https://www.ietf.org/mailman/listinfo/tcpm>
>
Will clarify that there are two suggested techniques possible and send 
out an updated version of the document.

-- 

*Mahesh Jethanandani*
*Technical Leader*
**ERBU*
*
[email-masked] <mailto:[email-masked]>
Phone :*+1 408 527-8230*
Fax :*+1 408 853-0762*

**
170 West Tasman Drive
San Jose, CA 95070
United States
www.cisco.com <http://www.cisco.com>

This e-mail may contain confidential and privileged material for the 
sole use of the intended recipient. Any review, use, distribution or 
disclosure by others is strictly prohibited. If you are not the intended 
recipient (or authorized to receive for the recipient), please contact 
the sender by reply e-mail and delete all copies of this message.

---

From: "Agarwal, Anil" <[email-masked]>
Date: Mon, 31 Jan 2011 15:04:15 -0500

Mahesh, Wes et al,

Few more points -

I don't think an application-initiated close() will terminate a tcp connection in persist state. The RFCs don't mention it and most (all?)
implementations don't do it either.

I have a zombie tcp connection On Linux 2.4.20, in FIN_WAIT state, 
which is sending TCP probes for the past 7 hours,
even though the application has been killed.
I suspended the receiving peer during a tcp transfer, so its tcp
is alive and merrily sending ACKs to tcp probes.
Same results with Linux 2.6.19. Checked the Linux source code too.

Section 7 explicitly refers to "close". The application probably needs
to do an "ABORT"; don't know if OSs support TCP ABORT operation on sockets from userland.

Also, it should be noted that application-initiated close will require changes to every application. I suspect most OS vendors will choose the second solution and implement a system-configurable timeout or retries value and have TCP abort the connection on its own.

Anil

________________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mahesh Jethanandani
Sent: Monday, January 31, 2011 12:37 PM
To: [email-masked]
Subject: Re: [tcpm] WGLC on draft-ietf-tcpm-persist

Will clarify that there are two suggested techniques possible and send out an updated version of the document.
-- 

Mahesh Jethanandani
Technical Leader
ERBU

[email-masked]
Phone :+1 408 527-8230
Fax :+1 408 853-0762

170 West Tasman Drive
San Jose, CA 95070
United States
www.cisco.com

This e-mail may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message.

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Mon, 31 Jan 2011 12:15:16 -0800

> Sent: Monday, January 31, 2011 12:04 PM
> To: Mahesh Jethanandani (mahesh); [email-masked]
> I don't think an application-initiated close() will terminate a tcp
> connection in persist state. The RFCs don't mention it and most (all?)

This is the way the Linux implementation chose to behave. In particular,
if the system runs out of resources, Linux has the algorithm that
targets the orphaned sockets, it would abort these TCP connections stuck
in FINWAIT1/2 state. Pl take a look at tcp_out_of_resources() function
in tcp_timer.c. 

-Anantha

<snip>

---

From: John Heffner <[email-masked]>
Date: Mon, 31 Jan 2011 18:04:58 -0500

First, apologies for missing the last call deadline.  But I'd like for
the record to state that I don't believe the authors have addressed
the substantive criticisms raised on the list.  I'm also not sure that
consensus has been reached on this document, but that is for the
working group chairs to decide.

To summarize my position in a single phrase, closing a TCP connection
because it is in the persist state is a mistake.  And this document
describes a mechanism to do exactly that.  If the language in RFC1122
needs to be clarified, the clarification should be that a connection
may be aborted while in the persist state.  It should not be aborted
*because* it is in the persist state.

To be more specific, the majority of the doc describes a DoS attack by
resource exhaustion; however, the persist condition is only a small
subset of this class of resource exhaustion attacks.  Addressing only
connections in the persist state ignores the rest of the problem.  The
document does not address this very simple well-know attack:
<http://shlang.com/netkill/>.  This document also does not address
connections where, for example, one byte of window is opened every 60
seconds, and how they might be prioritized relative to connections in
the persist state.  Using the mechanism described in the document
leaves a system completely vulnerable to resource exhaustion by such a
slow-acking attacker.  As a developer, why would I choose to implement
this?  Unless these simple variations are addressed, I think it's a
waste of time.

Additionally, most implementations already provide an API for handling
closing of sockets in FIN_WAIT or CLOSING states, with the SO_LINGER
option.  I don't think this document makes a sufficient justification
for why this existing mechanism is inadequate, or why the proposed API
is necessary or sufficient.

Thanks,
  -John

On Sun, Jan 30, 2011 at 11:27 PM, Wesley Eddy <[email-masked]> wrote:
> Since this last call has concluded, I'm planning to do the PROTO writeup and
> send up the document to the IESG late this week.  Since this document is
> fairly short, includes relatively simple technical material, and was
> well-discussed earlier in its history, I wasn't concerned by the lack of
> WGLC feedback, but will note it in the writeup.
>
> If there's any disagreement on this, or someone wants more time to review,
> please shout in the next couple days, before I submit the document to the
> IESG for publication.
>
> --
> Wes Eddy
> MTI Systems

---

From: Mahesh Jethanandani <[email-masked]>
Date: Mon, 31 Jan 2011 15:34:11 -0800

John,

I think you missed more than the last call deadline on the document :-)

If there was ever the intent of this document to define all possible 
solutions to resource exhaustion DoS attacks, it was dropped quite some 
time ago. For the last couple of drafts when it was adopted as WG 
document, the intent was only to clarify the language in RFC 1122. The 
example given in the document was to demonstrate how the attack could 
happen. It was not meant to encompass all possible attacks.

We agreed after numerous discussions that documenting and providing 
solutions to the resource exhaustion attacks that you refer to should be 
taken up in a separate document and is out of the scope of this document.

HTH.

On 1/31/11 3:04 PM, John Heffner wrote:
> On Sun, Jan 30, 2011 at 11:27 PM, Wesley Eddy<[email-masked]>  wrote:

-- 

*Mahesh Jethanandani*
*Technical Leader*
**ERBU*
*

**
170 West Tasman Drive
San Jose, CA 95070
United States

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Mon, 31 Jan 2011 15:34:42 -0800

John,
    We have been through this in the past and the authors have clarified the reasons for some of the questions being posed by you and others. Here is yet another attempt. Pl see inline responses :-

> John Heffner
> Sent: Monday, January 31, 2011 3:05 PM
> To: Wesley Eddy
> Cc: [email-masked]

Pl go read the emails threads on this list on this subject. The only issue, if I recall correctly was, the mention of a socket API in the document. Now, this is an informational document and the mention of the socket API is purely for illustrations only and thus doesn't mandate any new protocol changes. We have also noted very clearly that it is up to the implementation to chose whatever API it wants to. 

> because it is in the persist state is a mistake.  And this document

Nobody is advocating that. We are just clarifying that it is OK to close a connection at any time, persist state in no exception to this rule. These 2 are different things. What makes you carry an impression that we are asking implementations to close the connection stuck in persist state? We are simply clarifying the it is inline with RFC's spirits if the decision is made consciously. 

> *because* it is in the persist state.

Where did you get the impression? this has been your stance since day one, I suggest this as a way to move forward. The authors have tried their best put the verbiage to clarify the intent. Now, you seem to be not convinced, it would be nicer if you can :-

- read the document and state which sections are not conformant.
- suggest a modified text.

It is a very short document and hence I guess this shouldn't be a big deal.

> this?  Unless these simple variations are addressed, I think it's a

Well, we removed all the resource management stuff from the document since the WG wanted this document as a simple clarification document. I thought we added the case of one byte window being opened etc., and why addressing such a thing is beyond the scope of this document. I can check the document again...

> is necessary or sufficient.

We explained this earlier. SO_LINGER applies only during connection close. Applications would like to know the reason for connection not making any progress which includes persist. Now, application is simply telling TCP layer that if the connection is in persist for more then a stipulated time, TCP can go ahead and kill the connection in the same manner like retransmit timeout. Now, I don't want the above statement to say that RTO and persist need to be treated the same way, we are just saying that "as long as application closes the connection or has indicated its willingness to close the connection" it can do so and it is in spirits of RFC 1122.  

I am at loss to understand your concers on this very simple document. It would be nicer if you can point to the exact sections where this document is vague or isn't clear.

Side note :- now this is typical, someone is going to snip portions of my response and get hung up one statement (may be a language weakness or something that is not intended etc., ) hence I am urging folks to read the response in its entirety while responding and not pick (nitpick) on some text.

-Anantha

> review,

---

From: Fernando Gont <[email-masked]>
Date: Mon, 31 Jan 2011 20:38:37 -0300

> this?  Unless these simple variations are addressed, I think it's a

+1

For the record, I raised the same issues a while ago.

FWIW, at the point in which resources are exhausted, the system needs to
perform heuristics as to which connections are achieving anything
useful, which one might make sense to terminate, etc. And to a large
extent it becomes irrelevant whether "TCP MAY/SHOULD/MUST NOT abort a
connection". -- It becomes a resource management problem, and the system
should do what is best to keep offering the best possible service.

A connection being in the "persist state" is just *one* of the
parameters that may be employed when performed the aforementioned
heuristics -- progress on the connection, amount of data successfully
transferred so far, etc., etc. are other valuable parameters.

Giving that these issues have been publicly discussed for quite a while
(e.g., they have been part of a disclosure carried about by CERT-FI as
part of the "Sockstress vulnerabilities"), one would expect attack tools
to implement the "open the window a little bit, once in a while"
(suggested by John Heffner, and also present in
draft-ietf-tcpm-tcp-security) and other variants, that would defeat this
"mitigation". -- FWIW, I was in the process of implementing this attack
vector a while ago (and will resume when I have the time).

Thanks,
-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: John Heffner <[email-masked]>
Date: Mon, 31 Jan 2011 18:49:08 -0500

On Mon, Jan 31, 2011 at 6:34 PM, Mahesh Jethanandani <[email-masked]> wrote:
> If there was ever the intent of this document to define all possible solutions to resource exhaustion DoS attacks, it was dropped quite some time ago. For the last couple of drafts when it was adopted as WG document, the intent was only to clarify the language in RFC 1122. The example given in the document was to demonstrate how the attack could happen. It was not meant to encompass all possible attacks.
>
> We agreed after numerous discussions that documenting and providing solutions to the resource exhaustion attacks that you refer to should be taken up in a separate document and is out of the scope of this document.

It should address the general problem, or not attack the problem at
all.  Addressing only a single special case in isolation does no real
good, and I would argue actually does harm.  As an operating system
developer, I cannot see any justification to spend time and resources
implementing the described changes.

I have no objection to a very simple, straightforward clarification to
the language in RFC1122, stating that connections MAY be ABORTed while
in the persist state.

I don't think it will be productive to re-hash all the old
conversations, as neither of us have changed our positions.  I'm just
re-stating my position for the record, as concisely as I can.  If
there is in fact working group consensus then let's move forward.

Thanks,
  -John

---

From: Mahesh Jethanandani <[email-masked]>
Date: Mon, 31 Jan 2011 15:53:51 -0800

> the language in RFC1122, stating that connections MAY be ABORTed while
That is why the informational draft with a title "Clarification of 
sender behavior in persist condition".

-- mj

---

From: "Anantha Ramaiah (ananth)" <[email-masked]>
Date: Mon, 31 Jan 2011 17:48:08 -0800

Fernando,

> vector a while ago (and will resume when I have the time).

Well, the best way to address this is to write a separate document which
addresses these issues and proposes solutions to the same. This is
definitely beyond of the current document which is a simple
clarification. The API is part of the clarification.

-Anantha