Subject: [tcpm] AccECN semantics

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sun, 16 Mar 2014 11:17:20 +0000

Hi Mirja, Brian, group,

I'm currently testing, trying to follow what Mirja and you have described here:

http://tools.ietf.org/html/draft-kuehlewind-tcpm-ecn-fallback-01

but extending it a bit into the ambiguous region against a ECN receiver.

RFC3168 stipulates that pure control packets MUST NOT have ECT (thus no CE) (section 6.1.4), but doesn't explicitly specify the receiver reaction, if a control packet is nevertheless received with CE...

With one ECN TCP stack I happened to have under test, the receiver actually suppresses and conceals the CE information. 

IMHO, the receiver should not make assumptions or remove information (a path still using TOS rather than DiffServ might be the cause - but it's up to the sender to detect that misbehavior).

I think the requirement given in 3168 about control packets MUST NOT carry ECT is sound, it probably would have been better a SHOULD NOT; but IMHO, the receiver MUST reflect the ECT/CE once the session has negotiated ECN support - to allow future extensions without having to modify the receiver again.

However, this might make the accounting more complex, as CE marks on pure ACKs, while no data is being sent, don't necessarily have to invoke a CC reaction (other than ACK-CC) immediately.

Perhaps that should go into either AccEcn requirements, or one of the AccECN mechanism drafts?

Any thoughts?

Richard Scheffenegger

---

From: Brian Trammell <[email-masked]>
Date: Mon, 17 Mar 2014 10:24:15 +0100

Hi Richard, all,

> but extending it a bit into the ambiguous region against a ECN receiver.

Very cool. Backing all this up with information about reality in the network is essential. :)

FWIW, though the draft expired last week, we do intend to develop it further. I have a student project that started last week to revisit the proportion of usable and unusable endpoints among popular servers (the 2013 PAM paper), as well as to catalog ECN failure modes, and to determine the likelihood of mid-flow transition from ECN-usable to ECN-unusable as discussed in Vancouver. We intend to update the draft backed up by data produced by this work.

> RFC3168 stipulates that pure control packets MUST NOT have ECT (thus no CE) (section 6.1.4), but doesn't explicitly specify the receiver reaction, if a control packet is nevertheless received with…

It does point (non-normatively) in the direction of "future research":

                                      ... [m]echanisms for responding to
   congestion on the ACK-path are areas for current and future research.
   (One simple possibility would be for the sender to reduce its
   congestion window when it receives a pure ACK packet with the CE
   codepoint set).

> With one ECN TCP stack I happened to have under test, the receiver actually suppresses and conceals the CE information. 

Interesting to see the stack didn't take the hint there. :) "suppresses and conceals" means no ECE, no reduction, and no information to the application layer?

> IMHO, the receiver should not make assumptions or remove information (a path still using TOS rather than DiffServ might be the cause - but it's up to the sender to detect that misbehavior).

Given how many flows we observed on the open Internet that were clearly misusing the ECT bits, the receiver should probably always assume that the sender is broken.

> Perhaps that should go into either AccEcn requirements, or one of the AccECN mechanism drafts?

Well, "mechanisms for responding to congestion on the ACK-path are areas for ... future research" so maybe splitting that work out into another draft might scope the work a bit better. AccECN seems more complete than any of this at the moment...

Cheers,

Brian

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 17 Mar 2014 09:44:54 +0000

Hi Brian,

> > RFC3168 stipulates that pure control packets MUST NOT have ECT (thus no
> CE) (section 6.1.4), but doesn't explicitly specify the receiver reaction,
> if a control packet is nevertheless received with CE...
> 
> > With one ECN TCP stack I happened to have under test, the receiver
> actually suppresses and conceals the CE information.
> 
> Interesting to see the stack didn't take the hint there. :) "suppresses
> and conceals" means no ECE, no reduction, and no information to the
> application layer?

Well, I was only looking at the network layer without checking application sockets directly; Also, I'm concered at this point about the receiver behavior rather than the sender behavior.

ECE's reflected during phases where the sender only ACKs data from the receiver - that is, data is only flowing in the reverse direction - wouldn't need to actually reduce the cwnd on the sender (which is not sending data anyway, at that point. But they could reveal ACK-path congestion, which *may* help during cwnd validation phases etc.

> 
> > IMHO, the receiver should not make assumptions or remove information (a
> path still using TOS rather than DiffServ might be the cause - but it's up
> to the sender to detect that misbehavior).
> 
> Given how many flows we observed on the open Internet that were clearly
> misusing the ECT bits, the receiver should probably always assume that the
> sender is broken.

I take this as a statement of irony :)

ECN signaling is seriously impeded, but I think with we have a chance to get the end hosts right with the deployment of AccECN, we should be clear what the expected behavior *of the signaling alone* should be. Not leaving things ambiguous in the specs should make it easier to have implementations that follow the spirit of the spec... 

> Well, "mechanisms for responding to congestion on the ACK-path are areas
> for ... future research" so maybe splitting that work out into another
> draft might scope the work a bit better. AccECN seems more complete than
> any of this at the moment...

True, but we haven't yet any text that makes it clear if there should be a distinction between data and control segments. At least I assumed that the AccECN signal should be agnostic what type of segment the CE (ECT(0) /ECT(1)) was sent with...

Richard Scheffenegger

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 20 Mar 2014 22:25:21 +0000

Richard, Brian,

> >    congestion window when it receives a pure ACK packet with the CE

That's because Sally had in mind to work on ACK-CC [RFC5690].

>Well, I was only looking at the network layer without checking 
>application sockets directly; Also, I'm concered at this point about 
>the receiver behavior rather than the sender behavior.
>
>ECE's reflected during phases where the sender only ACKs data from 
>the receiver - that is, data is only flowing in the reverse 
>direction - wouldn't need to actually reduce the cwnd on the sender 
>(which is not sending data anyway, at that point. But they could 
>reveal ACK-path congestion, which *may* help during cwnd validation phases etc.
>

I agree in principle, but this is hard in practice (see below).

>I take this as a statement of irony :)

Seriously, in case its not obvious, it would be wrong to /always/ 
assume the sender is always broken, just because it always is now. 
Otherwise, if senders get fixed, we have a load of broken receivers.

> > > I think the requirement given in 3168 about control packets 
> MUST NOT carry ECT is
> > sound, it probably would have been better a SHOULD NOT; but IMHO, 
> the receiver MUST
> > reflect the ECT/CE once the session has negotiated ECN support - 
> to allow future
> > extensions without having to modify the receiver again.
>
> > > However, this might make the accounting more complex, as CE 
> marks on pure ACKs, while
> > no data is being sent, don't necessarily have to invoke a CC 
> reaction (other than ACK-
> > CC) immediately.

[Bob: I added the above 2 paras back into the thread]
+1 to allowing future extensions without mod'ing the rcvr.

But how?...
a) If we tried to define feedback of CE-marked bytes, then a 
CE-marked pure ACK would trigger zero feedback (feedback of the 
number zero could be semantically different from no feedback).
b) If we tried to define feedback of CE-marked packets, then a 
CE-marked pure ACK would increment the feedback count by 1, and the 
sender would usually be able to infer that over the same packet the 
ACK counter had incremented by zero bytes.

If the receiver got a CE-marked rexmt, and fed this back either way, 
the sender would read it as more bytes CE marked. But that's safe - 
IMHO the argument in 3168 for making re-transmissions Not-ECT is less 
clear-cut than with pure ACKs. If you lose a packet you respond to 
the congestion. If the repair is then CE-marked, that means more 
congestion AFA I'm concerned, so another congestion response seems 
reasonable to me.

So either way, the sender would (at least statistically) be able to 
work out what had happened, and decide how and whether to respond 
(ie. do nothing if the receiver feeds back that there was CE on a 
control packet). Superficially, option b) seems more efficient and a) 
doesn't give much more info in return for the greater overhead. 
Either way, the sender could at least change its behaviour 
unilaterally in the future.

>ECN signaling is seriously impeded, but I think with we have a 
>chance to get the end hosts right with the deployment of AccECN, we 
>should be clear what the expected behavior *of the signaling alone* 
>should be. Not leaving things ambiguous in the specs should make it 
>easier to have implementations that follow the spirit of the spec...

In this spirit I wrote my recent email tightening up the definition 
of the ACE field when the ACK flag is cleared for 
draft-kuehlewind-tcpm-accurate-ecn (I'll late cc to Brian too).

>True, but we haven't yet any text that makes it clear if there 
>should be a distinction between data and control segments. At least 
>I assumed that the AccECN signal should be agnostic what type of 
>segment the CE (ECT(0) /ECT(1)) was sent with...

AccECN requires mods to both sender and receiver (it is negotiated), 
so we can require that the receiver reflects as much info as it can 
as faithfully as possible, and that the sender attempts to do nothing 
in response to any feedback of CE on control packets.

Bob

>Richard Scheffenegger

________________________________________________________________
Bob Briscoe,                                                  BT

---

From: Brian Trammell <[email-masked]>
Date: Fri, 21 Mar 2014 12:35:24 +0100

hi Bob, all,

On 20 Mar 2014, at 23:25, Bob Briscoe <[email-masked]> wrote:

<snip>

> Seriously, in case its not obvious, it would be wrong to /always/ assume the sender is always broken, just because it always is now. Otherwise, if senders get fixed, we have a load of broken receivers.

I was making a more general point, along the lines of the principle of liberal acceptance. Given the prevalence of traffic (at least as of 2012) misusing these codepoints, ECN-capable endpoints should be designed and implemented defensively. This is the point of the ECN fallback draft.

Receivers should also note whether they saw CE on the SYN, and mark those peers/paths ECN-unusable. This probably needs to go in the ECN fallback draft too. It's not clear how to handle this misuse in-network. At this point in 2014, I think I'm happy giving guidance that it's okay to break TOS-byte traffic on the open Internet by marking when appropriate, since at least in our observations the vast majority of such misuse came from a very small number of providers.

Once ECN is negotiated, the sender/receiver should still be prepared to back off in case the path becomes ECN-impaired. Currently running research should shed some light on how often we expect this to happen.

>> > > I think the requirement given in 3168 about control packets MUST NOT carry ECT is
>> > sound, it probably would have been better a SHOULD NOT; but IMHO, the receiver MUST
>> > reflect the ECT/CE once the session has negotiated ECN support - to allow future
>> > > However, this might make the accounting more complex, as CE marks on pure ACKs, while
>> > no data is being sent, don't necessarily have to invoke a CC reaction (other than ACK-
>> > CC) immediately.
> 
> a) If we tried to define feedback of CE-marked bytes, then a CE-marked pure ACK would trigger zero feedback (feedback of the number zero could be semantically different from no feedback).
> b) If we tried to define feedback of CE-marked packets, then a CE-marked pure ACK would increment the feedback count by 1, and the sender would usually be able to infer that over the same packet the ACK counter had incremented by zero bytes.
> 
> If the receiver got a CE-marked rexmt, and fed this back either way, the sender would read it as more bytes CE marked. But that's safe - IMHO the argument in 3168 for making re-transmissions Not-ECT is less clear-cut than with pure ACKs. If you lose a packet you respond to the congestion. If the repair is then CE-marked, that means more congestion AFA I'm concerned, so another congestion response seems reasonable to me.

Yep.

> So either way, the sender would (at least statistically) be able to work out what had happened, and decide how and whether to respond (ie. do nothing if the receiver feeds back that there was CE on a control packet). Superficially, option b) seems more efficient and a) doesn't give much more info in return for the greater overhead. Either way, the sender could at least change its behaviour unilaterally in the future.

Also yep.

Cheers,

Brian