Subject: [conex] An individual for your review

From: Zhu Lei <[email-masked]>
Date: Tue, 06 Jul 2010 18:53:17 +0800

Hi all,
 
We already had some discussions which may allow wireless access networks to
notify congestions in mail list. 
 
I personally think that Conex charter is asking some solutions by stating
"However, the CONEX WG will initially focus on one use case, where the end
hosts and the network that contains the destination end host are
CONEX-enabled but other networks need not be." 
 
I have an individual draft which introduces some ideas for next f2f meeting.
I'd like to ask you to review this short document and help identify the
problems which could be resolved in Conex wg. Thank you!
 
draft-lei-ecn-localised-congestion-notification-00

Hope have your feedback soon.
 
Best regards,
Lei Zhu

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 14 Jul 2010 10:55:26 +0100

Zhu,

Hi. I read your individual draft last night.
draft-lei-ecn-localised-congestion-notification-00

First, we should be clear that working on new data plane congestion 
notification technology is not part of the ConEx charter. So, those 
focusing solely on the charter can stop reading now if they want.

But I want to briefly review your proposal anyway, mainly to record 
on the mailing list what I believe is the current status of ECN 
deployment problems.

If I can summarise your proposal, I believe the main proposed new 
requirement is that we should work on a congestion notification 
architecture that looks something like this:

+--------------------------------------------------------------<---+
|   +---<----+                                                     ^
|  /         ^                                                     |
| |          |ECN_S                                    ECN_R       |
V V          |                                     --------------> |
  S ---> S's access net ---> core ... core ---> R's access net ---> R

Where ECN_S and ECN_R are congestion notification channels from the 
respective access networks of the sender S and receiver R directly to 
S and R. I believe the proposal is currently agnostic to whether 
ECN_R is in-band or out-of-band, but clearly ECN_S must be out-of-band.

Is the stated problem solved by this proposal?
----------------------------------------------
The stated problem this proposal aims to solve is:
- congestion is mostly in access networks
- ECN sometimes does not get through certain middleboxes that do not 
support ECN.

I agree with both statements, but there seems to be a misconception 
in the document about what it means for a router, switch or middlebox 
to 'not support' ECN.

ECN incremental deployment design
---------------------------------
ECN was designed for incremental deployment where some, most or all 
routers or middleboxes do not do ECN marking. If both endpoints 
support ECN, they should be able to exchange packets between 
themselves that are ECN-capable (with an ECN-capable transport or ECT 
codepoint set) whatever network support there is for ECN between them.

ECN was designed so that network equipment that has not been upgraded 
to mark the ECN field during congestion would silently forward 
ECN-capable packets when not congested and drop them during 
congestion - no different to any non-ECN-capable packets with the ECN 
field cleared (Not-ECT).

Ways to 'not support' ECN
-------------------------
Network equipment can 'not support' ECN in three ways:
1) lacks ECN: not yet upgraded to do ECN marking.
2) prevents ECN (a bug): middlebox tampers with endpoint attempts to 
send ECN-capable packets. Three known sub-cases:
  2a) ECN-TCP drop: drops SYNs attempting to negotiate ECN in the TCP header
  2b) ECN-IP drop: drops packets with ECN-capable codepoint in IP header
  2c) ECN-IP normalise: reverts ECN-capable packets to Not-ECT in IP header
3) crashes with ECN (a bug): an endpoint attempt to negotiate ECN in 
the TCP header crashes a middlebox

Case 1) is normal and is by far the most prevalent. ECN's incremental 
deployment design (above) copes with this just fine.

Case 2) happens in a small number of cases:
* Until about 2005-6 there were a number of enterprise-grade 
firewalls and NATs that did this, but tests showed their rules got 
upgraded over time and it was no longer a problem.
* It is however still a problem with about five popular models of 
home gateway from those days that will take some time to be discarded 
from most homes.
* Recently, a new problem has appeared on paths that were previously 
OK. It is thought this may be a buggy attempt to zero the Diffserv 
codepoint, which is also accidentally clearing the ECN field in the IP header.

Case 3) happens in one known case of one model of a home gateway that 
was popularly deployed a few years ago.

Equipment on the path exhibiting Case 2) behaviour can be detected by 
ECN black-hole detection code on the end-system, reverting a flow to 
Not-ECT in order to get packets through. There are black-hole 
detection patches for Linux, but black-hole detection is not 
configured by default with the mainline code. The

The home gateway that crashes on seeing ECN in the TCP header of a 
SYN (Case 3) is more problematic. ECN is turned off by default for 
the client end of TCP connections in Windows and Linux because of 
this single popular bugged home gateway. It should be possible to do 
some heuristic test to detect if the offending model is on your home 
network (perhaps using uPNP or checking whether the MAC address falls 
within a known range). But AFAIK, no code is available to do this.

Again, is the stated problem solved by this proposal?
-----------------------------------------------------
The proposal seems to be focused particularly on 3GPP, where AFAIK 
there are no ECN deployment problems that are not under the control 
of the network operators who could upgrade and fix them quickly.

I believe that introducing a new way to signal ECN would encounter as 
many deployment problems, if not more, than persevering with 
deployment of the existing way to do ECN.

Is backward ECN a good idea anyway?
-----------------------------------
The backward messages I have termed ECN_S in your proposal are 
reminiscent of ICMP source quench messages. SQ was deprecated for a 
number of reasons that also all apply to your proposal:
* SQ violates layering (congestion control is assumed to be end to 
end in the Internet Architecture), so it can interact badly with 
IPsec encryption and tunneling, making it difficult to address the 
message back to the process on the host generating the load.
* ICMP packet creation is normally (always) implemented on the 
ingress interface of a router. Congestion is detected on the egress, 
where it is too late, and too expensive wrt. critical processing 
time, to trigger the creation and sending of an ICMP packet, without 
hugely re-working the implementation architecture of most routers.
* SQ creates more data (although admittedly in the other direction) 
during congestion, which is generally to be avoided.
* SQ may get lost, and there's no 'end-middle' reliable channel 
between source and bottleneck to recover losses.
* SQ messages can be maliciously sent to hosts as if they came from 
an on-path router, thus creating a DoS vulnerability. To solve this 
would require a complicated security association between the SQ 
message and the packet stream it refers to.
* SQ messages are invariably discarded by firewalls, mostly because 
they can be spoofed (previous point).

See 
<http://www.ietf.org/mail-archive/web/ternli/current/msg00036.html> 
for the history of ICMP SQ.

In summary, I don't believe your proposal is a good way to solve the 
problems it identifies.

Cheers

Bob

At 11:53 06/07/2010, Zhu Lei wrote:
>Hi all,
>
>We already had some discussions which may allow wireless access 
>networks to notify congestions in mail list.
>
>I personally think that Conex charter is asking some solutions by 
>stating "However, the CONEX WG will initially focus on one use case, 
>where the end hosts and the network that contains the destination 
>end host are CONEX-enabled but other networks need not be."
>
>I have an individual draft which introduces some ideas for next f2f 
>meeting. I'd like to ask you to review this short document and help 
>identify the problems which could be resolved in Conex wg. Thank you!
>
>_______________________________________________
>conex mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/conex

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Zhu Lei <[email-masked]>
Date: Fri, 16 Jul 2010 15:47:00 +0800

Hi Bob and all,
 
Thank you for identifing some problems of ECN, especially mentioned some
historical technical discussions which I never heard previously.
 
The problems of ECN at the moment are clearly summerised and clarified by
Bob. I should follow the directions together with issues which is to be
solved.
 
I have to say the understanding of the "proposal" is mainly correct. I
highly appreciate the draft proposal and problem statement in previous
e-mail. We at least need some statement in use cases draft.
 
The possible technical proposal of my mind is little bit different from the
picture. The major difference is that the ECN_S could be not out-of-band.
And, I also don't think that the ICMP is the approach. I personlly want the
in-band trasnportation of ECN mark.
 
The document seems hint the proposal, but the intention of the submitted
document is to introduce the requirements. I would try to write a technical
proposal in ML soon. The slides for presentation take me time due to my
native language.
 
I appologize for some delay of replying e-mail since I should have some
works for internal guys in parallel.
 
Best regards,
Lei

  _____  

发件人: Bob Briscoe [mailto:[email-masked]] 
发送时间: 2010年7月14日 17:55
收件人: Zhu Lei
抄送: [email-masked]
主题: Re: [conex] An individual for your review

Zhu,

First, we should be clear that working on new data plane congestion
notification technology is not part of the ConEx charter. So, those focusing
solely on the charter can stop reading now if they want. 

But I want to briefly review your proposal anyway, mainly to record on the
mailing list what I believe is the current status of ECN deployment
problems.

If I can summarise your proposal, I believe the main proposed new
requirement is that we should work on a congestion notification architecture
that looks something like this:

Where ECN_S and ECN_R are congestion notification channels from the
respective access networks of the sender S and receiver R directly to S and
R. I believe the proposal is currently agnostic to whether ECN_R is in-band
or out-of-band, but clearly ECN_S must be out-of-band.

- ECN sometimes does not get through certain middleboxes that do not support
ECN.

I agree with both statements, but there seems to be a misconception in the
document about what it means for a router, switch or middlebox to 'not
support' ECN. 

ECN incremental deployment design
---------------------------------
ECN was designed for incremental deployment where some, most or all routers
or middleboxes do not do ECN marking. If both endpoints support ECN, they
should be able to exchange packets between themselves that are ECN-capable
(with an ECN-capable transport or ECT codepoint set) whatever network
support there is for ECN between them.

ECN was designed so that network equipment that has not been upgraded to
mark the ECN field during congestion would silently forward ECN-capable
packets when not congested and drop them during congestion - no different to
any non-ECN-capable packets with the ECN field cleared (Not-ECT).

Ways to 'not support' ECN
-------------------------
Network equipment can 'not support' ECN in three ways:
1) lacks ECN: not yet upgraded to do ECN marking.
2) prevents ECN (a bug): middlebox tampers with endpoint attempts to send
ECN-capable packets. Three known sub-cases:
 2a) ECN-TCP drop: drops SYNs attempting to negotiate ECN in the TCP header
 2b) ECN-IP drop: drops packets with ECN-capable codepoint in IP header
 2c) ECN-IP normalise: reverts ECN-capable packets to Not-ECT in IP header
3) crashes with ECN (a bug): an endpoint attempt to negotiate ECN in the TCP
header crashes a middlebox

Case 2) happens in a small number of cases:
* Until about 2005-6 there were a number of enterprise-grade firewalls and
NATs that did this, but tests showed their rules got upgraded over time and
it was no longer a problem. 
* It is however still a problem with about five popular models of home
gateway from those days that will take some time to be discarded from most
homes. 
* Recently, a new problem has appeared on paths that were previously OK. It
is thought this may be a buggy attempt to zero the Diffserv codepoint, which
is also accidentally clearing the ECN field in the IP header.

Case 3) happens in one known case of one model of a home gateway that was
popularly deployed a few years ago.

Equipment on the path exhibiting Case 2) behaviour can be detected by ECN
black-hole detection code on the end-system, reverting a flow to Not-ECT in
order to get packets through. There are black-hole detection patches for
Linux, but black-hole detection is not configured by default with the
mainline code. The 

The home gateway that crashes on seeing ECN in the TCP header of a SYN (Case
3) is more problematic. ECN is turned off by default for the client end of
TCP connections in Windows and Linux because of this single popular bugged
home gateway. It should be possible to do some heuristic test to detect if
the offending model is on your home network (perhaps using uPNP or checking
whether the MAC address falls within a known range). But AFAIK, no code is
available to do this.

Again, is the stated problem solved by this proposal?
-----------------------------------------------------
The proposal seems to be focused particularly on 3GPP, where AFAIK there are
no ECN deployment problems that are not under the control of the network
operators who could upgrade and fix them quickly.

I believe that introducing a new way to signal ECN would encounter as many
deployment problems, if not more, than persevering with deployment of the
existing way to do ECN.

Is backward ECN a good idea anyway?
-----------------------------------
The backward messages I have termed ECN_S in your proposal are reminiscent
of ICMP source quench messages. SQ was deprecated for a number of reasons
that also all apply to your proposal:
* SQ violates layering (congestion control is assumed to be end to end in
the Internet Architecture), so it can interact badly with IPsec encryption
and tunneling, making it difficult to address the message back to the
process on the host generating the load. 
* ICMP packet creation is normally (always) implemented on the ingress
interface of a router. Congestion is detected on the egress, where it is too
late, and too expensive wrt. critical processing time, to trigger the
creation and sending of an ICMP packet, without hugely re-working the
implementation architecture of most routers.
* SQ creates more data (although admittedly in the other direction) during
congestion, which is generally to be avoided.
* SQ may get lost, and there's no 'end-middle' reliable channel between
source and bottleneck to recover losses.
* SQ messages can be maliciously sent to hosts as if they came from an
on-path router, thus creating a DoS vulnerability. To solve this would
require a complicated security association between the SQ message and the
packet stream it refers to.
* SQ messages are invariably discarded by firewalls, mostly because they can
be spoofed (previous point).

See <  <http://www.ietf.org/mail-archive/web/ternli/current/msg00036.html>
http://www.ietf.org/mail-archive/web/ternli/current/msg00036.html> for the
history of ICMP SQ.

Cheers

Bob

Hi all,

 

Best regards,
Lei Zhu

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 16 Jul 2010 12:45:54 +0100

Lei,

The reason I said ECN_S must clearly be out of 
band is that, by definition in-band means within 
the packets travelling from S to R.

If you mean in-band in the packets travelling 
from R to S, the congested node may not be on the 
route of packets between R and S, which need not 
follow the reverse route of the forward path between S and R.

I think you will find that this expired I-D from 
Pasi Sarolahti is very relevant:
Transport-layer Considerations for Explicit Cross-layer Indications
<https://datatracker.ietf.org/doc/draft-sarolahti-tsvwg-crosslayer/>

Bob

At 08:47 16/07/2010, Zhu Lei wrote:
>Hi Bob and all,
>
>Thank you for identifing some problems of ECN, 
>especially mentioned some historical technical 
>discussions which I never heard previously.
>
>The problems of ECN at the moment are clearly 
>summerised and clarified by Bob. I should follow 
>the directions together with issues which is to be solved.
>
>I have to say the understanding of the 
>"proposal" is mainly correct. I highly 
>appreciate the draft proposal and problem 
>statement in previous e-mail. We at least need 
>some statement in use cases draft.
>
>The possible technical proposal of my mind is 
>little bit different from the picture. The major 
>difference is that the ECN_S could be not 
>out-of-band. And, I also don't think that the 
>ICMP is the approach. I personlly want the in-band trasnportation of ECN mark.
>
>The document seems hint the proposal, but the 
>intention of the submitted document is to 
>introduce the requirements. I would try to write 
>a technical proposal in ML soon. The slides for 
>presentation take me time due to my native language.
>
>I appologize for some delay of replying e-mail 
>since I should have some works for internal guys in parallel.
>
>Best regards,
>Lei
>
>
>
>----------
>·¢¼þÈË: Bob Briscoe [mailto:[email-masked]]
>·¢ËÍÊ±¼ä: 2010Äê7ÔÂ14ÈÕ 17:55
>ÊÕ¼þÈË: Zhu Lei
>³ËÍ: [email-masked]
>Ö÷Ìâ: Re: [conex] An individual for your review
>
>Zhu,
>
>First, we should be clear that working on new 
>data plane congestion notification technology is 
>not part of the ConEx charter. So, those 
>But I want to briefly review your proposal 
>anyway, mainly to record on the mailing list 
>what I believe is the current status of ECN deployment problems.
>
>If I can summarise your proposal, I believe the 
>main proposed new requirement is that we should 
>work on a congestion notification architecture that looks something like this:
>
>Where ECN_S and ECN_R are congestion 
>notification channels from the respective access 
>networks of the sender S and receiver R directly 
>to S and R. I believe the proposal is currently 
>agnostic to whether ECN_R is in-band or 
>out-of-band, but clearly ECN_S must be out-of-band.
>
>- ECN sometimes does not get through certain 
>middleboxes that do not support ECN.
>
>I agree with both statements, but there seems to 
>be a misconception in the document about what it 
>means for a router, switch or middlebox to 'not support' ECN.
>
>ECN was designed for incremental deployment 
>where some, most or all routers or middleboxes 
>do not do ECN marking. If both endpoints support 
>ECN, they should be able to exchange packets 
>between themselves that are ECN-capable (with an 
>ECN-capable transport or ECT codepoint set) 
>whatever network support there is for ECN between them.
>
>ECN was designed so that network equipment that 
>has not been upgraded to mark the ECN field 
>during congestion would silently forward 
>ECN-capable packets when not congested and drop 
>them during congestion - no different to any 
>non-ECN-capable packets with the ECN field cleared (Not-ECT).
>
>2) prevents ECN (a bug): middlebox tampers with 
>endpoint attempts to send ECN-capable packets. Three known sub-cases:
>3) crashes with ECN (a bug): an endpoint attempt 
>to negotiate ECN in the TCP header crashes a middlebox
>
>Case 1) is normal and is by far the most 
>prevalent. ECN's incremental deployment design 
>(above) copes with this just fine.
>
>* Until about 2005-6 there were a number of 
>enterprise-grade firewalls and NATs that did 
>this, but tests showed their rules got upgraded 
>over time and it was no longer a problem.
>* It is however still a problem with about five 
>popular models of home gateway from those days 
>that will take some time to be discarded from most homes.
>* Recently, a new problem has appeared on paths 
>that were previously OK. It is thought this may 
>be a buggy attempt to zero the Diffserv 
>Case 3) happens in one known case of one model 
>of a home gateway that was popularly deployed a few years ago.
>
>Equipment on the path exhibiting Case 2) 
>behaviour can be detected by ECN black-hole 
>detection code on the end-system, reverting a 
>flow to Not-ECT in order to get packets through. 
>There are black-hole detection patches for 
>Linux, but black-hole detection is not 
>The home gateway that crashes on seeing ECN in 
>the TCP header of a SYN (Case 3) is more 
>problematic. ECN is turned off by default for 
>the client end of TCP connections in Windows and 
>Linux because of this single popular bugged home 
>gateway. It should be possible to do some 
>heuristic test to detect if the offending model 
>is on your home network (perhaps using uPNP or 
>checking whether the MAC address falls within a 
>known range). But AFAIK, no code is available to do this.
>
>The proposal seems to be focused particularly on 
>3GPP, where AFAIK there are no ECN deployment 
>problems that are not under the control of the 
>network operators who could upgrade and fix them quickly.
>
>I believe that introducing a new way to signal 
>ECN would encounter as many deployment problems, 
>if not more, than persevering with deployment of the existing way to do ECN.
>
>The backward messages I have termed ECN_S in 
>your proposal are reminiscent of ICMP source 
>quench messages. SQ was deprecated for a number 
>of reasons that also all apply to your proposal:
>* SQ violates layering (congestion control is 
>assumed to be end to end in the Internet 
>Architecture), so it can interact badly with 
>IPsec encryption and tunneling, making it 
>difficult to address the message back to the 
>* ICMP packet creation is normally (always) 
>implemented on the ingress interface of a 
>router. Congestion is detected on the egress, 
>where it is too late, and too expensive wrt. 
>critical processing time, to trigger the 
>creation and sending of an ICMP packet, without 
>* SQ creates more data (although admittedly in 
>the other direction) during congestion, which is generally to be avoided.
>* SQ may get lost, and there's no 'end-middle' 
>reliable channel between source and bottleneck to recover losses.
>* SQ messages can be maliciously sent to hosts 
>as if they came from an on-path router, thus 
>creating a DoS vulnerability. To solve this 
>would require a complicated security association 
>between the SQ message and the packet stream it refers to.
>* SQ messages are invariably discarded by 
>firewalls, mostly because they can be spoofed (previous point).
>
>See < 
>http://www.ietf.org/mail-archive/web/ternli/current/msg00036.html> 
>In summary, I don't believe your proposal is a 
>good way to solve the problems it identifies.
>
>Cheers
>
>
>Bob
>
>>We already had some discussions which may allow 
>>wireless access networks to notify congestions in mail list.
>>
>>I personally think that Conex charter is asking 
>>some solutions by stating "However, the CONEX 
>>WG will initially focus on one use case, where 
>>the end hosts and the network that contains the 
>>destination end host are CONEX-enabled but other networks need not be."
>>
>>I have an individual draft which introduces 
>>some ideas for next f2f meeting. I'd like to 
>>ask you to review this short document and help

---

From: Zhu Lei <[email-masked]>
Date: Mon, 19 Jul 2010 11:12:36 +0800

Hi,
 
If possible, I want to see more historical drafts related to the crosslayer ideas and thoughts.
 
I think that S must be ECN enabled in any cases. It is important for ECN_S knows it and is capable to notify access congestion by sending ECN marks with TCP respones.
 
The problem meet is how to know possibilities to receive congestion notification sent by access node(s) by UE. I personally want to oen it for access network(e.g. link layer) in Conex working group. For example, a statement would be useful for people working in access network who know the ECN solution might be used to notifiy access network congestion only.
 
Best regards,
Zhu Lei

  _____  

发送时间: 2010年7月16日 19:46

Lei,

The reason I said ECN_S must clearly be out of band is that, by definition in-band means within the packets travelling from S to R.

If you mean in-band in the packets travelling from R to S, the congested node may not be on the route of packets between R and S, which need not follow the reverse route of the forward path between S and R. 

I think you will find that this expired I-D from Pasi Sarolahti is very relevant:
Transport-layer Considerations for Explicit Cross-layer Indications
<  <https://datatracker.ietf.org/doc/draft-sarolahti-tsvwg-crosslayer/> https://datatracker.ietf.org/doc/draft-sarolahti-tsvwg-crosslayer/ >

Bob

Hi Bob and all,
 
Thank you for identifing some problems of ECN, especially mentioned some historical technical discussions which I never heard previously.
 
The problems of ECN at the moment are clearly summerised and clarified by Bob. I should follow the directions together with issues which is to be solved.
 
I have to say the understanding of the "proposal" is mainly correct. I highly appreciate the draft proposal and problem statement in previous e-mail. We at least need some statement in use cases draft.
 
The possible technical proposal of my mind is little bit different from the picture. The major difference is that the ECN_S could be not out-of-band. And, I also don't think that the ICMP is the approach. I personlly want the in-band trasnportation of ECN mark.
 
The document seems hint the proposal, but the intention of the submitted document is to introduce the requirements. I would try to write a technical proposal in ML soon. The slides for presentation take me time due to my native language.
 
I appologize for some delay of replying e-mail since I should have some works for internal guys in parallel.
 
Best regards,
Lei

  _____  

·¢¼þÈË: Bob Briscoe [  <mailto:[email-masked]> mailto:[email-masked]] 
·¢ËÍÊ±¼ä: 2010Äê7ÔÂ14ÈÕ 17:55
ÊÕ¼þÈË: Zhu Lei
³ËÍ: [email-masked]
Ö÷Ìâ: Re: [conex] An individual for your review

Zhu,

First, we should be clear that working on new data plane congestion notification technology is not part of the ConEx charter. So, those focusing solely on the charter can stop reading now if they want. 

But I want to briefly review your proposal anyway, mainly to record on the mailing list what I believe is the current status of ECN deployment problems.

If I can summarise your proposal, I believe the main proposed new requirement is that we should work on a congestion notification architecture that looks something like this:

Where ECN_S and ECN_R are congestion notification channels from the respective access networks of the sender S and receiver R directly to S and R. I believe the proposal is currently agnostic to whether ECN_R is in-band or out-of-band, but clearly ECN_S must be out-of-band.

- ECN sometimes does not get through certain middleboxes that do not support ECN.

I agree with both statements, but there seems to be a misconception in the document about what it means for a router, switch or middlebox to 'not support' ECN. 

ECN incremental deployment design
---------------------------------
ECN was designed for incremental deployment where some, most or all routers or middleboxes do not do ECN marking. If both endpoints support ECN, they should be able to exchange packets between themselves that are ECN-capable (with an ECN-capable transport or ECT codepoint set) whatever network support there is for ECN between them.

ECN was designed so that network equipment that has not been upgraded to mark the ECN field during congestion would silently forward ECN-capable packets when not congested and drop them during congestion - no different to any non-ECN-capable packets with the ECN field cleared (Not-ECT).

Ways to 'not support' ECN
-------------------------
Network equipment can 'not support' ECN in three ways:
1) lacks ECN: not yet upgraded to do ECN marking.
2) prevents ECN (a bug): middlebox tampers with endpoint attempts to send ECN-capable packets. Three known sub-cases:
 2a) ECN-TCP drop: drops SYNs attempting to negotiate ECN in the TCP header
 2b) ECN-IP drop: drops packets with ECN-capable codepoint in IP header
 2c) ECN-IP normalise: reverts ECN-capable packets to Not-ECT in IP header
3) crashes with ECN (a bug): an endpoint attempt to negotiate ECN in the TCP header crashes a middlebox

Case 1) is normal and is by far the most prevalent. ECN's incremental deployment design (above) copes with this just fine.

Case 2) happens in a small number of cases:
* Until about 2005-6 there were a number of enterprise-grade firewalls and NATs that did this, but tests showed their rules got upgraded over time and it was no longer a problem. 
* It is however still a problem with about five popular models of home gateway from those days that will take some time to be discarded from most homes. 
* Recently, a new problem has appeared on paths that were previously OK. It is thought this may be a buggy attempt to zero the Diffserv codepoint, which is also accidentally clearing the ECN field in the IP header.

Case 3) happens in one known case of one model of a home gateway that was popularly deployed a few years ago.

Equipment on the path exhibiting Case 2) behaviour can be detected by ECN black-hole detection code on the end-system, reverting a flow to Not-ECT in order to get packets through. There are black-hole detection patches for Linux, but black-hole detection is not configured by default with the mainline code. The 

The home gateway that crashes on seeing ECN in the TCP header of a SYN (Case 3) is more problematic. ECN is turned off by default for the client end of TCP connections in Windows and Linux because of this single popular bugged home gateway. It should be possible to do some heuristic test to detect if the offending model is on your home network (perhaps using uPNP or checking whether the MAC address falls within a known range). But AFAIK, no code is available to do this.

Again, is the stated problem solved by this proposal?
-----------------------------------------------------
The proposal seems to be focused particularly on 3GPP, where AFAIK there are no ECN deployment problems that are not under the control of the network operators who could upgrade and fix them quickly.

I believe that introducing a new way to signal ECN would encounter as many deployment problems, if not more, than persevering with deployment of the existing way to do ECN.

Is backward ECN a good idea anyway?
-----------------------------------
The backward messages I have termed ECN_S in your proposal are reminiscent of ICMP source quench messages. SQ was deprecated for a number of reasons that also all apply to your proposal:
* SQ violates layering (congestion control is assumed to be end to end in the Internet Architecture), so it can interact badly with IPsec encryption and tunneling, making it difficult to address the message back to the process on the host generating the load. 
* ICMP packet creation is normally (always) implemented on the ingress interface of a router. Congestion is detected on the egress, where it is too late, and too expensive wrt. critical processing time, to trigger the creation and sending of an ICMP packet, without hugely re-working the implementation architecture of most routers.
* SQ creates more data (although admittedly in the other direction) during congestion, which is generally to be avoided.
* SQ may get lost, and there's no 'end-middle' reliable channel between source and bottleneck to recover losses.
* SQ messages can be maliciously sent to hosts as if they came from an on-path router, thus creating a DoS vulnerability. To solve this would require a complicated security association between the SQ message and the packet stream it refers to.
* SQ messages are invariably discarded by firewalls, mostly because they can be spoofed (previous point).

See < http://www.ietf.org/mail-archive/web/ternli/current/msg00036.html> for the history of ICMP SQ.

In summary, I don't believe your proposal is a good way to solve the problems it identifies.

Cheers

Bob

Hi all,
 
We already had some discussions which may allow wireless access networks to notify congestions in mail list. 
 
I personally think that Conex charter is asking some solutions by stating "However, the CONEX WG will initially focus on one use case, where the end hosts and the network that contains the destination end host are CONEX-enabled but other networks need not be." 
 
I have an individual draft which introduces some ideas for next f2f meeting. I'd like to ask you to review this short document and help identify the problems which could be resolved in Conex wg. Thank you!

Best regards,
Lei Zhu

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 19 Jul 2010 19:40:51 +0100

Lei,

At 04:12 19/07/2010, Zhu Lei wrote:
>ï»¿
>Hi,
>
>If possible, I want to see more historical 
>drafts related to the crosslayer ideas and thoughts.

The refs in Pasi's draft cover all I know of, AFAIK.

>I think that S must be ECN enabled in any cases. 
>It is important for ECN_S knows it and is 
>capable to notify access congestion by sending ECN marks with TCP respones.

I agree that the current design cannot use ECN in 
a flow of packets for which the sender is ECN-capable but the receiver is not.

But I don't believe it will make deployment any 
easier if we introduce backward ECN_S feedback. 
Then packets will have to signal whether the 
router should do forward or backward feedback. 
Because a router must not do both, otherwise it 
will double the amount of congestion signalled. 
We don't have room in the packet to switch 
between forward and backward feedback. And if we 
did take a new approach, it would require another 
decade to wait for routers to be deployed that understand this new feature.

If deployment is taking a long time, we need to 
understand *why* before proposing changes that 
will make deployment take even longer and be more complicated.

But most importantly, I listed problems with 
backward feedback, which ALL must to be solved 
first. And they are ALL very good reasons against 
backward feedback. So please address them all before taking this further.

>
>The problem meet is how to know possibilities to 
>receive congestion notification sent by access 
>node(s) by UE. I personally want to oen it for 
>access network(e.g. link layer) in Conex working group.

The ConEx group is not chartered to work on 
changes to ECN. Chartering is a careful process 
that takes account of everyone's views and 
results in a precise set of work items.

>For example, a statement would be useful for 
>people working in access network who know the 
>ECN solution might be used to notifiy access network congestion only.

I wrote my my original review email very 
carefully to explain how ECN incremental 
deployment is designed to work (to save you 
reading RFC3168). There is nothing in the 
original ECN design to stop only access network 
nodes being ECN enabled and other network nodes 
not being ECN enabled. Indeed, that's how I would 
expect ECN to be deployed first - by enabling ECN 
on the nodes that are most likely to be congested.

Then, if there is any unexpected congestion (e.g. 
in the core), the endpoints will see the drop 
from that, as well as ECN markings from the access networks.

As I said, the problem (at least as stated in 
your draft) is already solved by the original design of ECN.

Bob

>
>Best regards,
>Zhu Lei
>
>
>----------
>å‘ä»¶äºº: Bob Briscoe [mailto:[email-masked]]
>å‘é€æ—¶é—´: 2010å¹´7æœˆ16æ—¥ 19:46
>æ”¶ä»¶äºº: Zhu Lei
>æŠ„é€: [email-masked]
>ä¸»é¢˜: Re: [conex] An individual for your review
>
>Lei,
>
>from R to S, the congested node may not be on 
>the route of packets between R and S, which need 
>not follow the reverse route of the forward path between S and R.
>
>< https://datatracker.ietf.org/doc/draft-sarolahti-tsvwg-crosslayer/ >
>
>
>Bob
>
>>summerised and clarified by Bob. I should 
>>follow the directions together with issues which is to be solved.
>>
>>little bit different from the picture. The 
>>major difference is that the ECN_S could be not 
>>introduce the requirements. I would try to 
>>write a technical proposal in ML soon. The 
>>slides for presentation take me time due to my native language.
>>
>>Â·Â¢Â¼Ã¾ÃˆÃ‹: Bob Briscoe [ mailto:[email-masked]]
>>Â·Â¢Ã‹ÃÃŠÂ±Â¼Ã¤: 2010Ã„Ãª7Ã”Ã‚14ÃˆÃ• 17:55
>>ÃŠÃ•Â¼Ã¾ÃˆÃ‹: Zhu Lei
>>Â³Ã‹Ã: [email-masked]
>>Ã–Ã·ÃŒÃ¢: Re: [conex] An individual for your review
>>
>>Zhu,
>>
>>data plane congestion notification technology 
>>is not part of the ConEx charter. So, those 
>>notification channels from the respective 
>>access networks of the sender S and receiver R 
>>directly to S and R. I believe the proposal is 
>>currently agnostic to whether ECN_R is in-band 
>>I agree with both statements, but there seems 
>>to be a misconception in the document about 
>>what it means for a router, switch or middlebox to 'not support' ECN.
>>
>>do not do ECN marking. If both endpoints 
>>support ECN, they should be able to exchange 
>>packets between themselves that are ECN-capable 
>>(with an ECN-capable transport or ECT codepoint 
>>set) whatever network support there is for ECN between them.
>>
>>3) crashes with ECN (a bug): an endpoint 
>>attempt to negotiate ECN in the TCP header crashes a middlebox
>>
>>flow to Not-ECT in order to get packets 
>>through. There are black-hole detection patches 
>>for Linux, but black-hole detection is not 
>>the client end of TCP connections in Windows 
>>and Linux because of this single popular bugged 
>>home gateway. It should be possible to do some 
>>The proposal seems to be focused particularly 
>>on 3GPP, where AFAIK there are no ECN 
>>deployment problems that are not under the 
>>control of the network operators who could upgrade and fix them quickly.
>>
>>ECN would encounter as many deployment 
>>problems, if not more, than persevering with 
>>would require a complicated security 
>>association between the SQ message and the packet stream it refers to.
>>>We already had some discussions which may 
>>>allow wireless access networks to notify congestions in mail list.
>>>
>>>I personally think that Conex charter is 
>>>asking some solutions by stating "However, the 
>>>CONEX WG will initially focus on one use case, 
>>>where the end hosts and the network that 
>>>contains the destination end host are

---

From: Zhu Lei <[email-masked]>
Date: Tue, 20 Jul 2010 15:22:19 +0800

Hi Bob,
 
I send this e-mail for clarfication of the solutions in the mind. 
 
I am try to make sure that the impact on existing solution is minimum, even no needs to change endpoints behaviors. The backward ECN is also not expected to be used. You may see the following text in slides for f2f meeting. Others related to background and statements to be concluded are more difficult. To be honest, I am thinking your suggestions in last e-mail and if we have a proper solution satisfy everybody.
  
•Highlight: 
§Negotiation of ECN only for local access network is kept to be OPEN for access network system; 
§Others(no changes): 
qUse  IPv4 TOS or IPv6 Traffic Class Octet to encode one of the ECN values 
qFor TCP, use two flags in the TCP header to signal the sender to reduce the amount of information it sends. 
q
•Impacts on existing solutions 
§No changes to existing ECN mechanism: 
qNo changes to endpoints, no changes to intermediate nodes. 
§NOTE: only impact on the endpoints and access networks which are proposed to notify localized congestion to endpoints. 

Best regards,
Zhu Lei

  _____  

发送时间: 2010年7月20日 2:41

Lei,

ï»¿ 
Hi,

I agree that the current design cannot use ECN in a flow of packets for which the sender is ECN-capable but the receiver is not.

But I don't believe it will make deployment any easier if we introduce backward ECN_S feedback. Then packets will have to signal whether the router should do forward or backward feedback. Because a router must not do both, otherwise it will double the amount of congestion signalled. We don't have room in the packet to switch between forward and backward feedback. And if we did take a new approach, it would require another decade to wait for routers to be deployed that understand this new feature.

If deployment is taking a long time, we need to understand *why* before proposing changes that will make deployment take even longer and be more complicated.

But most importantly, I listed problems with backward feedback, which ALL must to be solved first. And they are ALL very good reasons against backward feedback. So please address them all before taking this further.

The problem meet is how to know possibilities to receive congestion notification sent by access node(s) by UE. I personally want to oen it for access network(e.g. link layer) in Conex working group. 

The ConEx group is not chartered to work on changes to ECN. Chartering is a careful process that takes account of everyone's views and results in a precise set of work items. 

For example, a statement would be useful for people working in access network who know the ECN solution might be used to notifiy access network congestion only.

I wrote my my original review email very carefully to explain how ECN incremental deployment is designed to work (to save you reading RFC3168). There is nothing in the original ECN design to stop only access network nodes being ECN enabled and other network nodes not being ECN enabled. Indeed, that's how I would expect ECN to be deployed first - by enabling ECN on the nodes that are most likely to be congested.

Then, if there is any unexpected congestion (e.g. in the core), the endpoints will see the drop from that, as well as ECN markings from the access networks.

As I said, the problem (at least as stated in your draft) is already solved by the original design of ECN.

Bob

Best regards,
Zhu Lei

  _____  

å‘ä»¶äºº: Bob Briscoe [  <mailto:[email-masked]> mailto:[email-masked]] 
å‘é€æ—¶é—´: 2010å¹´7æœˆ16æ—¥ 19:46
æ”¶ä»¶äºº: Zhu Lei
æŠ„é€: [email-masked]
ä¸»é¢˜: Re: [conex] An individual for your review

Lei,

Bob

Hi Bob and all,

 

 
 
Best regards,
Lei

  _____  

Zhu,

Cheers

Bob

Hi all,

 

Best regards,
Lei Zhu

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 20 Jul 2010 18:05:08 +0100

Lei,

May-be I had better wait until your presentation 
- I don't understand the bullets without the extended words behind them.

But again, I don't think ConEx is the right place 
for this. If there's something to change in ECN 
that affects the IP layer, it would probably be done in tsvwg.

The main point of my original mail was that the 
problem you *seem* to be trying to solve is not a 
problem. So I would first like to see your 
statement of precisely what problem you are 
trying to solve, before discussing solutions to 
something that may not need solving.

Bob

At 08:22 20/07/2010, Zhu Lei wrote:
>I am try to make sure that the impact on 
>existing solution is minimum, even no needs to 
>change endpoints behaviors. The backward ECN is 
>also not expected to be used. You may see the 
>following text in slides for f2f meeting. Others 
>related to background and statements to be 
>concluded are more difficult. To be honest, I am 
>thinking your suggestions in last e-mail and if 
>we have a proper solution satisfy everybody.
>
>•Highlight:
>Â§Negotiation of ECN only for local access 
>network is kept to be OPEN for access network system;
>Â§Others(no changes):
>qFor TCP, use two flags in the TCP header to 
>signal the sender to reduce the amount of information it sends.
>q
>Â§No changes to existing ECN mechanism:
>Â§NOTE: only impact on the endpoints and access 
>networks which are proposed to notify localized congestion to endpoints.
>
>
>
>Best regards,
>Zhu Lei
>
>
>----------
>å‘é€æ—¶é—´: 2010å¹´7æœˆ20æ—¥ 2:41
>æ”¶ä»¶äºº: Zhu Lei
>æŠ„é€: [email-masked]
>>Ã¯Â»Â¿
>>I think that S must be ECN enabled in any 
>>cases. It is important for ECN_S knows it and 
>>is capable to notify access congestion by sending ECN marks with TCP respones.
>
>I agree that the current design cannot use ECN 
>in a flow of packets for which the sender is 
>ECN-capable but the receiver is not.
>
>did take a new approach, it would require 
>another decade to wait for routers to be 
>deployed that understand this new feature.
>
>first. And they are ALL very good reasons 
>against backward feedback. So please address 
>them all before taking this further.
>
>>
>>The problem meet is how to know possibilities 
>>to receive congestion notification sent by 
>>access node(s) by UE. I personally want to oen 
>>it for access network(e.g. link layer) in Conex working group.
>
>not being ECN enabled. Indeed, that's how I 
>would expect ECN to be deployed first - by 
>enabling ECN on the nodes that are most likely to be congested.
>
>Then, if there is any unexpected congestion 
>(e.g. in the core), the endpoints will see the 
>drop from that, as well as ECN markings from the access networks.
>
>>Ã¥Ââ€˜Ã¤Â»Â¶Ã¤ÂºÂº: Bob Briscoe [ mailto:[email-masked]]
>>Ã¥Ââ€˜Ã©€ÂÃ¦—Â¶ÃÂ¶Ã©—Â´: 2010Ã¥Â¹Â´7Ã¦œˆ16Ã¦—Â¥ 19:46
>>Ã¦â€Â¶Ã¤Â»Â¶Ã¤ÂºÂº: Zhu Lei
>>Ã¦Š„Ã©€Â: > [email-masked]
>>Ã¤Â¸Â»Ã©Â¢˜: Re: [conex] An individual for your review
>>
>>Lei,
>>
>>band is that, by definition in-band means 
>>within the packets travelling from S to R.
>>
>>the route of packets between R and S, which 
>>need not follow the reverse route of the forward path between S and R.
>>
>>>major difference is that the ECN_S could be 
>>>not out-of-band. And, I also don't think that 
>>>the ICMP is the approach. I personlly want the 
>>>Ã‚Â·Ã‚Â¢Ã‚Â¼ÃƒÂ¾ÃƒˆÃƒâ€¹: Bob Briscoe [ mailto:[email-masked]]
>>>Ã‚Â·Ã‚Â¢Ãƒâ€¹ÃƒÂÃƒŠÃ‚Â±Ã‚Â¼ÃƒÂ¤: 2010Ãƒ„ÃƒÂª7Ãƒâ€Ãƒ‚14ÃƒËÃƒË†Ãƒ• 17:55
>>>ÃƒŠÃƒ•Ã‚Â¼ÃƒÂ¾ÃƒˆƒË†Ãƒâ€¹: Zhu Lei
>>>Ã‚Â³Ãƒâ€¹ÃƒÂ: [email-masked]
>>>Ãƒ–ÃƒÂ·ÃƒŒÃƒÂ¢:¢: Re: [conex] An individual for your review
>>>
>>>Zhu,
>>>
>>>If I can summarise your proposal, I believe 
>>>the main proposed new requirement is that we 
>>>should work on a congestion notification 
>>>packets between themselves that are 
>>>ECN-capable (with an ECN-capable transport or 
>>>ECT codepoint set) whatever network support there is for ECN between them.
>>>
>>>ECN was designed so that network equipment 
>>>that has not been upgraded to mark the ECN 
>>>field during congestion would silently forward 
>>>ECN-capable packets when not congested and 
>>>drop them during congestion - no different to 
>>>2) prevents ECN (a bug): middlebox tampers 
>>>with endpoint attempts to send ECN-capable packets. Three known sub-cases:
>>>this, but tests showed their rules got 
>>>* It is however still a problem with about 
>>>five popular models of home gateway from those 
>>>days that will take some time to be discarded from most homes.
>>>* Recently, a new problem has appeared on 
>>>paths that were previously OK. It is thought 
>>>this may be a buggy attempt to zero the 
>>>Diffserv codepoint, which is also accidentally 
>>>clearing the ECN field in the IP header.
>>>
>>>through. There are black-hole detection 
>>>patches for Linux, but black-hole detection is 
>>>not configured by default with the mainline code. The
>>>
>>>and Linux because of this single popular 
>>>bugged home gateway. It should be possible to 
>>>do some heuristic test to detect if the 
>>>offending model is on your home network 
>>>(perhaps using uPNP or checking whether the 
>>>MAC address falls within a known range). But 
>>>AFAIK, no code is available to do this.
>>>
>>>quench messages. SQ was deprecated for a 
>>>creation and sending of an ICMP packet, 
>>>without hugely re-working the implementation architecture of most routers.
>>>>asking some solutions by stating "However, 
>>>>the CONEX WG will initially focus on one use 
>>>>case, where the end hosts and the network 
>>>>that contains the destination end host are 
>>>>ask you to review this short document and 
>>>>help identify the problems which could be resolved in Conex wg. Thank you!
>>>>