Subject: Re: [ledbat] MIN_CWND, was: ECN/AQM, was: 2 week last call for draft-ietf-ledbat-congestion-06

From: Arjuna Sathiaseelan <[email-masked]>
Date: Fri, 1 Jul 2011 11:03:03 +0100

Dear Lachlan - thanks for your reply..

> No.  This confuses "better" with "gives a higher throughput to this
> user".  The whole point of congestion control is to do what is good
> for the network rather than maximizing the flow's own throughput.
> Throwing away information that the network is currently congested is
> not a *better* response.

Ofcourse I agree with you - that was the whole point of CC..However we
still have the liberty to perform actions (for e.g. opening multiple
TCP connections, or stopping and starting a new connection when we
find the Internet is too slow in responding etc)..And they all seem to
be legitimate actions  although not good for the network..App
designers want better performance for their apps - so they end up
thinking of new ways of circumventing the law of the Internet CC..and
their actions sound legitimate within the realm. Bob and others have
been trying to say this for the past few years - that there is no real
notion of fairness in the Internet today and Matt Mathis (who came out
with the notion of TCP-friendliness) agrees with it now.

So my belief is that next generation TCP could have a more aggressive
approach providing incentive for app designers but at the same time
being more reactive to congestion. Coupled with mechanisms such as
ECN, CONEX etc they could be effective..

But when it comes to LEDBAT, we cannot take this approach. We need to
be more conservative..

Regards
Arjuna

---

From: Arjuna Sathiaseelan <[email-masked]>
Date: Fri, 1 Jul 2011 11:06:43 +0100

Dear Jana,
  I really think introducing the notion of a timer would be a great
idea to mitigate some of the problems..I am happy to discuss this.

Regards
Arjuna

On 30 June 2011 23:32, Janardhan Iyengar <[email-masked]> wrote:
> Arjuna,
>
> Thank you for raising this question --- it is an important one that we
> missed but, IMHO, need to resolve.  LEDBAT does not currently account for
> rate reduction in the case of extreme congestion, as both TFRC and TCP do.
>  A couple of thoughts offhand:
>
> -  Under extreme congestion where TCP would have timed-out and TFRC would
> have backed-down significantly, LEDBAT will only halve its sending rate.  If
> LEDBAT gets a good delay measurement, then the sender _could_ set its
> congestion window to 0, but what do we do when the congestion is extreme and
> detection is loss-based?
>
> -  There is no notion of a timer in LEDBAT so far.  If we decide to
> introduce a timer, we could use the current RTT estimate to seed it, or go
> with a fixed 1 second as the initial seed time, with a backoff of the timer
> on a subsequent loss.  We could then use the timer to decide how long to go
> idle before sending out the next packet.
>
> I'll try to hash this out some more and come up with something useful
> tonight.
> - jana
>
>
> On 6/30/11 1:23 PM, Arjuna Sathiaseelan wrote:
>>
>> Dear Mirja,
>>   We need to think about this..The current nature of transport
>> protocols seem to be overly conservative to me - for e.g. in the
>> presence of a RTO, the ssthresh can reduce to a very low value and it
>> can take ages for a cwnd to grow - so it would be better to stop and
>> start a new TCP connection! So based on such a theory, we could say
>> why not use two packets/RTT during severe congestion - because this is
>> no worse than a LEDBAT starting a new connection with two packets/RTT.
>> However, the problem here is the lower than best effort nature of
>> these methods and I dont think we can go in the direction of ICCRG. So
>> probably we need to stick to conservative methods.
>>
>> IMHO - I think the best option for LEDBAT during severe congestion is
>> not to send any data (thats the reason why I asked you the question
>> about timers and timeouts in my earlier email). However I also agree
>> that if we dont that we have no way of figuring out the necessary
>> metrics for LEDTBAT without sending any data.
>>
>> So howabout a restart mechanism - when the cwnd becomes 0 - can LEDBAT
>> be allowed to try transmitting one or two packets every second  - and
>> if this is successful (noting the feedback indicates there is no
>> congestion - based on the observed delay samples) then it can follow
>> the normal slow-start procedure? If it still notices congestion, it
>> stops sending for another second and restarts again..and so on..
>>
>> Just thoughts..
>>
>> Regards
>> Arjuna
>>
>>
>>
>> On 30 June 2011 12:17, Mirja Kuehlewind
>> <[email-masked]>  wrote:
>>>
>>> Hi Arjuna, hi Gorry,
>>>
>>> I was not aware that there is any recommendation to send only one packet
>>> per
>>> (up to) 64 sec. That seems a very high value for me. It actually means
>>> you
>>> are not sending any data for 64sec no matter what is happening in this 64
>>> sec
>>> (the link might be empty already after one second)...?
>>>
>>> The recommendation of 2 packet per RTT was because we need to monitor
>>> changes
>>> in one-way delay with LEDBAT. That's why we though there should be at
>>> least
>>> two samples per RTT.
>>>
>>> Thinking about it now, you can even get measurement samples by sending
>>> packets
>>> without any data. But that makes the description more complicated.
>>>
>>> One more remark: The draft says "MIN_CWND SHOULD be 2". That means if
>>> there is
>>> a good reason (in certain scenarios or with a certain framing protocol),
>>> another value can be used. In case of allowing a CWND of 0, there has to
>>> some
>>> mechanism to restart the transmission. We did not regard such as
>>> mechanism as
>>> part of the LEDBAT algorithm (so far).
>>>
>>> How should be proceed here now?
>>>
>>> Mirja
>>>
>>>
>>> On Wednesday 29 June 2011 20:36:42 Arjuna Sathiaseelan wrote:
>>>>
>>>> Dear Mirja,
>>>>   I am not sure whether cwnd of 2 packets/RTT during periods of heavy
>>>> congestion is a good response - because other methods support lower
>>>> rates for e.g TFRC would end up at a rate of 1 packet every 64 seconds
>>>> or so. So I am wondering whether its possible to reduce the cwnd to a
>>>> more lower rate as a response to sustained congestion.
>>>>
>>>>
>>>> I agree with Bob and Gorry that allowing LEDBAT to support ECN is not
>>>> going to hurt.
>>>>
>>>> Regards
>>>> Arjuna
>>>>
>>>> On 29 June 2011 12:39, Mirja Kuehlewind
>>>>
>>>> <[email-masked]>  wrote:
>>>>>
>>>>> Hi Arjuna,
>>>>>
>>>>> the minimum cwnd windows is 2 (MIN_CWND). That means LEDBAT will send
>>>>> at
>>>>> least two 2 packets per RTT (which is 'basically' zero). LEDBAT will
>>>>> not
>>>>> wait for any timers here but send continuously data with a very, very
>>>>> slow rate.
>>>>>
>>>>> Mirja
>>>>>
>>>>> On Tuesday 28 June 2011 21:28:12 Arjuna Sathiaseelan wrote:
>>>>>>
>>>>>> Dear Mirja,
>>>>>>   Sorry- I am a newcomer to LEDBAT - and I am trying to understand
>>>>>> this. So when you say LEDBAT reduces the sending rate to zero - does
>>>>>> that mean LEDBAT doesnt have the notion of timers/timeouts? So if
>>>>>> LEDBAT knows that the queues are being utilised could it wait for any
>>>>>> amount of time without sending data?
>>>>>>
>>>>>>> And when LEDBAT is working correctly, there shouldn't occur any ECN
>>>>>>> marking (and of course no losses neither) as LEDBAT will reduce it's
>>>>>>> sending rate to basically zero before any queues overflow.
>>>>>>
>>>>>> Regards
>>>>>> Arjuna
>>>>>
>>>>> --
>>>>> -------------------------------------------------------------------
>>>>> Dipl.-Ing. Mirja Kühlewind
>>>>> Institute of Communication Networks and Computer Engineering (IKR)
>>>>> University of Stuttgart, Germany
>>>>> Pfaffenwaldring 47, D-70569 Stuttgart
>>>>>
>>>>> tel: +49(0)711/685-67973
>>>>> email: [email-masked]
>>>>> web: www.ikr.uni-stuttgart.de
>>>>> -------------------------------------------------------------------
>>>
>>>
>>>
>>> --
>>> -------------------------------------------------------------------
>>> Dipl.-Ing. Mirja Kühlewind
>>> Institute of Communication Networks and Computer Engineering (IKR)
>>> University of Stuttgart, Germany
>>> Pfaffenwaldring 47, D-70569 Stuttgart
>>>
>>> tel: +49(0)711/685-67973
>>> email: [email-masked]
>>> web: www.ikr.uni-stuttgart.de
>>> -------------------------------------------------------------------
>>>
>> _______________________________________________
>> ledbat mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/ledbat
>
> --
> Janardhan Iyengar
> Assistant Professor, Computer Science
> Franklin & Marshall College
> http://www.fandm.edu/jiyengar
>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 01 Jul 2011 12:30:34 +0100

Lachlan,

I'm trying to work out the mental model you have that leads to this 
conclusion. Perhaps you are thinking ECN happens at a shallower point 
in the queue than loss?

I was also thinking the response to ECN could be less - until I 
thought it through a couple of days ago. I posted the logical 
progression on this list which concluded that LEDBAT has to respond 
the same to ECN and to loss.

See <STEP_BY_STEP_LOGIC> within 
<http://www.ietf.org/mail-archive/web/ledbat/current/msg00479.html>

As you are proposing otherwise, if you say where you disagree with 
that logical progression we might be able to get to the bottom of this?

Bob

At 05:47 01/07/2011, Lachlan Andrew wrote:
>Greetings
>
>On 1 July 2011 14:00, Janardhan Iyengar <[email-masked]> wrote:
> >
> >> I haven't been following Letbat, and this proposal may be heretical,
> >> but it is easy to give a Letbat flow an incentive to be ECN capable,
> >> by specifying that it should back off more when it sees loss than when
> >> it sees ECN.
> >
> > That seems like an artificial incentive.  Why should a LEDBAT flow reduce
> > its rate more than what TCP does on loss?
>
>No -- I suggested there could/should be a difference between how we
>respond to loss and to ECN.  I didn't mention TCP.
>
> >> As long as it backs off at least as much as Reno does on loss, then
> >> this does not violate the spirit of RFC 3168.  It does violate the
> >> requirement that response to ECN be the same as loss, but that was
> >> written at a time when loss was considered synonymous with congestion;
> >> really ECN should be treated more like extra delay for a delay-based
> >> algorithm.
> >
> > So, if I understand you correctly, your suggestion is not so much 
> to _reduce
> > more_ on loss, but to _reduce lesser_ on CE.
>
>Yes, they're equivalent.
>
> > While the idea seems
> > intriguing, what you are suggesting seems to be a rewrite of RFC 3168.
>
>Not at all.  Most of 3168 is about the mechanism.  That doesn't need
>to change.  I'd only challenge the "congestion is equivalent to loss"
>prescription.
>
> > The
> > receipt of CE signal is an event that may not be accompanied by any clear
> > delay increase;  it is not clear to me how a CE received in the 
> absence of a
> > clear delay signal can be interpreted as extra delay and not loss.
> > In other words, how do you translate a CE signal to delay?
>
>You don't need to "translate a CE signal to delay".  My point was just
>that you can treat it as another congestion signal whose severity is
>less than loss.
>
> > Isn't the equivalent argument for TCP-like flows to reduce their windows by
> > lesser than than 1/2 on CE marks and by 1/2 on loss  (or am I missing
> > something)?
>
>No.  It is more equivalent to the argument that it should be *OK* for
>TCP-like flows to reduce their windows by 1/2 on CE marks, and by
>*more* than 1/2 on loss (if they choose to).
>
>The requirement that CE marks be treated as loss was (presumably) to
>prevent flows from using it as a way to be more aggressive than
>standard TCP / TFRC.  Ledbat flows are already being much less
>aggressive than Reno.  There is no logical reason that something that
>backs of more timidly on loss should also back off more timidly on CE
>(although there is a logical reason that something should back off on
>CE by at least as much as "the universal congestion control when ECN
>was introduced" backed off on "the universal congestion signal before
>ECN was introduced").
>
>The primary argument in RFC 3168 for making the response to ECN
>essentially the same as that to a loss was to allow incremental
>deployment of 3168.  If making the response identical actually
>*stifles* the deployment of ECN by prohibiting the correct incentives,
>then perhaps the wording of 3168 should be fixed...
>
>Cheers,
>Lachlan
>
>--
>Lachlan Andrew  Centre for Advanced Internet Architectures (CAIA)
>Swinburne University of Technology, Melbourne, Australia
><http://caia.swin.edu.au/cv/landrew>
>Ph +61 3 9214 4837

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 01 Jul 2011 18:40:12 -0400

Bob,

Here's what I'm adding in the Applicability Section; please let me know what you think.

"
    When used with an ECN-capable framing protocol, LEDBAT should react
    to an ECN mark as it would to a loss, as specified in [RFC3168].

    LEDBAT is designed to reduce build-up of a standing queue by long-
    lived LEDBAT flows at a link with a tail-drop FIFO queue, so as to
    avoid persistently delaying other flows sharing the queue.  If Active
    Queue Management (AQM) is configured to drop or ECN-mark packets
    before the LEDBAT starts reacting to persistent queue build-up,
    LEDBAT reverts to TCP behavior, rather than yield to other TCP flows.
    However, such an AQM is still desirable since it keeps queuing delay
    low, achieving an outcome that is in line with LEDBAT's goals.
    Additionally, a LEDBAT transport that supports ECN enjoys the
    advantages that an ECN-capable TCP enjoys over an ECN-agnostic TCP;
    avoiding losses and possible retransmissions.  Weighted Fair Queuing
    (WFQ), as employed by some home gateways [Schneider10],  seeks to
    isolate and protect delay-sensitive flows from delays due to
    standing queues built up by concurrent long-lived flows.
    Consequently, while it prevents LEDBAT from yielding to other flows,
    it again achieves an outcome that is in line with LEDBAT's
    goals [Schneider10].

    Further study is required to fully understand the behaviour of LEDBAT
    with non-drop-tail, non-FIFO queues.
"

Thanks,
- jana

On 6/30/11 8:08 PM, Bob Briscoe wrote:
> Janar,
>
> I take Mirja's point that this isn't the right doc to recommend that the LEDBAT transport uses ECN, it's solely appropriate to say how LEDBAT should respond to ECN CE.
>
> Earlier revs of this draft said a design goal was to use ECN, but that got removed. The applicability section currently says:
> "
> LEDBAT seeks to operate in networks with FIFO queues and with tail-
> drop queue management. Further study is required to understand the
> implications of Active Queue Management (AQM) schemes on LEDBAT
> mechanisms.
> "
> Thinking about this, it's rather oddly worded. LEDBAT doesn't know what sort of queues there will be where it "seeks to operate".
>
> How about saying something like the following:
>
> "LEDBAT has been designed so that long-running flows should not build a standing queue that would persistently delay TCP flows that share a relatively slow FIFO queue using tail drop.
>
> If active queue management (AQM) is configured to start dropping or ECN-marking packets before the averaged queue has reached LEDBAT's target delay, it will make LEDBAT revert to TCP behaviour, rather than yield to other TCP flows. However the AQM will keep the queuing delay low, which achieves an outcome similar to that LEDBAT was aiming for. Additionally, if the LEDBAT transport supports ECN, it will enjoy the same minor advantages that an ECN-TCP enjoys over a non-ECN-TCP; avoiding losses and the consequent timeouts.
>
> Fair queuing will prevent LEDBAT yielding to other TCP flows. Nonetheless, fair queueing isolates brief TCP flows from any standing queue caused by concurrent long-running flows, which again achieves an outcome that is not much worse than that LEDBAT was aiming for [Schneider10].
>
> Further study is required to better fully understand the behaviour of LEDBAT with non-drop-tail non-FIFO queues.
> "
>
>
> [Schneider10] Joscha Schneider. Joerg Wagner, Rolf Winter , Hans-Joerg Kolbe, "Out of my Way -- Evaluating Low Extra Delay Background Transport in an ADSL Access Network" . In Proceedings of the 22nd International Teletraffic Congress (ITC22), Amsterdam, The Netherlands, September 7 - 9 2010.
> <http://www.i-teletraffic.org/fileadmin/ITCBibDatabase/2010/schneider10.pdf>
>
>
> At 16:43 30/06/2011, Janardhan Iyengar wrote:
>> Dear all,
>>
>> There is no question that LEDBAT flows will experience loss in the wild, especially, as Bob so eloquently pointed out, when congestion is deeper in the core.
>>
>> It seems to me that we are all in agreement that ECN marks must be respected. I don't think that the LEDBAT draft can do more than state what a LEDBAT flow does in the presence of an ECN mark, as Mirja has already included in the Applicability Section. After all, LEDBAT does not get into protocol mechanisms per se, and cannot really _specify_ anything more with ECN.
>>
>> For LEDBAT to actively "encourage" ECN adoption, there should be some explicit benefit to the ECN-capable LEDBAT flows over the non-ECN capable ones. And within the scope of the draft, I don't see that we can point to any such benefits. However, perhaps we could say something in the applicability section about how ECN and AQM can benefit the applications that we expect to use LEDBAT. Any suggestions?
>>
>> - jana
>>
>> On 6/30/11 7:41 AM, Mirja Kuehlewind wrote:
>>> Hi everybody,
>>>
>>> I try to recap now on the ECN issue to see which action is actually needed for
>>> the draft. (Btw. I believe we DO have the same opinion):
>>>
>>> There are two question:
>>>
>>> 1. Should LEDBAT react on ECN marks as it would on loss?
>>>
>>> A very clear YES here! Nobody wants to ignore ECN marks. The reaction to ECN
>>> is already specified in RFC3168 to be the same than to loss (for every
>>> congestion control). This was not mention explicit in the current draft
>>> version. I've been adding the following sentence for the next version (in
>>> 2.2. Applicability):
>>>
>>> "If LEDBAT is used with a framing protocol that is
>>> ECN-capable, it should react to ECN marks as it would to losses
>>> as specified in RFC3168."
>>>
>>>
>>> 2. Should we recommend that transports using LEDBAT should use/enable ECN
>>> as well?
>>>
>>> Here are the responses of everybody:
>>>
>>> Bob:
>>> "Sending ECN-capable packets ought to be recommended for LEDBAT, so
>>> that if ever ECN is deployed in the network, LEDBAT can take advantage of it."
>>>
>>> Gorry:
>>> "To be clear, I think LEDBAT should indicate that ECN is a useful
>>> mechanism for congestion control, and that instantiations of the LEDBAT
>>> method in transports should be encouraged to support ECN functionality."
>>>
>>> Arjuna:
>>> "I agree with Bob and Gorry that allowing LEDBAT to support ECN is not
>>> going to hurt."
>>> Agree, ECN does not hurt...
>>>
>>> I (Mirja):
>>> -> I was saying that this question does not need to be answered in this draft,
>>> as this draft is not talking about ANY framing protocol issues (beside some
>>> hard requirement on the framing protocol e.g. any kind of feedback is need).
>>> It only documents the LEDBAT algorithm for informational purpose (its not a
>>> protocol spec nor an implementation guide).
>>>
>>> How should we continue here? Should we add some text? If yes, where should we
>>> add the text? Somebody wants to make a proposal for text?
>>>
>>> Mirja
>>>
>>>
>>> On Thursday 30 June 2011 10:40:21 Gorry Fairhurst wrote:
>>>> Indeed I missed a "NOT".
>>>>
>>>> To be clear, I think LEDBAT should indicate that ECN is a useful
>>>> mechanism for congestion control, and that instantiations of the LEDBAT
>>>> method in transports should be encouraged to support ECN functionality.
>>>>
>>>> Gorry
>>>>
>>>> On 29/06/2011 20:02, Bob Briscoe wrote:
>>>>> Gorry,
>>>>>
>>>>> At 17:44 29/06/2011, Gorry Fairhurst wrote:
>>>>>> On 28/06/2011 19:25, Mirja Kuehlewind wrote:
>>>>>>> So in general, I'm not sure if we should talk about an TCP/IP
>>>>>>> mechanism (ECN)
>>>>>>> in this draft when we only specify the algorithm and not talking about
>>>>>>> anything TCP specific...?
>>>>>>
>>>>>> I'm not saying the group should define the mechanism, but I am saying
>>>>>> that from my point of view, it would be wise that ECN is ignored, and
>>>>>> that what matters is proper response to "congestion indications", not
>>>>>> to loss.
>>>>>
>>>>> Did you miss a 'not' in the sentence?
>>>>> "it would be wise that ECN is NOT ignored"
>>>>>
>>>>>
>>>>> Bob
>>>>>
>>>>>>> Mirja
>>>>>>
>>>>>> Gorry
>>>>>
>>>>> Bob Briscoe, BT Innovate& Design
>>>
>>>
>>
>> --
>> Janardhan Iyengar

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 01 Jul 2011 20:47:51 -0400

Hi Lachlan,

I think I understand (and am sympathetic to) your general point -- having a weaker congestion response to an ECN mark than to a loss seems intriguing.  That said ...

> *more* than 1/2 on loss (if they choose to).

.. this part I don't understand.  This seems like an artificial incentive.  Why should TCP-like flows choose to reduce by more than 1/2 on loss, when there's no way of knowing whether ECN marking is turned ON at the point of congestion?

> aggressive than Reno.
>There is no logical reason that something that

LEDBAT does not back off more timidly on loss.  It currently behaves as TCP (i.e., reduce rate by 1/2) when loss is experienced.  Caveat: It is possible that the LEDBAT reduction is ultimately more than TCP's under the same circumstances due to cumulative reduction with delay and loss *if the LEDBAT sender witnesses a delay increase first*; with a rate reduction due to the increasing queue, and then a cutting-back by 1/2 when loss is experienced.

> then perhaps the wording of 3168 should be fixed...

I think I see how have different reactions might set up the right incentives for deployment of network-wide ECN... however, endpoints cannot start with the assumption of ECN in the network.  Expecting endpoints to reduce by more than 1/2 on loss seems like an artificial incentive that might get easily ignored in implementations.

But then again, that seems like an update to 3168, and not just to LEDBAT.

- jana

---

From: Lachlan Andrew <[email-masked]>
Date: Sat, 2 Jul 2011 10:55:24 +1000

Greetings Bob,

Our point of divergence is probably step 1 in your argument.

I was still thinking of LEDBAT as less than best effort.  If it
(permanently) reverts to standard TCP on loss, then it is not longer
less than best effort, since LEDBAT flows will typically be long and
experience at least one loss before most of the transfer is complete.

However, if it only *temporarily* reverts to standard TCP, then step 1
of the argument is misleading.  ECN and loss can differ in how long
they cause the LEDBAT flow to remain aggressive.  If ECN causes it to
remain aggressive longer than loss does, then there is an incentive to
enable ECN (if throughput gives an incentive).  If higher throughput
is not an incentive (that's why we're not using TCP), then smoothness
may be an incentive; ECN could cause a more mild immediate backoff but
revert more quickly to non-TCP behaviour.

You may say that these alternatives are too complex to be worthwhile.
If so, I'd probably agree.  However, I'm arguing that it is *possible*
to give an incentive to use ECN without breaking other design goals.

Cheers,
Lachlan

On 1 July 2011 21:30, Bob Briscoe <[email-masked]> wrote:
> conclusion. Perhaps you are thinking ECN happens at a shallower point in the
> queue than loss?
>
> I was also thinking the response to ECN could be less - until I thought it
> through a couple of days ago. I posted the logical progression on this list
> which concluded that LEDBAT has to respond the same to ECN and to loss.
>
> As you are proposing otherwise, if you say where you disagree with that
> logical progression we might be able to get to the bottom of this?

---

From: Lachlan Andrew <[email-masked]>
Date: Sat, 2 Jul 2011 11:21:14 +1000

Greetings Jana,

On 2 July 2011 10:47, Janardhan Iyengar <[email-masked]> wrote:
>>
>> It is more equivalent to the argument that it should be *OK* for
> .. this part I don't understand.  This seems like an artificial incentive.
>  Why should TCP-like flows choose to reduce by more than 1/2 on loss, when
> there's no way of knowing whether ECN marking is turned ON at the point of
> congestion?

The reason would be to be less-than-best-effort.

> LEDBAT does not back off more timidly on loss.  It currently behaves as TCP
> (i.e., reduce rate by 1/2) when loss is experienced.  Caveat: It is possible
> that the LEDBAT reduction is ultimately more than TCP's under the same
> circumstances due to cumulative reduction with delay and loss *if the LEDBAT
> sender witnesses a delay increase first*; with a rate reduction due to the
> increasing queue, and then a cutting-back by 1/2 when loss is experienced.

Yes.  I would argue that the Caveat should be the "usual" case when a
TCP flow happens to use the same low-capacity bottleneck.  Treating
ECN differently in that single case is enough to give an incentive to
enable it, even if the response to a loss in the core/distribution
network is the same.

> I think I see how have different reactions might set up the right incentives
> for deployment of network-wide ECN... however, endpoints cannot start with
> the assumption of ECN in the network.  Expecting endpoints to reduce by more
> than 1/2 on loss seems like an artificial incentive that might get easily
> ignored in implementations.

If the RFC says to do it, then an implementer who wants to be
RFC-compliant will do it.  An implementer who doesn't want to be RFC
compliant could choose to ignore any other part of the spec too.  If
you mean it might be *overlooked* in an implementation, then that
depends on how the text is written.

The point is that, as long as the response to ECN is not more
aggressive than TCP's, then you are free to specify whatever response
to loss you like. (Even if the immediate response is halving the
window, the whole response includes when to become more timid again.
That can differ for loss and ECN, as I said in my reply to Bob.)

> But then again, that seems like an update to 3168, and not just to LEDBAT.

True.  A requirement that ECN be treated *identically* to loss
basically prohibits incentives to use ECN.  Perhaps I'm arguing for a
3168-compatible spec, rather than a 3168-compliant one.  As long as
LEDBAT remains no more aggressive than TCP, we know it won't cause the
Internet to melt down.

Cheers,
Lachlan

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 01 Jul 2011 21:37:38 -0400

Bob, Lachlan,

(Lachlan, I'd like to hear what you have to say too)

Even if the ECN happened at a shallower point in the queue than loss, wouldn't a lesser reduction imply that the LEDBAT flow backs off lesser than a competing ECN-capable TCP, in the absence of a delay signal?

So, a question that I pointed out in my earlier email (and I wonder, Lachlan, if this is what you had in mind) -- how should a LEDBAT flow reduce its rate on ECN or loss, if it does observe a queueing delay increase preceding that event?  The current LEDBAT mechanism would be more aggressive in its reaction to the congestion "event" than TCP.  We could simply say that LEDBAT ought to be more conservative anyways....  that would be simplest.  But can we do better?

- jana

On 7/1/11 7:30 AM, Bob Briscoe wrote:
> Lachlan,
>
> I'm trying to work out the mental model you have that leads to this conclusion. Perhaps you are thinking ECN happens at a shallower point in the queue than loss?
>
> I was also thinking the response to ECN could be less - until I thought it through a couple of days ago. I posted the logical progression on this list which concluded that LEDBAT has to respond the same to ECN and to loss.
>
> See <STEP_BY_STEP_LOGIC> within <http://www.ietf.org/mail-archive/web/ledbat/current/msg00479.html>
>
> As you are proposing otherwise, if you say where you disagree with that logical progression we might be able to get to the bottom of this?
>
>
> Bob
>
>> > So, if I understand you correctly, your suggestion is not so much to _reduce
>> > delay increase; it is not clear to me how a CE received in the absence of a

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 01 Jul 2011 23:10:42 -0400

Hi Lachlan,

I understand now what your proposal is.   More inline.

> experience at least one loss before most of the transfer is complete.

No, LEDBAT does not revert to TCP per se, it only reduces its rate by 1/2 on loss.  That is, a "temporary revert".

> revert more quickly to non-TCP behaviour.

So, this is interesting, and it certainly warrants considering ideas that may not be too complex.  One that comes to mind immediately relates to a point I made earlier:  currently, LEDBAT will end up getting a  cumulative reduction from increasing delay and ECN/loss when both are observed from the same congestion event.  We could consider leaving it as such with loss---that is our pathological case after all---but we could be a bit more careful with ECN.

Since LEDBAT's cwnd decreases with increasing queueing delay, a LEDBAT sender could track the highest cwnd within the past RTT (high_cwnd).
On loss: the sender simply reduces its cwnd to curr_cwnd/2.
On ECN: the sender reduces its window to min(curr_cwnd, high_cwnd/2).

This makes it so that if there is no delay increase observed (as Bob suggests, in the core), the new cwnd should be roughly equivalent for both loss and ECN.  If there is a delay increase, however, LEDBAT behaves conservatively under loss and a bit more experimentally with ECN.

This seems safe enough, does not seem too complex, and certainly provides the incentive we're looking for.  What do others think?  Is this an avenue worth pursuing?

- jana

---

From: Murari Sridharan <[email-masked]>
Date: Sat, 2 Jul 2011 09:47:22 +0000

Just caught up on the thread after my vacation. I think it's a bad idea to fudge modified responses to ECN this late in the game into Ledbat. It is largely speculative and clearly lacks experimentation and data to justify our decisions. 

There are several cases here as discussed by others that we need to be careful about. Depending on how low the marking threshold is ledbat may never observe queue build ups that are reaching its target. If other on the bottleneck are reacting but ledbat doesn't that could be an issue, however switches when they are congested (per marking threshold)  tend to drop packets that dont have the ECT code point set, especially for IP traffic (meaning it may allow ARP etc). It may very well be that ledbat encounters more packet loss compared to ECN capable transports that are "protected" as marking can be done on them without necessarily dropping them. If we think we can't rely on such switch behavior then we may have a problem. However I'd go back to the fact that ledbat isn't a protocol and simply an algorithm on how to achieve lower than best effort. So my vote is to strongly keep any recommendations in text rather than changing the algorithm in any way. 

Thanks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Janardhan Iyengar
Sent: Friday, July 01, 2011 8:11 PM
To: Lachlan Andrew
Cc: [email-masked]
Subject: Re: [ledbat] ECN/AQM

Hi Lachlan,

- jana

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 04 Jul 2011 00:04:12 +0100

Murari,

inline

At 10:47 02/07/2011, Murari Sridharan wrote:
>Just caught up on the thread after my vacation. I think it's a bad 
>idea to fudge modified responses to ECN this late in the game into 
>Ledbat. It is largely speculative and clearly lacks experimentation 
>and data to justify our decisions.
>
>There are several cases here as discussed by others that we need to 
>be careful about. Depending on how low the marking threshold is 
>ledbat may never observe queue build ups that are reaching its 
>target. If other on the bottleneck are reacting but ledbat doesn't 
>that could be an issue, however switches when they are congested 
>(per marking threshold)  tend to drop packets that dont have the ECT 
>code point set, especially for IP traffic (meaning it may allow ARP 
>etc). It may very well be that ledbat encounters more packet loss 
>compared to ECN capable transports that are "protected" as marking 
>can be done on them without necessarily dropping them. If we think 
>we can't rely on such switch behavior then we may have a problem.

Surely this is not a problem? RFC3168 ECN was designed knowing we 
can't rely on switches having implemented ECN. The transport gets 
drop signals instead of ECN from such switches, and responds the same 
to ECN or drop. No problem.

>However I'd go back to the fact that ledbat isn't a protocol and 
>simply an algorithm on how to achieve lower than best effort. So my 
>vote is to strongly keep any recommendations in text rather than 
>changing the algorithm in any way.

Sorry, don't understand this last sentence.

bob

>No, LEDBAT does not revert to TCP per se, it only reduces its rate 
>by 1/2 on loss.  That is, a "temporary revert".
>
>So, this is interesting, and it certainly warrants considering ideas 
>that may not be too complex.  One that comes to mind immediately 
>relates to a point I made earlier:  currently, LEDBAT will end up 
>getting a  cumulative reduction from increasing delay and ECN/loss 
>when both are observed from the same congestion event.  We could 
>consider leaving it as such with loss---that is our pathological 
>case after all---but we could be a bit more careful with ECN.
>
>Since LEDBAT's cwnd decreases with increasing queueing delay, a 
>LEDBAT sender could track the highest cwnd within the past RTT (high_cwnd).
>This makes it so that if there is no delay increase observed (as Bob 
>suggests, in the core), the new cwnd should be roughly equivalent 
>for both loss and ECN.  If there is a delay increase, however, 
>LEDBAT behaves conservatively under loss and a bit more 
>experimentally with ECN.
>
>This seems safe enough, does not seem too complex, and certainly 
>provides the incentive we're looking for.  What do others think?  Is 
>this an avenue worth pursuing?
>
>- jana
>
>--
>Janardhan Iyengar

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 04 Jul 2011 00:24:06 +0100

Janar,

There's a wider point against this.

I doubt developers of transports and apps will jump to use ECN more 
because the _IETF_ allows them more leeway. They will (rightly) only 
care about what leeway networks allow them. The IETF isn't in the 
equation at run-time.

The IETF is however seen as a source of advice on what is OK/safe/stable.

If the IETF says it's OK/stable/safe for LEDBAT to respond less to 
ECN, all that would say to an implementer is that it will also be 
OK/stable/safe for LEDBAT (or any transport for that matter) to 
respond less to losses as well.

Reason: losses and ECN marks are meant to be identical signs of the 
condition of a queue. So if the queue can cope with me responding 
less to ECN, it would have been able to cope with me turning off ECN 
and responding less to the equivalent loss.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 04 Jul 2011 00:39:16 +0100

Jana,

I can't get past the first 'if' without choking on it I'm afraid...

If ECN happened at a shallower point in the queue than loss, imagine 
what happens when ECN-capable and non-ECN-capable flows are sharing the queue.

When the queue reaches the shallower ECN threshold, an arriving ECN 
packet will be marked, but an arriving non-ECN packet will be 
enqueued without marking or loss.

WIthout any loss indications yet, the non-ECN flows will be able to 
drive the queue up to the deeper point where losses start. Now the 
queue is above the ECN operating point, every time an ECN packet 
arrives in such a queue, it gets marked with high probability.

So the ECN flows get lots of marks and back away. The non-ECN flows 
get few losses, so they continue to fill the space left by the ECN 
flows, continually trying to drive the queue to their loss-based 
operating point. Over time, this continues, ratchetting down all the 
ECN flows and ratchetting up all the non-ECN flows.

...Starving ECN flows isn't a good way to create incentives to use ECN.

Bob

At 02:37 02/07/2011, Janardhan Iyengar wrote:
>Even if the ECN happened at a shallower point in the queue than 
>loss, wouldn't a lesser reduction imply that the LEDBAT flow backs 
>off lesser than a competing ECN-capable TCP, in the absence of a delay signal?
>
>So, a question that I pointed out in my earlier email (and I wonder, 
>Lachlan, if this is what you had in mind) -- how should a LEDBAT 
>flow reduce its rate on ECN or loss, if it does observe a queueing 
>delay increase preceding that event?  The current LEDBAT mechanism 
>would be more aggressive in its reaction to the congestion "event" 
>than TCP.  We could simply say that LEDBAT ought to be more 
>conservative anyways....  that would be simplest.  But can we do better?
>
>- jana
>
>>conclusion. Perhaps you are thinking ECN happens at a shallower 
>>point in the queue than loss?
>>