Subject: [tcpm] New Draft: TCP Alternative Backoff with ECN (draft-khademi-alternativebackoff-ecn)

From: Naeem Khademi <[email-masked]>
Date: Mon, 13 Jul 2015 17:48:57 +0200

Hi all

Please note the new Internet draft "TCP Alternative Backoff with ECN (ABE)"
on https://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-00

We have been working to get better than current ECN and thus providing a
path for ECN deployment using a very simple sender-side modification (this
is not intended to compete with work on Accurate ECN) and only the existing
router ECN marking with the AQM mechanisms being deployed today.

Please read the draft, and send comments and questions to the list. Also
note that a technical report on ABE's evaluation is available at:
http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf

Cheers,
Naeem

---

From: Mark Allman <[email-masked]>
Date: Mon, 13 Jul 2015 14:19:02 -0400

> Please note the new Internet draft "TCP Alternative Backoff with ECN
> (ABE)" on
> https://tools.ietf.org/html/draft-khademi-alternativebackoff-ecn-00

I read this draft.  It doesn't hurt that the list of authors is the
biggest section in the document! :)

I have two thoughts ...

  - Rather than saying the backoff should be 0.8, why not just say
    the reaction should be whatever the reaction to loss is?  I.e.,
    why specify some number here?  The spirit of the original spec
    is that an ECN mark is the network saying "I should have dropped
    your packet, but I am being nice" and so the reaction should be
    the exact same as a loss.  The RFC doesn't exactly say that
    because at the time there was only one reaction and yeah I guess
    we should have had better foresight with the words, but we
    didn't.  But, why now not just leave it as "just as loss" and
    elide the specifics and hence avoid the mistake a second time?

  - Maybe instead of saying the backoff has to be just as it is for
    a loss, why not say the multiplier is (oh I dunno)
    1.1 * loss_multiplier.  So, if the backoff is usually cwnd *=
    0.7 on a loss then on an CE mark it is cwnd *= 0.77.  That
    provides a tangible incentive to use ECN (rather than the
    nebulous and dubious incentives in the original version).

Just two quick thoughts ... FWIW ...

allman

---

From: Naeem Khademi <[email-masked]>
Date: Mon, 13 Jul 2015 20:39:21 +0200

Hi Mark

Please see inline:

>     elide the specifics and hence avoid the mistake a second time?

An ECN mark is an *explicit* indication of an AQM presence on the
bottleneck link, whereas a packet loss simply is not!

ECN in term of routers doing CE-marking has not been used since its
introduction approx. 15 years ago, therefore a CE-mark today on the
Internet would *most likely* to indicate the presence of an AQM that is
trying to keep the latency low (in order of 5ms~20ms on the average for
(FQ_)CoDel and PIE as shipped) and therefore a beta_{ecn}=0.5 is too
conservative for such low thresholds. With a packet loss that's not the
case since packet loss may have been caused by many other factors such as
tail-loss, channel errors, etc.

>     nebulous and dubious incentives in the original version).

Assuming tail-loss, the maximum size of the buffer (which produces
tail-loss) has little to do with the AQM threshold being deployed on that
buffer so they are quite unrelated. We have (almost) clear indications of
where (at which thresholds) CE-marking happens with the current AQMs. With
loss, that indication isn't present whether the loss is a tail-loss or
caused by other factors.

> Just two quick thoughts ... FWIW ...

I hope that answers your comments clearly.

Cheers,
Naeem

---

From: Mark Allman <[email-masked]>
Date: Mon, 13 Jul 2015 14:58:10 -0400

>     is that an ECN mark is the network saying "I should have
>     dropped your packet, but I am being nice" and so the reaction
>     should be the exact same as a loss. The RFC doesn't exactly
>     say that because at the time there was only one reaction and
>     yeah I guess we should have had better foresight with the
>     words, but we didn't. But, why now not just leave it as "just
>     as loss" and elide the specifics and hence avoid the mistake a
>     second time?
> 

Sure.

> ECN in term of routers doing CE-marking has not been used since
> its introduction approx. 15 years ago, therefore a CE-mark today
> on the Internet would *most likely* to indicate the presence of an
> AQM that is trying to keep the latency low (in order of 5ms~20ms
> on the average for (FQ_)CoDel and PIE as shipped) and therefore a
> beta_ {ecn}=0.5 is too conservative for such low thresholds. With
> a packet loss that's not the case since packet loss may have been
> caused by many other factors such as tail-loss, channel errors,
> etc.

So, my take was that your 0.8 backoff is "do what CUBIC does"
whereas the original ECN is "do what 5681 does".  Except in both
cases it is specified as what CUBIC or 5681 does and not as a
pointer to those documents.  And, my suggestion is to not re-specify
the CC response, but instead say "do what the underlying CC does" so
we don't have to write another document for each CC response we
might ever see.

Now, above it seems you are saying 0.5 is simply inappropriate as a
response when we know we are responding to congestion in a place
that is using some sort of AQM (what kind of course TCP doesn't
know).  Perhaps that is correct.  So, why is 0.8 appropriate?  When
I read the draft I thought you were rooting 0.8 in CUBIC's response.
But, if you are not then you have to tell me why 0.8 (or whatever
you choose) is reasonable.

> tail-loss) has little to do with the AQM threshold being deployed
> on that buffer so they are quite unrelated. We have (almost) clear
> indications of where (at which thresholds) CE-marking happens with
> the current AQMs. With loss, that indication isn't present whether
> the loss is a tail-loss or caused by other factors.

I don't have any idea what any of that means or how it speaks to my
point.

All I am saying is that perhaps we incentivize ECN deployment by
making the response softer than the response to loss as a general
principle.  I.e., as you note, a CE is an explicit indication of AQM
and hence that the link is not completely full and so perhaps we
don't have to backoff as dramatically.  If ECN deployment is a
chicken and egg problem then perhaps this helps us break free from
that.  (And, in fact, perhaps this provides an actual reason to
deploy ECN ... because when ECN marks are the same as loss there
isn't really much point in ECN at all relative to the costs.  As
evidence, I would point you at ... reality! :) )

allman

---

From: Michael Welzl <[email-masked]>
Date: Mon, 13 Jul 2015 23:42:15 +0200

> On 13. jul. 2015, at 20.58, Mark Allman <[email-masked]> wrote:
> 
> 

CUBIC went from 0.8 to 0.7 at some point, god knows why. Anyway, indeed CUBIC isn't our reason for choosing 0.8; for a BDP worth of queuing, 0.7 is too much too - but then, who knows how large queues really are these days...  we're not debating that point, just the fact that the signal from AQM comes from a shorter queue

As for the reasoning for 0.8, I'm sorry that it wasn't available until now, but here it is, at last:
http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf

> evidence, I would point you at ... reality! :) )

Yes!

Cheers,
Michael

---

From: John Leslie <[email-masked]>
Date: Mon, 13 Jul 2015 18:36:37 -0400

Mark Allman <[email-masked]> wrote:
> [Naeem Khademi <[email-masked]> wrote]
>> 
>>... a CE-mark today on the Internet would *most likely* to indicate
>> the presence of an AQM that is trying to keep the latency low...
> 

   Indeed, we should specify and justify what we mean to do. No reader
should have to guess at what we mean.

> know).

   Indeed: we can only know "some sort of AQM". But IMHO, we can say
that ECN-CE _should_ indicate an AQM as opposed to tail-drop. Also IMHO,
we should say somewhere that ECN-CE _should_not_ be used when the buffer
is essentially full.

> Perhaps that is correct.  So, why is 0.8 appropriate?  When

   I agree. (And I look at the issue quite differently and wouldn't
recommend simply "s/0.5/0.8/")

> I don't have any idea what any of that means or how it speaks to my

   I also have trouble understanding it: it sounds as if the sender
is supposed to guess what the AQM is doing; and I don't see why the
sender has enough information to make a good guess.

   It appears to be arguing that this is safe to deploy because CUBIC
has been doing it for years and is widely deployed. The weakness of
that argument is that CUBIC _also_ does things to make sure the
sending rate doesn't grow back as quickly as RENO does. We can't pick
and choose just a few pieces of a congestion control scheme.

> don't have to backoff as dramatically.

   We know that's true -- but if that's all we graft on, folks will
claim that ECN gets an unfair share. We also need an argument that
our ECN scheme won't grow sending rate as fast as RENO -- at least
for a little while.

> If ECN deployment is a chicken and egg problem then perhaps this
> helps us break free from that. (And, in fact, perhaps this provides
> an actual reason to deploy ECN ...

   More deployment is good...

--
John Leslie <[email-masked]>

---

From: Michael Welzl <[email-masked]>
Date: Tue, 14 Jul 2015 07:26:44 +0200

Hi all,

Sorry for the confusion that was apparently caused by:
1) the draft being very short
2) the technical report with results  ( http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf )  appearing with a bit of delay

As I wrote in my response to Mark, CUBIC wasn't our reason for picking 0.8, the choice is based on the results in the technical report. About the lack of detail in the draft (item 1), our intention was simply to start minimal and gradually add information that people request.

So, about the choice of 0.8, our technical report concludes:

***
Setting βecn ∈ [0.7, 0.85] for NewReno and βecn ≈ 0.85 for CUBIC seems to provide reasonable trade-offs between latency, throughput and fairness across a wide range of scenarios.
***

...and we picked 0.8 out of this interval for no other reason than: it's a middle ground but leaning more towards the upper ( = more effect ) end. We could also generally recommend 0.85, for instance, and I guess none of us authors has strong feelings about a particular choice of value from the interval above.

Look at it as a parameter change. RFC 6928 changes IW from 3 to 10 (for 1500-byte packets), we suggest to change the backoff factor from 0.5 to 0.8 for the ECN response.

Cheers,
Michael

> On 14. jul. 2015, at 00.36, John Leslie <[email-masked]> wrote:
>

---

From: Mark Allman <[email-masked]>
Date: Tue, 14 Jul 2015 13:21:59 -0400

> that ECN-CE _should_ indicate an AQM as opposed to tail-drop. Also
> IMHO, we should say somewhere that ECN-CE _should_not_ be used
> when the buffer is essentially full.

Well, I agree with the sentiment, but you'd have to come up with
some better guidelines than that.  I am reminded of the old gentle
version of RED that AQM-ed all the way until it couldn't anymore and
had to tail drop because there was no queue space.  So, when should
that stop marking and start dropping?

> > making the response softer than the response to loss as a
> > general principle.  I.e., as you note, a CE is an explicit
> > indication of AQM and hence that the link is not completely full
> > and so perhaps we don't have to backoff as dramatically.
> 
> claim that ECN gets an unfair share.

So what?  Two things ...

  - As I framed it I was talking about a *bit* less multiplicative
    decrease.  E.g., cwnd *= 0.55 instead of cwnd *= 0.50 for a mark
    vs. a loss.  That was a complete WAG.  But, something like that
    provides some incentive without going massively overboard, if
    you ask me.

  - And, so what if it gets an "unfair share".  We have to show some
    judgment for sure, but if we keep doing what we're doing just
    because old crap exists then we can never make any progress.
    SACK allows a TCP to recover from multiple losses more rapidly
    and without the RTO (in many case) and hence gets an "unfair
    share" compared with non-SACK TCP.  That doesn't mean we decided
    that SACK was somehow not worthwhile.

allman

---

From: Mark Allman <[email-masked]>
Date: Tue, 14 Jul 2015 13:30:56 -0400

> our
> intention was simply to start minimal and gradually add
> information that people request. 

That's annoying.  Why don't you take a stab at answering some of the
obvious questions someone reading that would have?

> Look at it as a parameter change. RFC 6928 changes IW from 3 to 10
> (for 1500-byte packets), we suggest to change the backoff factor
> from 0.5 to 0.8 for the ECN response.

In broad terms I think I agree.  That is, leveraging the additional
fact that a congested queue is not full seems useful to me.  A
parameter change that is (a) a big change from the standard
parameter and (b) is based on simulations of two AQM variants seems
like it might be a bit much.  I.e., sure the queue isn't full and
dropping packets, but it is congested.  But, I do like the general
idea. 

allman

---

From: Michael Welzl <[email-masked]>
Date: Tue, 14 Jul 2015 20:33:37 +0200

> On 14. jul. 2015, at 19.30, Mark Allman <[email-masked]> wrote:
> 
> 
>> our

Much is in the technical report, which we intended to make available at the same time. There was a 1 week delay which we're sorry about - it was a holidays / draft deadline situation. Yes the tech.rep. is a long document but I believe it's well organized and clearly written, so I guess it won't take you long to find your answers there. I just thought starting this way is more efficient. I may be wrong, now you tell us it's annoying - well ok, I'll take that as a vote to include more next time.

> dropping packets, but it is congested.  But, I do like the general

About (a): okay, I offer 0.75.
About (b): many real life tests, not just simulations. Only two AQM variants, yes.

Cheers,
Michael

---

From: Mark Allman <[email-masked]>
Date: Tue, 14 Jul 2015 14:39:10 -0400

> > A parameter change that is (a) a big change from the standard
> > parameter and (b) is based on simulations of two AQM variants
> > seems like it might be a bit much.  I.e., sure the queue isn't
> > full and dropping packets, but it is congested.  But, I do like
> > the general idea.
> 

I will have to better understand the tests.  That seems like a
significant change to me.

> About (b): many real life tests, not just simulations. Only two

Apologies.  I only skimmed very quickly.

allman

---

From: Michael Welzl <[email-masked]>
Date: Tue, 14 Jul 2015 20:52:07 +0200

> On 14. jul. 2015, at 20.39, Mark Allman <[email-masked]> wrote:
> 
> 

Okay - but - while we haven't used CUBIC as an argument before, I feel like doing it now:  the Internet has survived CUBIC, which - in response to loss, i.e. in response to potentially much larger queues - first used 0.8, then 0.7 ( I'd be quite curious about why that downward change was made, but found nothing ).

0.75 or 0.8 seemed to be pretty good values in our tests and using this in response to ECN is much less drastic than CUBIC's general behavior as it's only going to kick in when you get an ECN signal and then standard TCP doesn't increase as fast as CUBIC does, too.

Cheers,
Michael

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 14 Jul 2015 13:44:52 -0700

> Okay - but - while we haven't used CUBIC as an argument before, I feel like doing it now:  the Internet has survived CUBIC, which - in response to loss, i.e. in response to potentially much larger…

This email conversation may help.

On Wed, Sep 5, 2012 at 3:29 PM, Sangtae Ha <[email-masked]> wrote:
> Hi Yuchung,
>
> We used the beta parameter of BIC (0.8) for CUBIC until that time. But we
> decided to give more room for other flows to ramp up for better convergence.
> We also removed the window increase limit of smax to probe an available
> bandwidth rather aggressively when there is no known capacity
> (last_max_cwnd) of the link.
> The following summary may be useful.
>
> BIC: beta (0.8) and the window increase is limited to smax (which is an
> issue when the smax is relatively small compared to the BDP of a path)
> CUBIC: beta (0.7) and the window increase is governed by the cubic function
> itself.
>
> Let me know if you have any more questions.
>
> Have a good day!
>
> Best Regards,
> Sangtae
>
>
> On Wed, Sep 5, 2012 at 5:27 PM, Yuchung Cheng <[email-masked]> wrote:
>>
>> Hi Santae,
>>
>> I noticed the beta of cubic is changed from 0.8 (as in the paper) to
>> 0.7 in this commit.

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Wed, 15 Jul 2015 07:04:26 +0000

I really think Yuchungs's explanation below should be considered in draft-ietf-tcpm-cubic...

Thanks

Michael

> -----Original Message-----
> From: tcpm [mailto:[email-masked]] On Behalf Of Yuchung Cheng
> Sent: Tuesday, July 14, 2015 10:45 PM
> To: Michael Welzl
> Cc: [email-masked] Extensions; [email-masked]
> Subject: Re: [tcpm] New Draft: TCP Alternative Backoff with ECN (draft-
> khademi-alternativebackoff-ecn)
> 
> > Okay - but - while we haven't used CUBIC as an argument before, I
> feel like doing it now:  the Internet has survived CUBIC, which - in
> response to loss, i.e. in response to potentially much larger queues -
> first used 0.8, then 0.7 ( I'd be quite curious about why that downward
> change was made, but found nothing ).
> 
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Michael Welzl <[email-masked]>
Date: Wed, 15 Jul 2015 09:38:20 +0200

+1 !

this is great information from Yuchung, thanks!

> On 15 Jul 2015, at 09:04, Scharf, Michael (Michael) <[email-masked]> wrote:
>

---

From: "Agarwal, Anil" <[email-masked]>
Date: Wed, 15 Jul 2015 19:21:51 +0000

Michael, Naeem,

1. Would be useful if the draft rfc could describe the interaction between the alternative and
    the "standard" backoff schemes for new-RENO.
    What are the fairness properties, when connections of each type are present in the network?
    How long does it take for cwnd to reach its steady-state value for connections starting at different times?
       - For mixed connections with standard and alternative backoff factors.
       - As well as for connections that use the alternative backoff factor only.
       - For different RTT values.

2. Would be useful if the simulations were complemented with some analysis, 
    showing properties of the new backoff factor.

3. How does the suggested backoff factor of 0.8 affect queue size and delays for connections in slow start?
    Let's take a simple example -
    Assume a single TCP connection and a link with bandwidth-delay product = W bytes.
    The connection will begin in slow-start and double its cwnd every RTT.
    Let's assume that the AQM marks a packet with ECN soon after cwnd reaches W.
    The ECN-echo is received by the sender one RTT later.
    At that time cwnd = 2W.
    Using the cwnd reduction factor of 0.8, cwnd will be reduced to 1.6W.
    This is still too high. So, one RTT later, it will be further reduced to 1.28W.
    This is still too high. So, one RTT later, it will be further reduced to 1.024W.
    This is still too high. So, one RTT later, it will be further reduced to 0.82W.
    Hence, for 4 RTTs, cwnd is above W, causing substantial queue build-up at the bottleneck.
    ECN will probably be received for a few more RTT because of the queue build-up; alternatively, tail-drop
    might kick in.

  Part of the problem, I think, is a weakness in the TCP cwnd reduction algorithm - it reduces the current
  cwnd by a constant factor (cwnd = cwnd * f) on congestion detection; ideally, it should reduce cwnd
  to its value **one RTT earlier** multiplied by the constant factor.
  What this implies is that during slow-start cwnd should get reduced as cwnd = cwnd * 0.5 * f.
  During congestion avoidance, cwnd should get reduced as cwnd = (cwnd - MSS) * f,
      which is close to cwnd * f anyways.
  (We have been using this scheme within our geo-satellite network segment with TCP-PEPs
   for a long time to good effect. We have also been using f = 0.67 with ECN).
  If it makes sense, we could suggest this in the rfc as well, perhaps after you evaluate it in your testbeds.

Regards,
Anil

-----Original Message-----
From: tcpm [mailto:[email-masked]] On Behalf Of Michael Welzl
Sent: Tuesday, July 14, 2015 2:52 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [tcpm] New Draft: TCP Alternative Backoff with ECN (draft-khademi-alternativebackoff-ecn)

Cheers,
Michael

_______________________________________________
tcpm mailing list
[email-masked]
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_tcpm&d=AwICAg&c=jcv3orpCsv7C4ly8-ubDob57ycZ4jvhoYZNDBA06fPk&r=FyvaklKYrHaSCPjbBTdviWIW9uSbnxdNSheSGz1Jvq4&m=qyGZB6w2Tt1WdYgoDX6uzUCsfiqK0vBlqF_CyzdrDZ0&s=C4SVZsO-w0cXlOOpKnu-8b3Ma3adPsHHQY_xRtg9urI&e=

---

From: Michael Welzl <[email-masked]>
Date: Thu, 16 Jul 2015 00:20:47 +0200

Hi Anil,

Thanks for your interest!

You'll find that the technical report at http://caia.swin.edu.au/reports/150710A/CAIA-TR-150710A.pdf
answers most of your questions. More below:

> On 15. jul. 2015, at 21.21, Agarwal, Anil <[email-masked]> wrote:
> 
> Michael, Naeem,
> 

Page 9, Figures 14/15. Real life tests. Short answer: ratio of roughly 2:1, no starvation.

>       - For mixed connections with standard and alternative backoff factors.

We did not evaluate this because the convergence with similar backoff factors was not significantly and consistently harmed by increasing the backoff factor.

>       - As well as for connections that use the alternative backoff factor only.

Pages 8/9, figures 12/13, real life tests.

>       - For different RTT values.

We did not document this in the paper because convergence is already hard to understand with similar RTT values, and changing beta did neither consistently improve nor worsen the situation. Some point tests showed that convergence of TCP with backoff factor 0.5 does by itself not show a clear trend as a result of the RTT ratio, and the same is true for a different beta factor. In other words, we're sometimes better and sometimes worse than 0.5, which by itself is sometimes better and sometimes worse, depending on RTT ratios.

>    showing properties of the new backoff factor.

A rather simple model is presented on page 3. Equation 6 gives you the long-term average throughput in congestion avoidance as a function of a DropTail queue's buffer size (or marking ratio of at least CoDel, as Fig. 4 on the next page shows (simulations here) ), the capacity, the BDP and the backoff factor. Figure 2 gives a simple overview of the buffer size / backoff factor / throughput relationship.

>    ECN will probably be received for a few more RTT because of the queue build-up; alternatively, tail-drop

This is illustrated by Fig. 6 on page 6 and explained in Section 3 A (page 5). Section 4 D (pages 10-11) evaluates flow completion times of short flows in detail. These were simulations. Short answer: you're right, one can get unlucky and it can turn out like you describe, but one can also be luckier - it depends on the amount by which we overshot the capacity+queue limit in the first place. What you describe has a negative effect (for a limited time but for 0.8 it was outweighed by the positive effect (the "lucky" case). The number of RTTs this can take is bounded by log_backoffFactor (0.5)  (see page 5).

>  If it makes sense, we could suggest this in the rfc as well, perhaps after you evaluate it in your testbeds.

From our results and the explanations in Section 3 A, this strikes me as too conservative.

I hope this helps,

cheers,
Michael

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 15 Jul 2015 15:34:51 -0700

>>    the "standard" backoff schemes for new-RENO.
Kinda unrelated to the discussion, but I can't help to point out a
misnomer that many people even networking experts get it wrong.

NewReno (RFC6582) is NOT really a congestion control. It's just a
small enhancement on fast recovery in the absence of SACK. For
example, Linux uses NewReno on non-sack connections to send
fast-retransmit. It works with any congestion control (cubic, dctcp,
xxx). But it's kinda irrelevant b/c 99.9% conns use SACK.

Reno is the AIMD we know. There is not a new version of it called NewReno :)