Subject: Re: [conex] WG energy

From: Caitlin Bestler <[email-masked]>
Date: Thu, 3 Mar 2011 10:05:51 -0800

On Feb 24, 2011, at 2:21 AM, Scheffenegger, Richard wrote:

> 
> 
> 
>>> In particular, the degree to which an implementation could be
> synergistic
>>> with 802.1 DCB ccongestion control is still very vague.
>> 
>>  Hmm...
>> 
>>  I suspect most of us on this list aren't terribly familiar with
>> 802.1Qau. (That is what you're referring to, right?)
>> 
>>> An ideal conex draft, in my evaluation, would allow forwarding
> elements
>>> to implement *both* DCB and Conex, and suggest a method by which DCB
>>> congestion could be reported to the IP stack as though it had been
>>> reported by a CONEX device.
>> 
>>  I'm guessing here... 
>> 
>>  We've been writing in terms of something very like ECN to mark
>> congestion, which marks according to a queue filling.
>> 
>>  802.1Qau has an out-of-band mechanism to report "percent congestion"
>> from receiving switch to sender (if I read it correctly, I presume
> that
>> to be between two routers). This does sound like promising
> information,
>> but it's not immediately clear how it should get reported to a Layer 3
>> device.
> 
> Isn't 802.1Qau supposed to throttle / stop all traffic of an offending
> flow (as classified by 802.1Qau - which could be of variable granularity
> depending on the capabilities of the device IIRC) when congestion is
> experienced?
> 
> Also, the sender in the 802.1Qau case is a L2 MAC device - which is
> doing the throttling. 
> 
> The only thing a L3+ sender would notice would either be queues building
> at the L3/L2 interface, or calls to send data would block instead of
> immediately return. 
> 
> Obvioulsy, filling some queue right in the sender (be it a hardware or
> software queue) is  a bad thing (www.bufferbloat.net), so there might be
> some merit in signaling this between layers. However, as all that is
> required happens within the same host, it's more of an API question
> between device driver and software stack, rather than a protocol issue
> concerning the explicit signaling in a L3 header... 
> 
> Of course, if the egress side of a Router is 802.1Qau, but the ingress
> side is not, the router may want to inform the flow - but standard ECN
> marking (available with IPv4 and IPv6) should do that... 
> 
> Or am I missing something more?
> 

If bufferbloat in switches/routers is bad, it is nothing compared to the buffering
available in hosts.

A classic L4 congestion algorithm will respond only to drops or ECNs, which
a successful QCN deployment will prevent. So unless L4 is responsive to 
queuing delays the predictable impact of QCN on L4 is for L4 to continue to
"floor it" until things finally break.

Explicitly sharing the information that a flow is being throttled by some form
of queue management, as opposed to wireless retransmissions, would be
useful in helping the L4 algorithms take the optimal response. Ideally this
would be an abstract measurement, rather than requiring the L4 algorithm
understand the specifics of whatever form of Queue Management is being
employed.

--
Caitlin Bestler
[email-masked]

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 03 Mar 2011 23:01:44 +0000

Caitlin,

> >>> reported by a CONEX device.

As well as my response to your most recent posting (later) I wanted 
to pick up on these sentences, which I let go at the time...

I think you meant an ECN device not a ConEx device? Or do I misunderstand?

ConEx is an e2e signal in the L3 internetwork protocol, enabling the 
source transport to provide a measure of congestion on the whole path 
between L4 endpoints. Forwarding elements don't implement ConEx. A 
device that *monitors* ConEx might also be a forwarding element, but 
it doesn't "report ConEx signals to the IP stack".

The aim is to take congestion signalling from L2 subnets and 
propagate it upwards into IP-ECN at L3 alongside this e2e ConEx 
signal. Then ConEx & ECN are over comparable scopes (both between the 
L4 endpoints).

>"floor it" until things finally break.

The second sentence seems to agree with Richard, but the first 
sentence seems to disagree. The 2nd sentence says that the 802.1Qau 
L2 scheme can only prevent traffic building up at the source of this 
L2 subnet, while the L3 sitting over this L2 source and feeding it 
data will just back up into its own buffer unless somehow something 
can stop the remote L4 sending into this DCB subnet.

But then the first sentence cannot be true. QCN alone cannot prevent 
drops or ECN. QCN can prevent them *at L2*, but not in the L3 above 
this L2, which is feeding this DC Bridging system from the (probably 
remote) L4. QCN can only reduce ECN or drops if it has some way to 
signal up the layers (i.e. ECN or drop). Otherwise, yes, things will 
finally just break.

>understand the specifics of whatever form of Queue Management is being

Much as this is interesting to most of us in the ConEx w-g, we need 
to be clear that *generating* congestion signals at forwarding 
devices isn't in the ConEx charter. Generally congestion notification 
in IP and in lower layers (e.g. MPLS) has been handled in tsvwg.

The ADs might want to change that. But that's how it is at the moment.

I just posted a new -00 draft to tsvwg:
"Guidelines for Adding Congestion Notification to Protocols that 
Encapsulate IP"
<http://tools.ietf.org/id/draft-briscoe-tsvwg-ecn-encap-guidelines>

It's mostly about propagating congestion signalling from L2 to L3.

I tried to deal with QCN alongside similar schemes, but none in great detail.

After posting it, I realised it will need a section on the possible 
different layouts of the layered feedback loops. They're all in 
there: QCN, ATM, Frame Relay etc, but I didn't explicitly bring out 
the distinction between these and the way the loops are laid out in 
ECN-MPLS [RFC5129], for instance.

Frankly, given L3 generally uses forward signalling, feedback to the 
source of a L2 subnet would add unnecessary delay, if that source was 
then going to signal forwards at L3 towards the L4 destination. The 
L2 destination might as well signal up to L3 directly at its end.

Incidentally, Lei Zhu has been talking about this in ConEx too:
<http://www.ietf.org/proceedings/78/slides/conex-1/conex-1_files/v3_document.htm>
Altho again, it's been pointed out that this is off-charter for ConEx.

Perhaps we should have a little ad hoc bar BoF in Prague about L2 & 
ECN? Taking Lars's advice, we should make sure we have it in a real bar.

Bob

>--
>Caitlin Bestler
>[email-masked]
>
>
>
>
>_______________________________________________
>conex mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/conex

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 09 Mar 2011 11:30:25 +0000

Caitlin,

Still catching up on parts of past threads...

At 04:41 23/02/2011, Caitlin Bestler wrote:

>On Feb 17, 2011, at 10:04 AM, John Leslie wrote:
>
> > Caitlin Bestler <[email-masked]> wrote:
>The other area where convergence would be useful is coming up with 
>an abstract way to report "congestion
>detected" with more precision than just an ECN bit, so that L4 can 
>utilize this information wihtout having to
>learn the specifics of L2 congestion control protocols.

This as relevant to tsvwg & iccrg as to ConEx, but I'll keep it to 
ConEx for now...

Matt Mathis taught me something important in respect of how many bits 
you need per frame/packet for congestion control. The critical metric 
is the amount of signalling information per *window*, not per 
packet/frame. You can maintain the same amount of signalling 
information per window with just 1 bit per packet as long as the 
congestion control keeps the packet rate proportional to 1/p, where p 
is the marking probability.

Intuition: with a window of 100 frames, if the marking probability is 
1% you will get one mark per round trip. If when the marking 
probability reduces /100 to 0.01% you speed up x100 to 10,000 frames 
per window, you will still get one mark per RTT. So you will still 
have the same rate of control information per time.

See slide 8 of this Anaheim ICCRG presentation jointly from Matt & I:
<http://www.bobbriscoe.net/present.html#1003iccrg>

DCTCP, Relentless TCP and Scalable TCP all aim for 1/p (this is why 
Scalable TCP was called scalable). TCP NewReno's window is 
proportional to 1/sqrt(p), but more modern algorithms like these are 
moving closer to 1/p.

Bob