Subject: [GGIE] Low latency and high bandwidth

From: Bob Briscoe <[email-masked]>
Date: Mon, 22 Jul 2019 15:35:32 -0400

Justin, and MPOS-ers, if that's a thing.

Here's links to the work on making sure your low latency high throughput 
video isn't screwed up by intermittent other traffic sharing the queue.

L4S = Low Latency Low Loss Scalable throughput (L4S) is nearing 
completion in the transport area.

      1/ Demo of L4S at the Jul 2015 IETF that kicked off the work

Still worth seeing. It shows a panoramic cloud-rendered video of a live 
football match, panned and zoomed by finger-gestures on a tablet, over a 
data centre - core - backhaul - 40Mb/s broadband DSL access - home 
network (7ms base delay), with multiple TCP downloads (L4S and non-L4S) 
running through the same queue:
     https://riteproject.eu/dctth/#1511dispatchwg
You'll see it appears to stick to the finger (sub-1ms queuing delay).

      2/ Low Latency DOCSIS
      <https://www.cablelabs.com/technologies/low-latency-docsis>

IETF L4S was adopted for DOCSIS a couple of years ago. The updated 
DOCSIS specs were published in Jan'19, with vendor products now planned 
for early 2020 (operators in various countries have deployment plans).

See the annex of the white paper 
<https://community.cablelabs.com/wiki/plugins/servlet/cablelabs/alfresco/download?id=0affb960-25f4-44f9-b747-74ad8555fd06&__hstc=217413636.3b29bbcaa0b5b68b88c232833241caa7.1556811205144.1561623882501.1563819684414.6&__hssc=217413636.1.1563819684414&__hsfp=751612915> 
at the end of the above page entitled "Low Latency AND High Throughput: L4S"
There's an IETF draft of the same paper, but the pictures aren't so good ;)

      3/ Google BBRv2 will use L4S

At the Mar'19 IETF, Neal Cardwell announced 
<https://datatracker.ietf.org/meeting/104/materials/slides-104-iccrg-an-update-on-bbr-00#page=9> 
that BBRv2 will switch to exploiting L4S where the path supports it 
("DCTCP-inspired ECN").

      4/ Even more impressive demo
      <http://bobbriscoe.net/pubs.html#uld4all-demo>

At the MMSYS'16 demo session.

      5/ Latency Performance

Our most evil test traffic mix: 600 web session arrivals per second and 
two TCP downloads (all half and half L4S and non-L4S).
99th percentile queuing delay: 2ms for /all/ the L4S packets: web and 
downloads [see log CCDF plot 
<https://datatracker.ietf.org/meeting/104/materials/slides-104-iccrg-implementing-the-prague-requirements-in-tcp-for-l4s-01#page=22>].

      6/ Code?

All the pieces via the code section of the landing page 
<https://riteproject.eu/dctth/#code>
TCP Prague is most mature, but there's an L4S variant of rmcat SCReAM, 
and QUIC there too.

      7/ Are there ops issues to work through?

Sure. See the ops & management sections of the drafts 
<https://riteproject.eu/dctth/#stds-specs>.
Let's discuss.

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Justin Uberti <[email-masked]>
Date: Mon, 22 Jul 2019 15:46:41 -0700

Thanks Bob. It looks like the impending rollouts will reduce typical
residential internet latency caused by the physical network from 10ms to 1
ms, which is a meaningful improvement..

How much improvement is predicated on marking traffic with the NQB diffserv
bits? Our experience has been that any sort of diffserv marking leads to
significant deployment problems, and as such I don't see this being a
near-term option.

On Mon, Jul 22, 2019 at 12:35 PM Bob Briscoe <[email-masked]> wrote:

> L4S = Low Latency Low Loss Scalable throughput (L4S) is nearing completion
> in the transport area.
>
> 1/ Demo of L4S at the Jul 2015 IETF that kicked off the work Still worth
> seeing. It shows a panoramic cloud-rendered video of a live football match,
> panned and zoomed by finger-gestures on a tablet, over a data centre - core
> - backhaul - 40Mb/s broadband DSL access - home network (7ms base delay),
> with multiple TCP downloads (L4S and non-L4S) running through the same
> queue:
> <https://www.cablelabs.com/technologies/low-latency-docsis> IETF L4S was
> adopted for DOCSIS a couple of years ago. The updated DOCSIS specs were
> published in Jan'19, with vendor products now planned for early 2020
> (operators in various countries have deployment plans).
>
> <http://bobbriscoe.net/pubs.html#uld4all-demo> At the MMSYS'16 demo
> session.
>
> 5/ Latency Performance Our most evil test traffic mix: 600 web session
> arrivals per second and two TCP downloads (all half and half L4S and
> non-L4S).
> 6/ Code? All the pieces via the code section of the landing page
> TCP Prague is most mature, but there's an L4S variant of rmcat SCReAM, and
> QUIC there too.
>
> 7/ Are there ops issues to work through? Sure. See the ops & management
> sections of the drafts <https://riteproject.eu/dctth/#stds-specs>.
> Let's discuss.
>
>
>
> Bob
>
>
>
> --

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 22 Jul 2019 19:42:25 -0400

Justin,

On 22/07/2019 18:46, Justin Uberti wrote:
> residential internet latency caused by the physical network from 10ms 
> to 1 ms, which is a meaningful improvement..
I assume by physical you mean the MAC layer scheduling (plus the 
propagation delay). If so yes, - down to about 1ms.

>
> How much improvement is predicated on marking traffic with the NQB 
> diffserv bits? Our experience has been that any sort of diffserv 
> marking leads to significant deployment problems, and as such I don't 
> see this being a near-term option.

I think you've misunderstood. There's two ways to get out of the queue 
for all the old 'Classic' traffic that hopefully will gradually all go 
away (prob over decades):

1) L4S-ECN
For high bandwidth low delay, like Stadia you would use the ECN ECT(1) 
codepoint. Then you're cutting out about 100ms of queuing under load at 
the 99th percentile, reducing that to 1ms.

2) NQB
The Non-Queue Building Diffserv Codepoint is for a different purpose. 
it's for unresponsive sparse traffic, e.g. traditional game sync 
packets, DNS, etc.

The deployment problems would have been with the ISPs that are offering 
the service, and they are making sure there won't be the deployment 
problems across their networks. 'Cos NQB is unlike other diffserv 
codepoints. It describes the sender behaviour, not needs or wants. So we 
can objectively test whether the traffic is complying with that 
behaviour (I've just posted an I-D documenting that for the IETF - 
draft-briscoe-q-protection). Packets of non-compliant flows that would 
cause the queue to exceed a threshold get ejected into the 'Classic' 
queue. ...That's the brief version.

HTH

Bob
>
> On Mon, Jul 22, 2019 at 12:35 PM Bob Briscoe <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Here's links to the work on making sure your low latency high
>     throughput video isn't screwed up by intermittent other traffic
>     sharing the queue.
>
>     Still worth seeing. It shows a panoramic cloud-rendered video of a
>     live football match, panned and zoomed by finger-gestures on a
>     tablet, over a data centre - core - backhaul - 40Mb/s broadband
>     DSL access - home network (7ms base delay), with multiple TCP
>     downloads (L4S and non-L4S) running through the same queue:
>     DOCSIS specs were published in Jan'19, with vendor products now
>     planned for early 2020 (operators in various countries have
>     deployment plans).
>
>     that BBRv2 will switch to exploiting L4S where the path supports
>     it ("DCTCP-inspired ECN").
>
>
>     Our most evil test traffic mix: 600 web session arrivals per
>     second and two TCP downloads (all half and half L4S and non-L4S).
>     99th percentile queuing delay: 2ms for /all/ the L4S packets: web
>     and downloads [see log CCDF plot
>     TCP Prague is most mature, but there's an L4S variant of rmcat
>     SCReAM, and QUIC there too.
>
>
>     Bob Briscoehttp://bobbriscoe.net/
>
>
> _______________________________________________
> GGIE mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ggie

---

From: Justin Uberti <[email-masked]>
Date: Tue, 23 Jul 2019 09:00:29 -0700

On Mon, Jul 22, 2019 at 4:42 PM Bob Briscoe <[email-masked]> wrote:

> diffserv bits? Our experience has been that any sort of diffserv marking
> leads to significant deployment problems, and as such I don't see this
> being a near-term option.
>
>
> I think you've misunderstood. There's two ways to get out of the queue for
> all the old 'Classic' traffic that hopefully will gradually all go away
> (prob over decades):
>
> 1) L4S-ECN
> codepoint. Then you're cutting out about 100ms of queuing under load at the
> 99th percentile, reducing that to 1ms.
>
> 2) NQB
> The Non-Queue Building Diffserv Codepoint is for a different purpose. it's
> for unresponsive sparse traffic, e.g. traditional game sync packets, DNS,
> etc.
>
> OK, I was confused by the text below:

*Thus, applications such as online games will be able to tag their packets
with the NQB DiffServ value to indicate that they behave in a
non-queue-building way, so that LLD will be able to classify them into the
Low Latency Service Flow. *

ECN/ECT may work, we'll have to see if those markings get eaten before they
reach ISP networks.

> can objectively test whether the traffic is complying with that behaviour
> (I've just posted an I-D documenting that for the IETF -
> cause the queue to exceed a threshold get ejected into the 'Classic' queue.
> ...That's the brief version.
>
> HTH
>
>
> Bob
>
>
> GGIE mailing [email-masked]://www.ietf.org/mailman/listinfo/ggie
>
>
> --