Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

From: Peter Thatcher <[email-masked]>
Date: Sat, 15 Oct 2016 22:20:27 -0700

Assuming ICE checks are around 100 octets (which they are, almost), then a
MaxPackets of 150 with a HTO of 500ms gives a maximum of about 300
packets/sec.  But a Ta of 5ms gives a maximum of 200 packets/sec.  So the
Ta value would end up being more strict.

Does that mean the Ta value of 5ms is sufficiently strict to render the
HTO/MaxPacket rules unnecessary?  Could we just reduce the ruleset down to
a min Ta value of 5ms?  Or do you think those rules still have value?

And as for the packet rate, previous to discussing with you "transport
guys", there was a lot of discussion in the WG about packet rates, in
particular rates of opening new NAT bindings with residential equipment.
After a lot of discussion and experimenting in the the real world, we
agreed that a lower bound of 5ms on the Ta was sufficient to cover packet
rates and rates of opening NAT bindings.  I can give more history on that
if you'd like.    But that still left a concern over bit rates, which is
how we ended up talking to you :).

On Wed, Sep 28, 2016 at 8:45 AM, Black, David <[email-masked]> wrote:

> I'm apparently one of the "transport guys" referred to in the subject line
> ;-).
>
> In https://www.ietf.org/mail-archive/web/ice/current/msg00378.html, Peter
> Thatcher
> described the current proposal:
>
> ----------------------
> 1.  A maximum number of outstanding request packets, call it MaxPackets
> which SHOULD be 10.
>
> 2.  A timeout for request packets, call it handshake timeout or HTO which
> SHOULD be 2*RTT if the RTT is known and 500ms otherwise.
>
> 3.  A global pacing rate of 5ms (like a global Ta).
>
> If all packets are dropped and the RTT is not known, then 1+2 are
> effectively 20 packets per second,
> well below what you would get with the bitrate limits we were asking for.
> ----------------------
>
> There's been some further discussion with the "transport guys" about
> packet size and bit rate.   In essence, the rationale for 10 as the maximum
> number of outstanding request packets (item 1 above) is based on the
> typical TCP segment size of about 1500 bytes.  ICE request packets tend to
> be much smaller, 100 bytes or less, which suggests that a factor of 15
> could be available if bit rate matters but packet count doesn't.    That
> may well be the case ...
>
> RFC 7141 (2014 BCP on AQM configuration/implementation) supports that view
> by recommending that AQM implementations count octets (i.e., forwarding
> resources are bit-congestible) as opposed to counting packets (i.e.,
> forwarding resources are not packet-congestible).  That RFC effectively
> asserts that the current Internet is bit-congestible only:
> https://tools.ietf.org/html/rfc7141
>
> That would suggest a value of 150 for MaxPackets *when* all the packets
> are 100 octets or less, as should be case for ICE request packets.  A
> possible area of concern (based on ICE use by Web RTC) is residential
> gateway/NAT boxes, but the 5ms pacing (item 3 above) ought to be sufficient
> to avoid overrunning them.
>
> What do people with more knowledge of and/or experience with residential
> gateway boxes think of this line of reasoning?
>
> Thanks, --David
>
> _______________________________________________
> Ice mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ice
>

---

From: "Black, David" <[email-masked]>
Date: Sun, 16 Oct 2016 12:22:15 +0000

It looks like the Ta value suffices, and the prior discussion provides assurance that residential equipment won't be overrun ... but given the subtlety of the discussion that reached this result, I strongly recommend that we show our work ;-).   In other words, the draft should provide a complete explanation of our reasoning, as both the ICE packet size and the 500ms value are specific to this discussion, making the conclusion *not* generally applicable.

Thanks, --David ++Sent from my Android not-so-smartphone ...

-------- Original message --------
From: Peter Thatcher <[email-masked]>
Date: 10/16/16 7:21 AM (GMT+01:00)
To: "Black, David" <[email-masked]>
Cc: [email-masked]
Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

Assuming ICE checks are around 100 octets (which they are, almost), then a MaxPackets of 150 with a HTO of 500ms gives a maximum of about 300 packets/sec.  But a Ta of 5ms gives a maximum of 200 packets/sec.  So the Ta value would end up being more strict.

Does that mean the Ta value of 5ms is sufficiently strict to render the HTO/MaxPacket rules unnecessary?  Could we just reduce the ruleset down to a min Ta value of 5ms?  Or do you think those rules still have value?

And as for the packet rate, previous to discussing with you "transport guys", there was a lot of discussion in the WG about packet rates, in particular rates of opening new NAT bindings with residential equipment.  After a lot of discussion and experimenting in the the real world, we agreed that a lower bound of 5ms on the Ta was sufficient to cover packet rates and rates of opening NAT bindings.  I can give more history on that if you'd like.    But that still left a concern over bit rates, which is how we ended up talking to you :).

On Wed, Sep 28, 2016 at 8:45 AM, Black, David <[email-masked]<mailto:[email-masked]>> wrote:
I'm apparently one of the "transport guys" referred to in the subject line ;-).

In https://www.ietf.org/mail-archive/web/ice/current/msg00378.html, Peter Thatcher
described the current proposal:

----------------------
1.  A maximum number of outstanding request packets, call it MaxPackets which SHOULD be 10.

2.  A timeout for request packets, call it handshake timeout or HTO which SHOULD be 2*RTT if the RTT is known and 500ms otherwise.

If all packets are dropped and the RTT is not known, then 1+2 are effectively 20 packets per second,
well below what you would get with the bitrate limits we were asking for.
----------------------

There's been some further discussion with the "transport guys" about packet size and bit rate.   In essence, the rationale for 10 as the maximum number of outstanding request packets (item 1 above) is based on the typical TCP segment size of about 1500 bytes.  ICE request packets tend to be much smaller, 100 bytes or less, which suggests that a factor of 15 could be available if bit rate matters but packet count doesn't.    That may well be the case ...

RFC 7141 (2014 BCP on AQM configuration/implementation) supports that view by recommending that AQM implementations count octets (i.e., forwarding resources are bit-congestible) as opposed to counting packets (i.e., forwarding resources are not packet-congestible).  That RFC effectively asserts that the current Internet is bit-congestible only: https://tools.ietf.org/html/rfc7141

That would suggest a value of 150 for MaxPackets *when* all the packets are 100 octets or less, as should be case for ICE request packets.  A possible area of concern (based on ICE use by Web RTC) is residential gateway/NAT boxes, but the 5ms pacing (item 3 above) ought to be sufficient to avoid overrunning them.

What do people with more knowledge of and/or experience with residential gateway boxes think of this line of reasoning?

Thanks, --David

_______________________________________________
Ice mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ice

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 19 Oct 2016 09:16:59 -0700

Jana, are you OK with a global Ta value with text explaining how we arrived
at that value?

On Sun, Oct 16, 2016 at 5:22 AM, Black, David <[email-masked]> wrote:

> It looks like the Ta value suffices, and the prior discussion provides
> assurance that residential equipment won't be overrun ... but given the
> subtlety of the discussion that reached this result, I strongly recommend
> that we show our work ;-).   In other words, the draft should provide a
> complete explanation of our reasoning, as both the ICE packet size and the
> 500ms value are specific to this discussion, making the conclusion *not*
> generally applicable.
>
>> RFC 7141 (2014 BCP on AQM configuration/implementation) supports that
>> view by recommending that AQM implementations count octets (i.e.,
>> forwarding resources are bit-congestible) as opposed to counting packets
>> (i.e., forwarding resources are not packet-congestible).  That RFC
>> effectively asserts that the current Internet is bit-congestible only:

---

From: Jana Iyengar <[email-masked]>
Date: Wed, 19 Oct 2016 12:26:50 -0700

Sounds good to me. Thanks for taking the time to think this through, even
though it seems to have "collapsed" to the same answer. I definitely think
it's worthwhile putting the reasoning into the draft.

On Wed, Oct 19, 2016 at 9:16 AM, Peter Thatcher <[email-masked]>
wrote:

> Jana, are you OK with a global Ta value with text explaining how we
> arrived at that value?
>
>
>
>> Assuming ICE checks are around 100 octets (which they are, almost), then
>> a MaxPackets of 150 with a HTO of 500ms gives a maximum of about 300
>>> 2.  A timeout for request packets, call it handshake timeout or HTO
>>> which SHOULD be 2*RTT if the RTT is known and 500ms otherwise.
>>>

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 26 Oct 2016 07:00:29 -0700

OK, based on the discussion, I have made the following PR for 5245bis:

https://github.com/ice-wg/rfc5245bis/pull/19

Please let me know if the logic and wording are right.

On Wed, Oct 19, 2016 at 12:26 PM, Jana Iyengar <[email-masked]> wrote:

>>>> What do people with more knowledge of and/or experience with
>>>> residential gateway boxes think of this line of reasoning?
>>>>
>>>> Thanks, --David
>>>>

---

From: "Black, David" <[email-masked]>
Date: Thu, 27 Oct 2016 20:49:08 +0000

Hi Peter,

Some comments ...

+  <t>Start with the following rules which would be generally
+  applicable to transport protocols:
+  <list style="numbers">
+    <t>Let MaxPackets be the maximum number of transactions allowed to be outstanding in the network at any time, which SHOULD be 10.</t>
+    <t>Let HTO be the transaction timeout, which SHOULD be 2*RTT if RTT is known and 500ms otherwise.</t>
+    <t>Let MinPacing be the minimum pacing interval between transactions, which SHOULD be 5ms.</t>
+  </list>

I’d cite references for at least the first two of the list items, if not all three.  The first item is based on TCP Initial Window of 10 segments, which I believe is RFC 6928 (Jana will know for certain).  The second item is *not* general - 500ms is specific to STUN, and comes from RFC 5389 (credit to Bernard Aboba for finding this: https://www.ietf.org/mail-archive/web/ice/current/msg00379.html) - that’s important because the generally applicable number is larger - both RFC 6298 and draft-ietf-tsvwg-rfc5405bis use 1 second for TCP and UDP, respectively.  For the third item, I hope the rationale for the 5ms pacing interval is explained elsewhere in the draft.

+    Observe that ICE transactions are typically around 100 bytes or
+    less, which makes it reasonable to increase MaxPackets to about
+    150.

Add “because 1500 bytes is a typical TCP segment size” to the end of this.

Thanks, --David

From: Peter Thatcher [mailto:[email-masked]]
Sent: Wednesday, October 26, 2016 10:00 AM
To: Jana Iyengar
Cc: Black, David; [email-masked]
Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

On Wed, Oct 19, 2016 at 12:26 PM, Jana Iyengar <[email-masked]<mailto:[email-masked]>> wrote:
Sounds good to me. Thanks for taking the time to think this through, even though it seems to have "collapsed" to the same answer. I definitely think it's worthwhile putting the reasoning into the draft.

On Wed, Oct 19, 2016 at 9:16 AM, Peter Thatcher <[email-masked]<mailto:[email-masked]>> wrote:
Jana, are you OK with a global Ta value with text explaining how we arrived at that value?

On Sun, Oct 16, 2016 at 5:22 AM, Black, David <[email-masked]<mailto:[email-masked]>> wrote:
It looks like the Ta value suffices, and the prior discussion provides assurance that residential equipment won't be overrun ... but given the subtlety of the discussion that reached this result, I strongly recommend that we show our work ;-).   In other words, the draft should provide a complete explanation of our reasoning, as both the ICE packet size and the 500ms value are specific to this discussion, making the conclusion *not* generally applicable.

-------- Original message --------
From: Peter Thatcher <[email-masked]<mailto:[email-masked]>>
Date: 10/16/16 7:21 AM (GMT+01:00)
To: "Black, David" <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

----------------------

Thanks, --David

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 27 Oct 2016 15:19:41 -0700

On Thu, Oct 27, 2016 at 1:49 PM, Black, David <[email-masked]> wrote:

> +    <t>Let MaxPackets be the maximum number of transactions allowed to be
> outstanding in the network at any time, which SHOULD be 10.</t>
>
> I’d cite references for at least the first two of the list items, if not
> all three.  The first item is based on TCP Initial Window of 10 segments,
> which I believe is RFC 6928 (Jana will know for certain).  The second
>

Yes, RFC 6928 is right. I think this text could be made more precise: "Let
MaxBytes be the maximum number of bytes allowed to be outstanding in the
network at start-up, which SHOULD be 14600 bytes [RFC6928]."

> item is **not** general - 500ms is specific to STUN, and comes from RFC
> 5389 (credit to Bernard Aboba for finding this: https://www.ietf.org/mail-
> archive/web/ice/current/msg00379.html) - that’s important because the
> generally applicable number is larger - both RFC 6298 and
> draft-ietf-tsvwg-rfc5405bis use 1 second for TCP and UDP, respectively.
>

Additionally, my reasoning for this was as follows (if you want to add this
to the rationale.) The TCP initial RTO is 1 sec (RFC 6298), and the HTO can
be less conservative, since an expected case is that no responses are
received even if the network is perfect.

For the third item, I hope the rationale for the 5ms pacing interval is
> explained elsewhere in the draft.
>
>
>

If you use the byte-limit from RFC 6928 above you won't need to specify TCP
segment size, and you would change MaxPackets to MaxBytes.

- jana

> Thanks, --David
>
>
>
> *From:* Peter Thatcher [mailto:[email-masked]]
> *Sent:* Wednesday, October 26, 2016 10:00 AM
> *To:* Jana Iyengar
> *Cc:* Black, David; [email-masked]
>
> *Subject:* Re: [Ice] Continue discussion of global pacing based on
> feedback from "transport guys"
>
>
>

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 31 Oct 2016 13:23:30 +0000

Hi,

Can we expect an update to the pull request today, so I can generate and submit a new version of the draft before the submission deadline?

Yes, there are still some other issues that we need to do, but at least we’d have a version with the Ta issue (which has been ongoing for a while) solved.

Regards,

Christer

From: Ice <[email-masked]<mailto:[email-masked]>> on behalf of Jana Iyengar <[email-masked]<mailto:[email-masked]>>
Date: Friday 28 October 2016 at 01:19
To: "Black, David" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

On Thu, Oct 27, 2016 at 1:49 PM, Black, David <[email-masked]<mailto:[email-masked]>> wrote:
Hi Peter,

Some comments ...

I’d cite references for at least the first two of the list items, if not all three.  The first item is based on TCP Initial Window of 10 segments, which I believe is RFC 6928 (Jana will know for certain).  The second

Yes, RFC 6928 is right. I think this text could be made more precise: "Let MaxBytes be the maximum number of bytes allowed to be outstanding in the network at start-up, which SHOULD be 14600 bytes [RFC6928]."

item is *not* general - 500ms is specific to STUN, and comes from RFC 5389 (credit to Bernard Aboba for finding this: https://www.ietf.org/mail-archive/web/ice/current/msg00379.html) - that’s important because the generally applicable number is larger - both RFC 6298 and draft-ietf-tsvwg-rfc5405bis use 1 second for TCP and UDP, respectively.

Additionally, my reasoning for this was as follows (if you want to add this to the rationale.) The TCP initial RTO is 1 sec (RFC 6298), and the HTO can be less conservative, since an expected case is that no responses are received even if the network is perfect.

For the third item, I hope the rationale for the 5ms pacing interval is explained elsewhere in the draft.

If you use the byte-limit from RFC 6928 above you won't need to specify TCP segment size, and you would change MaxPackets to MaxBytes.

- jana

Thanks, --David

From: Peter Thatcher [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Wednesday, October 26, 2016 10:00 AM
To: Jana Iyengar
Cc: Black, David; [email-masked]<mailto:[email-masked]>

-------- Original message --------
From: Peter Thatcher <[email-masked]<mailto:[email-masked]>>
Date: 10/16/16 7:21 AM (GMT+01:00)
To: "Black, David" <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Ice] Continue discussion of global pacing based on feedback from "transport guys"

----------------------

Thanks, --David

---

From: Peter Thatcher <[email-masked]>
Date: Mon, 31 Oct 2016 07:48:38 -0700

Yes, I can update the PR today.  Then you can merge it and push an update.

On Mon, Oct 31, 2016 at 6:23 AM, Christer Holmberg <
[email-masked]> wrote:

> Hi,
>
> Can we expect an update to the pull request today, so I can generate and
> submit a new version of the draft before the submission deadline?
>
> Yes, there are still some other issues that we need to do, but at least
> we’d have a version with the Ta issue (which has been ongoing for a while)
> solved.
>
> Regards,
>
> Christer
>
> From: Ice <[email-masked]> on behalf of Jana Iyengar <[email-masked]
> >
> To: "Black, David" <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <
> [email-masked]>
>
>> +    <t>Let MaxPackets be the maximum number of transactions allowed to
>> be outstanding in the network at any time, which SHOULD be 10.</t>
>>
>> 5389 (credit to Bernard Aboba for finding this:
>> https://www.ietf.org/mail-archive/web/ice/current/msg00379.html) -
>> that’s important because the generally applicable number is larger - both
>> RFC 6298 and draft-ietf-tsvwg-rfc5405bis use 1 second for TCP and UDP,
>> respectively.
>>
>
> Additionally, my reasoning for this was as follows (if you want to add
> this to the rationale.) The TCP initial RTO is 1 sec (RFC 6298), and the
> HTO can be less conservative, since an expected case is that no responses
> are received even if the network is perfect.
>
> If you use the byte-limit from RFC 6928 above you won't need to specify
> TCP segment size, and you would change MaxPackets to MaxBytes.
>
> - jana
>
>
>> Thanks, --David
>>
>>
>>