Subject: Re: [Ice] Proposal for ICE "network cost"

From: Emil Ivov <[email-masked]>
Date: Thu, 7 Apr 2016 12:50:28 -0500

On 21.03.16 г. 19:07, Peter Thatcher wrote:
> As an author (not a chair), I just submitted a draft for an idea I'd
> like to propose for ICE: network cost.
>
> Here's the draft:
>
> https://datatracker.ietf.org/doc/draft-thatcher-ice-network-cost/
>
>
> The idea is basically to allow the controlled side to tell the
> controlling side which candidates "cost" more than others (such as
> cellular costing more than Wi-Fi).  This is different than candidate
> priority for reasons explained in the draft.

I see the value here but I have a few problems with the current state.

To begin with, I find the "cost" name overly confusing. I realize the 
draft already says this is not necessarily real cost but it still leaves 
people with the wrong impression (and I've confirmed this in 
conversations these days). So let's just go for bias or something (I 
also like fondness).

The draft is not currently very explicit about exactly how this works in 
the selection of candidates but I assume it basically means: "if it's 
all the same to you, I'd much rather you get in touch with me here than 
there."

The trickiest part here is how to assign importance to this new bias as 
opposed to existing metrics such as RTT for example. We could say it's 
up to implementations or try to come up with some recommendation for a 
formula.

The other thing is that I cannot at all see the value of the network ID. 
Why is this necessary?

Emil

-- 
https://jitsi.org

---

From: Peter Thatcher <[email-masked]>
Date: Thu, 7 Apr 2016 14:27:20 -0700

On Thu, Apr 7, 2016 at 10:50 AM, Emil Ivov <[email-masked]> wrote:

> people with the wrong impression (and I've confirmed this in conversations
> these days). So let's just go for bias or something (I also like fondness).
>

​There are two question here:  the name and the direction.  The choice of
direction is more important than the name.  I prefer the direction where 0
means "free/use-freely" and a big number means
"expensive/restrained/last-resort".  This is because if we ever expand the
range, I think it would be better in the direction of "I really don't want
to use this" rather than "this is even more free to use".  You'll note that
this direction is the opposite of fondness/preference/priority.

In other words, I think modeling this as "badness' is better than as
"goodness".

So if we're going to debate names, let's debate synonyms of "badness".  The
best I could come up with is cost.  But I'd love to hear good synonyms.​
​

> the selection of candidates but I assume it basically means: "if it's all
> the same to you, I'd much rather you get in touch with me here than there."
>
>
Close.  0 means "use this freely".  A small number means "I'd rather not
use this".  A large number means "Only use this if nothing else works".​
​

> opposed to existing metrics such as RTT for example. We could say it's up
> to implementations or try to come up with some recommendation for a formula.
>

​That's exactly what I think we should do: standardized how information is
conveyed from controlled to controlling, but then let the controlling side
decided how to select and nominate.​
​

> The other thing is that I cannot at all see the value of the network ID.

​Technically, I could make it a separate draft, but the kind of go well
together (they both fit in one nice STUN attribute, for example).

The value of network ID is the following:  if the remote side's network
interface changes, it's wise to notify your bandwidth estimator, because
the change may have been drastic (wifi to 2g).  ​ It could change either by
TURN mobility, or by continual gathering, or by the use of backup candidate
pairs (I realized the last two are not yet standardized).

---

From: Emil Ivov <[email-masked]>
Date: Thu, 7 Apr 2016 17:14:03 -0500

On 7.04.16 г. 16:27, Peter Thatcher wrote:
>
>
> On Thu, Apr 7, 2016 at 10:50 AM, Emil Ivov <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     To begin with, I find the "cost" name overly confusing. I realize
>     the draft already says this is not necessarily real cost but it
>     still leaves people with the wrong impression (and I've confirmed
>     this in conversations these days). So let's just go for bias or
>     something (I also like fondness).
>
>
> ​There are two question here:  the name and the direction.  The choice
> of direction is more important than the name.  I prefer the direction
> where 0 means "free/use-freely" and a big number means
> "expensive/restrained/last-resort".  This is because if we ever expand
> the range, I think it would be better in the direction of "I really
> don't want to use this" rather than "this is even more free to use".
> You'll note that this direction is the opposite of
> fondness/preference/priority.
>

OK, this makes sense.

> So if we're going to debate names, let's debate synonyms of "badness".
> The best I could come up with is cost.  But I'd love to hear good synonyms.​

Let's go with "handicap" then? (If not we always have aversion and 
reluctance).

>     The draft is not currently very explicit about exactly how this
>     works in the selection of candidates but I assume it basically
>     means: "if it's all the same to you, I'd much rather you get in
>     touch with me here than there."
>
>
>     The trickiest part here is how to assign importance to this new bias
>     as opposed to existing metrics such as RTT for example. We could say
>     it's up to implementations or try to come up with some
>     recommendation for a formula.
>
>
> ​That's exactly what I think we should do: standardized how information
> is conveyed from controlled to controlling, but then let the controlling
> side decided how to select and nominate.​

The problem here is that unless we give directions about how it should 
be used in practice, then we greatly limit the usefulness and 
interoperability.

I actually think we can be very helpful without going into the details 
that much. Like for example:

0: use freely as you say.
h>0: you MAY use if no valid pairs exist where the remote candidate has 
a lower handicap but you SHOULD/MUST switch if such a pair becomes valid.

Also, unless your nominated pair has pair.remoteCandidate.handicap==0, 
then you MUST continue running checks until either you run through all 
of them or at least one pair with zero handicap becomes valid. When that 
happens you must switch.

Also if your selected pair has a remoteCandidate with non-zero handicap 
H and another pair becomes valid where the remoteCandidate has a lower 
handicap h (i.e., h<H) then you SHOULD switch to that new pair.

>     The other thing is that I cannot at all see the value of the network
>     ID. Why is this necessary?
>
>
> the change may have been drastic (wifi to 2g).  ​ It could change either
> by TURN mobility, or by continual gathering, or by the use of backup
> candidate pairs (I realized the last two are not yet standardized).

I am not a huge fan of this. I think it requires the remote side to know 
more than it needs to. If you want to reinit your bwe engine then we 
should have a message that says that (and it should probably be in 
RTCP). There could be other reasons for this that cannot be directly 
mapped to changing interfaces.

An interface ID mainly bothers me as it delivers more knowledge than one 
needs for that sort of stuff and it encourages implementers to make 
assumptions ... and ICE is all about avoiding them.

Emil

-- 
https://jitsi.org

---

From: Peter Thatcher <[email-masked]>
Date: Tue, 12 Apr 2016 11:42:56 -0700

> Let's go with "handicap" then? (If not we always have aversion and

​I'm not a fan of any of those words.  I'd prefer "badness" over those.
​

> The problem here is that unless we give directions about how it should be
> used in practice, then we greatly limit the usefulness and interoperability.

> 0: use freely as you say.

​h​
> >0: you MAY use if no valid pairs exist where the remote candidate has a
> lower handicap but you SHOULD/MUST switch if such a pair becomes valid.
>

I think such a rule is too restrictive of cost/badness > 0.  Perhaps such a
rules for cost/badness == MAX would make sense.  Basically, MAX would mean
"last resort".

However, we do need to be careful about the case where there are only two
candidate pairs available, one of which is (0, MAX) and the other (MAX,
0).  In other words, either the controlling side or the controlled side has
to end up using a "last resort".  Do we standardize which is picked, or
leave that up to the controlling side to decide (let it decide to be
gracious or not).
 ​

> then you MUST continue running checks until either you run through all of
> them or at least one pair with zero handicap becomes valid. When that
> happens you must switch.
>
>
​So, basically, don't prune less-bad networks just because a more-bad
network worked first?  That seems reasonable.  I'm not sure if it's
MUST-worthy, though.

And, again, be careful with the "must switch"​, since you need to remember
the case where only two candidate pairs exist: (N, 0) and (0, N) and the
controlling side needs to pick who pays more (and whether to be gracious or
not).

> Also if your selected pair has a remoteCandidate with non-zero handicap H
> and another pair becomes valid where the remoteCandidate has a lower

​Well, as mentioned, the badness of both sides needs to be balanced.  A
given candidate pair might be slight better for you but a lot worse for me.​
​

> more than it needs to. If you want to reinit your bwe engine then we should
> have a message that says that

​It's not as simple as just "reinit the BWE".  The BWE wants to know *why*,
and knowing its because the remote network changed is important
information.​
​
And the other problem with message sent from the remote side is that the
local/controlling side may switch candidate pairs, and it needs to know if
doing so changes the remote network or not, so it can know if it should
reset or not.  If it had to wait for a message from the remote side, it
would have to wait for the remote side to receive a message from the local
side, and then for the remote side to send back a message saying "by the
way, that candidate pair changed my local network", which just adds an
extra round trip for no good reason.

(and it should probably be in RTCP).

​That's not going to work for data channels, which also use ICE.​

There could be other reasons for this that cannot be directly mapped to
> changing interfaces.

​In which case, the BWE would like to know which case it is, because it
might behave differently between them. ​

> needs

for that sort of stuff and it encourages implementers to make assumptions
> ... and ICE is all about avoiding them.

​It's basically the minimal information to know that the remote network
path is changing between two candidate pairs.  And the assumption that can
be made is that if the ID is different, the network is different, which is
exactly the information it is meant to convey.
​

> Emil

​I mostly disagree here.  The controlled side is always at the mercy of the
controlling side, and t
he status-quo is that the controlled side has no way to say anything about
the networks it finds costly/bad.  Being able to communicate
0/max/something-in-between is very useful for the controlled side with any
reasonably implemented controlling side.

But for the values of things in between 0 and max, I agree.  ​The
difference between 100 and 200 are hard to nail down.  We tried to nail it
down according to how we think it would be useful, and we realized two
things:
1.  Everyone might not agree with how we were thinking to nail it down.
2.  We might change our minds 6-12 months from now anyway.

Being able to signal these things is valuable even if we haven't nailed
down what the difference between 100 and 200 means (if we ever nail it
down).  So I'd prefer standardizing the signaling of these things
(signaling here includes the STUN attribute) and the range of values, and
then leave the interpretation of the difference of particular values to
implementations or future specs once we have more implementation experience.

---

From: Emil Ivov <[email-masked]>
Date: Wed, 13 Apr 2016 00:04:50 -0500

Hey Peter,

On 12.04.16 г. 13:42, Peter Thatcher wrote:
>
>
> On Thu, Apr 7, 2016 at 3:14 PM, Emil Ivov <[email-masked]
>         <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>

And I guess I'd still prefer "handicap" (I can also be talked into 
preferring "lousiness" ;) )

>     The problem here is that unless we give directions about how it
>     should be used in practice, then we greatly limit the usefulness and
>     interoperability.
>
>
>     I actually think we can be very helpful without going into the
>     details that much. Like for example:
>
>      >0: you MAY use if no valid pairs exist where the remote candidate
>     has a lower handicap but you SHOULD/MUST switch if such a pair
>     becomes valid.
>
>
> I think such a rule is too restrictive of cost/badness > 0.Perhaps
> such a rules for cost/badness == MAX would make sense.  Basically, MAX
> would mean "last resort".
It sounds like the algorithm you have in mind basically cares about 
three sets of values: 0, anything, MAX_HANDICAP. If that is the case we 
might as well go for 0,1 and 2 and not allow other values.

I personally like the idea of allowing all integers but then that has to 
have a meaning in order to be useful. Otherwise it would just sloppy.

I think we are talking about different things though. My bad(ness). I 
was only referring to remote handicap comparison. Here's how the above 
rule should look like.

How about this:

0:   user freely
h>0: you MAY use if no valid pairs exist *for the same local candidate* 
where the remote candidate has a lower handicap but you SHOULD/MUST 
switch if such a pair becomes valid.

> However, we do need to be careful about the case where there are only
> two candidate pairs available, one of which is (0, MAX) and the other
> (MAX, 0).  In other words, either the controlling side or the controlled
> side has to end up using a "last resort".  Do we standardize which is
> picked, or leave that up to the controlling side to decide (let it
> decide to be gracious or not).

Agreed. We  don't need to stipulate it, but we can say that 
(Controlling.0, Controlled.MAX) is the most likely outcome.

>     Also, unless your nominated pair has
>     pair.remoteCandidate.handicap==0, then you MUST continue running
>     checks until either you run through all of them or at least one pair
>     with zero handicap becomes valid. When that happens you must switch.
>

As a matter of fact I am wondering whether even SHOULD makes sense. It 
is very hard to reconcile these things with other constraints such as 
RTT. I see no easy way where application developers could know whether 
an RTT of 5 seconds would be preferable to using an LTE interface ...

You may want to avoid LTE because it would exhaust your 5GB plan (but 
you'd still rather not have to suffer through a 5 second RTT) or you may 
want to avoid it because you are on roaming in Congo and having an HD 
call on it would cost more than your house ...

So maybe we could write all the rules without any 2119 language.

> And, again, be careful with the "must switch"​, since you need to
> remember the case where only two candidate pairs exist: (N, 0) and (0,
> N) and the controlling side needs to pick who pays more (and whether to
> be gracious or not).

Yes, rectified the rule above.

>     Also if your selected pair has a remoteCandidate with non-zero
>     handicap H and another pair becomes valid where the remoteCandidate
>     has a lower handicap h (i.e., h<H) then you SHOULD switch to that
>     new pair.
>
>

The minute we start comparing local and remote badness we will go into 
an arms race. All ICE implementations will become moody teens that find 
all candidates disgusting except for their favourite ones. I think we 
really should keep comparisons only for local2local and remote2remote.

In other words, the controlling pick their favourite local candidate L1 
and pick a pair [L1, R.DECENT] such that R.DECENT is the least crappy 
handicap that was ever validated against L1. Then they have to continue 
checks if those checks could lead to validating [L1, R.MIN] where MIN < 
DECENT

>     I am not a huge fan of this. I think it requires the remote side to
>     know more than it needs to. If you want to reinit your bwe engine
>     then we should have a message that says that
>
>
> ​It's not as simple as just "reinit the BWE".  The BWE wants to know
> *why*, and knowing its because the remote network changed is important
> information.​

And we can communicate the *why* just as easily.

> local/controlling side may switch candidate pairs, and it needs to know
> if doing so changes the remote network or not, so it can know if it
> should reset or not.  If it had to wait for a message from the remote
> side, it would have to wait for the remote side to receive a message
> from the local side, and then for the remote side to send back a message
> saying "by the way, that candidate pair changed my local network", which
> just adds an extra round trip for no good reason.

True. Before we go on though ... can you think of many cases out there 
(other than TURN mobility) where a change of interface is not directly 
visible to the remote end through the change of address?

> ​That's not going to work for data channels, which also use ICE.​

I agree ... although the rest of the BWE communications today don't seem 
to be bothered by that.

>     There could be other reasons for this that cannot be directly mapped
>     to changing interfaces.
>
>

My point exactly.

>     An interface ID mainly bothers me as it delivers more knowledge than
>     one needs
>
>     for that sort of stuff and it encourages implementers to make
> path is changing between two candidate pairs.  And the assumption that
> can be made is that if the ID is different, the network is different,
> which is exactly the information it is meant to convey.
> ​
>
>
>     Emil
>
>     --
>     https://jitsi.org
>
>
>

I am not sure what the following responds to but still ... :)

> ​I mostly disagree here.  The controlled side is always at the mercy of
> the controlling side, and t
> he status-quo is that the controlled side has no way to say anything
> about the networks it finds costly/bad.  Being able to communicate
> 0/max/something-in-between is very useful for the controlled side with
> any reasonably implemented controlling side.
>
> difference between 100 and 200 are hard to nail down.  We tried to nail
> it down according to how we think it would be useful, and we realized
> two things:
> (signaling here includes the STUN attribute) and the range of values,
> and then leave the interpretation of the difference of particular values
> to implementations or future specs once we have more implementation
> experience.

There have been other similar discussions on the IETF. One that I can 
remember happened around Dan's draft for Spam score SIP headers. IIRC, 
what the authors were suggesting was: we don't know how SPAM scores will 
be generated, but we want to standardize how it's signalled.

There was pretty strong opposition to that. Many people felt that unless 
you know exactly what the informations means, it is unreasonable to 
standardize it.

I didn't necessarily agree one way or another back then but this does 
serve as an IETF precedent.

I feel it is even more of an argument for ICE. The whole point of 
standardizing this is so that Chrome can send that information to other 
ICE implementations and get it back from them. Yet, other ICE 
implementations would neither know how to handle it nor how to generate 
it for you unless they know exactly what it means.

Emil

-- 
https://jitsi.org

---

From: Peter Thatcher <[email-masked]>
Date: Wed, 13 Apr 2016 11:11:38 -0700

> preferring "lousiness" ;) )
"lousiness" is ... lousy :).

​How about "woe" or "dislike"?

Maybe we should just call it "preference" but then only allow negative (or
zero) values.

> It sounds like the algorithm you have in mind basically cares about three
> sets of values: 0, anything, MAX_HANDICAP. If that is the case we might as
> well go for 0,1 and 2 and not allow other values.
>

​That was our original design, but it fails in an important way: if the
controlling side wants to do something more sophisticated, like factor in
RTT values into the selection algorithm, it can't do that if the range of
values has such low precision.  In other words, that might work for now,
but it doesn't leave any flexibility for the future.​

> I think we are talking about different things though. My bad(ness). I was
> only referring to remote handicap comparison. Here's how the above rule
> should look like.
>
> How about this:
>
> 0:   user freely
> where the remote candidate has a lower handicap but you SHOULD/MUST switch
> if such a pair becomes valid.

​That sounds reasonable, except for leaving open the possibility of
factoring in things like RTT (which means that SHOULD might be better than
MUST).

Oh, and instead of keying in on the same local candidate, perhaps just key
in on the same local network cost (multiple local candidates might have the
same local network cost).
​

> (Controlling.0, Controlled.MAX) is the most likely outcome.
​I was thinking that if we were to write a SHOULD, it should be
(Controlling=MAX, Controlled=0).  In other words, the controlling side
SHOULD be gracious.​

> As a matter of fact I am wondering whether even SHOULD makes sense. It is
> very hard to reconcile these things with other constraints such as RTT. I
> see no easy way where application developers could know whether an RTT of 5
> seconds would be preferable to using an LTE interface ..
> ​.​
>

> want to avoid it because you are on roaming in Congo and having an HD call
> on it would cost more than your house ...
>
​Yeah, now you see where this gets tricky.  We want to be able to balance
against RTT and other metrics, but there's no common unit of network
badness that we can use.

> The minute we start comparing local and remote badness we will go into an
> arms race. All ICE implementations will become moody teens that find all
> candidates disgusting except for their favourite ones. I think we really
> should keep comparisons only for local2local and remote2remote.
>

​It's not really an arms race, though, because the controlling side has all
the power.  The controlling side can always just ignore what the remote
side wants.  It's more like the controlled side saying what it dislikes
least and hopes the controlling side will be nice about it.

But I think you are right that whatever rules we right (if any) should be
in terms of when the local network cost is fixed/the-same (similar to what
you were saying about a fixed local candidate above).

> visible to the remote end through the change of address?

​
Yes, continual gathering.  If the selected candidate pair is over remote
wifi, and then the remote wifi goes away and the remote side gathers a
cellular candidate and then starts pinging from it, and it works (hopefully
very quickly) the local side needs to know whether the remote side is on a
different network interface or not.  It's not robust to just assume any new
candidate is a different network, because then you may over-react,
especially if you get a random peer reflexive candidate.

A similar situation comes up with "backup" candidate pairs where if a
backup candidate pair is kept alive and then either clients switches to
that one, the other side needs to know if that involved a network interface
change or not.

I realize that neither continual gathering nor backup candidate pairs
aren't even proposed yet (but we plan to propose them), which is why I'm
using TURN mobility as the example for now.  But our real use case is with
continual gathering and backup candidate pairs.

> I agree ... although the rest of the BWE communications today don't seem
​I think you're overly assuming that everything will be using RTCP.  ICE is
useful outside of the context of RTCP.​

> I am not sure what the following responds to but still ... :)

​Maybe I had some email authoring fail.  Sorry about that.
​

> remember happened around Dan's draft for Spam score SIP headers. IIRC, what
> the authors were suggesting was: we don't know how SPAM scores will be
> generated, but we want to standardize how it's signalled.
>
​That seems kind of silly to me.  If our choices are:

A:  Standardize all behavior completely.
B.  Standardize how to exchange information, but not behavior.
C.  Standardize nothing.​

If A is not possible (and I don't think it is), then our choices are
between B and C.  And B seems better than C.  Otherwise you'll just end up
with endpoints using non-standard mechanisms.

> standardizing this is so that Chrome can send that information to other ICE
> implementations and get it back from them. Yet, other ICE implementations
> would neither know how to handle it nor how to generate it for you unless
> they know exactly what it means.
>
>
​I think the meaning of "all other things being equal, use this network
instead of that one" is pretty clear.  And a value of 0 meaning "feel free
to use this with no restrictions" and a value of max meaning "please don't
use this unless your really have to" is all pretty clear.  It's just the
values in between and their relative weight that don't have clear meaning.
But, as I mentioned, we can leave a large range of possibilities for future
expansion.

​