Subject: Re: [v6ops] simplfying Happy Eyeballs further

From: Ray Hunter <[email-masked]>
Date: Mon, 20 Jun 2011 18:42:30 +0200

+1
> On Mon, Jun 20, 2011 at 11:44:57PM +1000, Mark Andrews wrote:
>    
>> >  >  DNS response time has absolutely nothing to do with which address family
>> >  >  is best to use.
>>      
On Tue, 21 Jun 2011 00:30:59 +1000, Mark Andrews wrote:
> Also it is not hard to get nameservers that work. If a site is slow 
> because they are running broken nameservers should we let them off the 
> hook. Perhaps having customers complain that their site is slow will 
> get them to fix their end.
Further, I like the idea of a straight race between IPv4 and IPv6 with 
some simple local storage of recent history of which address family to 
prefer based on measured round trip response times to the real server 
(not DNS).

IMVHO If network operators want to turn off IPv4/CGN later in a 
controlled manner, they can achieve the effect of discouraging the use 
of IPv4/CGN compared to IPv6 by introducing some delay or configuration 
in their own particular CGN box / IPv4 service at a time and in a manner 
that suits their own particular IPv6 migration requirements, rather than 
trying to use an RFC to artificially cripple/influence behavior of 100% 
of end clients in some way on 100% of networks.

Some additional queuing of IPv4 SYN packets compared to other traffic at 
some point within the network cloud (e.g. on the CPE/ ingress router, or 
the CGN service router itself) wouldn't seem that hard to do IMHO, 
rather than changing 100% of clients. That way IPv6 could get a head 
start, but only when needed at some point in time of the migration, and 
only in some networks that need this feature, but neither IPv4 nor IPv6 
is delayed once/if the IPv4 or IPv6 session is established.

At least one leading manufacturer already seems to do a good job of 
coupling users/ user groups to IPv4/CGN services via user profiles, so 
that any CGN switch off won't have to be an all or nothing process for 
all users at once. They could also build in that (artificial) IPv4 SYN 
delay on one particular CGN profile, so you can switch over to IPv6 by 
groups/ batches of users without changing the client at all.

I recently did a complex disentanglement of interconnects where people 
weren't really certain what residual traffic was still running and why. 
As a last step, the only practical operational way forward was to really 
cut the wire, and have a rapid-response team in place to restore any 
important functionality when "important" people "peeped" to the help 
desk, whilst "non-important" people were told to reconfigure their apps 
or find another solution themselves. No technology can tell you in 
advance of a cut off with 100% certainty what traffic is "important", or 
why it's still running. People and processes are a far better solution 
to this type of problem.

regards,
RayH

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 20 Jun 2011 11:29:23 -0700

On Mon, Jun 20, 2011 at 9:42 AM, Ray Hunter <[email-masked]> wrote:
>>> >  >  DNS response time has absolutely nothing to do with which address
>>> > family
>>> >  >  is best to use.
>>>
>
>> because they are running broken nameservers should we let them off the hook.
>> Perhaps having customers complain that their site is slow will get them to
>> fix their end.
>
> Further, I like the idea of a straight race between IPv4 and IPv6 with some
> simple local storage of recent history of which address family to prefer
> based on measured round trip response times to the real server (not DNS).
>
>
> IMVHO If network operators want to turn off IPv4/CGN later in a controlled
> manner, they can achieve the effect of discouraging the use of IPv4/CGN
> compared to IPv6 by introducing some delay or configuration in their own
> particular CGN box / IPv4 service at a time and in a manner that suits their

This is very unlikely that an SP would inject latency like this.  In
fact, it is completely unrealistic from a business and instrumentation
perspective.

> own particular IPv6 migration requirements, rather than trying to use an RFC
> to artificially cripple/influence behavior of 100% of end clients in some
> way on 100% of networks.
>

Generally, i like e2e solutions, not middle box manipulation.

> some point within the network cloud (e.g. on the CPE/ ingress router, or the
> CGN service router itself) wouldn't seem that hard to do IMHO, rather than
> changing 100% of clients. That way IPv6 could get a head start, but only
> when needed at some point in time of the migration, and only in some
> networks that need this feature, but neither IPv4 nor IPv6 is delayed
> once/if the IPv4 or IPv6 session is established.
>
> At least one leading manufacturer already seems to do a good job of coupling
> users/ user groups to IPv4/CGN services via user profiles, so that any CGN
> switch off won't have to be an all or nothing process for all users at once.
> They could also build in that (artificial) IPv4 SYN delay on one particular
> CGN profile, so you can switch over to IPv6 by groups/ batches of users
> without changing the client at all.
>
> weren't really certain what residual traffic was still running and why. As a
> last step, the only practical operational way forward was to really cut the
> wire, and have a rapid-response team in place to restore any important
> functionality when "important" people "peeped" to the help desk, whilst
> "non-important" people were told to reconfigure their apps or find another
> solution themselves. No technology can tell you in advance of a cut off with
> 100% certainty what traffic is "important", or why it's still running.
> People and processes are a far better solution to this type of problem.
>
> regards,
> RayH
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Ray Hunter <[email-masked]>
Date: Mon, 20 Jun 2011 22:01:50 +0200

Thanks for the reply

> fact, it is completely unrealistic from a business and instrumentation
I agree. My suggestion is only meant as a tool for those people who have 
expressed a requirement to be able to force behavior of the client to 
prefer one AF family over another (e.g. during phase out of CGN).

Standard behavior on the majority of networks and clients would be to 
choose the fastest session set up with no handicapping at all.

If it is not acceptable for the SP to add any artificial latency within 
the network for IPv4 SYN packets to prefer IPv6 during migration, why 
would we think that adding any (artificial) latency in every single 
client on every single network (whether they use CGN or not) would be 
any more acceptable? [current Chrome behavior?]

Or indeed waiting for one AF family session to fail completely before 
trying another? [most current browsers]

> <snip>
And CGN isn't middle box manipulation? ;)

The IPv4 SYN packets may anyway need special treatment (to form a new 
NAT association in the CGN box) so I don't see this as being too onerous 
IMVHO.

regards,
RayH

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 20 Jun 2011 13:18:03 -0700

On Mon, Jun 20, 2011 at 1:01 PM, Ray Hunter <[email-masked]> wrote:
> expressed a requirement to be able to force behavior of the client to prefer
> one AF family over another (e.g. during phase out of CGN).
>
> Standard behavior on the majority of networks and clients would be to choose
> the fastest session set up with no handicapping at all.
>

Can you cite the standard you refer to? On my Windows and Linux
systems, i seem to always go to the IPv6 / AAAA destination first, if
it is available.

> If it is not acceptable for the SP to add any artificial latency within the
> network for IPv4 SYN packets to prefer IPv6 during migration, why would we
> think that adding any (artificial) latency in every single client on every
> single network (whether they use CGN or not) would be any more acceptable?
> [current Chrome behavior?]
>
> Or indeed waiting for one AF family session to fail completely before trying
> another? [most current browsers]
>

Does this contradict what you said about "standard behavior"?

> And CGN isn't middle box manipulation? ;)

It is. The only way to get the flow off the middle box is for the
end-point to initiate the flow on IPv6.  The smarts should be in the
end points, not the middle box.

> The IPv4 SYN packets may anyway need special treatment (to form a new NAT
> association in the CGN box) so I don't see this as being too onerous IMVHO.
>
> regards,
> RayH
>

---

From: Ray Hunter <[email-masked]>
Date: Mon, 20 Jun 2011 23:30:00 +0200

"Standard" was perhaps an unfortunate choice of wording on my part. 
"Vanilla"?

Cameron Byrne wrote:
> On Mon, Jun 20, 2011 at 1:01 PM, Ray Hunter<[email-masked]>  wrote:
>    
See below.
> Does this contradict what you said about "standard behavior"?
Assuming "amended Happy Eyeballs" becomes a standard address selection 
algorithm based on a simple straight race with a cached result, as I 
suggested.

RFC 3484 (and RFC 3484 bis) are also standard address selection algorithms.

I see no contradiction personally. Address selection algorithms to me 
are an application level choice. They do not have to preclude each 
other, even though all are "standard."

If a developer chooses to implement "amended Happy Eyeballs" (and no 
RFC3484) then the browser app would start two sessions in parallel for 
the first IPv4 and first IPv6 returned by DNS (if there was no cached 
entry), and use the first one to complete the SYN ACK handshake in a 
straight race. Once the cache is filled there'd be no need to start two 
sessions. The cache would flush regularly, and also on interface 
changes. This is for me the easiest vanilla form to implement and is 
just a simplified version of the current Happy Eyeballs draft (with IH 
=0 and no handicapping algorithm).

If a developer chooses to implement just RFC3484, then the app could try 
one session to the first entry in the orderd list from the source / 
destination address selection process. If that session fails it starts a 
second session to the second entry and so on. [most current browsers]

But I don't read anything fundamental in RFC3484 that would stop a 
developer from treating the result of the address selection rules as an 
ordered list, to be manipulated further, rather than an absolute selection.

If a developer chooses to implement both RFC 3484 and amended Happy 
Eyeballs, the app can still select "the best" IPv6 and "the best" IPv4 
(from RFC3484) and kick two sessions off in parallel for IPv4 and IPv6 
in a race with no initial delay. The app can then wait as long as the 
developer likes for both sessions to complete (implementation 
dependent). If only one session completes in a time acceptable for this 
application, job done, and the cache can be filled. If both session 
start ups complete within an acceptable time of each other then the 
developer can still use the list ordering from RFC3484 in any tie break 
for selecting the appropriate address family. I personally see this as 
minor difference to Chrome (perhaps a few more SYN packets until a 
cached entry is learned). I also think it's relatively simple, whilst 
still giving system/network managers a chance of influencing choice of 
AF via the existing RFC3484 mechanism, rather than any external timers/ 
additional config. Others may disagree.

Perhaps I'm being dumb and have missed something in the text that you 
have seen.
> end points, not the middle box.
I humbly disagree that it's the "only way."

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 20 Jun 2011 16:34:42 -0700

I think I'm with Ray and the others who liked the idea of a straight
race when both IPv4 and IPv6 addresses are available.

Thanks - Fred
[email-masked]

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Ray Hunter
Sent: Monday, June 20, 2011 2:30 PM
To: Cameron Byrne
Cc: [email-masked] WG
Subject: Re: [v6ops] simplfying Happy Eyeballs further

 "Standard" was perhaps an unfortunate choice of wording on my part. "Vanilla"?

Cameron Byrne wrote:

On Mon, Jun 20, 2011 at 1:01 PM, Ray Hunter <[email-masked]><mailto:[email-masked]> wrote:

See below.

Assuming "amended Happy Eyeballs" becomes a standard address selection algorithm based on a simple straight race with a cached result, as I suggested.

I see no contradiction personally. Address selection algorithms to me are an application level choice. They do not have to preclude each other, even though all are "standard."

If a developer chooses to implement "amended Happy Eyeballs" (and no RFC3484) then the browser app would start two sessions in parallel for the first IPv4 and first IPv6 returned by DNS (if there was no cached entry), and use the first one to complete the SYN ACK handshake in a straight race. Once the cache is filled there'd be no need to start two sessions. The cache would flush regularly, and also on interface changes. This is for me the easiest vanilla form to implement and is just a simplified version of the current Happy Eyeballs draft (with IH =0 and no handicapping algorithm).

If a developer chooses to implement just RFC3484, then the app could try one session to the first entry in the orderd list from the source / destination address selection process. If that session fails it starts a second session to the second entry and so on. [most current browsers]

But I don't read anything fundamental in RFC3484 that would stop a developer from treating the result of the address selection rules as an ordered list, to be manipulated further, rather than an absolute selection.

If a developer chooses to implement both RFC 3484 and amended Happy Eyeballs, the app can still select "the best" IPv6 and "the best" IPv4 (from RFC3484) and kick two sessions off in parallel for IPv4 and IPv6 in a race with no initial delay. The app can then wait as long as the developer likes for both sessions to complete (implementation dependent). If only one session completes in a time acceptable for this application, job done, and the cache can be filled. If both session start ups complete within an acceptable time of each other then the developer can still use the list ordering from RFC3484 in any tie break for selecting the appropriate address family. I personally see this as minor difference to Chrome (perhaps a few more SYN packets until a cached entry is learned). I also think it's relatively simple, whilst still giving system/network managers a chance of influencing choice of AF via the existing RFC3484 mechanism, rather than any external timers/ additional config. Others may disagree.

Perhaps I'm being dumb and have missed something in the text that you have seen.

<snip>

---

From: Mark Andrews <[email-masked]>
Date: Tue, 21 Jun 2011 11:06:22 +1000

In message <[email-masked]
> race when both IPv4 and IPv6 addresses are available.

If CGN added enough delay to make IPv4 slower I might agree with
you but they won't.  Additionally race to win always results in
redundant connection being made in a correctly working network which
is totally unneeded.

HE needs to work well enough with a broken IPv6 network and not
cause problems with a working IPv6 network.  This working group
seems to be too focused on making a broken network work well verses
not making a working network worse.

Applications supporting HE will be in use long after the need for
HE to address large scale IPv6 brokenness has gone and the failure
mode is "this address is unreachable" rather than "IPv6 is unreachable"
and all addresses are reachable 99.999% of the time.

Let getaddrinfo() do its job and sort the addresses.  Try them in
the order returned with a short timeout.  Re-use the working address
if you need to re-connect to the same host.  This will give you
reasonable performance with a broken/tunneled IPv6 or broken/tunneled
IPv4 network and good performance with when both stacks are working.

If we don't let getaddrinfo() do its job and sort the addresses
there will be complaints that the applications don't honour the
address selection rules in a few years time.

Mark

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Philip Homburg <[email-masked]>
Date: Tue, 21 Jun 2011 10:22:01 +0200

>redundant connection being made in a correctly working network which

How can you be so sure that CGN doesn't add any delays? If CGN is perfect from
a performance point of view, then there is no (performance) reason to try IPv6
at all.

I'm not so sure that all ISPs will be able to afford buying CGN capacity for
peak load. 

I agree that just letting IPv4 and IPv6 connection race is not the right way
to do things. 

>not making a working network worse.

Yes. It may be able to do more. But this should be the core functionality.

>and all addresses are reachable 99.999% of the time.

I think both are needed. A dual stack server may have a problem with IPv6, so
it would be good to avoid connecting over IPv6 to that server. But locally, 
there may also be a failure that affects just IPv6. 

>address selection rules in a few years time.

I don't care at all about the default policy table. I don't see why HE would
have to respect that.

It would be different if an admin specifically installs a different policy
table. So (at least conceptually) we may need some flags in the getaddrinfo 
result that specify whether HE should be active at all or whether it is allowed
to reorder addresses.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 21 Jun 2011 22:02:29 +1000

In message <[email-masked]>, Philip Homburg writes:
> How can you be so sure that CGN doesn't add any delays? If CGN is perfect
> from a performance point of view, then there is no (performance) reason
> to try IPv6 at all.

Please stop trying to put words in my mouth that I didn't say.  I
didn't say that it added no delay.  I said that I don't think CGN's
will have enough delay to cause race to win to favour IPv6 by any
significant amount.  Certainly not enough to shift the majority of
traffic to IPv6.

> I'm not so sure that all ISPs will be able to afford buying CGN capacity 
> for peak load. 

I suspect ISP's will put low volume users onto CGN's and leave high
volume users with public IP addresses.  That will give the biggest
address savings for the smallest outlays.  It also reduces the peak
loads that have to be met.

> result that specify whether HE should be active at all or whether it is
> allowed to reorder addresses.

Why is it different?  The policy table is supposed to be updatable.  There
is the reason it is called a "default policy table" not "the policy table"
as it was well known that there could be valid reasons to override it.

Mark

---

From: Philip Homburg <[email-masked]>
Date: Tue, 21 Jun 2011 14:35:11 +0200

>significant amount.  Certainly not enough to shift the majority of

I think the same comment remains. If CGN perform well enough that (from a 
performance point of view) is is almost indistinguishable from native IPv6,
then why bother with IPv6?

>loads that have to be met.

Sounds like a great way to fail. 'If you become a high volume user, you get a
public IP'. Great. Let's start downloading.

>as it was well known that there could be valid reasons to override it.

Yes. But with the default policy table in place, there is very good chance
that the 'admin' (in many cases just the owner of the device) didn't think
about it and mostly likely is also quite unaware of the very existance of the
policy table. No need to be overly strict about honoring that table.

So what we need is a way for an admin to disable HE when it is not wanted.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2011 00:37:12 +1000

> then why bother with IPv6?

Because "speed" is not the sole determinate about whether CGN is good or
not.
 
> public IP'. Great. Let's start downloading.

For lots of us that would just result in a throttled connection
after you have exceeded your montly allowance or you start paying
extra $/MB.

> So what we need is a way for an admin to disable HE when it is not wanted.

Why add more knobs than are actually needed.  Have you tried using
applications connecting from a dual stack box to a dual stack box
with one of the address families down and using a 100ms timer to
switch to the next address returned by getaddrinfo?

Mark

---

From: Philip Homburg <[email-masked]>
Date: Tue, 21 Jun 2011 17:00:39 +0200

>Because "speed" is not the sole determinate about whether CGN is good or

Yes. But low latency without losing packets is hard to get right. So it seems
a good metric for how well a device functions.

>after you have exceeded your montly allowance or you start paying

Yes, that's true. 

>switch to the next address returned by getaddrinfo?

I don't know where that 100ms comes from. Many sites need much longer to
connect than that. If you want to avoid spurious connections to IPv4 then
Google's 300 ms is much closer to what you need.

Some sites already put 4 or 5 IPv4 addresses in a DNS reply. If they would do
the same for IPv6 then we talking about a latency of upto 1500 ms. I don't
want to wait that long.

---

From: Andrew Yourtchenko <[email-masked]>
Date: Tue, 21 Jun 2011 20:13:16 +0200 (CEST)

> the order returned with a short timeout.  Re-use the working address

getaddrinfo is too optimistic about insisting on use of IPv6.

There's an interesting concept of label:

#label ::1/128       0
#label ::/0          1
#label 2002::/16     2
#label ::/96         3
#label ::ffff:0:0/96 4
#label fec0::/10     5
#label fc00::/7      6
#label 2001:0::/32   7

These seem to generalize the "address family" in the pure IPv4<->IPv6 
dichotomy.

So, how about this:

take the getaddrinfo() results, and reshuffle the sequence into several 
subsequences, such that each subsequence does not have more than 1 
occurence of an address within a particular label.

Then try to connect with short timeout, in this order.

This way you both honor the order of the 3484 and fail quickly in case the 
entire "class" of the addresses is broken ?

(of course, we're second-guessing what is more likely in case the first 
IPv6 address does not work - that you do not have a connectivity on IPv6 
at all or that it's only the specific IPv6 address that does not work. In 
case of having a chance of trying over more diverse options, I'd rather 
assume the former.)

As a user, I don't care whether the water arrives to my kitchen over 
plastic tubes or the brass tubes - as soon as it is reasonably clean and 
with good pressure.

cheers,
andrew

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 21 Jun 2011 11:57:49 -0700

Hi Mark, 

> -----Original Message-----
> From: Mark Andrews [mailto:[email-masked]] 
> Sent: Monday, June 20, 2011 6:06 PM
> To: Templin, Fred L
> Cc: Ray Hunter; Cameron Byrne; [email-masked] WG
> In message 
> <[email-masked]

Come to think of it, having a properly sorted list of
candidate addresses would also satisfy my needs, where
sometimes I'd like the applications to prefer an almost
certainly lower latency IPv4 service over an almost
certainly higher latency IPv6 service without having
to go to the trouble of running a race.

I am noticing that this discussion is largely failing to
take into account the *scope* of communications. It may
well be that communcaitions in which both endpoints
reside within the same site may have very different
address selection considerations than when the endpoints
are within different sites.

Thanks - Fred

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2011 08:02:26 +1000

In message <[email-masked]>, Philip Homburg writes:
> >> Sounds like a great way to fail. 'If you become a high volume user, you ge
> t 
> >a

I'm well aware of the speed of light in glass.  I live far for most
of the rest of the world's population.  100ms is a allowable tunnel
penalty.  It also is good enough for most in continent/country
connections.  Whatever figure we use won't be perfect.

Sydney - Amsterdam is about 220 ms rtt from where I sit in Sydney.
A trans Pacific tunnel adds up to 150-200ms delay depending apon
the destination location.

> the same for IPv6 then we talking about a latency of up to 1500 ms. I don't
> want to wait that long.

Only if you assume a 100ms per timeout.  Use 100, 50, 25, 12, 6,
3, 2, 1, 0 which sees you trying all addresses within 200ms.  Or
you could start with 150 and then 75, 37, 18, 9, 4, 2, 1, 0 to have
all address being attempted within 300ms. 

As for your 1500ms that is worst case with the last address tried
succeeding out of 10 addresses.

Mark

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 21 Jun 2011 16:28:05 -0700

> all address being attempted within 300ms.

Most of the IETF's protocols, for good reasons, have exponential backoff
when something is non-responsive.  This is the exact opposite -- if the
network is non-responsive, the network is punished with more aggressive
connection attempts.  Would it contribute (significantly?) to congestion 
on an already-congested network?  Could the idea pass IETF/IESG review?

-d

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2011 09:54:21 +1000

> on an already-congested network?  Could the idea pass IETF/IESG review?

Your not re-attempting the same addresses.  You are attempting
alternate addresses.  You exponentially back of subsequent attempts
to connect to the host if this group fails.

You still end up waiting 30 seconds or so if the host is down for
the proceedure to complete.

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 21 Jun 2011 16:58:06 -0700

> the proceedure to complete.

We had difficulties getting ICE (RFC5245) through IESG and transport 
area review, and it is doing similar things (sending probes to various 
addresses, and wants to send them fast to provide good service to 
the user).  This feels similar.

-d