Subject: [v6ops] Comments on draft-wing-v6ops-happy-eyeballs-ipv6-01

From: "Bjoern A. Zeeb" <[email-masked]>
Date: Thu, 27 Jan 2011 14:28:14 +0000 (UTC)

Hi,

(ignore the things that have been said since October already)

General comments upfront:

- I don't like the name of compnay-of-the-days being used as samples
   in standards, especially when there is no reference attached to it
   and if it's just one in a zillion.  You should generalize this.

- I left some private notes (XXX-BZ) in there that I had at the time
   of reading.  They may either provide information or be addressed at
   the end.

> 1.  Introduction
>
>    In order to use HTTP successfully over IPv6, it is necessary that the
>    user enjoys nearly identical performance as compared to IPv4.  A
>    combination of today's applications, IPv6 tunneling and IPv6 service
>    providers, and some of today's content providers all cause the user
>    experience to suffer (Section 3).  For IPv6, Google ensures a

The thing that makes me currently suffer the most wrt to IPv6 is
`magic (or call it bug) in a commercial OS resolver tralala` forcing me
to flush caches up to 70 times a day to be able to get to ipv6 enabled
sites.  This comment to my best understanding is very related to the
roots of your draft (but I have yet limited understanding of the actual
problem causing this bug to show up for people; it's been reported by
multiple, yet not fixed).

>   While the application recommendations in this document are described
>   in the context of HTTP clients ("web browsers"), but is useful and
>   applicable to other time-sensitive applications.

I'd remove the `other` here as "web browsing" is all but time-sensitive
in the classical meaning.  The 300ms you will not really notice for a
page-load-time can for some other network usage ruin entire entire
companies or cost them fortunes.

> 3.  Problem Statement
> ..
>   As discussed in more detail in Section 3.2, IPv6 connectivity is
>   sometimes broken entirely or slower than native IPv4 connectivity.

Contrary to my findings over the last years, where IPv6 often was
faster or more reliable especially on trans-continental links between
the same two dual-stack systems (as seen from DE to CA.US, DE to CA and
DE to AU).
I have even found that going via some 3rd party IPv6 rather than directly
into the destination network at the next IXP using their internal IPv4
transport, IPv6 was still faster.  I also like the variance, especially
as in the following samples the IPv6 is actually tunnelled and still
better.

(DE - AU)
round-trip min/avg/max/stddev  = 332.616/357.183/408.802/28.150 ms
round-trip min/avg/max/std-dev = 336.952/338.048/340.461/0.973 ms

(DE - CA [east coast])
round-trip min/avg/max/stddev = 129.852/136.686/145.283/5.634 ms
round-trip min/avg/max/std-dev = 118.636/120.272/121.628/0.996 ms

[I wonder if someone may have the data of a lot more probes somewhere?]

So I think, even though pointing to the other section, the above
should be a lot more precise or limit things to "very special
circumstances" as I think it's just the very obvious brokeness you try
to address but you'll also catch the 10ms deltas in other areas
unfortunately.

> 3.2.  IPv6
> 
> ...
>   Reasons for such failure include no connection to the IPv6 Internet,
>   broken 6to4 or Teredo tunnels, and broken IPv6 peering.

Hmm, if there is no connection you should get the almost instant
notification back and move on.  Broken IPv6 peering as much in
broken IPv4 peering (I don't think the legacy world is any better).

>         1.    |<--www.example.com A?-----|                       |
>         2.    |<--www.example.com AAAA?--|                       |
>         3.    |---192.0.2.1------------->|                       |
>         4.    |---2001:dba::1----------->|                       |

Hmm, www.example.com has A and AAAA records; either use those or
better 2001:db8:: rather than 2001:dba:: (here and throughout the document).

The problem described here, really is not that much IPv6 specific and
not much different to just multiple-A records, is it?

>   The client attempts to connect using IPv6 to the server, but the IPv6
>   path is broken (6-8), which consumes several seconds of time.

The entire thing strikes me as as general TCP behaviour (often tunable
by global MIB variables or per socket options depending on the OS)
told how to handle since at least RFC1122 (STD3) section 4.2.3.5.

> 4.1.  IPv6

I think the section title, if not already for 3.2, at least here is
misleading.  The very best it descripes "Dual Stack" behaviour.

>  In diagram above, ...

Add 'the'.

XXX-BZ revisit (what if there is more than one record returned per address
family (AF))

>   using IPv4.  The IPv6 path is retried until the application gives up
>   (10).

I think given the diagram you should s/the application gives up/TCP
times out/ here.

>   The absolute value of
>   P is the measure of a delay before initiating a connection attempt on
>   the other address family.

A 'connection attempt' in literature seems to be more the one of the
real upper layer protocoal (ULP) connection but what you want to say is
"the DNS lookup and connection attempt ..."

>   The TCP client application starts two threads in order to minimize
>   the user-noticeable delay ("dead time") during the connection
>   attempts:

I think the draft should not enforce how to implement things.
Threads are just one way to implement the expected (concurrent)
behaviour.  I would love to see the word 'thread' to go away from the
draft completely or detailed that other ways to implement the expected
behaviour may equally be chosen.

>       *  If P<0, wait for absolute value of p*10 milliseconds

Mix of 'P' and 'p'.  Should be all upper-case?

XXX-BZ revist - which P is this - the per destination or the global?
(should be disambiguated throughout the entire draft Ps (P socket) and
Pa (P application) or similar and one or the other or both mentioned
explicitly where appropriate.

>   The value of P
>   is incremented (decremented) each time an IPv6 (IPv4) connection is
>   successfully made.

I felt this is inaccurate.  Both IPv4 and IPv6 connections could
succeed, even before you can cancel the other (connection|thread),
so I think you mean s/is successfully made/wins the race/.

>   When a connection using the less-preferred
>   address family is successful, it indicates the wrong address family
>   was used and the P is halved:
>  ...

For the two items:
>   o  If P>0  ...
>   o  If P<0  ...

I'd prefer to have the exact same wording with just s/[46]/[64]/
s/[<>]/[><]/.

>   o  If P=0 (indicating equal preference), P is incremented if the
>      first thread to complete was the IPv6 thread, or decremented if
>      the first thread to complete was the IPv4 thread.

"incremented by 1" "decremented by 1"

>   which
>   is similar to the value used by many IPv6-capable TCP client
>   applications to switch to an alternate A or AAAA record.

Again, it's not the application usually (unless using setsockopt) but
the intial connection setup timeout of the TCP stack.

>      Note:  Proof of concept tests on fast networks show that even
>      smaller value (around 0.5 seconds) is practical.  More extensive
>      testing would be useful to find the best upper boundary that still
>      ensures a good user experience.

I think in both cases here and the paragraph above a value of P should
not be in seconds but the absolute value of 400 or 50 (given it's
|P|==400 [ * 10ms = 4s ] ).

I feel that |P|==50 is optimistic given that it can easily be 450+ms
from Europe to Australia meaning losing the race twice the application
would immediately favour the other AF.

XXX-BZ revist and explain the general problem with the maths in here
as I understand it; general failover, larger caches, bad expiry

XXX-BZ again, what about multiple AAAA or multiple A records and how
to count that if the first records wins but we move on to the second
for an application failure on that AF?

> 4.2.3.  Flush or Expire Cache
> ...
>   However, in some instances the
>   application and the host are unaware the network connectivity has
>   changed so it is RECOMMENDED that per-destination values expire after
>   10 minutes of inactivity.

XXX-BZ see further down

> 4.2.4.  Determining Address Type

I would remove that section entirely.  It would mean just more magic
that might not be needed at all.  If replacement transition technolgy
would work better than native, so be it;-)

> 4.2.5.  Debugging and Troubleshooting
> ...
> mechanism to temporarily use only one address family.

The much I'll love once we are there, this is somewhat out of scope I fear
as you can write an entire draft about the pros and cons of just that if
you have the time.  I'd much rather like to see the fallback to revert to
the well-known beviour without this draft (whatever it is, mostly usually
disabling the concurrency and trying on after the other).
If you disable one address family a debug trace will usually only show you
half of the information you may want/need.

Ok, some of the XXX-BZ to be addressed still in general outside the
scope of a specific paragraph:

"Which P?"

I feel the two 'P's need to be better explained, as in when to update
which one and when to pick which one.  If I had to guess I would go
for a specific implementation but that does not mean anyone else would
go for the same.  Be more explicit about that.

"What about multiple AAAA or A records?"

I know the abstract says "dual stack client" to "dual stack server"
but while in a majority of the cases that might still be true, it's an
ideal world that seems to be especially untrue for the top n sites.

What should an application do if it gets back only two AAAA records
or 2 AAAA and 4 A records?

It may well be one of the machines is in Europe and the other is in the
Asia?  Do you want to try those in sequence only if one fails or do
you also want to measure the 150ms out that the other one could be
faster?  When, in your model, will you then start more threads?

What if the first A wins your match above but the ULP errors? How will
the retry work or the moving on to another record? Will we try the
other AF then depsite having clsoed the conneciton there?

Depsite they usually should do not expect application to behave exactly
the same no matter which AF you are using.

"general failover, larger caches, bad expiry"

Flushing entries from the cache after 10 min. of inactivity has multiple
problems:

- Say my browser has about the same uptime as my machine.
- Say I keep random-modern-ajax-website open in a window all the time.
- The website will refresh itself every minute or even have "pseudo"
   persistent connections (there's a lot of variance in there what
   you can observe).

The result of this will be kind of like: my preference will grow to the
max until there is a network hickup.  The question then is will it
last long enough so the preference will flip or will I just see the
3s delay?  Or will it cause the converse and put me on the other
address family for the next week despite my conenctivity in my
actually prefered address family would be all fine again after a
couple of minutes?

Next problem is that you are creating a shadow cache in each
application on (hostname,port,P,timeout) which can easily get
into the thousands in 10 minute sliding window of inactivity depending
on the application.  That seems wrong to me but I guess people will fix
it with a more intelligent purging strategy (depending on application) --
say I close my browser tab for a site, purge the entry if it's the last
tab with a connection to the host.  Too bad if it was my "home page".
People will be creative once they hit an in-application specific "upper
bound" of cache size despite memory being cheap.

Another thing I am worried is that the experience in the non-"fast"
networks in the non-ideal world will kill IPv6 more often than you
wished, especially in the early transition time as it will be for
most users.   The reason for that is:  while these days you often
get a local A mirror close by for the larger sites, the AAAA machine(s)
are more often further away (which can be in another state or country or
even continent).  This will impose extra delay for IPv6
"lookup+connection attempt" times possibly quickly degrading P to be
IPv4 friendly.  It's unpredictable how the content will move to IPv6
yet, as I see it but I am almost sure the world wide global "IPv6 day"
turn on switch everywhere will not happen.  The preference will slowly
change towards v6 but in the mean time you want to push it rather than
slowing it down.
Again, as said at the beginning, it will work well for the really broken
cases but it will also affect the "10 ms difference" setups.

So will it be a better user experience for me if I'll go through a CGN
which may still is quicker than the native IPv6?

One way to circumvent the aforementioned weekly flap would be to flush
every cached record every n minutes (a time also long enough for a route
flap to happen and propagate, or just an hour or a day or worst the
TTL of the record returned the application will never learn).  It's
something that you can spend a year of research on to properly tune that
for most places around the globe.  Time noone has, which makes this bad.

I said that a |P|==50 sounds too optimistic for me. What will be a good
absolute value of P?  Should the be a static recommendation or not?

What about non-TCP?  The first mention of TCP is in 4.1 (ignoring the
earlier diagram).  Should it equally apply to non-TCP or should TCP be
mentioned already in the abstract?

The very last thing is that the winning address you can connect to
fastest does not have to be the address that gives you the best user
experience as it could be the least loaded machine for other reasons:
the application could hang there, ... the application could still have
problems with connections on this address family, ...  Will that help
user experience?  Yes it's another problem domain but "user
experience" doesn't see the difference.

/bz

-- 
Bjoern A. Zeeb                                 You have to have visions!
         <ks> Going to jail sucks -- <bz> All my daemons like it!
   http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html

---

From: "Frank Bulk" <[email-masked]>
Date: Thu, 27 Jan 2011 09:41:01 -0600

In regards to IPv6 speed, these sites:
	http://www.ipv6matrix.org/ (IPv6 Faster? tab)
	http://speedlab01.cmc.co.ndcwest.comcast.net:8088/monitor/ (choose
one of the two download times from the drop down)
has something a bit more quantitative.

Frank

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Bjoern A. Zeeb
Sent: Thursday, January 27, 2011 8:28 AM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: [v6ops] Comments on draft-wing-v6ops-happy-eyeballs-ipv6-01

<snip>

(DE - AU)

(DE - CA [east coast])

<snip>

-- 
Bjoern A. Zeeb                                 You have to have visions!
         <ks> Going to jail sucks -- <bz> All my daemons like it!
   http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html
_______________________________________________
v6ops mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/v6ops