Subject: Re: [v6ops] Lion/Snow Leaopard side-by-side on an IPv6 enabled ADSL2+ home line..

From: Philip Homburg <[email-masked]>
Date: Mon, 01 Aug 2011 09:51:26 +0200

In your letter dated Mon, 1 Aug 2011 07:37:19 +0200 you wrote:
>On Mon, Aug 01, 2011 at 11:20:02AM +1000, George Michaelson wrote:
>> All three connect on IPv6 for Snow Leopard.
>> All three connect on IPv4 for Lion.
>
>Excellent, so operators have to make sure that their LSN setups will
>artificially introduce some delay to bias clients to use IPv6 to
>offload the expensive NATs. We shall raise feature requests with the
>vendors.

What I did in my HE implementation is that when sorting the output of
getaddrinfo I give the first address a 'bonus' of 30ms. The net effect is that
as long as the first address is within 30ms of the best address, the first
address will be used.

I think this is good compromise between on the one hand following the
destination selection policy and address order in a RR set and on the
other hand protecting the user from badly performing addresses.

---

From: Sander Steffann <[email-masked]>
Date: Mon, 1 Aug 2011 09:47:46 +0200

Hi Daniel,

> offload the expensive NATs. We shall raise feature requests with the

I think the NAT will do that automatically all by itself.

Met vriendelijke groet,
Sander Steffann

---

From: Daniel Roesen <[email-masked]>
Date: Mon, 1 Aug 2011 16:39:49 +0200

> I think the NAT will do that automatically all by itself.

Judging from what I'm seeing: Not really. We're talking about
sub-millisecond added latency which probably is deep in the noise level
of DNS resolution time variance anyway (caveat: I haven't measured and
compared that myself).

So from this perspective, a HE implementation will (unless the ISP uses
substandard hardware to implement the NAT) probably not notice the NAT
by any added delay.

Best regards,
Daniel

-- 
CLUE-RIPE -- Jabber: [email-masked] -- dr@IRCnet -- PGP: 0xA85C8AA0

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 1 Aug 2011 08:29:46 -0700

> substandard hardware to implement the NAT) probably not notice the NAT

+1 to confirming CGN/LSN does not add latency in a meaningful way.

I also find that injecting latency at the nat will be hard sell to
management and possibly difficult to instrument.  I suggest users use Chrome
or Firefox which produces the desired results at the host.

Cb

> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: Philip Homburg <[email-masked]>
Date: Mon, 01 Aug 2011 18:26:44 +0200

>or Firefox which produces the desired results at the host.

As a user, why would I do that?

I've no idea how Chrome and Firefox interact with getaddrinfo. Do they
actually move an IPv6 to the front of the list. Or do they implement their
own solver, bypassing getaddrinfo?

Anyhow, assuming Chrome and Firefox would have their own getaddrinfo
implementation, you get a 200ms delay whever IPv6 doesn't work. It doesn't
even have to be at the user's end, some people manage to misconfigure IPv6
on their servers as well.

So what kind of horrors can we expect from CGN that would make Chrome and
Firefox a better deal?

---

From: Lorenzo Colitti <[email-masked]>
Date: Mon, 1 Aug 2011 09:31:05 -0700

> So what kind of horrors can we expect from CGN that would make Chrome and

One example is the NAT intercepting your HTTP request and showing you a nice
"you've run out of ports" error message instead of the web page you wanted
to see.

---

From: Philip Homburg <[email-masked]>
Date: Mon, 01 Aug 2011 18:36:44 +0200

>"you've run out of ports" error message instead of the web page you wanted

I'm sure somebody is going to do something as stupid as that. So let's assume
that it will happen.

Then a slightly more advanced NAT implementation might take that situation
as a signal it should start delaying SYN+ACKs to avoid overload and to
allow HE to do its trick.

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 1 Aug 2011 09:57:48 -0700

> allow HE to do its trick.

The first thing that comes to mind here is an arms races.

I believe the scope of LSN/CGN is "big and dumb" so that it can scale,
what you are describing is yet another level of state and complexity
in a box that we are trying to keep simple (no ALG ...)

Cameron

---

From: Philip Homburg <[email-masked]>
Date: Mon, 01 Aug 2011 19:07:57 +0200

>in a box that we are trying to keep simple (no ALG ...)

It seems to me that the easiest way to deal with lack of ports is to simply
drop the SYN.

To be able to send an HTTP error message, would require faking a SYN+ACK plus
an HTTP reply. And that only to screw up anything automated that runs over
http.

Plus that just dropping the SYN may just slow things down enough to let
requests go through. Instead of users recognizing fake error replies and
writing proxies that hammer the NAT box until they do get a useful reply.

---

From: Simon Perreault <[email-masked]>
Date: Mon, 01 Aug 2011 13:18:16 -0400

Lorenzo Colitti wrote, on 08/01/2011 12:31 PM:
> "you've run out of ports" error message instead of the web page you wanted to see.

Note that this would break REQ-13 from draft-ietf-behave-lsn-requirements.

It's tempting to add this as a MUST NOT sub-bullet...

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: Hagen Paul Pfeifer <[email-masked]>
Date: Mon, 1 Aug 2011 19:55:52 +0200

* George Michaelson | 2011-08-01 11:20:02 [+1000]:

>www.apnic.net:    29.668/ 30.224/ 30.511/ 0.305 ms   27.484/ 33.900/ 55.400/ 9.697 ms 
>www.ripe.net:    384.082/427.672/474.278/32.085 ms  345.837/379.814/450.711/37.940 ms
>ipv6actnow.org:  371.395/409.083/457.154/29.726 ms  346.257/384.443/434.757/30.477 ms
>
>I think that from a difference/variance point of view, all three sites were
>rationally 'better' on IPv4
>
>So, in that sense "it worked". -The selection alg as encoded in Lion, is
>doing a reasonable job.

To quote Josh Graessley from Apples IPv6 ML
http://web.archiveorange.com/archive/v/Wh4rykklKYyRDNa0EJWU#70wFtRVnk32QeQ4:

<begin>

There are some significant changes in Lion.

Results from getaddrinfo are now sorted using routing statistics (destination
with the lowest min round trip time wins). If the statistics can not determine
which destination is better, an implementation of RFC3484 is used. The default
RFC3484 policy is read only.

CF and NS layer frameworks that use CFSocketStream do not use getaddrinfo.
Those APIs use something similar to happy eyeballs. The A and AAAA queries are
started at the same time but the responses are handled as they are received.
When an answer is received, it is sorted in to a list of destination
addresses. If there are no more addresses coming in (this was the last answer
in the DNS packet or mDNSResponder has no more answers in the cache), a
connection is started to the first destination on the sorted list. The DNS
resolve operation is left running and more answers are processed as they
arrive. A timer is setup for a period of time in which we would expect the
connection to complete, based on the routing statistics. If the timer fires
before the connection is established, a connection to the next best address
will be started while the existing connection continues to try and make
progress. A similar timer is setup and the process repeats until a connection
is established or we run out of addresses to try. The code keeps track of
whether or not it has received both A and AAAA response (whether the answer
was a list of addresses or no address). If the connection is established
before both A and AAAA responses come back, the resolve is kept open for up to
a second to allow mDNSResponder to receive a slow response and store it in the
cache. This way, subsequent connections to the same host in a short period of
time will have all answers in the cache.

Most users that aren't on this list don't care if they connect over IPv4 or
IPv6. They just care that they connect. The code in CF and NS aims to make
sure the user always gets a connection quickly. The code above also addresses
issues where a AAAA response is never received (doesn't hold up connecting to
IPv4) and issues where the user has some busted equipment that sends routing
advertisements for a prefix even though the equipment has no way to route IPv6
traffic. The trade off is that it may be hard to predict whether a connection
will occur over IPv6 or IPv4. If an option were added to prefer one address
family over the other, it would need to indicate how much longer the user was
willing to wait to get the address family of their choice.

You can use the command line tool "nettop -n -m route" to dump a live view of
the routing statistics. You can use "nettop -n" to dump a live view of all TCP
and UDP sockets on the system.

-josh

<end>

Cheers, Hagen

-- 
Hagen Paul Pfeifer            ||  http://protocollabs.com
Telephone: +49 174 5455209    ||  Key Id: 0x98350C22
Key Fingerprint: 490F 557B 6C48 6D7E 5706 2EA2 4A22 8D45 9835 0C22

---

From: Daniel Roesen <[email-masked]>
Date: Tue, 2 Aug 2011 14:25:45 +0200

On Mon, Aug 01, 2011 at 08:29:46AM -0700, Cameron Byrne wrote:
> management and possibly difficult to instrument.

It's not a hard sell at all, given that the alternative is to
unnecessarily scale up the LSN setups for IPv4 traffic (which could be
native, non-NATted IPv6 traffic).

Looking at economics, there can be only one strategy: get as much
traffic as feasible onto IPv6 as quickly as possible to avoid exploding
cost of LSN infrastructure (be it NAT444, DS-Lite, or whatever else).
Supporting IPv4 traffic will be much more expensive in the future than
before.

Keeping traffic on NATted IPv4 more than really necessary (read:
acceptable delay penalty) harms us all in multiple ways.

Best regards,
Daniel

---

From: Philip Homburg <[email-masked]>
Date: Tue, 02 Aug 2011 14:51:48 +0200

>native, non-NATted IPv6 traffic).

At least one alternative is to pressure vendors into implementing a form of
HE that prefers IPv6 when the performance of IPv6 and IPv4 is roughly equal.

---

From: Cameron Byrne <[email-masked]>
Date: Tue, 2 Aug 2011 08:04:12 -0700

> HE that prefers IPv6 when the performance of IPv6 and IPv4 is roughly
equal.

Hopefully they will do as you say because it is the right thing to do, as
chrome and firefox have already done. Sans that, the isp can promote an
appropriet HE implementation that fits everyone's need to be off the LSN.
After all, LSN costs and service quality are passed on to the customer.

Cb

---

From: Philip Homburg <[email-masked]>
Date: Tue, 02 Aug 2011 17:17:24 +0200

>After all, LSN costs and service quality are passed on to the customer.

At the moment there is almost no content on IPv6 and also about no ISPs
providing their users with IPv6.

So, vendors have plenty of time to get their act together in this respect.

---

From: james woodyatt <[email-masked]>
Date: Tue, 02 Aug 2011 12:27:15 -0700

On Aug 2, 2011, at 05:51 , Philip Homburg wrote:

> At least one alternative is to pressure vendors into implementing a form of HE that prefers IPv6 when the performance of IPv6 and IPv4 is roughly equal.

I don't see that working very well.  The key phrase there is "roughly equal" and I don't think it has any useful meaning in practice here.  Exhibiting no preference whatever between IPv4 and IPv6 is simpler in practice and delivers more consistent results.  Besides, I have a hard time imagining such pressure originating in demands on Apple from its paying customers.  What other form of pressure do you have in mind?

--
james woodyatt <[email-masked]>
member of technical staff, core os networking

---

From: Philip Homburg <[email-masked]>
Date: Tue, 02 Aug 2011 23:05:30 +0200

In your letter dated Tue, 02 Aug 2011 12:27:15 -0700 you wrote:
>I don't see that working very well.  The key phrase there is "roughly equal" a
>nd I don't think it has any useful meaning in practice here.  Exhibiting no pr
>eference whatever between IPv4 and IPv6 is simpler in practice and delivers mo
>re consistent results.  

I'd say that preferring IPv6 is more consistent. At the moment I'm using
30ms as the cut off for 'roughly equal' and that seems to be working well.

When my HE implementation connects to IPv4 I know that IPv6 is off by at least
30ms.

For quite a while I didn't want to this either. I had the same idea that
connecting to whatever is faster is best. But when you think about it,
the whole world trying to connect to whatever system is fastest. You deny
operators an imporant tool for essentially no gain for the customer. I apply
the same 30ms to first record in the RR set as well, so authorative DNS
servers also have some control (assuming intermediate DNS resolvers, etc.
do not reorder the set).

>Besides, I have a hard time imagining such pressure or
>iginating in demands on Apple from its paying customers.  What other form of p
>ressure do you have in mind?

That could be all kinds of things. I don't think it would be good if ISPs
were openly hostile to Apple or if people write how badly Apple implemented
certain features.

But it may take a while for enough dual-stack content to arrive to be able to
actually measure the effect.

---

From: james woodyatt <[email-masked]>
Date: Tue, 02 Aug 2011 16:12:32 -0700

On Aug 2, 2011, at 14:05 , Philip Homburg wrote:
> 
> I'd say that preferring IPv6 is more consistent. At the moment I'm using 30ms as the cut off for 'roughly equal' and that seems to be working well.

Those 30ms of additional latency contribute significantly to the bandwidth delay product.

You may find them acceptable, but Apple engineers have looked pretty closely at it and decided that no time interval that could possibly make a difference would be tolerated by users.  I don't want to make users pay with a delay factor in the bandwidth delay product just to push an agenda that operators may or may not actually share, depending on with whom you are talking this week.

> operators an imporant tool for essentially no gain for the customer.

I don't see it that way.  I see those 0-30ms of additional latency as an unacceptable penalty that we would be asking our users to pay, in exchange for nothing of any measurable value to them.  Do you see where our different perspectives leads us to reach different conclusions?

---

From: Ted Lemon <[email-masked]>
Date: Tue, 2 Aug 2011 23:23:53 +0000

On Aug 2, 2011, at 7:12 PM, james woodyatt wrote:
Those 30ms of additional latency contribute significantly to the bandwidth delay product.

Can you expand on that?   I hear this a lot, but it doesn't make much sense, first because the ratio of SYNs to data-bearing packets is typically large, and secondly because an adaptive algorithm should automatically eliminate significant delays.

Have you done any kind of data collection and analysis that supports your conclusion?

---

From: Fred Baker <[email-masked]>
Date: Tue, 2 Aug 2011 16:37:11 -0700

On Jul 31, 2011, at 6:20 PM, George Michaelson wrote:

> I had a moment in time to test a Snow Leopard, and Lion OSX setup @home, before completing my upgrade to Lion on all boxes.
> 
> I tested http://www.apnic.net/ http://ipv6actnow.org/ and http://www.ripe.net/
> 
> The ping6/ping delay differences:
> 
>                              IPv6                     IPV4
> I think that from a difference/variance point of view, all three sites were rationally 'better' on IPv4

I'm missing something. with APNIC and RIPE, the average RTT was less with IPv6 than IPv4, and in all three cases the standard deviation of the delay was less with IPv6.

You must be looking at the data in a way that isn't related to your statistics... I don't understand your analysis.

> So, in that sense "it worked". -The selection alg as encoded in Lion, is doing a reasonable job.
> 
> -George

---

From: George Michaelson <[email-masked]>
Date: Wed, 3 Aug 2011 09:46:41 +1000

> You must be looking at the data in a way that isn't related to your statistics... I don't understand your analysis.

the figures are min/avg/max/std-dev

Column 1 set is IPv6. Column 2 set is IPv4. 

in two, the IPv6 figure is worse RTT in avg compared to IPv4. I cannot read this as average RTT less with 6 than 4. 

apnic 30.244 vs 33.900 is the only one with an apparently better avg and its the one with the biggest variance in std-dev (0.0305 vs 9.697) and I suspect, given how *few* packets the HE has to work with, thats a consequence.

ripe, the avg is 427.672 (v6) vs 379.814 (v4) and the std dev is 32.085 vs 37.940. much closer std-dev. HOw did you see the RIPE average RTT as less in V6?

I agree the std dev is less. For short ping runs, I suspect its not relevant. I should of course have done far far longer pingruns.

Why not get a student to test? I think this is well within the capabilities of an undergrad networks course!

-G

---

From: james woodyatt <[email-masked]>
Date: Tue, 02 Aug 2011 17:53:27 -0700

> Have you done any kind of data collection and analysis that supports your conclusion?

Yes, we collect data.  We've collected a *lot* of data.  Please remember that we didn't implement the happy-eyeballs draft.  We're not talking about just the 3-way handshake completion time.  We collect statistical data from measurements of the round-trip times stored in the protocol control structures, and we prefer destinations with shorter average round-trips over longer and/or unknown ones.

The main reason Apple implemented its concurrent TCP connections SPI to the CFNetwork layer is *not* to support IPv6 transition, but instead to help solve an unrelated problem that arises entirely in IPv4-only networks.  It's just gravy that it also happens to help in the IPv6 transition case.

We only use the order specified in the RFC3484-revised-02 default policy table for the remaining destinations after we have finished initiating connections to all the destinations with statistically relevant round-trip time measurements.

---

From: Michael Newbery <[email-masked]>
Date: Wed, 3 Aug 2011 13:42:05 +1200

> Have you done any kind of data collection and analysis that supports your conclusion?

Looking at http://www.apple.com I count
32 Images
11 Scripts
5 Stylesheets
and the base HTML document itself. 

49 URLs to be fetched

Apple/s front page is if anything atypically spare in terms of elements. The New York TImes has 126 URLs.

If we were just looking at delaying the SINGLE first SYN, and if in addition we perform no parellisation or other optimisation (and of course modern browsers do both, but anyway) then 30ms would add 1.5 seconds to the load time of Apple's page and 3.8 to the NYT. Just from the bandwidth delay product.

As someone who is 75ms from the West Coast of the USA as the photon flies I am acutely aware of what bandwidth delay product does to my customers. The SYN:Data ratio is completely irrelevant here. TCP Windowing can't help during the TCP handshake, and that's what the customers perceive as slow page loading. Of course, for speed of light latency it's even worse as we suffer the delay for the full 3-way TCP handshake, not just the first SYN delay as proposed here.

Modern software goes to great lengths  to counter this with smart loading etc, but don't discount even a single extra ms as not mattering.

---

From: George Michaelson <[email-masked]>
Date: Wed, 3 Aug 2011 11:56:01 +1000

Its important not to over-estimate the amount of parallelism available in the browser. And, for that matter, under-estimate!

I assumed we got it all over HTTP 1.1 pipes. Turns out, not widely deployed. 

Some say, its 5 channels of parallelism max.

Some say, thats not honoured in practice.

In the various NAT wars and competing 444/666/464/646/6ot4/<insert random number> arguments, many have said that per user, 1000 ports is not unreasonable and I believe IVI and like documents discuss the relative numbers per customer, and at what point they begin to impact things..

-G

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 09:24:59 +0200

Le 2 août 2011 à 17:17, Philip Homburg a écrit :
...
>> After all, LSN costs and service quality are passed on to the customer.

Indeed.

That's why stateless solutions, where they apply, can be better for users than LSN-based ones: they tend to be less expensive than LSN-based solutions, in capex and opex (tools.ietf.org/html/draft-operators-softwire-stateless-4v6-motivation-02).

> providing their users with IPv6.

Note that Google is a very significant IPv6-enbled content provider.
IETF is less significant, but also a useful one.
There may be, I suppose, many others we don't know.

> So, vendors have plenty of time to get their act together in this respect.

Whatever the amount of time found to be left for ubiquitous IPv6 enablement, the earlier the better.

Regards,
RD

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 3 Aug 2011 00:35:37 -0700

On Aug 3, 2011 12:25 AM, "Rémi Després" <[email-masked]> wrote:
> That's why stateless solutions, where they apply, can be better for users
than LSN-based ones: they tend to be less expensive than LSN-based
solutions, in capex and opex (
tools.ietf.org/html/draft-operators-softwire-stateless-4v6-motivation-02).

Do you have data to support the claim that stateless is less expensive ? I
have not seen a kit price comparison ? projected cost comparison of
stateless vs. Stateful ?

Cb

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 09:55:50 +0200

Hi, James,

100% agreement on points you make.

May I add that:

- To promote IPv6, that is generalizing good IPv6 service that will be efficient (not wasting energy and money to further damage the IPv4 service). 
With IPv4, the lack of global addresses available everywhere, and the uncertainty as to how many ports applications can use, are already good reasons to generalize IPv6-service availability.

- How damageable to users a voluntary impediment would be cannot be reliably estimated from ping-based measurements, made at a specific time from specific locations.

King regards,
RD

Le 3 août 2011 à 01:12, james woodyatt a écrit :

> You may find them acceptable, but Apple engineers have looked pretty closely at it and decided that no time interval that could possibly make a difference would be tolerated by users.

+1

>  I don't want to make users pay with a delay factor in the bandwidth delay product just to push an agenda that operators may or may not actually share, depending on with whom you are talking this week.

+1

> I don't see it that way.  I see those 0-30ms of additional latency as an unacceptable penalty that we would be asking our users to pay, in exchange for nothing of any measurable value to them.  

+1

> Do you see where our different perspectives leads us to reach different conclusions?
> 
> 
> --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 3 Aug 2011 09:57:10 +0200 (CEST)

On Wed, 3 Aug 2011, Cameron Byrne wrote:

> Do you have data to support the claim that stateless is less expensive ? 
> I have not seen a kit price comparison ? projected cost comparison of 
> stateless vs. Stateful ?

Compare 6to4 wirespeed support in Cisco 7600/Sup720 for instance, where 
it'll do 10-15Mpps (or so) of translation, and this on 5+ year old kit. If 
they would have known about 6RD back then, I'm sure they could have 
implemented it at little extra cost.

Even today, buying similar capacity stateful gear I'd say is more 
expensive, and they also have quite limited session setup rates compared 
to this nonexistant problem in stateless implementations.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Philip Homburg <[email-masked]>
Date: Wed, 03 Aug 2011 10:11:39 +0200

In your letter dated Tue, 02 Aug 2011 16:12:32 -0700 you wrote:
>> I'd say that preferring IPv6 is more consistent. At the moment I'm using 30ms
> as the cut off for 'roughly equal' and that seems to be working well.
>
>You may find them acceptable, but Apple engineers have looked pretty closely at
> it and decided that no time interval that could possibly make a difference wou
>ld be tolerated by users.  

I don't think I can follow you. What context do you have in mind?

In my mind, HE is something that works for many, short-lived connections.
HE only gets it right on average. With many short-lived connections that is
fine. If you start a large download or if it is a long ssh session,
a vnc session, etc., and HE gets it wrong, then it will have a significant
impact.

Taking your statement literally, 30ms delay on a 1Gbit/s links requires extra
buffering of 3.75 MByte. I'm sure you can find products with a 1 Gbit/s 
interface that are short on memory, but I don't think Apple products are among
them. If we drop down to more realistic WAN speeds, then at 100Mbit/s, the
extra buffer requirements are 375kByte. 

So, I'm quite curious what customers you have that would notice an extra 4MByte
memory used per 1 Gbit/s of WAN speed.

I don't have enough experience with Apple products to know if Apple does
something exceedingly clever, but when links start filling up, overall delay
increases.

Your statement suggest that Apple can do large downloads and still keep the
impact on total latency far less than 30ms. This is very impressive. That
would mean that VoIP etc, would not require any special treatment because
just buffering for tens of milliseconds would be enough. Great.

Is that delay control technology already in Snow Leopard or do I have to upgrade
to Lion? I guess I can do some experiments to see how it works in Snow Leopard.

>push an agenda that operators may or may 
>not actually share, depending on with whom you are talking this week.

I'm quite curious what ISPs prefer CGN over IPv6. But there is an easy way out
for them: just don't roll out IPv6.

>I don't see it that way.  I see those 0-30ms of additional latency as an unacce
>ptable penalty that we would be asking our users to pay, in exchange for nothin
>g of any measurable value to them.  Do you see where our different perspectives
> leads us to reach different conclusions?

I think CGNs have many risks. Apple ships products worldwide. So customers
do not just connect to ISPs who have enough money to do CGN perfectly. There
will be plenty will do CGN just barely good enough. And my experience with
bad NATs tells me that I would gladly except a 30ms extra delay to get an 
IPv6 connection without NAT. Life is too short to deal with flaky network
connections.

In an organization that actually has its act together, the difference in latency
between IPv6 and IPv4 is close to zero. So another question is what customers
you are thinking about who have IPv6 connections that are actually 30ms
worse than IPv4 and still care that much about performance. It doesn't
seem to add up.

---

From: Rémi Després <[email-masked]>
Date: Wed, 3 Aug 2011 10:14:49 +0200

Le 3 août 2011 à 09:35, Cameron Byrne a écrit :

> Do you have data to support the claim that stateless is less expensive ? I have not seen a kit price comparison ? projected cost comparison of stateless vs. Stateful 
> 
No precise data, but confidence in past experience in other contexts, and in reasons expressed by operators who signed the draft.
The point is only that, since users always pay at the end, they have a stake in having operators choosing solutions that are the less expensive to them.

If a particular operator has competitive prices for LSN solutions, that's a good choice for it (I appreciate arguments you made and which apply to your 3GPP network). 
It is clear, however, that the draft is from operators having different constraints.
That's why I only said "tend to be" less expensive.

Regards,
RD

---

From: Philip Homburg <[email-masked]>
Date: Wed, 03 Aug 2011 10:21:07 +0200

In your letter dated Wed, 3 Aug 2011 11:56:01 +1000 you wrote:
>Its important not to over-estimate the amount of parallelism available in the b
>rowser. And, for that matter, under-estimate!
>

I've been looking quite a bit a web traffic recently. As far as I can tell,
it is almost exclusively http/1.1. Sometimes there is an http/1.0 box, but
is is very rare.

>Some say, thats not honoured in practice.

Ah, you mean HTTP pipelining? 

I don't think it is used very much. I don't think it is very important either.
Firefox defaults to 15 parallel connections to a single server. At that point
adding piplining doesn't give you much extra.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Wed, 03 Aug 2011 10:36:25 +0200

Similar experience here. Since installed Lion, I've difficulty to access
with IPv6, where it worked before, and still works with Windows.

I think Apple is doing very wrong with this implementation. A small RTT
difference should not give preference to IPv4, and the user needs to have
a way to chose it.

Being so strict with RTT will damage IPv6 deployment.

Regards,
Jordi

-----Mensaje original-----
De: George Michaelson <[email-masked]>
Responder a: <[email-masked]>
Fecha: Wed, 3 Aug 2011 09:46:41 +1000
Para: Fred Baker <[email-masked]>
CC: IPv6 Operations <[email-masked]>
Asunto: Re: [v6ops] Lion/Snow Leaopard side-by-side on an IPv6 enabled
ADSL2+ home line..

>>> I had a moment in time to test a Snow Leopard, and Lion OSX setup
>>>@home, before completing my upgrade to Lion on all boxes.
>>> 
>> I'm missing something. with APNIC and RIPE, the average RTT was less
>>with IPv6 than IPv4, and in all three cases the standard deviation of
>>the delay was less with IPv6.
>> 
>> You must be looking at the data in a way that isn't related to your
>>statistics... I don't understand your analysis.
>
>in two, the IPv6 figure is worse RTT in avg compared to IPv4. I cannot
>read this as average RTT less with 6 than 4.
>
>apnic 30.244 vs 33.900 is the only one with an apparently better avg and
>its the one with the biggest variance in std-dev (0.0305 vs 9.697) and I
>suspect, given how *few* packets the HE has to work with, thats a
>consequence.
>
>ripe, the avg is 427.672 (v6) vs 379.814 (v4) and the std dev is 32.085
>vs 37.940. much closer std-dev. HOw did you see the RIPE average RTT as
>less in V6?
>
>
>I agree the std dev is less. For short ping runs, I suspect its not
>relevant. I should of course have done far far longer pingruns.
>
>Why not get a student to test? I think this is well within the
>capabilities of an undergrad networks course!
>
>-G
>
>> 

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.consulintel.es
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited.

---

From: Philip Homburg <[email-masked]>
Date: Wed, 03 Aug 2011 10:45:38 +0200

In your letter dated Wed, 3 Aug 2011 13:42:05 +1200 you wrote:
>Apple/s front page is if anything atypically spare in terms of elements. >The New York TImes has 126 URLs.
>
>If we were just looking at delaying the SINGLE first SYN, and if in >addition we perform no parellisation or other optimisation (and of >course modern browsers do both, but anyway) then 30ms would add 1.5 >seconds to the load time of Apple's page and 3.8 to the NYT. Just from >the bandwidth delay product.

The delay bandwidth product is the amount of data that has to be kept in
flight to fill the 'tube'.

You can't just add up delays of different connections. It is the delay
across a single connection that matters.

Anyhow, browsers do use parallelism to reduce the impact of latency. Why do
you then present a calculation that assumes they don't?

For 49 URLs at 15 URLs in parallel to a single server, this would give an
additional delay of 120ms. Hardly worth writing home about. For 126 URLs
it would be 270ms.

>As someone who is 75ms from the West Coast of the USA as the photon >flies I am acutely aware of what bandwidth delay product does to my >customers. The SYN:Data ratio is completely irrelevant here. TCP >Windowing can't help during the TCP handshake, and that's what the >customers perceive as slow page loading. Of course, for speed of light >latency it's even worse as we suffer the delay for the full 3-way TCP >handshake, not just the first SYN delay as proposed here.

Slow start is an issue here. And some content providers play tricks with that.

>Modern software goes to great lengths  to counter this with smart >loading etc, but don't discount even a single extra ms as not mattering.

Not all content is available at your local IX. In my experience, page design
is a major factor. Bloated, badly designed pages take forever to load on
anything that is not on the page designer's LAN. Well designed pages load
quickly even across continents.

---

From: Michael Newbery <[email-masked]>
Date: Thu, 4 Aug 2011 12:39:56 +1200

On 3/08/2011, at 8:45 PM, Philip Homburg wrote:

>> and the base HTML document itself.=20
>> 
>> 49 URLs to be fetched
>> 
>> Apple/s front page is if anything atypically spare in terms of elements. =
>> The New York TImes has 126 URLs.
>> 
>> If we were just looking at delaying the SINGLE first SYN, and if in =
>> addition we perform no parellisation or other optimisation (and of =
>> course modern browsers do both, but anyway) then 30ms would add 1.5 =
>> seconds to the load time of Apple's page and 3.8 to the NYT. Just from =
>> the bandwidth delay product.
> 

Yes, and windowing can address that. But you have to open the initial connection. That first handshake is subject to latency.
> across a single connection that matters.

When someone opens a web page, it goes:
1. Start a TCP session.
2. Fetch the page

#1 depends on latency, but it's only one connection. #2 depends more on bandwidth, slightly on latency and windowing helps here.

3. Parse the page and extract lots of embedded URLs.
Some of these will be to the initial site, but a lot won't. Many are to ad servers, web trackers, assorted web resources, etc.. For each of these, the browser has to open a session and fetch the resource. If the browser is at all simple-minded and does things in a mostly serial fashion, then all those initial latencies will sum. Note that even infinite bandwidth does not help here, it's the latency which is the killer. So, you DO have to add up the delays of different connections. And on most modern web pages, there are a lot of them.
> additional delay of 120ms. Hardly worth writing home about. For 126 URLs

Not all browsers behave as you describe. Some do, in fact that's where a lot of browsers are making improvements, simply because as bandwidth has risen, latency has become the great enemy and parallelism is one of the few client options available to combat it.

However, I see no reason to make the browser's job any harder. If latency is to be deliberately introduced, then it needs to be done because there is a greater benefit to the user than by not introducing it.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 04 Aug 2011 11:29:03 +1000

In message <[email-masked]>, Michael Newbery writes:
> 
> --Apple-Mail-1--896246449
> Content-Transfer-Encoding: quoted-printable
> Content-Type: text/plain;
> 	charset=us-ascii
> 
> 
> >> and the base HTML document itself.=3D20
> >>=20
> >> 49 URLs to be fetched
> >>=20
> >> Apple/s front page is if anything atypically spare in terms of
> elements. =3D
> >>=20
> >> If we were just looking at delaying the SINGLE first SYN, and if in =3D
> 
> >> addition we perform no parellisation or other optimisation (and of =3D
> >> course modern browsers do both, but anyway) then 30ms would add 1.5 =3D
> 
> >> seconds to the load time of Apple's page and 3.8 to the NYT. Just
> from =3D
> >=20
> Yes, and windowing can address that. But you have to open the initial
> connection. That first handshake is subject to latency.
> >
> #1 depends on latency, but it's only one connection. #2 depends more on
> bandwidth, slightly on latency and windowing helps here.
> 
> Some of these will be to the initial site, but a lot won't. Many are to
> ad servers, web trackers, assorted web resources, etc.. For each of
> these, the browser has to open a session and fetch the resource. If the
> browser is at all simple-minded and does things in a mostly serial
> fashion, then all those initial latencies will sum. Note that even
> infinite bandwidth does not help here, it's the latency which is the
> killer. So, you DO have to add up the delays of different connections.
> And on most modern web pages, there are a lot of them.
> >
> Not all browsers behave as you describe. Some do, in fact that's where a
> lot of browsers are making improvements, simply because as bandwidth has
> risen, latency has become the great enemy and parallelism is one of the
> few client options available to combat it.
> 
> However, I see no reason to make the browser's job any harder. If
> latency is to be deliberately introduced, then it needs to be done
> because there is a greater benefit to the user than by not introducing
> it.
 
Engineering is often about making decisions about what is best for
the whole rather than what is best for one component of the system.

If IPv4 and IPv6 will be of equal cost to provide then taking the
best RTT is the right choice but we know the IPv4 is going to be
more costly in the long term to provide and that the additional
cost will be proportional to the amount of IPv4 traffic so adding
a small IPv6 bias in the destination selection will reduce the
overall costs to the user.  The costs of deploying LSN will ultimately
be borne by the customer in one way or another.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Lorenzo Colitti <[email-masked]>
Date: Wed, 3 Aug 2011 18:32:30 -0700

> be borne by the customer in one way or another.

I think that what you're saying is that ISPs want users to use IPv6.
But James is saying is that if ISPs wanted their users to use IPv6, they
would deploy IPv6. So I doubt that you will agree :-)

---

From: Mark Andrews <[email-masked]>
Date: Thu, 04 Aug 2011 12:47:01 +1000

In message <[email-masked]>
> would deploy IPv6. So I doubt that you will agree :-)

No. I'm arguing about what the behaviour should be when the ISP
*has* deployed IPv6 along side IPv4 or is IPv6 only with DS-Lite
or NAT64/DNS64 (DNS64 already has some IPv6 bias built in).

IPv6 is basically shuffle packet.  IPv4 is shuffle and mangle
packets.  The mangle packets parts is the additional cost but it
may not have a noticable time penalty.  Often it will be a IPv6
router sitting next to a LSN/AFTR/NAT64 boxes so there isn't even
a extra hop penalty.

Mark

---

From: Lorenzo Colitti <[email-masked]>
Date: Wed, 3 Aug 2011 20:06:49 -0700

> or NAT64/DNS64 (DNS64 already has some IPv6 bias built in).

And I suspect what James is saying is that today, what you describe is such
a small percentage of the Internet that there's no point optimizing for it
in current operating systems.

I don't necessarily agree - for example, I think a 10ms latency penalty is
still worth it if real IPv6 is available - but I can see his point.

---

From: George Michaelson <[email-masked]>
Date: Thu, 4 Aug 2011 13:26:00 +1000

On 04/08/2011, at 1:06 PM, Lorenzo Colitti wrote:

> And I suspect what James is saying is that today, what you describe is such a small percentage of the Internet that there's no point optimizing for it in current operating systems.
> 
> I don't necessarily agree - for example, I think a 10ms latency penalty is still worth it if real IPv6 is available - but I can see his point.

I entirely agree with what Apple has done so far. I think its made sense to me, as a user of their product.

I could ask for a tuneable (and I did in fact ask James) but, pragmatically, given online update cycles, this is within the scope of Apple to fix later on at very low pain.

I know others disagree, but I think Apple has made a reasonable, justifiable decision for now, and I agree with Lorenzo that it reflects v4/v6 as we see it NOW. Not what we WANT. What we SEE.

Mark, you can't expect people to flag-wave for the future on their flagship. They put the smallest flag they can on the backstaff. Wait until the wind rises, bigger flags will come

-George

---

From: Mark Andrews <[email-masked]>
Date: Thu, 04 Aug 2011 13:36:06 +1000

In message <CAKD1Yr15PEJcS32t8B_ps=[email-masked]>
, Lorenzo Colitti writes:
> --000e0cdf1c56c9b5d204a9a546ef
> Content-Type: text/plain; charset=ISO-8859-1
> 

It all depends on when you think deployment of CGN/AFTR/NAT64 boxes
will happen.  My feeling is you will see these boxes being widely
deployed within 3 years and ISP's scrambling to move IPv4 only
customers to IPv4 + IPv6 or IPv6 only to reduce the number of
CGN/AFTR/NAT64 boxes they need to deploy.  Currently shipping code
shouldn't make the problem worse when this happens.

---

From: George Michaelson <[email-masked]>
Date: Thu, 4 Aug 2011 14:10:11 +1000

> shouldn't make the problem worse when this happens.

Mark, consider both Apple, and Microsofts codebase 3 years ago, and the frequency of updates.

Lion has been out precisely <1month, and has had one update already.

Vista gets regular self-applied updates. 

You cannot seriously be saying that the consequential 'damage' to the v6 message stems from this deployment, because it only applies to people WHO GO ONLINE and if they go online with Apple they get FREE UPDATES.

What am I missing? Do you think Apple owners stay on factory defaults?

Really?

-george

---

From: Mark Andrews <[email-masked]>
Date: Thu, 04 Aug 2011 14:38:05 +1000

In message <[email-masked]>, George Michaelson 
> Mark, consider both Apple, and Microsofts codebase 3 years ago, and the

I'm also looking at what hasn't happened for those that don't make
the jump to the next major os release.  You get large numbers stuck
at whatever feature set there is at the of the release.

Neither of us know if this will be addressed in Lion before the
next major release of Mac OS X or not, and there is a major release
will it only be addressed in that and not Lion?

> You cannot seriously be saying that the consequential 'damage' to the v6
> message stems from this deployment, because it only applies to people
> WHO GO ONLINE and if they go online with Apple they get FREE UPDATES.
> 
> -george=

---

From: "Dale W. Carder" <[email-masked]>
Date: Wed, 03 Aug 2011 23:59:47 -0500

On Aug 3, 2011, at 11:10 PM, George Michaelson wrote:

> What am I missing?

What we've learned w/ 6to4: having an exit strategy ready for when we 
measure there is enough harm being done.  

I believe we need to be thinking about the exit strategy for LSN before 
it gets us stuck in a costly loop.  Biasing v6 would be one approach.
Economics for continued v4 service would be another, but inconsistent
in applicability and outside of our scope.  

Dale

---

From: "John Mann (ITS)" <[email-masked]>
Date: Thu, 4 Aug 2011 15:37:52 +1000

Hi,

+1

Look at PowerPC Macs stuck on 10.4 / 10.5 that still trust rogue Windows
boxes advertising 6to4 routes that don't work.  See e.g. the brown and
yellow lines on the Mac OS X version
distribution<http://fud.no/ipv6/gnuplot/osxversions.png> graph
on http://fud.no/ipv6/ -- still about 20% usage.

And now single-core Intel Macs stuck on Snow Leopard that can't get even
Lion's initial IPv6 changes.

There *can* be a very long tailed distribution for old versions of software,
even if software upgrades are free or cheap.

Now, if Apple were to back-port IPv6 improvements into all previous releases
... ;-) ;-)

---
In other news, Windows XP market share dips below 50
percent<http://news.cnet.com/8301-10805_3-20086776-75/windows-xp-market-share-dips-below-50-percent/>
.
Another long tail here.  I don't know if WinXP will get RFC3484-bis updates.
Or get a hypothetical IE update that implements Happy Eyeballs that could
work-around underlying O/S behaviour ...

    John

---

From: George Michaelson <[email-masked]>
Date: Thu, 4 Aug 2011 16:14:02 +1000

John, I could understand if this was a 6to4 deprecation issue.

Its a v4/v6 PREFERENCE issue.

So, for a long tail, some number (20% but I doubt it) of Lion instances on the global net will continue to prefer V4 over V6.

And if there is no V4 in that timeframe, while I am cashing in my lotto cheque, I will care why? Because, in the absence of V4... guess what the system will do ...

Go on John.. guess

20% of the system will weight V6 below V4, if it detects an RTT preference under a HE like algorithm favouring V4.

What exactly is the 'fail' moment here?

-G

---

From: Philip Homburg <[email-masked]>
Date: Thu, 04 Aug 2011 08:48:15 +0200

>would deploy IPv6. So I doubt that you will agree :-)

It may actually be worse.

If you look at the following article about RIPE measurements during ipv6day
<http://labs.ripe.net/Members/emileaben/measuring-world-ipv6-day-comparing-ipv4-and-ipv6-performance>

Then if all vendors would be like Apple, IPv6 would essentially be left
unused.

So this is telling ISPs: do not bother with IPv6 because our client side
operating systems are no going to use it anyhow (unless you make IPv6 a
couple of sigmas faster than IPv4).

---

From: "John Mann (ITS)" <[email-masked]>
Date: Thu, 4 Aug 2011 16:54:30 +1000

George,

On 4 August 2011 16:14, George Michaelson <[email-masked]> wrote:

> So, for a long tail, some number (20% but I doubt it) of Lion instances on
> the global net will continue to prefer V4 over V6.
>
> And if there is no V4 in that timeframe, while I am cashing in my lotto
> cheque, I will care why? Because, in the absence of V4... guess what the
> system will do ...
>
> Go on John.. guess
>

I was trying to agree with Mark and comment just on your assertion that
regular vendor updates will/can change the settings of previously released
software.

The vendor 'fail' moment is when they stop feature updates for older
software, users can't upgrade (e.g. because of old hardware), and the
sub-optimal behaviour gets frozen forever.

[ I illustrated my point using the example of IPv6 code that isn't being
updated on old Macs and old PCs. I wasn't trying to examine the merits or
otherwise of that code. Sorry if you read too much into my message. ]

    John

---

From: George Michaelson <[email-masked]>
Date: Thu, 4 Aug 2011 17:08:40 +1000

On 04/08/2011, at 4:54 PM, John Mann (ITS) wrote:

> 
> I was trying to agree with Mark and comment just on your assertion that regular vendor updates will/can change the settings of previously released software.
> 
> The vendor 'fail' moment is when they stop feature updates for older software, users can't upgrade (e.g. because of old hardware), and the sub-optimal behaviour gets frozen forever.
> 
> [ I illustrated my point using the example of IPv6 code that isn't being updated on old Macs and old PCs. I wasn't trying to examine the merits or otherwise of that code. Sorry if you read too much into my message. ]
> 
>     John

John I know my answer was a bit rough but, we can't always take negatives on positive actions. Apple did a thing, and its a pragmatic thing, and while I hate acting like  fanboi, I think they did a good thing.

Marks position is basically wrong. This is not 6to4, there is no inherent fail moment in this. this is not something which stops V6 working at all. It de-preferences V6 in dual stack when the V4 is demonstrably better.

Yes, software fails to be updated in the field. I know this. I am not naive. 

But, of the choices in front of us, I think this pragmatic position is a very good 'do least harm' place to be.

I expect others to take other stances. If you want V6 to be better for your users, de-tuning V4 to make it look better isn't where I think I want to be, or you want to be. So don't do that.

Reducing RTT in V6 to being within tolerance of V4 is actually doable. Its pretty much what Internode has done, and while I posted a rough and ready 'look: it prefers V4' thats not frozen in stone. When the V6 RTT is better, I am willing to look at my Lion preferencing V6. I expect it will. Good! I look forward to some V6 low latency paths from home, and when I find them, I will say so.

Meantime, on my Vista box, one node at home can't get to a really important Anaphalaxis website which has put up a V6 AAAA and isn't responding. Thats pretty close to a major, MAJOR V6 fail to me.

An Anaphalaxis information site with a broken Dual-stack?

You *bet* I want V4 to work better on that site.

-G

---

From: Philip Homburg <[email-masked]>
Date: Thu, 04 Aug 2011 09:26:44 +0200

In your letter dated Thu, 4 Aug 2011 17:08:40 +1000 you wrote:
>I expect others to take other stances. If you want V6 to be better for your us
>ers, de-tuning V4 to make it look better isn't where I think I want to be, or 
>you want to be. So don't do that.
>
>Reducing RTT in V6 to being within tolerance of V4 is actually doable. Its pre
>tty much what Internode has done, and while I posted a rough and ready 'look: 
>it prefers V4' thats not frozen in stone. When the V6 RTT is better, I am will
>ing to look at my Lion preferencing V6. I expect it will. Good! I look forward
> to some V6 low latency paths from home, and when I find them, I will say so.

Without bias, when IPv6 and IPv4 are equal, you can expect half of the
connections to go to IPv4.

It is only when you make IPv6 better then IPv4 by a couple of sigmas that
most of the traffic will go to IPv6.

Given that you can't really make IPv6 faster than IPv4, because IPv4 is
already as fast as it gets, to only opion left is...

To make IPv4 slower than IPv6.

(Or just don't bother with IPv6 all together and let the customers pay for
CGN).

---

From: Doug Barton <[email-masked]>
Date: Thu, 04 Aug 2011 01:15:05 -0700

> connections to go to IPv4.

That's a feature.

Apple is under no obligation to promote IPv6 adoption. Contrariwise,
they ARE under a fiduciary obligation to produce an OS that people will
want to buy.

At the point sometime in the future when IPv6 routinely IS better than
IPv4, the whole line of discussion will be moot so why not get ahead of
the rush?

Doug

-- 

	Nothin' ever doesn't change, but nothin' changes much.
			-- OK Go

	Breadth of IT experience, and depth of knowledge in the DNS.
	Yours for the right price.  :)  http://SupersetSolutions.com/

---

From: Philip Homburg <[email-masked]>
Date: Thu, 04 Aug 2011 10:25:43 +0200

>IPv4, the whole line of discussion will be moot so why not get ahead of

The only way to make IPv6 better than IPv4 is to make IPv4 worse. You can't
just change the laws of physics and make IPv6 packets travel faster than
the speed of light.

So whereas a bias in Apples software will statistically have no negative
effect on systems with equally good IPv4 and IPv6 performance, having ISPs or
content providers add a delay for IPv4 will have a negative effect on every
IPv4-only system.

It is so nice that Apple considers sending 50% of their users traffic through
a CGN as in the interest of their users. I'm glad I'm not one of their
customers.

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Aug 2011 07:43:20 -0400

> in applicability and outside of our scope.  

I'm a big fan of exit strategies.  I've often wished that v4 NAT had been designed with an exit strategy.

Note however that a global 'kill switch' is probably not a good exit strategy.  

Keith

---

From: Daniel Roesen <[email-masked]>
Date: Thu, 4 Aug 2011 13:45:14 +0200

On Thu, Aug 04, 2011 at 12:39:56PM +1200, Michael Newbery wrote:
> because there is a greater benefit to the user than by not
> introducing it.

- less brokeness
- less cost
- easier troubleshooting due to better predictability/reproducibility.

How does that sound to you as an end user - compelling reasons enough?
:-)

Best regards,
Daniel

---

From: Daniel Roesen <[email-masked]>
Date: Thu, 4 Aug 2011 13:55:42 +0200

> Lion has been out precisely <1month, and has had one update already.

http://fud.no/ipv6/gnuplot/osxversions.png

25% of the user base observed by Tore are not even on 10.6.

> You cannot seriously be saying that the consequential 'damage'
> to the v6 message stems from this deployment, because it only
> applies to people WHO GO ONLINE and if they go online with
> Apple they get FREE UPDATES.

No they don't - at least not always, as far as I'm told. People have
to pay for 10.7.

Best regards,
Daniel

---

From: james woodyatt <[email-masked]>
Date: Thu, 04 Aug 2011 12:24:31 -0700

On Aug 4, 2011, at 01:25 , Philip Homburg wrote:
> 
> It is so nice that Apple considers sending 50% of their users traffic through a CGN as in the interest of their users.

Once again, I am presented with an illustration of how my perspective is so very different from what seems to be the mainstream in the operator community.

I don't see how the new concurrent TCP connections behavior in Lion is "sending 50% of [traffic originating from hosts with Apple operating systems] through a CGN."  It's not like we have a CGN-detector in the stack that applies a limit of the form, "Is this destination on a path that requires a CGN?  No?  Sorry, unacceptable.  At least 50% of my outbound packets MUST go through a CGN or I'm not doing my job properly."

The way I see it, operators-- specifically Internet service providers-- are the ones who get to choose where to send arbitrary percentages of traffic originated by hosts on their networks, and it's my understanding that, when considering their alteratives, competent operators usually give some passing thoughts, occasionally, to the interests of their customers (where those interests are aligned with the interests of shareholders).

It's not the job of Apple's Core OS Networking group to tell Internet service providers whether and how to deploy CGN for IPv4 paths to achieve address amplification and deal with free pool exhaustion.  If they're gonna do it, then we're going to try to make it work for our customers as best we can.  But, it's their call... as I've been reminded quite forcefully, on this list, and others, again and again.

I'm just rather amused that suddenly it's *our* fault that, when you put a CGN into the network, it gets used.  Who could have predicted?

---

From: Sander Steffann <[email-masked]>
Date: Thu, 4 Aug 2011 22:10:39 +0200

Hi James,

> I'm just rather amused that suddenly it's *our* fault that, when you put a CGN into the network, it gets used.  Who could have predicted?

It's obvious that it is being used. That is what it's there for :-)  What annoys the operators is that it is used more than is necessary...

I think it boils down to:
- If it is used when it gives better performance to the user: great.
- If it is used when the performance is roughly equal to IPv6: please use IPv6.

It's cheaper for the operator, and it doesn't harm the user. And yes: the decisions Apple makes here can affect the cost in the operator network.

Met vriendelijke groet,
Sander Steffann

---

From: Doug Barton <[email-masked]>
Date: Thu, 04 Aug 2011 13:53:03 -0700

> a CGN as in the interest of their users. I'm glad I'm not one of their

As you stated in another message, avoiding nat is your goal. As a
result, I think your statement above is an excellent example of the
market forces at work. Thus we can *conclude* that apple has met their
fiduciary responsibilities.

--

---

From: james woodyatt <[email-masked]>
Date: Thu, 04 Aug 2011 14:00:18 -0700

> - If it is used when the performance is roughly equal to IPv6: please use IPv6.

I don't know how to say this differently, so I'm just going to repeat myself.

If you provision your networks so that IPv4 paths have longer round-trip times or are congested more often and/or to a greater degree, then we will prefer less congested IPv6 paths.

Furthermore, if you provision your networks so that IPv6 paths have longer round-trip times or are congested more often and/or to a greater degree, then we will prefer less congested IPv4/NAT paths.

Finally, if you provision your networks so that IPv4 and IPv6 exhibit "roughly equivalent" levels of path latency and congestion, then we're going to try very, very hard to use both IPv4 and IPv6 in "roughly equivalent" measure.

Isn't this what traffic engineers have been clamoring at us for years to give them?

---

From: Michael Newbery <[email-masked]>
Date: Fri, 5 Aug 2011 10:06:31 +1200

On 5/08/2011, at 7:24 AM, james woodyatt wrote:
> The way I see it, operators-- specifically Internet service providers-- are the ones who get to choose where to send arbitrary percentages of traffic originated by hosts on their networks,
[...]
> 
> It's not the job of Apple's Core OS Networking group to tell Internet service providers whether and how to deploy CGN for IPv4 paths to achieve address amplification and deal with free pool exhaustion.  If they're gonna do it, then we're going to try to make it work for our customers as best we can.
[...]

+1

Is this a classic SEP (Someone Else's Problem)? LSNs are Bad, but since some ISPs are going to deploy them anyway, can't we make the clients detect them and de-prefer them, so the ISPs won't have an economic motive to continue deploying them? I submit that is wrong. Let the ISPs make their decisions; let the clients work with what is there.

LSNs are doomed, because they can't provide IPv4 only access to IPv6. So, it might well be that an ISP that feels the need to deploy LSN or NAT64 temporarily could introduce latency to bias customers with a choice against. It might not be something the've thought about, but as a way to reduce the need for ongoing investment in interim hardware it has a certain attraction. But, it is entirely their decision. The last thing the ISPs need to to be having to second guess the client software, as it attempts to second guess the ISPs, ...

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 4 Aug 2011 15:19:28 -0700

On Thu, Aug 4, 2011 at 15:06, Michael Newbery <[email-masked]> wrote:

> LSNs are doomed, because they can't provide IPv4 only access to IPv6.
>

Nope. There will be no IPv6-only content that matters until IPv6 is widely
or ubiquitously deployed. So not having access to IPv6 will not matter.

---

From: Mark Andrews <[email-masked]>
Date: Fri, 05 Aug 2011 08:31:05 +1000

In message <[email-masked]>, james woodyatt wri
tes:
> If you provision your networks so that IPv4 paths have longer round-trip time
> s or are congested more often and/or to a greater degree, then we will prefer
>  less congested IPv6 paths.
> 
> Furthermore, if you provision your networks so that IPv6 paths have longer ro
> und-trip times or are congested more often and/or to a greater degree, then w
> e will prefer less congested IPv4/NAT paths.
> 
> Finally, if you provision your networks so that IPv4 and IPv6 exhibit "roughl
> y equivalent" levels of path latency and congestion, then we're going to try 
> very, very hard to use both IPv4 and IPv6 in "roughly equivalent" measure.
> 

In a world where the packets are just being routed that would be optimal,
however we are planning for a world where packets aren't just being routed.

We are planning for a world where IPv4 packets cost significantly
more to dollars to process than IPv6 packets.  We are saying that
when the paths are of roughly equal performance wise to use the
IPv6 path.  Even if a ISP puts enough CGN/AFTR/NAT64 equipement in
that they can process there entire traffic load over it with
performance parity they don't want any traffic to use it.  All of
these boxes need to log every flow through them for abuse traceback
/ law enforcement.

Mark

---

From: Michael Newbery <[email-masked]>
Date: Fri, 5 Aug 2011 10:45:37 +1200

On 5/08/2011, at 10:19 AM, Lorenzo Colitti wrote:

> Nope. There will be no IPv6-only content that matters until IPv6 is widely or ubiquitously deployed. So not having access to IPv6 will not matter.

I disagree. The Web is not the Internet. As soon as any significant number (and I suspect a number as little as 5%) of clients are IPv6 only, and direct communication is required, then the IPv6 have nots will start demanding action from their ISP.

Consider a hypothetical "Irritated Avians, Network Edition". Frag your friends---except *your* friend is on LTE and is IPv6 only... You now need IPv6.

Don't make the mistake of thinking that 'content' is only what sits on central servers. In particular, as an ISP, don't think that content is something that you own or control. Or understand, for that matter. The content of the Internet is its edge, its users. The bit in the middle, where we operators play, is an irritating irrelevancy to the customers. And that's how it should be. We are the plumbing, and people only care about plumbing when it doesn't work.

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Aug 2011 18:48:45 -0400

On Aug 4, 2011, at 6:45 PM, Michael Newbery wrote:

> Don't make the mistake of thinking that 'content' is only what sits on central servers.

+1

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 4 Aug 2011 16:03:22 -0700

On Thu, Aug 4, 2011 at 15:45, Michael Newbery <[email-masked]> wrote:

> I disagree. The Web is not the Internet. As soon as any
> significant number (and I suspect a number as little as 5%) of clients are
> IPv6 only, and direct communication is required, then the IPv6 have nots
> will start demanding action from their ISP.
>

I can't imagine my family, my friends or anyone outside the industry ever
calling their ISP asking why they don't have IPv6. You might as well assert
that users will call their ISP asking for PPPoA over PPPoE because it has
less overhead.

---

From: james woodyatt <[email-masked]>
Date: Thu, 04 Aug 2011 16:17:53 -0700

On Aug 4, 2011, at 15:31 , Mark Andrews wrote:
> 
> We are planning for a world where IPv4 packets cost significantly more dollars to process than IPv6 packets.  We are saying that when the paths are of roughly equal performance wise to use the IPv6 path.

When the additional costs of having IPv4 service available alongside IPv6 are passed along to subscribers-- where they have some hope of noticing anything remotely related to it-- then when we can have a conversation about who should pay, and who should be paid, for making sure that all the IPv4 network elements are only used by IPv4-only hosts.

Until then, I recommend trying to get used to the new tactical environment.  It's going to be with us for a long while yet to come.

---

From: Shane Amante <[email-masked]>
Date: Thu, 4 Aug 2011 18:45:03 -0600

On Aug 4, 2011, at 1:24 PM, james woodyatt wrote:
[--snip--]
> The way I see it, operators-- specifically Internet service providers-- are the ones who get to choose where to send arbitrary percentages of traffic originated by hosts on their networks, and it's…

There may be a problem with the aforementioned logic.  Specifically, an ISP's routers are only capable of applying (BGP) Traffic Engineering *within* and not /between/ address families.  IOW, an operator can control where/how IPv4 packets are routed, but an operator cannot control, for example, that IPv6 forwarding should be preferred to IPv4 forwarding.  At this point, given implementations of Happy Eyeballs or behavior like it, the only "control" an operator can feasibly 'exert' to change the preference of IPv4 compared to IPv6 is to drop packets of one address family over the other ... which is a extremely crude sledgehammer for a variety of reasons, that are hopefully obvious.

I think it would be more productive to discuss the feasibility of moving past the notion of a single, "fixed" address policy (e.g.: RFC 3484) to rule them all and, instead, potentially consider something like:
http://tools.ietf.org/html/draft-ietf-6man-addr-select-opt-01
... or, a similar/better mechanism in RA's.  (I'm not here to prescribe a specific protocol, rather to raise the more general point).

For example, if somehow an operator were able to _dynamically_ [and more strongly] hint to end-hosts that native IPv6 is preferred compared to IPv4, then _at the right time in the near future_ this would allow operators to start signaling to hosts, on an per-Metro or per-region (?) basis, that they believe a good quality, good performance, etc. deployment of IPv6 exists (in the access, the Backbone and peering/interconnect portions of the network) and they want hosts to start originating packets using IPv6, when possible.  This would also, potentially, absolve host stack vendors from making a difficult and/or wrong call on a single, static address selection policy and, instead, point the blame & pain of an address selection policy firmly back at the ISP, where it belongs.  :-) 

Note that this could more generally considered as a, hopefully, lightweight method of advertising "site" exit routing preferences/hints towards hosts.  This way, when hosts are multi-homed (have two or more IPv6 prefixes on a single link, but two exit routers to two different ISP's), an administrator can dynamically signal to hosts that they should prefer one GUA over another GUA, perhaps because the transit costs of ISP A are less than the transit costs of ISP B or, alternatively, because the administrator deems the 'performance' of ISP A to be substantially better than the 'performance' of ISP B.  (As one example see draft-baker-fun-multi-router-00).  Note that if the two ISP's are of equal cost and performance, administrators could signal a policy that an equal 'weight' applied to both GUA prefixes advertised to the host and the host can either more simply round-robin between them or, if it wants, implement something Happy Eyeball like to squeeze our more certainty as to a particular path's quality. 

In summary, it's time we stop thinking "the network" is omniscient and it will always, automatically know the intent/desire of end-hosts.  Likewise, hosts should only benefit from 'hints' learned from the network as to "preferred [exit] paths" so that they wouldn't need to always bother attempting to open connections just to keep finding out that a particular path/link is of consistently poor performance, (i.e.: the definition of insanity).  

> I'm just rather amused that suddenly it's *our* fault that, when you put a CGN into the network, it gets used.  Who could have predicted?

I wouldn't say it's Apple's fault; however, we probably all need to "think different".  :-)  Namely, we should all see if we can find a way to give operators some amount of control over address selection policy on hosts, since ultimately we [think we] know our networks best.  :-)

-shane

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Aug 2011 21:34:49 -0400

On Aug 4, 2011, at 7:03 PM, Lorenzo Colitti wrote:

> I can't imagine my family, my friends or anyone outside the industry ever calling their ISP asking why they don't have IPv6. 

Perhaps not.  But calling their ISP to ask "Why can't I run Irritated Avian Carriers?  I downloaded the app but it says I need something called ipveesix, and I should call my ISP and ask for it." is at least a distant possibility.

That assumes, of course, that there's enough IPv6 market _somewhere_ to allow such an app to gain a following.  For instance, if educational institutions generally had IPv6 and students got exposed to interesting IPv6-only apps, they'd demand those same applications in their homes and workplaces after leaving academia.

Granted, this is somewhat of a longshot.  But it's why I and a few others are still interested in better methods of providing IPv6 service over the existing IPv4 infrastructure. 

Keith

p.s. I remember when telcos didn't think anyone would actually want to use the Internet in preference to services that the telcos would provide.  They kept mumbling something about lack of guaranteed service.

---

From: "Dale W. Carder" <[email-masked]>
Date: Thu, 04 Aug 2011 21:26:53 -0500

> Finally, if you provision your networks so that IPv4 and IPv6 exhibit "roughly equivalent" levels of path latency and congestion, then we're going to try very, very hard to use both IPv4 and IPv6 i…

That could end up causing a lot of headache.  I personally believe it 
will be a horrible path for Internet engineering to follow when we 
know, today, that this could be a very, very bad host behavior.

<snip>
> When the additional costs of having IPv4 service available alongside IPv6 are passed along to subscribers
</snip>

Just like how Internet != http, this economic model is similarly limited 
in applicability.  For example, I do some work for a regional ISP that
doesn't do billing.  It also may not be applicable for many enterprises.

> Until then, I recommend trying to get used to the new tactical environment.  It's going to be with us for a long while yet to come.

And this approach may *exacerbate* the problem by treating the address 
families as equal when v4 will truly be encumbered.  

Dale

---

From: David Conrad <[email-masked]>
Date: Thu, 4 Aug 2011 07:26:09 -1000

> 25% of the user base observed by Tore are not even on 10.6.

I suspect a significant portion of that 25% is running PowerPC architecture (and hence unable to move to anything above 10.5 despite how much they might like to).  I'm guessing Apple will not be changing this situation, so that portion of the 25% will probably have a long tail (as machines die and/or migrate to PowerPC versions of *BSD or Linux).

> No they don't - at least not always, as far as I'm told. People have to pay for 10.7.

Yes. US$30 for up to 5 machines (at least in the US).  

Regards,
-drc

---

From: james woodyatt <[email-masked]>
Date: Thu, 04 Aug 2011 20:01:19 -0700

On Aug 4, 2011, at 7:26 PM, Dale W. Carder wrote:
> 
> That could end up causing a lot of headache.

How?  Please be specific.

The behavior we have now is proven to alleviate the killer throw-your-Macbook-at-the-wall migraine headaches that a lot of our paying customers have been complaining about quite publicly, and I'm not going to get very far inside Apple toward undoing what we've done without more technical detail.

---

From: Keith Moore <[email-masked]>
Date: Thu, 4 Aug 2011 23:37:25 -0400

> The behavior we have now is proven to alleviate the killer throw-your-Macbook-at-the-wall migraine headaches that a lot of our paying customers have been complaining about quite publicly, and I'm n…

I can think of cases where IPv4 will work better than many kinds of IPv6 service that we're likely to see anytime soon - e.g. large UDP/IPv6 packets that are tunneled over IPv4 over Ethernet (or worse, PPPoE) somewhere along the way.  PMTUD helps but can't always fix these issues.

I can think of cases where IPv6 will often, or even always, work better than IPv4 no matter what the relative performance of the two links, e.g. applications that do address referrals over NAT.

A metric that measures the behavior of a network path for one type of application may be very poor at predicting the behavior of a network path for a different application.   Some apps are latency sensitive, some are bandwidth sensitive, some are jitter sensitive, and some are NAT sensitive.   

But the clever API does not strike me as a problem as long as developers understand what the clever API optimizes for, i.e., when to use the clever API and when not to use it.

I'm very much in agreement with the idea that if you want hosts and applications to minimize use of the path through the NAT, pessimize the path through the NAT.   There are lots of ways to do that - add delay, randomly drop X% of SYN packets, etc..   You can even make the amount of pessimization adjustable, so that you can start out at zero and tweak the amount of pessimization over time to minimize the rate of support calls and/or reduce the cost associated with the NAT.   But don't expect hosts or apps to second guess what the network is doing as a matter of global policy; let them adapt to the specific network situation that they find themselves in.   A path selection policy that works well in your network might not work so well in a different network.

Keith

---

From: Mark Andrews <[email-masked]>
Date: Fri, 05 Aug 2011 13:38:06 +1000

In message <[email-masked]>, james woodyatt wri
tes:
> The behavior we have now is proven to alleviate the killer throw-your-Macbook
> -at-the-wall migraine headaches that a lot of our paying customers have been 
> complaining about quite publicly, and I'm not going to get very far inside Ap
> ple toward undoing what we've done without more technical detail.

James, I've got a Mac Book, running 10.6.8, wireless keyboard and
trackpad, external monitor.  I sit in front of it 8+ hours day.  It
migrates to the lounge room for several more hours many evenings.

A couple of weeks ago the IPv6 path between home and the IETF servers
was stuffed for over a week.  Safari was unusable for reaching the
IETF site because it took too long to fail over to IPv4 and it
didn't remember that it failed with IPv6.  Google Chrome on the
other hand was fine, a slight pause and off it went using IPv4.

I don't think what been done needs to be undone.  It just needs to
be tuned slightly differently.

Mark

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 5 Aug 2011 07:39:55 +0200 (CEST)

On Thu, 4 Aug 2011, Lorenzo Colitti wrote:

> I can't imagine my family, my friends or anyone outside the industry 
> ever calling their ISP asking why they don't have IPv6. You might as 
> well assert that users will call their ISP asking for PPPoA over PPPoE 
> because it has less overhead.

I don't agree. If I tell my mom that unless she gets a connection with 
IPv6 she won't be able to see her grandchild in high quality in <insert 
video application here>, she will get a connection with IPv6.

Personally I've had heaps of problems getting some video applications 
working when both parties were behind NAT, but since I work for an ISP I 
was actually able to make sure I got a larger GUA block at my end which 
made it work just fine. Most other people are not so fortunate.

Now, if just <insert almost any video application here> would support 
IPv6, that would be great :P

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 5 Aug 2011 07:58:55 +0200 (CEST)

> Isn't this what traffic engineers have been clamoring at us for years to 

I don't know who told you this, but personally I consider a NAT box to 
warrant a penalty when that path is considered.

As an ISP, when we consider implementing IPv6 we factor in the saving of 
not having to do IPv4 CGN for some of the traffic, and this factor should 
increase over time. This has been true all along but now it's changed. All 
along, the discussion regarding happy eyeballs have been (as far as I 
could discern) that IPv6 would still be preferred, and HE was going to be 
used to work around broken IPv6 connectivity. IPv4 would have a 
configurable penalty, and by default this penalty would be set to "medium" 
(50-200ms or so).

NAT makes the connectivity worse, I don't understand why this wouldn't be 
a factor in protocol choice for outgoing connections.

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 4 Aug 2011 23:27:03 -0700

On Thu, Aug 4, 2011 at 22:39, Mikael Abrahamsson <[email-masked]> wrote:

> I don't agree. If I tell my mom that unless she gets a connection with IPv6
> she won't be able to see her grandchild in high quality in <insert video
> application here>, she will get a connection with IPv6.
> [...]
> Now, if just <insert almost any video application here> would support IPv6,
> that would be great :P

And there you have the fallacy of the argument. You say that users will ask
for IPv6 to use cool apps, and barely three lines later you say that those
cool apps don't support IPv6.

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 4 Aug 2011 23:27:08 -0700

On Thu, Aug 4, 2011 at 22:58, Mikael Abrahamsson <[email-masked]> wrote:

> increase over time. This has been true all along but now it's changed.
>

It hasn't changed. Fortunately for you, most of your traffic will be from
Windows machines, and some Mac traffic will be using Firefox or Chrome, so
only a small part of it will prefer IPv4 if there is working IPv6 present.

I agree that if all implementations behaved like Safari on Lion,
transitioning to IPv6 would be that much more difficult.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 5 Aug 2011 09:20:15 +0200 (CEST)

On Thu, 4 Aug 2011, Lorenzo Colitti wrote:

> And there you have the fallacy of the argument. You say that users will 
> ask for IPv6 to use cool apps, and barely three lines later you say that 
> those cool apps don't support IPv6.

I still have high hopes that they will gain support in the not so distant 
future. When I read your earlier email it seemed like you indicated that 
you thought users would *never* ask for IPv6. I think they will, and it'll 
start in a year or two.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 05 Aug 2011 10:10:58 +0200

In your letter dated Thu, 04 Aug 2011 20:01:19 -0700 you wrote:
>The behavior we have now is proven to alleviate the killer throw-your-Macbook-
>at-the-wall migraine headaches that a lot of our paying customers have been co
>mplaining about quite publicly, and I'm not going to get very far inside Apple
> toward undoing what we've done without more technical detail.

So what it boils down to is that Apple came up with their own HE
implementation without telling anybody or asking for feedback. And now that
they have shipped it, it has to be defended as being the best solution. Sad,
but that's usually how it works.

There is no way that a slight preference for IPv6 is going result in 
'throw-your-Macbook-at-the-wall migraine headaches' when Chrome and Firefox 
have been shipping with quite crude HE implementations with a much stronger
IPv6 preference and (to the extent that they actually know about it) people
are happy.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 05 Aug 2011 10:22:19 +0200

In your letter dated Thu, 4 Aug 2011 18:45:03 -0600 you wrote:
>For example, if somehow an operator were able to _dynamically_ [and more >strongly] hint to end-hosts that native IPv6 is preferred compared to >IPv4, then _at the right time in the near future_ this would allow >operators to start signaling to hosts, on an per-Metro or per-region (?) >basis, that they believe a good quality, good performance, etc. >deployment of IPv6 exists (in the access, the Backbone and >peering/interconnect portions of the network) and they want hosts to >start originating packets using IPv6, when possible.  This would also, >potentially, absolve host stack vendors from making a difficult and/or >wrong call on a single, static address selection policy and, instead, >point the blame & pain of an address selection policy firmly back at the >ISP, where it belongs.  :-) 

I'm not sure in what situation that option would be required.

Let's assume that client operatings ship with a Happy Eyeballs implementation
with a slight preference for IPv6 then:
1) If the user doesn't have any kind of IPv6 it doesn't matter whether IPv6 is
   preferred or not.
2) If an ISP provides IPv6 and believes it actually works, then nothing needs
   to be doen because the systems already prefer IPv6.
3) If an ISP provides IPv6 but believes that it is broken, then this option
   would be useful.
4) The ISP doesn't provide IPv6 and doesn't want customers to have any other
   kind of IPv6 either (through tunnels).

4) doesn't make any sense (it may make sense in a corporate environment).
So 3) would be the only use case. Other than may be a brief period during
testing, I sort of can't imagine ISPs rolling out IPv6 and telling systems
not to use it because it doesn't actually work.

So, we end up with the first two cases, where from the ISP's perspective,
IPv6 should be preferred.

---

From: Keith Moore <[email-masked]>
Date: Fri, 5 Aug 2011 08:14:15 -0400

> but that's usually how it works.

Don't defend HE as being the best solution either.    HE is of pretty narrow applicability, really.    The document doesn't overstate its applicability, and the algorithm described in the document is not bad for web browsers, which is why I have no objection to it.   (and I'm happy to see IETF work in this area)   But the algorithm isn't going to work well for apps in general.

Also, my understanding is that Apple doesn't force programmers to use the clever API.     Programmers can still (and sometimes should) use simpler APIs and do their own style of path selection that better fits the needs of their specific application.

Also, don't assume that the v6ops group is in a good position to pass sole judgment on these algorithms.  

And as someone else pointed out, Apple has a small share of the installed base.   If this ends up creating a problem, Apple's customers, and Apple, will notice, and both will have incentive to fix it.  Meanwhile the impact on the rest of the net will be negligible. 

Architecturally speaking, putting a global preference policy based on address type into every host is a very dubious idea.  It cannot work well in general.

Keith

---

From: Ted Lemon <[email-masked]>
Date: Fri, 5 Aug 2011 13:23:57 +0000

In all fairness, Apple was working on this implementation before HE was a glimmer in anybody's eye, and indeed Stuart Cheshire described it in some detail in a v6ops meeting about two years ago.   I kind of think it might have inspired the development of HE, but I may be wrong about that.   In any case, I think throwing stones here is not really constructive.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 05 Aug 2011 15:53:45 +0200

In your letter dated Fri, 5 Aug 2011 13:23:57 +0000 you wrote:
>In all fairness, Apple was working on this implementation before HE was a g=
>limmer in anybody's eye, and indeed Stuart Cheshire described it in some de=
>tail in a v6ops meeting about two years ago.   I kind of think it might hav=
>e inspired the development of HE, but I may be wrong about that.   In any c=
>ase, I think throwing stones here is not really constructive.

He certainly deserves credit for presenting it.

But then you can't just go off to an island and just implement something.
It is always possible that as an operating system designer, you miss an
operator perspective.

Of course, it is possible that the implementation was frozen long before there
was any discussion here about the subject. But then you can just say that
you will try to get it fixed in a future release.

If Apple really has more than two years of experience with HE, then it would
have been nice if they would have shared some.

---

From: Philip Homburg <[email-masked]>
Date: Fri, 05 Aug 2011 16:06:52 +0200

In your letter dated Fri, 5 Aug 2011 08:14:15 -0400 you wrote:
>Don't defend HE as being the best solution either.    HE is of pretty =
>narrow applicability, really.    The document doesn't overstate its =
>applicability, and the algorithm described in the document is not bad =
>for web browsers, which is why I have no objection to it.   (and I'm =
>happy to see IETF work in this area)   But the algorithm isn't going to =
>work well for apps in general.
>
>Also, my understanding is that Apple doesn't force programmers to use =
>the clever API.     Programmers can still (and sometimes should) use =
>simpler APIs and do their own style of path selection that better fits =
>the needs of their specific application.

I could be wrong, but I think I read somewhere that Apple does reorder the
results of getaddrinfo. If that is true then application that use
the basic socket extensions would end up with HE whether they want it or
not.

>Architecturally speaking, putting a global preference policy based on =
>address type into every host is a very dubious idea.  It cannot work =
>well in general.

Note that the HE draft is smart enough to say that the address selection
policy should be followed unless there is evidence that a certain address
or protocol doesn't work all that well.

So there is the possibility of local overrides.

---

From: Daniel Roesen <[email-masked]>
Date: Fri, 5 Aug 2011 16:08:50 +0200

> It is always possible that as an operating system designer, you miss an

I get a distinct impression that the operator perspective wasn't of
relevant interest (and still isn't).

To make the IPv6 transission as painless as possible, all stakeholders
have to work together, based on a common "big picture".

Best regards,
Daniel

---

From: Ted Lemon <[email-masked]>
Date: Fri, 5 Aug 2011 14:12:52 +0000

On Aug 5, 2011, at 9:53 AM, Philip Homburg wrote:
But then you can't just go off to an island and just implement something.
It is always possible that as an operating system designer, you miss an
operator perspective.

I'm not sure v6ops is the place to have an argument about this, but I will remind you of the IETF motto: "running code and rough consensus."

---

From: Cameron Byrne <[email-masked]>
Date: Fri, 5 Aug 2011 09:14:22 -0700

> have to work together, based on a common "big picture".

I don't believe Apple's goal is IPv6 transition, they are locally
optimizing their own product and user experience.

Cameron

---

From: Warren Kumari <[email-masked]>
Date: Fri, 5 Aug 2011 14:02:43 -0400

[ Not responding specifically to Cameron here, more of a meta comment. ]

> optimizing their own product and user experience.

Wow! Imagine that, a company optimizing their user experience and providing the *users* what they want, instead of some religious ideology.

For shame.

For years the users have been living under the shackles of NAT, and their lives have been pure hell because of it -- they have not been able to use the applications they want, have not been able to access any content, send email, use VoIP, etc. Now is the time to liberate them from the tyranny of address translation and drag them, kicking and screaming into a new world, full of happiness, light and more bits in an address.

Sure, their lives may suck for a while while we add some latency to their IPv4 world, but, hey, this will just provide the necessary impetus to make them change... and, once they have, their dancing hamsters sure will look prettier, and they'll never look back...

W

---

From: Keith Moore <[email-masked]>
Date: Fri, 5 Aug 2011 14:08:55 -0400

On Aug 5, 2011, at 9:53 AM, Philip Homburg wrote:

> It is always possible that as an operating system designer, you miss an operator perspective.

True.  It is also always possible that operators miss the perspective of OS implementors, applications implementors, and users.

Keith

---

From: Mark Smith <[email-masked]>
Date: Sat, 6 Aug 2011 07:30:30 +0930

On Fri, 5 Aug 2011 14:08:55 -0400

+1

One of the best pieces of advice I got a long time ago was "think like
a user".

---

From: "Frank Bulk" <[email-masked]>
Date: Fri, 5 Aug 2011 16:10:16 -0700 (PDT)

In-Reply-To: <[email-masked]>
Subject: RE: [v6ops] Lion/Snow Leaopard side-by-side on an IPv6 enabled	ADSL2+ home line..
Date: Fri, 5 Aug 2011 18:10:26 -0500
Message-ID: <[email-masked]>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AcxSXHQWcADk5g7mToqWJeF+eWUzcABaE0Qg
Content-Language: en-us
X-Authenticated-User: [email-masked] 

Then why hasn't Microsoft addressed the IPv6 issues around accidental use of
ICS?  That's been known for a long time, MSFT is aware of it, yet I'm still
waiting for that patch...

Frank

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
George Michaelson
Sent: Wednesday, August 03, 2011 11:10 PM
To: Mark Andrews
Cc: IPv6 Operations
Subject: Re: [v6ops] Lion/Snow Leaopard side-by-side on an IPv6 enabled
ADSL2+ home line..

> shouldn't make the problem worse when this happens.

Mark, consider both Apple, and Microsofts codebase 3 years ago, and the
frequency of updates.

You cannot seriously be saying that the consequential 'damage' to the v6
message stems from this deployment, because it only applies to people WHO GO
ONLINE and if they go online with Apple they get FREE UPDATES.

Really?

-george

---

From: Rémi Després <[email-masked]>
Date: Sat, 6 Aug 2011 16:44:09 +0200

> Furthermore, if you provision your networks so that IPv6 paths have longer round-trip times or are congested more often and/or to a greater degree, then we will prefer less congested IPv4/NAT paths.

Full agreement on these two (at least for applications that express indifference about v4 or v6 being used).
 
> Finally, if you provision your networks so that IPv4 and IPv6 exhibit "roughly equivalent" levels of path latency and congestion, then we're going to try very, very hard to use both IPv4 and IPv6 i…

Serious disagreement on this one.
In this case, v6 MUST be preferred:
- If both paths are "good" (as they should be!), preference for IPv6 tends to increase the v6/v4 traffic ratio, which will lead to a visible fading out of IPv4 use.
- It's easier to implement, AFAIK, than trying hard to evenly split the traffic between v4 and V6.
Any solid argument against this?

Regards,
RD

---

From: Keith Moore <[email-masked]>
Date: Sat, 6 Aug 2011 11:55:03 -0400

> Any solid argument against this?

As far as I can tell, it's easier to implement if you don't try to bias in favor of IPv6.  Then the difference between v4 and v6 will amount to random noise, and the choice between a v4 path and a v6 path will essentially be random.  I don't know that it's necessary to "try very, very hard" to use both; I think this will happen fairly naturally.

But it might be reasonable for the software to do something like: if there's not a statistically significant difference between the measured performance of the different paths available, fall back to an rfc 3484-like policy table. 

Keith