Subject: [v6ops] CGN vs Native IPv6 latency

From: Cameron Byrne <[email-masked]>
Date: Mon, 14 May 2012 08:29:18 -0700

Hi,

I sometimes hear folks say that CGN adds latency or otherwise makes
the network slower, and this fact will mean that IPv6 will take more
traffic than the slower CGN IPv4 path.

I do not believe CGN adds any meaningful latency when the CGN path and
IPv6 are ostensibly the same for the internal network (something an
access network provider controls).  Sometimes, on the external network
(the internet), the IPv6 peering is less robust and results in longer
paths and therefore higher latency and lower performance for IPv6.

Here is some data i casually collect from my phone doing speedtest at
ipv6-test.com.   Yes, i know these test are flawed in many ways, but
this is just a casual observation.  The test was done using the
IPv6-only connection across an HSPA+ mobile network.  The IPv6 path is
native end to end.  The "IPv4 path" is IPv6 from the Samsung handset
to a NAT64 (CGN) and then IPv4 to the test server.  This is just a
casual test using production and easily available elements.

IPv4 / IPv6 speeds Mbit/s

10.2 /9.07
http://img2.ipv6-test.com/speedtest/result/2012/05/14/0b6984f7f589c0fdd10bce5bdaf7b80f.png

11.8/10.8
http://img2.ipv6-test.com/speedtest/result/2012/05/14/898322b8348a47dfe31d944bf3d8b154.png

11.8/12.0
http://img2.ipv6-test.com/speedtest/result/2012/05/14/ae2cb534d9d5e74020191d072f127456.png

As you can see, no noticeable difference in this casual observation.
I am sure someone with time and precision gear can show that CGN addes
~1ms of latency, but that is hardly impactful on today's internet

Does anyone have data showing CGN harms network performance from a
speed perspective?  Perhaps in such a way that it may impact a happy
eyeballs implementations that may do a "straight race" such as
Apple's?

CB

---

From: "Lee Howard" <[email-masked]>
Date: Mon, 14 May 2012 12:10:50 -0400

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Cameron
> Byrne
> Sent: Monday, May 14, 2012 11:29 AM
> To: IPv6 Ops WG
> Subject: [v6ops] CGN vs Native IPv6 latency
> 
> Hi,
> 
> I sometimes hear folks say that CGN adds latency or otherwise makes the
network slower,
> and this fact will mean that IPv6 will take more traffic than the slower
CGN IPv4 path.
> IPv6 are ostensibly the same for the internal network

Is that a universal architectural assumption?
That implies running CGN on your existing routers, which may not have the
capacity.  So
one could upgrade, or add additional devices in network hubs.  Or shunt
traffic to a data
center for the CGN.  The latter would add a couple of milliseconds.

> Sometimes, on the external network (the internet), the IPv6 peering is
less robust
> and results in longer paths and therefore higher latency and lower
performance for IPv6.

Also true.
 
> Does anyone have data showing CGN harms network performance from a speed
> perspective?  Perhaps in such a way that it may impact a happy eyeballs
implementations
> that may do a "straight race" such as Apple's?

I think Apple's implementation notices microseconds, which might change your
results.  

Thanks for sharing your results though; this is constructive conversation to
have.

Lee

> 
> CB
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: Tina TSOU <[email-masked]>
Date: Mon, 14 May 2012 16:37:26 +0000

Sent from my iPad

> Thanks for sharing your results though; this is constructive conversation to
Does iPAD support DHCPv6 already? I only see SLAAC supported. If anyone has the information how to use iPAD for DHCPv6, ping me off-list, thank u.

---

From: Fred Baker <[email-masked]>
Date: Mon, 14 May 2012 11:34:15 -0700

On May 14, 2012, at 8:29 AM, Cameron Byrne wrote:
> Does anyone have data showing CGN harms network performance from a speed perspective?  Perhaps in such a way that it may impact a happy eyeballs implementations that may do a "straight race" such as Apple's?

I personally would be very surprised to see CGN having an impact that is predictable and significant on an end to end basis.

An RTT is composed of the sum of the queuing delay, processing delay, serialization delay, and propagation delay on each hop end to end. For the most part, queuing delays will be nominal as bandwidth significantly exceeds demand; there will usually be one or two hops that act as a bottleneck, but most are undersubscribed. Processing delays tend to be nominal as well, and serialization and propagation delays are governed by the physical layer technology and physics. The argument has to be either that the CGN is located "somewhere else" and therefore experiences a different propagation delay (which might be better or worse than some other path), or that it involves more header reading and writing. 

The latter, it certainly does. Now, what does that constitute? Finding numbers that are public is of course an interesting problem, but I think we can all agree that available CGN products all maintain "gigabits" of throughput. If so, that translates to an order of magnitude of "millions" of packets per second, which inverted means "millionths" of a second per packet. If we were talking about tens-to-hundreds of milliseconds, it would be something a user would notice; if it's on the order of magnitude of milliseconds-to-tens-of-milliseconds, we're on the same order of magnitude as normal queuing delay variation.

And "millionths" isn't even "milliseconds". It's microseconds or portions thereof. 

Yes, the execution process of any translation is a bigger deal than a simple forwarding decision. Having written code that does both, I would be hard pressed to describe that in larger terms than "a subroutine".

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 14 May 2012 13:53:00 -0500

> I think Apple's implementation notices microseconds,

This begs the beaten-up question - should the happy eyeball algorithm
tolerate few 100s of usec to still favor IPv6?

Cheers,
Rajiv

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Lee Howard
> Sent: Monday, May 14, 2012 12:11 PM
> To: 'Cameron Byrne'; 'IPv6 Ops WG'
> Subject: Re: [v6ops] CGN vs Native IPv6 latency
> 
> 
> 
> > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> > Of
> Cameron
> > Byrne
> > the
> capacity.  So one could upgrade, or add additional devices in network
hubs.
> Or shunt traffic to a data center for the CGN.  The latter would add a
couple
> of milliseconds.
> > eyeballs
> I think Apple's implementation notices microseconds, which might
change
> your results.

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Mon, 14 May 2012 13:54:09 -0500

Are there any public/free servers for latency measurement similar to
that of speed measurement?

That would be helpful here.

Cheers,
Rajiv

> Of Cameron Byrne
the
> network slower, and this fact will mean that IPv6 will take more
traffic than
> the slower CGN IPv4 path.
> 
access
> network provider controls).  Sometimes, on the external network (the
> internet), the IPv6 peering is less robust and results in longer paths
and
> therefore higher latency and lower performance for IPv6.
> 
IPv6-only
> connection across an HSPA+ mobile network.  The IPv6 path is native
end to
> end.  The "IPv4 path" is IPv6 from the Samsung handset to a NAT64
(CGN)
> and then IPv4 to the test server.  This is just a casual test using
production
> and easily available elements.
> 
> http://img2.ipv6-
> test.com/speedtest/result/2012/05/14/0b6984f7f589c0fdd10bce5bdaf7b80f.
> png
> 
> 11.8/10.8
> http://img2.ipv6-
> test.com/speedtest/result/2012/05/14/898322b8348a47dfe31d944bf3d8b15
> 4.png
> 
> 11.8/12.0
> http://img2.ipv6-
> test.com/speedtest/result/2012/05/14/ae2cb534d9d5e74020191d072f12745
> 6.png
> 
Apple's?

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 14 May 2012 16:59:39 -0700

> Of Rajiv Asati (rajiva)
> Sent: Monday, May 14, 2012 11:53 AM
> To: Lee Howard; Cameron Byrne; IPv6 Ops WG

There is text in Happy Eyeballs explaining reasons to prefer IPv6
over IPv6 (*).  But today it does not matter at all to users,
especially users on dual-stack clients accessing dual-stack
servers.  IPv4-only clients accessing servers, or dual-stack
clients accessing IPv4-only servers may well experience port
starvation, but users will attribute that problem to their
"old" IPv4-only client or to the lack of IPv6 on the server.
All that assumes that port starvation will exist; that is 
entirely dependent on how many ports the ISP allocates at busy 
hour to their subscribers.

(*) "Delay IPv4", http://tools.ietf.org/html/rfc6555#section-4.1

-d

---

From: Cameron Byrne <[email-masked]>
Date: Mon, 14 May 2012 17:19:46 -0700

> (*) "Delay IPv4", http://tools.ietf.org/html/rfc6555#section-4.1

And port exhaustion should not be expected in the real world of stateful
translation that may use port overloading

CB

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 14 May 2012 17:44:41 -0700

...
> And port exhaustion should not be expected in the real world of
> stateful translation that may use port overloading

It can help, but if all the subscribers visit the same popular 
IPv4 address, port overloading can fail to provide service -- its
law of large numbers collapses.  There is a proposal to improve one 
of the problems (TCP server going into TIMEWAIT, see (*) and
RFC6191) but the number of active connections to a popular server 
remains a problem if the server is suddenly more popular than 
anticipated by the port allocation.  Stateless MAP ("A+P")
techniques can suffer significantly from attempts to use port
overloading, because their user base is smaller (often one 
house), and a stateful CGN suffers fewer problems (because
it can utilize more port numbers if there is sudden popularity
to a single IPv4 server).  The use of SPDY, which reduces the 
number of TCP connections to servers, will help all of these problems.

(*) http://www.ietf.org/proceedings/83/slides/slides-83-tcpm-2.ppt
    http://tools.ietf.org/html/draft-naito-nat-resource-optimizing-extension

Port overloading also interferes with, or breaks, peer to peer 
applications such as SIP, Skype, Bittorrent, Octoshape, and the 
upcoming WEBRTC.  I know those applications are often not well loved 
for a variety of reasons.  That is documented in REQ-1 of RFC5382, which 
says port overloading is bad.  I realize business reasons cause a 
lot of networks to be deployed violating that requirement.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 14 May 2012 19:20:20 -0700

> Sent: Monday, May 14, 2012 8:29 AM
> To: IPv6 Ops WG
> test.com/speedtest/result/2012/05/14/0b6984f7f589c0fdd10bce5bdaf7b80f.p
> ng
> 
> 11.8/10.8
> http://img2.ipv6-
> test.com/speedtest/result/2012/05/14/898322b8348a47dfe31d944bf3d8b154.p
> ng
> 
> 11.8/12.0
> http://img2.ipv6-
> test.com/speedtest/result/2012/05/14/ae2cb534d9d5e74020191d072f127456.p
> ng
> 
> speed perspective? 

For a NAT64, http://www.lightreading.com/document.asp?doc_id=216955 says 
average bi-directional latency is about 800 microseconds (0.8 milliseconds),

which includes all 7 devices in their test bed (not solely the NAT64).
Worst
case was about 108 milliseconds (bi-directional), also including all 7
devices in their test bed.

To my knowledge, that's the best test that has been published.

> Perhaps in such a way that it may impact a happy

Send synthetic traffic through the IPv4 path, so it stays busy and 
slows down?  :-)

-d

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 15 May 2012 11:38:00 +0100

On 15/05/2012 01:44, Dan Wing wrote:
> law of large numbers collapses.

I would be particularly concerned in this situation about CDNs (e.g.
akamai, limelight and all those guys) and other anycast addresses like as112.

Nick