Subject: [dispatch] RELOAD and UDP

From: Cullen Jennings <[email-masked]>
Date: Tue, 7 Dec 2010 16:02:03 -0500

I just wanted to point out that the RELOAD protocol in P2PSIP WG is getting close to being sent to IESG and I wanted to point out that it uses UDP in a way that relates to some of the recent discussions in this WG, in particular provision of reliable message transport over UDP.

WHY UDP?
Like many other protocols, RELOAD has the issue where two devices that are both behind different NATs need to talk to each other. It's impossible to ensure that direct is always possible so relays are used in some cases but it's optimal to have it be direct as often as possible. While most of the data
about NAT penetration success is confidential there is still enough public information available out of CMU and MIT to make the following points clear:  1) UDP works a fairly high percentage of the time with ICE-like mechanisms. Most claims for this seem to be in the 80% to low 90's for success rate, with some variation based on geography. 2) Success rates for TCP simultaneous open without port prediction are far lower. It is also very fragile if lots of simultaneous sessions are happening. 3) Port prediction drastically raises the success rate of both UDP and TCP in ideal conditions but fails  if the conditions are far from ideal. 4) Opinions vary widely on how much value port prediction ultimately ours but it's worth noting that most of the widely deployed software that has good nat traversal does not bother to even try port prediction. All of this leads to one conclusion, with todays NATs, UDP has significantly higher success rate than direct TCP or pinhole management protocols.

HOW TO USE UDP?
When you want to do a reliable congestion safe protocol between two clients behind NATs you have three major choice:

I will broadly put them in the following classes 

1) use UDP with a simple congestion / reliability protocol 
2) tunnel kernel TCP over application controlled UDP tunnel 
3) do a user land TCP stack in the application and tunnel over UDP 

I'll get back to #1 in a minute but let me start with #2. This seems attractive initially but from a software perspective it's a non-starter. The problem is that in order to get access to the IP datagrams or TCP segments on most operating systems you need to be in the kernel (either directly or as a kernel module). RELOAD is an application expected to be installed and run by users possibly without super user privileges. Historically, enterprise IT folks have quite bit wary about allowing users to install VPN clients, which have similar issues but are much easier to explain. [And don't get me started on the
potential interaction between RELOAD and a VPN client.] IMO, asking people to install kernel-level software to run a softphone is a nonstarter.

This leaves us with the second natural alternative, userland TCP. It's true that you can readily find ports of kernel TCP stacks, but all the ones I know of seem to be experimental or research projects. 
I'm not aware of anyone that has a rock solid version of such a stack, let alone one that's widely used. I'm also very hesitant on the long term implications of doing this. It very obvious that one could change a TCP stack in trivial ways to give the user far more than their "fair" share of the network, and right now that's mostly prevented by the major OS developers (Microsoft, Apple, the Linux Kernel folks, Google^H^H^H^H^H doing the right thing.)  Once we get into an environment where we're encouraging everyone to develop their own TCP stack the threat of misbehavior (including misbehavior due to misguided "optimizations") becomes much greater. So, while a user land TCP stack might be fine in theory, we have been discussing this since something like 2002 and we still don't have an example in practice that work well and is widely usable. Having worked with phones that had bugs in the TCP stack that had been fixed in all mainstream TCP stacks 15 years previously, I'm really not a fan of yet more poorly supported, tested, and written ports of of the BSD TCP stack. I can certainly imagine that someone could do a great job of this and prove me wrong - and I'd change my opinion at that point - but I've been waiting many years for this and no sign of solution yet. 

This gets us back to number #1 - writing something that is congestion safe and reliable is actually very easy and we do all the time in a wide range of IETF protocols. The catch is that is has no where near the performance of TCP. If you need TCP grade performance as well as safe and reliable, well that is hard. Luckily RELOAD did not need high throughput efficiency in the slightest so this was not a big deal.
The decision we made for RELOAD simplest thing that looked like it would work and leave extensibility points to put in something better. So there is a minimal algorithm required to run over UDP with negotiation that would allow insertion of much more complex algorithm, such as full blown user land TCP, in the future. To facilitate only one side needing to upgrade to get a better algorithm, the receiver reports back packet loss to the sender that would allow the sender to implement very advanced protocols. This may have been a bit more than needed but it was easy to implement and left a lot of flexibility for future. The design has been cleared with TSV, so we can at least have confidence it's safe.

Obviously, this isn't ideal and if the software environment were different, I suspect we would have made different choices, but IMO this is the best that can be done at this time.

Sorry for the long post of this but I wanted people here to be aware of what RELOAD does and why it got to that design. 

Cullen <in my individual contributor role>

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Tue, 07 Dec 2010 15:55:01 -0600

On 12/07/2010 03:02 PM, Cullen Jennings wrote:

> Cullen<in my individual contributor role>

(wearing my rose colored glasses)

If the 'environment being different' includes reasonably wide deployment 
of IPv6, and reduced need for NAT devices in the path in order to 
provide adequate addressing space, would that mean that RELOAD could 
have an option to operate over existing TCP-on-IPv6 stacks? Are the 
issues with being able to open end-to-end connections caused by the NAT 
devices doing address mapping, or by their firewall-like functionality, 
or both? I would think that the problems are primarily caused by the 
address mapping... and a network that had reduced need for address 
translation, along with pinhole management, could quite easily support 
end-to-end TCP connection establishment.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
skype: kpfleming | jabber: [email-masked]
Check us out at www.digium.com & www.asterisk.org

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Fri, 10 Dec 2010 09:36:06 -0800

This discussion of support for end-to-end TCP connection establishment
through firewalls/SBCs came up in the DISPATCH meeting at IETF 79. While
at least one vendor supports this, most do not, so it does not appear to
be trivial for implementations in general.
As another data point, support for a new application media line in SDP
for far end camera control (RFC 4573) is an example of the type of
change that appeared to be quite straightforward for support through
firewalls/SBCs.

Cheers,
Charles

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
Behalf Of Kevin P. Fleming
> Sent: Tuesday, December 07, 2010 1:55 PM
> To: [email-masked]
> Subject: Re: [dispatch] RELOAD and UDP
> 
> > Obviously, this isn't ideal and if the software environment were
different, I suspect we would have
> made different choices, but IMO this is the best that can be done at
this time.
> >
> > Sorry for the long post of this but I wanted people here to be aware
of what RELOAD does and why it
> got to that design.
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch

---

From: Cullen Jennings <[email-masked]>
Date: Sun, 12 Dec 2010 10:31:14 -0700

On Dec 7, 2010, at 2:55 PM, Kevin P. Fleming wrote:

> If the 'environment being different' includes reasonably wide deployment of IPv6, and reduced need for NAT devices in the path in order to provide adequate addressing space, would that mean that RELOAD could have an option to operate over existing TCP-on-IPv6 stacks?

Sure - if there were less NATs and firewalls. I will note that every major NAT vendors seems to be working on multiple variants of NAT for v6 and several IETF WG are working on v6 transition strategies that have many of the properties of a NAT. 

> Are the issues with being able to open end-to-end connections caused by the NAT devices doing address mapping, or by their firewall-like functionality, or both?

Both - even with no NATS, you need to be able to tell the FW to open a pinhole. And the mapping issues makes it much harder to know the address/port to make the pinhole go to.

> I would think that the problems are primarily caused by the address mapping... and a network that had reduced need for address translation, along with pinhole management, could quite easily support end-to-end TCP connection establishment.

I somewhat agree with you - I wish the the network was moving towards what you describe but I doubt that will happen any time real soon. One of the key points you said was a pinhole management. Many good solutions to this class of problem would involved the client behind a firewall being able to tell the firewall to open a specific pinhole. However this is extremely unlikely to fly in many cases because of the complexity of creating a trust relationship between the client and the firewall that makes the firewall administrator willing to allow the client to install new firewall rules. Note that an unauthenticated outgoing packet being used to create a pinhole may be good enough for many NAT scenarios but many firewall scenarios don't consider this strong enough. Furthermore, even discovering the many firewalls that a given flow goes through is a challenging problem. 

Now if only I had a PBX where NAT/FW traversal was a key part of the signaling protocol design :-)