Subject: Re: [tcpm] TCP, multiple addresses and soft errors when connecting

From: Pekka Savola <[email-masked]>
Date: Fri, 2 Apr 2004 14:30:55 +0300 (EEST)

I've been waiting for other comments on this, but none have emerged 
so...

What I propose as a way forward for this specific Informational 
document:

 1) state the reasons why this specific connect() etc. behaviour is 
desirable (read: why many vendors have already done that), and what 
drawbacks it could have (if any), and

 2) state the desire for getting a better higher-level API which would
handle all of this and much more (instead of/in addition to 1), but
mention why this is not an only practical solution.

Does this sound reasonable and well-balanced?

On Mon, 22 Mar 2004, Fernando Gont wrote:
> At 16:39 20/03/2004 +0200, Pekka Savola wrote:
> 
> > > Does it really make sense to handle the whole connection establishment
> > > issue inside the app, instead of inside the API itself?
> >Those are the core APIs we have today.  We could have better ones, but
> >IMHO I think we need to fix problems in the current ones as long as we
> >don't have a widely-deployed alternative.
> 
> IMHO, there are two points to note:
> 
> * If you fix the current ones so that their behavior is "acceptable" to the 
> application programmer, then it'll be harder to introduce a new API. People 
> is reluctant to change, so they must have a reason for learning a new API. 
> If you try to fix the current one, they will find fewer reasons for the change.
> 
> * You said "we don't have a widely-deployed alternative". Unless you meant 
> "we don't have a well-tested alternative", I'd say that it's a 
> chicken-and-egg problem. That means, for an API to be widely deployed, 
> people would have to use it. And for people to use it, they should be able 
> to find a good reason to do it.
> 
> Yes, I'm aware that my proposal my sound as a long-term solution. But, 
> IMHO, it will certainly be harder to introduce a new API if you actually 
> want to do it "in long term".
> I guess "timing" is an issue, here.
> 
> 
> 
> > > I think that if we're debating this, we're implicitly assuming that
> > > applications want to connect to say www.example.com, and not one specific
> > > IP address to which it maps to.
> > >
> > > So why not implement such a "higher-level" connect(), and handle all the
> > > relevant issues inside the API?
> >
> >Defining such APIs is probably beyond the scope of the work we can do,
> >and the things we can fix except in the 5+ year timeframe.. but if I
> >get you right, you're proposing that instead of:
> >
> >  1) getaddrinfo() or gethostbyname() and connect for one address,
> >[where, arguably, TCP retry timeout could be warranted -- I don't
> >think so myself, but you could argue for it]
> >
> >  2) getaddrinfo() / gethostbyname() loop to connect to one address
> >[where TCP retry timeout could not be warranted]
> >
> >You'd be proposing to create a new function to accomplish 2) in such a
> >manner that it would also react to ICMP messages, etc?
> 
> Exactly.
> 
> Then, within that function, each individual connect() could react to ICMP 
> messages a
> as you proposed, but if all addresses fail because of ICMP errors, it could 
> let TCP retry in the hope the transiet problem will dissappear.
> 
> 
> [....]
> > > Doesn't the problem really have to do with having applications performing
> > > such a low-level action?
> >
> >At some point, someone has to decide which behaviour is desirable.
> >Isn't this only hiding the same policy decision inside an abstraction
> >layer
> 
> Yes, but even when the difference may sound subtle, IMHO it's not.
> Having that abstraction layer is what let's you choose a policy, without 
> having the risk of causing any harm to existing apps.
> That way, if you ever need to change the low-level-connect() behavoir for 
> whatever reason, you could still do it. As far as the high-level connect() 
> behaves the same, there would be no problem.
> 
> Think about it. Why should an application programmer care about TCP 
> retries, ICMP erros, an the like?
> In the same way, why should he care about setting "strange" options such as 
> SO_REUSEADDR on his listening socket?
> Should *he* really handle byte-ordering issues  by means of htons() and the 
> like?
> 
> 
> >-- and the same could be achieved by e.g. a setsockopt() flag?
> 
> Strictly speaking, you *could* do that. But then you'd have all programmers 
> implementing themselves a function you could provide them.
> You'd have all them implementing the getaddrinfo()/gethostbyname() loop, 
> setting a strange socket option, etc. Why not provide them with a 
> well-tested function, and let them spend their time implementing their 
> application itself?
> 
> 
> 
> >(I don't deny that better APIs would not hurt, but we still seem to
> >require low-level C-like primitives, and unless there are additional
> >ones implemented there, we still have to fix the problems somehow..)
> 
> Well, the low-level ones could be provided as part of the new API.
> 
> Or along with the providing a new API, the current one could be modified as 
> you suggest. It wouldn't hurt too much if the *low-level* API aborted a 
> connection establishment because an ICMP error is received. Being a 
> *low-level* API, you'd be suppossed to know the details behind the function.
> 
> Best Regards,
> 
> 
> --
> Fernando Gont
> e-mail: [email-masked] || [email-masked]
> 
> 
> 
> 

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Fernando Gont <[email-masked]>
Date: Sat, 03 Apr 2004 15:33:41 -0300

>drawbacks it could have (if any), and

I agree.

>mention why this is not an only practical solution.

What do you mean "this is not an only practical solution"?

---

From: Pekka Savola <[email-masked]>
Date: Mon, 5 Apr 2004 09:25:41 +0300 (EEST)

> >drawbacks it could have (if any), and

Great.

> What do you mean "this is not an only practical solution"?

What I should have said if "but mention why this is would not be a 
practical solution, if there were no other solutions".

That is, new APIs would be great, but we cannot stop fixing problems
current APIs until we have something more concrete out there.  
Otherwise we'd end up with no practically (in short term) deployable
solution.

---

From: Fernando Gont <[email-masked]>
Date: Mon, 05 Apr 2004 23:04:08 -0300

>practical solution, if there were no other solutions".

Actually, I think it is the only real solution to the problem.
Having the API report ICMP errors to the application is moving the problem 
from the API to the application, but *not* solving it.

If you leave connect() as is, then you may have the delay problem you 
describe (i.e., you may suffer undesirable delays to use a service).
If you make it report ICMP errors to the application, you still have a 
problem I mentioned in the other e-mails. You might even say that you'll 
have application programmers (probably *not* knowledgeable on TCP/IP 
internals) handling issues of the underlying protocols.

So a real solution is to hide all this stuff from the application 
programmer. Let him just concentrate on actually using a service, and 
handle all the protocol-related issues inside the API.

Furthermore, if for whatever reason you need to change the behavior of 
connect() in the future, then, unless you provide a more "abstract" 
connect(), you'll have to "bother" application programmers again.

>Otherwise we'd end up with no practically (in short term) deployable

If you consider it useful, I could try to continue thinking on the 
"abstract" API. I'm just thinking about writting some stuff down, so that 
there's a place from which which can move things forward. It may take me 
some time (say, months), but...

(BTW, for those interested tcpm-only readers, I've just posted a message 
with subject "Some comments on draft-ietf-v6ops-v6onbydefault-01.txt" on 
the v6ops list, that's related to all this. I can forward it to to the tcpm 
list, if you want).

---

From: Pekka Savola <[email-masked]>
Date: Fri, 9 Apr 2004 01:24:23 +0300 (EEST)

> internals) handling issues of the underlying protocols.

I have to disagree here.  Writing getaddrinfo() loops which include
loops around socket() and even connect() are commonplace so this is
nothing new.  Actually this was discussed in
draft-ietf-v6ops-application-transition-02.txt which we just shipped
off to the IESG.  Section 6.3.2 gives a specific example and section
6.3 describes alternative means; the user does not -- necessarily --
need to know anything and the coder only needs to know to build the
right logic. (Which is required anyway for proper v6/v4 applications.)

Again, high-level API which makes it smoother to use either v4 or v6 
would not hurt either, and could be achieved in one shot.  But again, 
this is not here and now.

> handle all the protocol-related issues inside the API.

For long-term, yes, that would be preferable (even though I think 
still there should be low-level equivalents, but not necessarily 
integrated with connect()).

But the problem is that the IETF has avoided specifying APIs whereever 
it has been able to, and even if we made these here (or elsewhere), 
there would have to be significant vendor buy-in (so that they could 
be used to create portable code), and implementation.  Even if there 
would be significant interest, this would take 3 years (minimum).

So, for all the good having such APIs would do, I fail to see how it 
could provide sufficiently short-term fix to this particular problem, 
and which is why we're documenting the connect() semantics change some 
have already done.

> some time (say, months), but...

I certainly think this is interesting work (actually, I've seen some
abstractization libraries for Unix already a couple of years ago,
doing something like you say -- but I can't find it now; maybe it was
something along the lines of http://thf.ath.cx/snl), but as I stated
above, there are significant barriers to be torn down first..