Subject: RE: [Hipsec] WGLC: draft-ietf-hip-native-api-03.txt

From: Miika Komu <[email-masked]>
Date: Sat, 5 Jan 2008 20:51:17 +0200 (EET)

On Tue, 18 Dec 2007, Henderson, Thomas R wrote:

>> -----Original Message-----
>> From: Gonzalo Camarillo [mailto:[email-masked]]
>> Sent: Wednesday, November 28, 2007 1:22 PM
>> To: HIP
>> Subject: [Hipsec] WGLC: draft-ietf-hip-native-api-03.txt
>>
>> Folks,
>>
>> I would like to WGLC the following draft:
>> http://www.ietf.org/internet-drafts/draft-ietf-hip-native-api-03.txt
>>
>> This WGLC will end on December 19th.
>>
>> Send your comments to the author and this list.
>>
>> Thanks,
>>
>> Gonzalo
>> HIP co-chair
>>
>
> Miika,
> Here are some more comments.  In summary, I'd like to see more work on
> this draft on some points below.

Tom,

thanks for the comments. Please notice that some of the comments were 
already addressed in the draft-ietf-hip-native-api-04-pre1 which I had 
posted to the mailing list in the beginning of December:

http://www.iki.fi/miika/docs/draft-ietf-hip-native-api-04-pre1.txt

Tom, I updated the version based on your comments. The new draft is found 
from here:

http://www.iki.fi/miika/docs/draft-ietf-hip-native-api-04-pre2.txt

Please have a look at the up-to-date version and check if it is ok. More 
comments below.

> Overall
> ========
>
> I still don't think the document gives enough guidance to the
> implementor about where various aspects of HIP-related API are defined
> (there are two other API documents in shim6 and BTNS WGs).
> Specifically, the shim6 API draft specifies new socket options
> applicable to shim6 and HIP, but some of them are not applicable when
> using PF_HIP.  I would like to see more clearly stated which shim6
> options from Table 1 of the shim6 document are applicable/invalid when
> using PF_HIP.

I think this should be added to the SHIM6 draft or to a separate draft for 
the following reasons:

* I think it is out of scope when the abstract (and not the title :) of
   draft is conserned.
* The draft only requires only two simple socket options from the shim6
   document. Editorialwise, I believe it is a good idea to avoid
   intertwining the two drafts too  much.
* The scope of applicability is not clear yet (consider e.g. reap) and I
   think this needs to mature a bit.

> So I would like to see more explicit guidance of the form; e.g.:
> - this document specifies extensions to RFC3493 and 3542 to define a new
> socket address family and describe how the socket calls in those RFCs
> are adapted or extended as a result

Agree on RFC3493, but I am not convinced yet that RFC3542 has much to 
do with this draft. I fitted your comments to one paragraph in the intro:

"This document specifies extensions to RFC3493 to define a new socket 
address family, AF_HIP. The macro PF_HIP is used as an alias for AF_HIP in 
this document because the distinction between PF and AF has been lost in 
the practice.  The extensions also describe a new socket address structure 
for sockets using Host Identity Tags (HITs) explicitly and describe how 
the socket calls in "RFC3493 are adapted or extended as a result."

> - socket options for use with PF_HIP/PF_INET6 sockets are defined in the
> [shim6-api], and section X.X of this document clarifies which options
> are applicable to PF_HIP sockets.

I disagree on this, see above.

> - this document relates to the RFC5014 as follows...
> - APIs to manipulate IPsec bindings are defined in [btns-api]...

    There are two related API documents.  Multihoming and explicit
    locator-handling related APIs are defined in
    [I-D.ietf-shim6-multihome-shim-api].  IPsec related policy attributes
    and channel bindings APIs are defined in [I-D.ietf-btns-c-api].  The
    extensions defined in this document can be used independently of the
    two mentioned related API documents.

> Andrew also pointed out the need to align with or extend RFC5014 (source
> address selection).

I'll address these commments in a separate thread.

> The introduction should clearly state a design assumption:
> "This API allows a HIP-aware application to use the sockets API
> independent of any handling of IP addresses, but also allows an
> application to use HIP names in conjunction with IP addresses if so
> desired.  As a result, this API implies that the system may need to
> resolve HIP names to IP addresses, and describes how a system resolver
> might be able to cache such bindings during the name resolution
> process."

I believe this was stated already at least in 
draft-ietf-hip-native-api-04-pre1:

    The extensions in this document focus on the use of the resolver to
    map host names to HITs and locators in HIP-aware applications.  The
    resolver associates implicitly the the HIT with the locator(s).
    However, it is possible that an application operates directly with a
    peer HIT without interacting with the resolver.  In such a case, the
    application may resort to the system to map the peer HIT to an IP
    address.  Alternatively, the application can explicitly map the HIT
    to an IP address as specified in [I-D.ietf-shim6-multihome-shim-api]
    using the socket options SHIM_LOC_LOCAL_PREF and SHIM_LOC_PEER_PREF.

"How a system resolver might be able to cache such bindings" sounds more 
like implementation issue but I added a clarification into section 
"Interaction with the Resolver":

The resolver associates implicitly the HIT with the locator(s) by e.g. 
communicating the association to the HIP daemon.

> In section 4.1:
> "The HIT value is an IPv6 address and it is stored in network byte
> order."  I disagree, and I think other reviewers will too.  HITs are not
> IPv6 addresses.

Corrected to: "The ship_hit is the size of an IPv6 address and it is 
stored in network byte order."

> In section 4.3 (page 8) the document starts talking about ORCHIDs, in a
> manner that suggests at first glance that they are distinct from HITs.

I changed the text to this:

         The HIP_HIT_ANY_* macros reject non-HIP based
         communications. To distinguish between HIP <xref
         target="RFC4843" /> and non-HIP-based communications in the
         case of the HIP_ADDR_ANY macro, the application calls
         getsockname() and getpeername() to discover the actual
         identifiers used for the communications and verifies the
         prefix with HIP_IS_ORCHID macro. ..

> I think I understand what you are trying to do here, but I am not sure
> it is the right way.  If I understand correctly, you are allowing to
> bind sockets to wildcarded HIT macros, but then allowing the system to
> accept the communications even if it is not a HIT but is instead an IPv6
> address, unless it sets a flag to "ONLY_ORCHID" in the ship_flags
> (although it is unclear which socket call requires this flag to be set).

I move the ONLY_ORCHID flag functionality to the HIP_ADDR_ANY macro 
(seemed to be a more right way to do it):

    The application can use the HIP_HIT_ANY_* and HIP_ADDR_ANY macros to
    accept incoming communications to all of the HITs of the local host.
    Incoming communications refers here to the functions such as bind(),
    recvfrom() and recvmsg().  The HIP_HIT_* macros correspond to the
    sockets API macros INADDR_ANY and IN6ADDR_ANY_INIT, but they are
    applicable at the HIP layer.  After initial contact with the peer,
    the local and peer HITs can be discovered using getsockname() and
    getpeername() calls for connection oriented sockets.  The difference
    between the use of the HIP_HIT_* and HIP_ADDR_ANY macros here is that
    the former allows only HIP-based communications but the latter also
    allows communications without HIP.

    The application also uses the HIP_HIT_ANY or HIP_ADDR_ANY macros in
    ship_hit field to establish outgoing communications in Opportunistic
    mode [I-D.ietf-hip-base], i.e., when the application knows the remote
    peer locator but not the HIT.  Outgoing communications refers here to
    the use of functions such as connect(), sendto() and sendmsg().
    However, the application must first associate the socket with at
    least one IP address of the peer using SHIM_LOC_PEER_PREF socket
    option.  After initial contact with the peer, the application
    discovers local and peer HITs using getsockname() and getpeername()
    calls when it is using connection-oriented sockets.  The difference
    between the use of the HIP_HIT_ANY and HIP_ADDR_ANY macros here is
    that the former allows only HIP-based communications but the latter
    allows the socket to timeout and fall-back to communications without
    HIP.

I believe it should be now clear from the context that which socket 
functions require the HIP_ADDR_ANY macro.

> My opinion is that more work needs to be done on this part of the API.

Please be more verbose or suggest some text if there is something more 
needed to the above texts.

> I think you want to allow an application to set some wildcards in the
> socket calls and possibly permit the socket to open if the peer does not
> support HIP but is reachable via IPv6.  I think there will need to be
> some care taken to not violate semantics since we are mixing address
> families at this point.

The locator and its family is not actually visible to the application when 
it is using the wildcards. Also, I don't see why we should constrain only 
only to IPv6-based locators.

> Regarding the last paragraph of section 4.1, some systems support
> accept_filters that allow the system to do the access controls-- do we
> want to specify the same here?

This was new to me. Added a small note about this even though I am not 
sure yet how relevant this is:

         Applications may rely on system level access control, either
         implicit or explicit (such as accept_filter function() found on
         BSD-based systems), but such discussion is out of scope.
         However, applications can also implement access control
         themselves by using the HITs.

> In section 4.2 (resolver extensions), the description implies that a
> resolver will have to do two queries (nodename to HITs and nodename to
> IP addresses), and concatenate the results in the rres field, before
> returning from getaddrinfo().  Shouldn't the application instead issue
> two getaddrinfo's for each of the different queries?

Why complicate things unnecessarily for the developer? The current API is 
designed so that it can be used in existing apps with minimal changes 
(i.e. by adding some flags). See e.g. some example code from the FreeBSD 
man pages:

http://www.freebsd.org/cgi/man.cgi?query=getaddrinfo&sektion=3&apropos=0&manpath=FreeBSD+6.2-RELEASE

Is there a good reason for doing it this way? If the small delay of DNS is 
problem, this can certainly parallelized, but that is an implementation 
issue.

> Section 4.3 first paragraph could perhaps be developed further.  Maybe
> it warrants another major section in the document.

I think this comment is too generic for me to write anything concrete. 
Please suggest some text.

> Also, I did not quite understand the placement of the second paragraph 
> in this section. It seems like this guidance (what error to return if 
> the HIT cannot be resolved) is more appropriate somewhere in Section 
> 4.1.

Moved to the end of section 4.1.

> Editorial
> ==========
>
> The last sentence of the abstract has some missing words.
>
> Generally, Terminology section follows the Introduction.

Moved.

> It seemed to me that the third paragraph of the Introduction might be
> the one that you want to lead with.

Ok.

> I do not think that "HIP Layer" row needs to be in Table 2.  To me, this
> is akin to saying that there is an "ARP Layer" between Network and Link
> layer, and that is not how ARP is commonly depicted in the stack.

I think the uninitiated reader will question the presence of HIP in the 
stack if we remove it. Anyway, I think the whole section 3.1 is rather 
uninformative so I have removed it from the draft. Is this ok?

(As a side effect, I moved "Interaction with the Resolver" section upwards 
to replaced "Design Model" because it was the only section in the main 
section.

> I was surprised that there was no reference to the HIP DNS draft since
> this is heavily based on HIP DNS resolution.

It was probably removed when I removed one section based on some comments 
in an earlier version. Reintroduced now the reference to the beginning of 
the intro.

> In the paragraph after figure 2:
> s/published or not, but that/published or not, but note that/
>
> I thought that we agreed to delete the "IPV6_ADDR" portion from the
> macro HIP_IS_IPV6_ADDR_ANON_HIT?

This was already fixed in draft-ietf-hip-native-api-04-pre1.

> For document title, I would suggest changing from:
> Native Application Programming Interfaces (APIs) for Host Identity
> Protocol (HIP) to Basic Socket Interface Extensions for Host Identity 
> Protocol (HIP) which would align the title with RFC 3493.

Fine by me. I changed the abbreviation also to "Basic API Extensions for 
HIP". I won't change the name of the file, though.

> Lingering comment from my previous post
> =========
>
>>>>     Both of these two approaches may be more prone to errors
>>>> than the use
>>>>     resolver with host names.  Hence, HIP-aware applications should
>>>>     prefer to use the resolver with host names.
>>>
>>> Can you specify which errors the resolver-less approach is
>> more prone
>>> to?
>>
>> The main concern is wow to map a HIT to an IP address in the
>> current state
>> of HIP deployment in DNS. Do you suggest to move the last two
>> sentences?
>
> I just want to understand what the assumed error risk is.  You seem to
> be saying that indirecting through the DNS is less error prone than
> directly providing the HIT to IP address binding at the API.  Is that
> because you are assuming that handling HITs is an error prone operation
> (as compared with DNS names) for either applications or users?
>
> I think it should either be more specified to state what the presumed
> error is, or else remove the last two sentences.

This was already removed in draft-ietf-hip-native-api-04-pre1.

Thanks for the feedback!

-- 
Miika Komu                                       http://www.iki.fi/miika/

_______________________________________________
Hipsec mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/hipsec

---

From: "Henderson, Thomas R" <[email-masked]>
Date: Mon, 28 Jan 2008 08:41:25 -0800

Hi Miika,

Sorry for the latency in responding.  Partly this is because I was
thinking a bit about some of the larger issues with this draft:
http://www.iki.fi/miika/docs/draft-ietf-hip-native-api-04-pre3.txt

> (i.e. by adding some flags). 

This depends perhaps on the perception of what complicates things for
the developer.  I do not agree that trying to make the native HIP API
deviate as little as possible from the address-based API should be the
primary goal here.  Rather, the main goal should be to have an API with
clear semantics, which is somewhat in conflict with aligning with the
address-based API because functions and data structures get overloaded.
The sockets API is already roundly criticized for its semantic
ambiguity; I think we should not make things worse if we can help it.

Here are two main issues I have with the current draft:

- it presumes that DNS is the entry point for applications to use HIP.
Personally, I suspect that HIP-based applications will more often than
not use something else like a DHT, or other things we haven't thought of
yet.  But the main issue is that this draft delegates handling of the
HIT/IP address bindings to an intermediate layer; i.e., it is kind of a
higher-level API.  This might be a nice eventual goal in some cases but
I think that introducing the abstraction immediately (that applications
never need to handle locators) might be taking too big of an initial
step.  

- there is a mixture of IP addresses and HITs in some cases
(HIP_ADDR_ANY); a given data structure may hold an IP address or a HIT
depending on the context.  I understand the appeal for doing it this way
because it involves the least change to the current sockets API, but I
think it will lead to semantic difficulties later.  I would rather that
we consider the design that addresses are in address data structures and
HITs are in HIT data structures and not to mix these.

The current draft covers a lot of ground; not only is the basic API
(partially) covered but also resolver interactions and now (in the
latest version) source HIT selection policy.  My preference would be to
focus first on the basic raw API and then move to the other points.  I
would like to suggest the following steps.

1) try to write a draft that has no DNS/DHT dependency whatsoever.  The
starting assumption is that the application holds zero or more HITs and
zero or more locators for the host it is trying to communicate with, and
possibly additional information such as rendezvous server information.
In other words, the draft picks up at the point after any DNS or DHT
resolutions have occurred.

2) write down the various use cases of interest, and how the system
should respond from an application's perspective for each of them.  For
instance, what if the destination HIT field is wildcarded but there are
locators (opportunistic mode).  What if both destination HIT fields and
address fields are wildcarded (illegal).  What should the system do if
the locators field is empty and it doesn't have the binding?  For
instance, what error should be conveyed to the application if a HIT but
no locators are provided, and the system cannot resolve the locator?
How does this differ from the case when the locator is resolved but the
host at that address denies ownership of that HIT?  Note that the case
where the intelligent resolver caches address/locator bindings (i.e.,
the current draft) can be a particular use case elaborated.

3) once the WG agrees on the use cases and the policies that are
implicit therein, then adopt/adapt the existing sockets API as needed.

I would be willing to help work on such revisions, but I'd first like to
hear from the rest of the WG whether they agree with my concerns or
whether the current draft direction is fine with everyone else and I
should just stand down.  

Tom

---

From: Miika Komu <[email-masked]>
Date: Tue, 29 Jan 2008 11:03:18 +0200 (EET)

On Mon, 28 Jan 2008, Henderson, Thomas R wrote:

Hi Tom,

> http://www.iki.fi/miika/docs/draft-ietf-hip-native-api-04-pre3.txt

no problem and thanks for the comments! Some comments below and the 
next version based on your technical corrections is here:

http://www.iki.fi/miika/docs/draft-ietf-hip-native-api-04-pre4.txt

Tom, please send further comments and technical feedback to the list ASAP
so that I can post the draft to the official IETF archives.

>> Why complicate things unnecessarily for the developer? The current API 
>> is designed so that it can be used in existing apps with minimal 
>> changes (i.e. by adding some flags).
>

Have you thought about what the application would look like when it is 
going to do two resolver calls and can initiate both HIP and non-HIP 
connections?

> ambiguity; I think we should not make things worse if we can help it.

Can you be more specific what parts of the sockets API have received 
criticism? AFAIK, the main part I've heard of in the apps area is the 
resolver part which has not met its promises in the real world.
Also, it was not originally designed with multihoming in mind.

I believe you are suggesting here effectively an additional deployment 
obstacle. Even my futile attempt to introduce an additional abstraction 
("end-point descriptor") to the HIP sockets API extensions (in draft 02 
version) was shot down by the research, apps area and hip community. I 
have some serious doubts about the future of such an approach.

I think we need to follow the narrow path of sockets API to its end for 
two reasons. First, to facilititate deployment of HIP-enabled applications 
by introducing minimal changes to the current applications. Second, to get 
experimental-driven feedback on where to improve when designing better 
APIs for the future. Either from the scratch, or as 
python/java/c-library/etc middleware on top of the sockets API (apps-area 
folks think that the latter will be the reality). This is getting a 
little-bit off topic...

> never need to handle locators) might be taking too big of an initial

I disagree about the DNS presumptions partially. There is one section, 
4.5. Manual Handling of Locators, dedicated completely to non-lookup 
environments. I agree that the draft has more focus on DNS, but I believe 
think this is acceptable since most of the current applications rely on 
DNS.

Also, now that the draft is more compatible with RFC5014, there is more 
space for e.g. OpenDHT-related resolution flags.

> HITs are in HIT data structures and not to mix these.

I don't like this either, but HIP folks wanted to see HITs in the data 
structures instead of endpoint descriptors. Notice how the HIP_ADDR_ANY 
resembles that concept?

Anyway, I requested comments earlier on this list whether HIP folks want 
to see just HITs in sockaddr_hip structure, or both HITs and locators. The 
feedback was that people wanted only HITs. I think this was a reasonable 
call for four reasons. First, the locators in sockaddr_hip structure can 
get depracated quite soon in mobile environments and it is easier for the 
application (developer) to misuse them. Second, sockaddr structures have 
size limits and can hold only a number of locators. Third, when the 
resolver resolves multiple HITs, it does not make sense to have the 
locators spread redundantly in multiple socket address structures. Fourth, 
I think the current version of the draft is more compatible with the 
multihoming draft this way.

This design choice had a consequence on the opportunistic mode. Namely, I 
had to introduce the HIP_ADDR_ANY concept because there is no locator 
structure in sockaddr_hip. I don't like it either, but that is rough 
consensus ;)

> would like to suggest the following steps.

I was working on the assumption that you were concurring with Andrew on 
the source HIT selection policy because you repeated his request on your 
comments:

http://www1.ietf.org/mail-archive/web/hipsec/current/msg02323.html

Hence, I included the source HIT selection policy to the API.

> resolutions have occurred.

See Section 4.5. Manual Handling of Locators. Is the name of the section 
somehow misleading?

I have already given my statement that the defails of locator handling is 
specified in multihoming API. I believe that I have added enough 
references from the native API draft to the multihoming draft. Further, it 
does not make sense to me to specify the same thing multiple times.

> locators (opportunistic mode).
>
> What if both destination HIT fields and address fields are wildcarded 
> (illegal).

Right, should result in EDESTADDRREQ.

> What should the system do if the locators field is empty and it doesn't 
> have the binding?  For instance, what error should be conveyed to the 
> application if a HIT but no locators are provided, and the system cannot 
> resolve the locator?

Agree that this should be added to the draft and should result in 
EDESTADDRREQ error.

> host at that address denies ownership of that HIT?

Section 4.5: The function returns EINVALIDLOCATOR when the HIT is not 
reachable at the specified locator.

> Note that the case where the intelligent resolver caches address/locator 
> bindings (i.e., the current draft) can be a particular use case 
> elaborated.

I belive this is addressed in a minimal way now in section 4.5.

> whether the current draft direction is fine with everyone else and I

Tom, it seems like you are suggesting a complete rewrite of both native 
and multihoming drafts. Lately, we have spend quite many cycles on 
polishing the two drafts and a complete rewrite would result in a further 
slip in the chartered schedule.

It seems like you are mostly disagreeing about the structuring of 
draft(s), but there is more than one way to write about things. I think 
the current draft is split into sections and paragraphs in a reasonably 
understandable way even though I admit that there is some historical 
burden and I still have room for improvement as a writer.

There seems to be some disagreeing also in the area of the technical 
approach. I have proposed three different design alternatives and the 
current approach is based on the consensus on hip-wg. If you want to 
propose a clean slate approach for the sockets API, the proper place to do 
it is probably hipsec-rg, apps-area or some research conferences.

---

From: "Henderson, Thomas R" <[email-masked]>
Date: Tue, 29 Jan 2008 09:31:21 -0800

Miika,

It would probably be more productive if I just propose some specific
text changes at this point.  I understand your frustration with my
complaints, but IMO it is not critical that we get this API draft
finalized quickly (the NAT traversal draft seems to me to be much higher
priority).  I appreciate that you have been very responsive and
proactive to handling suggestions through this process.   

Regards,
Tom

---

From: "Henderson, Thomas R" <[email-masked]>
Date: Thu, 31 Jan 2008 23:20:49 -0800

Miika,
I owe you some more specific text proposals, but I thought I'd respond
to some points inline below. 

> From: Miika Komu [mailto:[email-masked]] 
> Sent: Tuesday, January 29, 2008 1:03 AM
> To: Henderson, Thomas R
> Cc: HIP; Gonzalo Camarillo
> Subject: RE: [Hipsec] WGLC: draft-ietf-hip-native-api-03.txt
> 

I thought we were going down that path already with the DNS draft (see
section 3.2).  I think that there may be cases where the application
gathers the information it wants through DNS or whatever means directly,
and then uses a low-level API, and cases where the application just
wants to use HITs alone and not have to worry about RVS and addresses.
IMO, we should strive for an API that allows either case, but to me, it
is important to get the first one done first.

> Also, it was not originally designed with multihoming in mind.

Plus, the mixture of stream and datagram semantics.

> folks think that the latter will be the reality). This is getting a 

Well, if we announce "this is the native HIP API" and then change it a
few years later, apps developers will not be happy.  That is why I'd
prefer to have the raw low-level API done first (which shouldn't have to
change in the future) and then start to engineer abstractions and
performance improvements-- those should be more experimental.

> think this is acceptable since most of the current 

As I commented previously, it requires reading/using the shim6 draft to
really implement the section 4.5, and here it was ambiguous (at least to
me) how these options might apply to HIP.  

For instance:

"5.6.  SHIM_LOC_PEER_PREF

   The SHIM_LOC_PEER_PREF option can be used to read or set preferred
   locator on peer side within a given context.  Hence this option is
   effective only when there is a shim context associated with the
   socket."

Does that apply to HIP?  When is there a HIP "shim context" present?  I
don't think the use of these options is as clear as it could be,
especially to people who do not know HIP and shim6.

> feedback was that people wanted only HITs. 

Yes, I am suggesting that HITs be in sockaddr_hip structures, and IP
addresses be elsewhere.

>I think this was a 
> reasonable 

The option is to always allow for a (possibly wildcarded) locator
structure.

> Hence, I included the source HIT selection policy to the API.

Yes, I am just mentioning that you have recently added the equivalent of
RFC 3484 and 5014, which makes it cover a lot of ground now.

I have not carefully thought about the choices in Table 3, although it
occurred to me that key (HI) length is not a criteria in the selection
(should it be?). 

> Agree that this should be added to the draft and should result in 

Yes, this is the point that I am trying to make-- we should try to
clarify these types of things now.

- Tom

_______________________________________________
Hipsec mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/hipsec