Subject: [core] Options: Uri-Query and Uri-Scheme

From: Peter Bigot <[email-masked]>
Date: Wed, 20 Oct 2010 11:06:12 -0500

Two option issues:  I think I suggested long ago that scheme be removed
because it was always coap, but I meant to be speaking of its use in
resource discovery.  For CoAP as a whole, I think it needs to be put back:
possibly in support of the potential for using a different scheme for
multicast CoAP as discussed on the last working group telecon, but certainly
because without it you can't redirect responses to a resource in a different
scheme (such as HTTP).

Also, we need a separate option for Uri-Query, per previous discussion on
this list.  Currently the only place for the query-part is within the
Uri-Path option, which is not consistent with URI syntax.  Absence indicates
no query-part in the URI; presence provides the text following the hook in
the reconstructed URI.

Related to that, the text in 2.5.2 has:

   CoAP splits the URI up into its three parts with the default coap://
   scheme, Uri-Authority and Uri-Path Options.  The full URI can be
   created by concatenating those parts (or their defaults if not
   present).

This is inaccurate.  Reconstruction of a full URI is done relative to a CoAP
message header, and is done by catenating:

   Uri-Scheme
   "://"
   Uri-Authority
   "/"
   Uri-Path
   ( "?"  Uri-Query ) if Uri-Query is present

Where reference to an option is replaced by the value of that option if the
option is in the header, and the default value for the option if it is not
present.  The default value for Uri-Scheme is "coap".  The default value for
Uri-Authority and Uri-Path is the empty string.  Uri-Query has no default
value.

Not sure how to format that in the text.

The same reconstruction is required for getting the full URI for the target
of a redirection response, except that the Location option is used instead
of the Uri-Path option.

Peter

---

From: Peter Bigot <[email-masked]>
Date: Wed, 20 Oct 2010 11:19:51 -0500

Need to correct this; reconstructing the URI should be:

   ( Uri-Scheme ":" )
   ( "//" Uri-Authority ) only if Uri-Authority is present
   ( "/" Uri-Path )
   ( "?"  Uri-Query ) only if Uri-Query is present

Note that this implicitly defines the syntax of strings valid for these
options, to wit:

Uri-Scheme conforms to the scheme rule in section 3.1 of RFC3986
Uri-Authority conforms to "host [ : port ]" (section 3.2 of RFC3986,
excluding use of userinfo)
Uri-Path confirms to the path-noscheme (path-rootless?) rule in section 3.3
of RFC3986
Uri-Query conforms to the query rule in section 3.4 of RFC3986

Delegation to existing definitions is the way to go.  This eliminates the
need for similar rules in section 5.3, which otherwise implicitly restrict
the Uri-Authority syntax.

There had been talk of an option for a binary encoding of Uri-Authority; I
assume that would be a new, critical option, the presence of which would
complicate this reconstruction a little.

Peter

---

From: Kerry Lynn <[email-masked]>
Date: Wed, 20 Oct 2010 13:56:56 -0400

Hi Peter,

In light of your clarifications, I wonder if we can discuss some of the
comments in your earlier email?

> URI references... may have been a source of confusion within the
> working group, including leading to some mistaken assumptions
> about how URIs with multicast addresses might work.

First, we're agreed that we're really talking about URLs, right?
"As this URI is used purely as a locator, CoAP only supports Universal
 Resource Locator features of [RFC3986] although throughout the
 document we refer to URI."

It believe there was a typo in the coap draft, as you pointed out.
However, it seems that in the later descriptions relative URIs
were everywhere disallowed by language such as "CoAP does not
support "." or ".." in URIs" and "The leading slash is assumed and
and MUST be omitted".

Jumping around a bit, you said:
> the scheme and any query and fragment parts do not contribute to
> this identification

I think from the perspective of a coap server this MIGHT be correct,
but I'm not positive.  The reason I'm agnostic is because I'm not
sure yet how the coap-to-http mapping will work.

>From outside the server, for location purposes, I don't agree.  In the
-bc draft, Peter van der Stok and I said that the scheme and authority
parts could be used by e.g. DNS-SD to locate instances of the coap
service.  IOW, you expect that a query for "http://foo.example.com"
would give a different result than "coap://foo.example.com" in terms
of returned port and other answers.

Before returning to group resource locators, I'd like to vet an idea
related to multi-function devices.  I think there's been an implicit
assumption that "functionality" (e.g. an object model) may be located
at any interior node in a URI hierarchy, and may be optionally named
by a short URI.  This is the thrust of draft-ietf-core-link-format.  But
modern HTTP servers also implement the concept of name-based
virtual hosts, in which multiple authorities can be mapped to the
same IP address.  Thus, http://foo.example.com and http://bar...
might be located at the same IP & port.  It seems to me that this
would be a natural way to express multi-function devices (as a 1:1
mapping of "device" to DNS host name) as well as lead to shorter
URIs.  I think this might be especially useful when devising
coap-to-foo legacy gateways.  The implication is that the Uri-
Authority Option would be used with some regularity in this case.

The idea of authorities mapping to muticast addresses was
introduced in -bc-01 and referenced in -groupcomm-00.  In the
latter, we say that a request directed to a group of nodes MUST
be directed to a /.well-known/core/... Uri-Path.  I was thinking
that a name-based approach would apply here, since the server
receiving such a request (and belonging to the IP group) could
be considered as a server for the group authority as well as for
any individual authorities it might serve.  However, it seems
the name-based approach might also create a problem for
group resources in the case of multiple vhosts per interface,
in that the /.well-known/core/... Uri-Path received in the request
must be resolved to an individual Uri-Authority at that interface
as well as a Uri-Path, and these would be returned in any
response.  How is this handled in the case where the same
absolute path is found in more than one virtual host?

I'd be very interested to hear other opinions about how group
resource naming might work.

Regards, -K-

On Wed, Oct 20, 2010 at 12:19 PM, Peter Bigot <[email-masked]> wrote:
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>
>

---

From: Peter Bigot <[email-masked]>
Date: Wed, 20 Oct 2010 14:09:41 -0500

> comments in your earlier email?

Certainly.

>  document we refer to URI."

At the moment, yes.  However, I can think of no technical justification for
this restriction.  If an IP address and port for the owner of a resource
identified by a URN could be found elsewhere (e.g., hard-coding or LDAP or
DNS-SD), and CoAP added a way to pass that URN within the request message (a
Uri-Name option), URNs would work too.

As a general rule, my understanding is that IETF, per text in RFC3305,
prefers use of the term URI, because the cases where URL and URN must be
distinguished are so rare.  In fact, I don't think CoAP is one of those
cases.  CoAP does not mandate how the location of a resource (in the sense
of an IP address and port) is determined.  Nor does it strictly require that
the path part be interpreted hierarchically.  I think the text "As this URI
is used purely as a locator" should be removed.

> of returned port and other answers.

Yes.  Actually, this relates to a question you raised on both CoAP WG
telecons: whether http://server/path denotes the same resource as
https://server/path.  Everybody who spoke up, including me, said "no",
because from a web-server perspective the two reach completely different
implementations.

I think from the REST perspective the answer is "yes".  That you can
legitimately get different representations of the resource from the
different servers is because of metadata provided as a consequence of going
through the different protocol, but RESTfully whatever you get really should
be representations of the same resource.  Perhaps one has more detail than
the other, but if they are truly unrelated, it is a poorly-designed system.

I stand by my assertion that, in REST, the scheme is irrelevant to the
resource identification.  If I have to, I'll go dig up the reference from
Fielding that supports that position.

However, I don't think it matters, because what DNS-SD is providing is not
the address of the REST resource but the address to a server that can be
used to manipulate that resource using a specific protocol.  All is good.

> Authority Option would be used with some regularity in this case.

Yes: this is exactly why I thought it so important to ensure Uri-Authority
was preserved end-to-end by CoAP-carried REST transactions.

> any individual authorities it might serve.

Yes, the problem being that all receiving hosts would believe they were the
(single) server responsible for that resource.  Separate issue, ignore it
for now.

> However, it seems

In HTTP, I don't think you'd even notice.  Each host has its own directory
namespace.  The authority (Host header) is not optional (though it can be
empty), so only one directory structure would be examined for a match to the
path part.  It would be there, or it wouldn't.  There would never be an
ambiguity.

Same thing in CoAP: if Uri-Authority is present, it's used to select which
of the directory namespaces hosted via CoAP on that server and port should
be used to resolve the path part.  If it's absent, there's got to be exactly
one default namespace that will be used.

> resource naming might work.

In my glance over the rahman draft I saw some potential approaches; I hope
to have comments on it sometime tomorrow.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 20 Oct 2010 22:01:13 +0200

A couple of random thoughts about this area:

I definitely agree that referring to the old ever so confusing "locator vs. name" discussion that is alluded to in section 1.1.3 of RFC 3986 does not really add value to the CoAP spec.  Let's talk about URIs only, and where we want to restrict them, about the specific restrictions we are making.

As with HTTP, we don't support the relative references "." and "..".
I think that is because we try to encode a URI, not a URI reference, i.e. section 5 of RFC3986 is performed at the client (if it needs to be performed at all), not at the server.

One discussion we have had is whether the authority should be sent when there isn't an expectation that the server is acting as a proxy.  The best choice may be different in HTTP because this is a constrained environment and authorities tend to be large, so there is more incentive to avoid them than there was in HTTP.

The Uri-Authority-Binary option proposed in section 3.2 of draft-bormann-coap-misc-06.txt mainly tries to make the case less heavyweight where the authority is already resolved to a transport address ([2001:...]:4711 is rather verbose).  Do we need that support?  I don't know.

We haven't defined yet how to translate the authority in a coap URI into the transport address used to contact the server.  There seems to be a common assumption between all current implementations that this goes through AAAA (and possibly A) records (and no SRV records) in the case of DNS names.  Virtual hosts only make sense if you have an indirection like DNS where several names can point to the same thing.

I completely agree that
	foo://he.re/it/is
is a different resource from
	bar://he.re/it/is
independent of whether foo/bar are http, https, or coap.

As with http and https, there *may* be an *intention* that resources that only differ in scheme may solve a similar problem (e.g., a http:// resource often just redirects to a related https:// resource and thus is completely different from that resource).

However, http://he.re:80/it/is may be completely different from http://he.re:81/it/is.  So which of these should coap://he.re:61616/it/is be similar to? coap://he.re:61617/it/is?
As a general idea, URIs are opaque identifiers; there is only very limited license a server gives to a client for deriving one URI from another one (e.g., by adding query parameters).

If we want to support something like coap://[ff02::1]/ (multicast authorities), we'll have to invent something -- so far REST doesn't have it.  The semantics may be moderately obvious for a PUT (but not for its response), and much less obvious for a GET.
A proxy translating a GET response from multicast back to unicast would have to combine all resource representations, probably indexed by something like a unicast URI somehow constructed from each source of a response (you want to know who sent the response and how to talk to that server).  Some kind of multipart/... media type, maybe.  Do we need to support that?  (I.e., what is the use case?)

Gruesse, Carsten

---

From: Peter Bigot <[email-masked]>
Date: Wed, 20 Oct 2010 15:49:24 -0500

Re whether scheme affects resource identification, see the first bullet
under:

http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

The rest of his post helps explain why I believe it is a mistake to believe
that CoAP, which is designed for M2M communications between constrained
devices, should be expected to be 100% compatible with REST, which was
designed for distributed management of hypermedia on the Internet.

Peter

On Wed, Oct 20, 2010 at 3:01 PM, Carsten Bormann <[email-masked]> wrote:

> I definitely agree that referring to the old ever so confusing "locator vs.
> name" discussion that is alluded to in section 1.1.3 of RFC 3986 does not
> really add value to the CoAP spec.  Let's talk about URIs only, and where we
> want to restrict them, about the specific restrictions we are making.
>
> I think that is because we try to encode a URI, not a URI reference, i.e.
> section 5 of RFC3986 is performed at the client (if it needs to be performed
> at all), not at the server.
>
> One discussion we have had is whether the authority should be sent when
> there isn't an expectation that the server is acting as a proxy.  The best
> choice may be different in HTTP because this is a constrained environment
> and authorities tend to be large, so there is more incentive to avoid them
> than there was in HTTP.
>
> The Uri-Authority-Binary option proposed in section 3.2 of
> draft-bormann-coap-misc-06.txt mainly tries to make the case less
> heavyweight where the authority is already resolved to a transport address
> ([2001:...]:4711 is rather verbose).  Do we need that support?  I don't
> know.
>
> We haven't defined yet how to translate the authority in a coap URI into
> the transport address used to contact the server.  There seems to be a
> common assumption between all current implementations that this goes through
> AAAA (and possibly A) records (and no SRV records) in the case of DNS names.
>  Virtual hosts only make sense if you have an indirection like DNS where
> several names can point to the same thing.
>
> I completely agree that
>        foo://he.re/it/is
> As with http and https, there *may* be an *intention* that resources that
> only differ in scheme may solve a similar problem (e.g., a http://resource often just redirects to a related https://resource and thus is completely different from that resource).
>
> However, http://he.re:80/it/is may be completely different from
> http://he.re:81/it/is.  So which of these should coap://he.re:61616/it/isbe similar to? coap://
> he.re:61617/it/is?
> As a general idea, URIs are opaque identifiers; there is only very limited
> license a server gives to a client for deriving one URI from another one
> (e.g., by adding query parameters).
>
> If we want to support something like coap://[ff02::1]/ (multicast
> authorities), we'll have to invent something -- so far REST doesn't have it.
>  The semantics may be moderately obvious for a PUT (but not for its
> response), and much less obvious for a GET.
> A proxy translating a GET response from multicast back to unicast would
> have to combine all resource representations, probably indexed by something
> like a unicast URI somehow constructed from each source of a response (you
> want to know who sent the response and how to talk to that server).  Some
> kind of multipart/... media type, maybe.  Do we need to support that?
>  (I.e., what is the use case?)
>
> Gruesse, Carsten
>
>