Subject: [core] proposal: header options may appear at most once

From: Peter Bigot <[email-masked]>
Date: Wed, 21 Jul 2010 13:59:39 -0500

I don't see any discussion on whether it is permitted to repeat a header
option with additional values (i.e., have option delta be zero).  I propose
that this be disallowed.

In practice, I want this because I'm working on a Python developer API for
CoAP, and I need a way to search a message instance for a particular
option.  It's easy to return either a null value (if the option is absent),
or the option value (if it is present).  If a given option could appear
multiple times, I really need the generic get_option operation to return a
sequence, which will in virtually all cases be either empty or contain a
single element.  Having to always dereference the sequence is an unpleasant
user experience.  But since unhappy Python users aren't really an IETF
concern, I'll back up my desire with the following:

   - None of the currently defined header options are meaningful if
   repeated.  The proposed Accepts option is, but multiple values can easily be
   encoded in one TLV element by providing a sequence of codes.4

   - An HTTP header field-name can appear multiple times only if the field
   content admits a sequence, and the header content could have been
   represented by a single field with the catenated content.

Thus, there is reason to believe that it is unnecessary to support
encoding/decoding multiple instances of any option type.

Peter

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 21 Jul 2010 21:17:36 +0200

On Jul 21, 2010, at 20:59, Peter Bigot wrote:

> I don't see any discussion on whether it is permitted to repeat a header option with additional values (i.e., have option delta be zero).  I propose that this be disallowed.

If we wanted to disallow that, we would encode the delta minus one, not the delta.
(Principle of making it impossible to speak what you don't want to have said.)

Indeed, the idea with encoding just the delta was to enable having multiple options of the same type (cf. "repeated" in protobuf, http://code.google.com/apis/protocolbuffers/docs/proto.html).

> In practice, I want this because I'm working on a Python developer API for CoAP, and I need a way to search a message instance for a particular option.  It's easy to return either a null value (if the option is absent), or the option value (if it is present).  If a given option could appear multiple times, I really need the generic get_option operation to return a sequence, which will in virtually all cases be either empty or contain a single element.  Having to always dereference the sequence is an unpleasant user experience.  But since unhappy Python users aren't really an IETF concern, I'll back up my desire with the following:
> 	• None of the currently defined header options are meaningful if repeated.  The proposed Accepts option is, but multiple values can easily be encoded in one TLV element by providing a sequence of codes.4

I think there was a suggestion using Content-Type repeatedly instead of a single Accept option.  But we haven't decided that yet.

> 	• An HTTP header field-name can appear multiple times only if the field content admits a sequence, and the header content could have been represented by a single field with the catenated content.

HTTP has commas there in its syntax.
We don't want to have the same kind of commas, I strongly believe.

> Thus, there is reason to believe that it is unnecessary to support encoding/decoding multiple instances of any option type.

We don't know that yet.
For instance, if we go for that Uri-Query idea, we will have to allow multiple instances.

BTW, the obvious solution in your Python interface is to offer an Array exactly when there are multiple instances of an option and the single instance otherwise.  (This is what I'll do in the Ruby version, if I ever get any more time to work on that.)

Gruesse, Carsten

---

From: Peter Bigot <[email-masked]>
Date: Wed, 21 Jul 2010 15:15:43 -0500

> For instance, if we go for that Uri-Query idea, we will have to allow

FWIW, I would strongly object to a repeated Uri-Query option.  A repeated
Query-Param option makes sense, but a URI has at most one query part, and it
does not necessarily encode a sequence of key/value pairs.

>       • An HTTP header field-name can appear multiple times only if the
> field content admits a sequence, and the header content could have been

Sure, not as a general rule, but what's wrong with defining it for specific
options?  Certain options have meaning when repeated, and it makes sense to
allow for that in their representation.  For options that make no sense if
repeated, the representation should disallow repetition.  As you said:
prevent the expression of nonsense.  Everything's good.

I see no reason why Query-Param could not itself encode its multiple values
as a sequence of keys with optional values, with the number of such blocks
determined implicitly or explicitly.  In fact, to define it as a sequence of
key/optional value blocks separated by ASCII ampersands would make it
trivial to encode from a URI, while leaving a higher-level API free to split
it into its components for the application developers' convenience without
affecting its interpretation.

Disallowing option repetition also relieves some of the pressure on the size
of the num_options field.

BTW, the obvious solution in your Python interface is to offer an Array
> exactly when there are multiple instances of an option and the single
> instance otherwise.  (This is what I'll do in the Ruby version, if I ever
> get any more time to work on that.)
>

As a developer my reaction is "No", but then I admit to a dislike for
duck-typing.  I don't want my code splattered with:

   ct = message.findOption(ContentType)
   if ct is None:
     # process as text/plain
     ContentType.Default.process(message)
   elif isinstance(ct, ContentType):
     # process as ct.value
     rv.process(message)
   else:
     # what the heck does this mean?
     for sub_ct in ct:
       sub_ct.process(message)

Especially since I'd then have to introduce error-handling at every
point-of-call for cases like Content-Type where it makes no sense for the
option to be repeated.  I can't have my server throw an uncaught exception
attempting to invoke a method on a list just because somebody sent me a
packet with decodable garbage and I didn't bother to check for the
situation; nor do I care for top-level "Something bad happened
somewhere"-style recovery.

But that's getting towards religion and is off-topic, except to the extent
that the same problem will occur in C where run-time type checking is not an
option and I will be returning either a null pointer or a pointer to a
single option value (which may have multiple elements, if appropriate).

Peter

---

From: Brian Frank <[email-masked]>
Date: Wed, 21 Jul 2010 17:40:53 -0400

I think option headers should only appear once.  So I like Carstens
suggestion that the delta is encoded minus one (so that you can never have
zero).

Multiple options seems really confusing from a semantic perspective
(especially given our current list of options such as Uri-Path, etc)

On Wed, Jul 21, 2010 at 4:15 PM, Peter Bigot <[email-masked]> wrote:

> On Wed, Jul 21, 2010 at 2:17 PM, Carsten Bormann <[email-masked]>wrote:
>
>> We don't know that yet.
> Disallowing option repetition also relieves some of the pressure on the
> size of the num_options field.
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>
>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 22 Jul 2010 00:14:42 +0200

> code splattered with:
>  [...] error-handling at every point-of-call

Well, this is not the list to discuss coding practices, but if you don't want that, then don't do that.
Do it once in the option parser then.

In the C code for a simple device, the processing is likely to happen inline instead through such an abstraction, so you know exactly what to do (e.g., put a mask bit into a byte for each of the very few content-types you actually support, and then match that against the ones available for the resource that you want to deliver).

But none of this discussion has much bearing on whether it is good protocol practice to allow multiple-instance options.

Not having them means that each time we need them we need to invent some array notation inside the option instead.
For Accept, that kind of worked once we made content-type one-size-fits-all, so a fixed-size (one-byte) array works fine.
(Unless we throw out Accept and replace it with Content-Type.)
It didn't look so nice when we still had multi-byte content-types.

So let's look at another example: what about Etags in requests?
These are variable-length, and we are proposing you can send multiple to allow a 304 response for any of them.
(Of course, you can now argue you don't like that proposal, but that would look like the encoding should determine what is possible semantically.)

Gruesse, Carsten

---

From: Peter Bigot <[email-masked]>
Date: Thu, 22 Jul 2010 07:01:57 -0500

No, Etags is the first example where the same option appearing multiple
times is clearly preferable to one option appearance that encodes multiple
values.  I can't assess whether the motivating use-case is valid, and leave
that to those who are concerned with caching CoAP.

Defining a mechanism for multiple values that is option-independent might
ultimately be a better approach, but for now I withdraw the proposal.

I request the following compensating clarifications in draft -02:

   - Each option definition should indicate whether the minimum number of
   occurrences is zero or one (optional or required);
   - Each option definition should indicate whether the maximum number of
   occurrences is one or unbounded;
   - Message recipients are not obligated to diagnose violation of
   multiplicity restrictions for any option (meaning, for example, that they
   can do whatever they want if they receive multiple Uri-Path options in a PUT
   or GET message).

(Motivation for that last one: unless a node is obligated to recognize an
option even if it doesn't support it, a requirement we don't currently have,
it can't diagnose multiplicity violations.  I sure hope this doesn't mean
people will start to retrieve multiple documents with a single request by
including multiple URIs....)

Peter

On Wed, Jul 21, 2010 at 5:14 PM, Carsten Bormann <[email-masked]> wrote:

> ...
But none of this discussion has much bearing on whether it is good protocol
> practice to allow multiple-instance options.
>
> Not having them means that each time we need them we need to invent some
> array notation inside the option instead.
> For Accept, that kind of worked once we made content-type
> one-size-fits-all, so a fixed-size (one-byte) array works fine.
> (Of course, you can now argue you don't like that proposal, but that would
> look like the encoding should determine what is possible semantically.)
>
> Gruesse, Carsten
>
>

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Thu, 22 Jul 2010 14:21:53 +0200

On Thu, Jul 22, 2010 at 14:01, Peter Bigot <[email-masked]> wrote:
> multiple Uri-Path options in a PUT or GET

In my opinion this is clearly a very useful use-case for multiple options...

GET /sensor/temperature /sensor/humidity /system/threshold etc. etc.

we can GET multiple resources with a single message..

Angelo

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 22 Jul 2010 17:05:43 +0200

> we can GET multiple resources with a single message..

I'd rather do this with payload-length (coap-misc-05) and multiple messages in one packet.
There is too much infrastructure that would be needed to carry multiple GETs in one message.

Gruesse, Carsten

---

From: Peter Bigot <[email-masked]>
Date: Thu, 22 Jul 2010 10:28:48 -0500

Agreed.  For the specific case, I'd simple GET /sensor and have the
implementation bundle its subcomponents appropriately.  The thought of
returning the values for /.well-known/r, /sensor, and /firmware.img in one
response body makes me shudder.

Peter

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 22 Jul 2010 18:59:25 +0200

> 	• Each option definition should indicate whether the minimum number of occurrences is zero or one (optional or required);

Sounds good. Actually, the answer would always be "optional", as we don't have any required options.
(I.e., a 4-byte CoAP message with oc=0 and meaningful ttype/code is a good message.)

> 	• Each option definition should indicate whether the maximum number of occurrences is one or unbounded;

An option definition must definitely say what it actually means to have multiple occurrences, if that is to be allowed.
(And the summary table could indeed have a column that indicates whether the option is "repeatable".)

> 	• Message recipients are not obligated to diagnose violation of multiplicity restrictions for any option (meaning, for example, that they can do whatever they want if they receive multiple Uri-Path options in a PUT or GET message).

"Be liberal in what you accept".  (But this is also the road to tag-soup hell.)
Still, I agree there should be no requirement on a node to detect and reject all violations of message syntax.
(My demo implementation certainly doesn't.)
But an implementation also shouldn't go out of its way to process invalid messages.

> (Motivation for that last one: unless a node is obligated to recognize an option even if it doesn't support it,

Elective options can always be safely ignored by definition.
Requiring to detect violations in an option you don't understand would indeed be absurd.

Gruesse, Carsten

---

From: Zach Shelby <[email-masked]>
Date: Mon, 26 Jul 2010 23:02:37 +0200

+1.

On Jul 21, 2010, at 11:40 PM, Brian Frank wrote:

> I think option headers should only appear once.  So I like Carstens suggestion that the delta is encoded minus one (so that you can never have zero).
> 
> Multiple options seems really confusing from a semantic perspective (especially given our current list of options such as Uri-Path, etc)
> 
> FWIW, I would strongly object to a repeated Uri-Query option.  A repeated Query-Param option makes sense, but a URI has at most one query part, and it does not necessarily encode a sequence of key/value pairs.
> 
> 
> Sure, not as a general rule, but what's wrong with defining it for specific options?  Certain options have meaning when repeated, and it makes sense to allow for that in their representation.  For options that make no sense if repeated, the representation should disallow repetition.  As you said: prevent the expression of nonsense.  Everything's good.
> 
> I see no reason why Query-Param could not itself encode its multiple values as a sequence of keys with optional values, with the number of such blocks determined implicitly or explicitly.  In fact, to define it as a sequence of key/optional value blocks separated by ASCII ampersands would make it trivial to encode from a URI, while leaving a higher-level API free to split it into its components for the application developers' convenience without affecting its interpretation.
> 
> Disallowing option repetition also relieves some of the pressure on the size of the num_options field.
> 
> As a developer my reaction is "No", but then I admit to a dislike for duck-typing.  I don't want my code splattered with:
> 
> Especially since I'd then have to introduce error-handling at every point-of-call for cases like Content-Type where it makes no sense for the option to be repeated.  I can't have my server throw an uncaught exception attempting to invoke a method on a list just because somebody sent me a packet with decodable garbage and I didn't bother to check for the situation; nor do I care for top-level "Something bad happened somewhere"-style recovery.
> 
> But that's getting towards religion and is off-topic, except to the extent that the same problem will occur in C where run-time type checking is not an option and I will be returning either a null pointer or a pointer to a single option value (which may have multiple elements, if appropriate).
> 
> Peter
> 

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: Zach Shelby <[email-masked]>
Date: Mon, 26 Jul 2010 23:13:33 +0200

> (And the summary table could indeed have a column that indicates whether the option is "repeatable".)

Yep, we should do this.

> But an implementation also shouldn't go out of its way to process invalid messages.

Agreed (except for the multiple Uri-Path options ides....).

---

From: Peter Bigot <[email-masked]>
Date: Tue, 27 Jul 2010 07:48:58 -0500

> look like the encoding should determine what is possible semantically.)

I admitted the value of multiple options for Etags but said I couldn't
assess whether the motivating use-case for multiple Etags was valid. If it's
the situation in section 3 of hartke-coap-observe I call "bogus".

For that to be useful the following conditions have to hold:

   - The use case requires that a message be sent anyway; the Etag only
   permits transfer of the etag instead of the whole state.  Therefore, the
   size of a state representation has to be sufficiently larger than the etag
   value that it's worth conveying the state as an etag.
   - The server must consistently use the same etag for the same
   representation.  Rules out one-up counters, the obvious etag on a
   constrained environment.
   - The set of expected representations has to be so small that it's
   reasonable to cache them all.  In this case, why wouldn't the representation
   be a label for the state, hence small?
   - All clients would need to use the Etags caching; any client that
   doesn't declare itself in possession of the cached value would still need to
   be sent the full state representation

Remembering that we're talking about constrained devices, this all seems
extremely unlikely.

Unless a compelling use case can be found, I re-assert my support for
options appearing at most once, with those few options for which multiple
values make sense represented either in an option-specific encoding (as with
the proposed Accepts) or by developing a generic LV or separator-based
representation.

Peter

---

From: Zach Shelby <[email-masked]>
Date: Tue, 27 Jul 2010 15:11:05 +0200

On Jul 27, 2010, at 2:48 PM, Peter Bigot wrote:

> Unless a compelling use case can be found, I re-assert my support for options appearing at most once, with those few options for which multiple values make sense represented either in an option-specific encoding (as with the proposed Accepts) or by developing a generic LV or separator-based representation.

I would also like to see us take this path.

Zach