Subject: [Cbor] CDDL Prelude

From: Christophe Lohr <[email-masked]>
Date: Sun, 18 Jun 2017 13:42:31 +0200

Hello,
  I have questions about the CDDL prelude.

First, why not defining uint8/16/32/64 such as for floatXX:

uint8  = #0.24
uint16 = #0.25
uint32 = #0.26
uint64 = #0.27

nint8  = #1.24
nint16 = #1.25
nint32 = #1.26
nint64 = #1.27

Then, how to restrict data to (in)definite structures?
Can I write the following?

bstr_indefinite  = #2.31
bytes_indefinite = bstr_indefinite
tstr_indefinite = #3.31
text_indefinite = tstr_indefinite

bstr_definite8  = #2.24
bstr_definite16 = #2.25
bstr_definite32 = #2.26
bstr_definite64 = #2.27
bstr_definite  = bstr_definite8
		/ bstr_definite16
		/ bstr_definite32
		/ bstr_definite64
bytes_definite = bstr_definite

tstr_definite8  = #2.24
tstr_definite16 = #2.25
tstr_definite32 = #2.26
tstr_definite64 = #2.27
tstr_definite = tstr_definite8
		/ tstr_definite16
		/ tstr_definite32
		/ tstr_definite64
text_definite = tstr_definite

Best regards
Christophe

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 18 Jun 2017 14:00:54 +0200

> First, why not defining uint8/16/32/64 such as for floatXX:

Hi Christophe,

it is not clear to me what you think the semantics of uint64 would be.  Should this be a data model level restriction?  Or are you further trying to restrict the serialization?  Or, in other words, if the value of a uint64 is 42, would 0x182a be an allowable serialization, or are you really trying to specify that the serialization should be 0x1b000000000000002a?

CDDL actually is a data model level specification language.  As a matter of style, a data model specification should not restrict the serialization choices that an implementation may make.

(We haven’t really solved the issue for float16/32/64 yet, either.
I think this would be a good discussion to have.
Much of the need for this comes from the few implementers that don’t want to add the 10 lines of code needed to decode float16.  But there also is a genuine need to be specific about float32 vs. float64 support in a format — except that it isn’t clear that this is needed at the data model level.)

I could very well imagine that
	uint32 = uint .size 4
might be a rule that you might want to use in your specification.
Should that rule be in the prelude?

So, I summary, yes, this is a discussion that we should continue to a conclusion.

> Then, how to restrict data to (in)definite structures?

Similar to the above, it is not really the data model that should be making these serialization choices.
I think it might be useful to have a (simple) mechanism to control the small number of serialization choices that CBOR provides at a global level.  I don’t think we want to attach serialization controls to the individual types of CDDL, but I would be interested in learning why we should.

In that vein, I’m curious what kind of application you had in mind when you wrote your message.

Grüße, Carsten

---

From: Christophe Lohr <[email-masked]>
Date: Sun, 18 Jun 2017 22:28:28 +0200

> it is not clear to me what you think the semantics of uint64 would be.  Should this be a data model level restriction?  Or are you further trying to restrict the serialization?  Or, in other words,…

Yes, this is what I had in mind:  uint64 should accept
0x1b000000000000002a and reject 0x182a

Well, to be honest I have no concrete example where forcing the size of
a "uint" could be mandatory.
It's not a big deal to do a "cast" if I receive 8 bits where I would
expected 64.

However, I wonder if in some circumstances having a stable predictable
size of data would be more important than having the smallest possible
or of an indefinite size.
So, how to specify it?
Or maybe this is a bad question. A concern for dinosaurs of
computer-science. ;-)

> CDDL actually is a data model level specification language.  As a matter of style, a data model specification should not restrict the serialization choices that an implementation may make.

Hum. There is a data description model level, and a data presentation
model level. The concern of CDDL is the former. Does I understand well?

> Should that rule be in the prelude?

I thought that this means "up to 4 bytes".
I had wished to express "exactly 4".

> I think it might be useful to have a (simple) mechanism to control the small number of serialization choices that CBOR provides at a global level.  I don’t think we want to attach serialization con…

Let's consider a communication protocol based on CBOR messages that are
specified with CDDL.
The emitter is free to produce definite or indefinite bytes/text
strings.  Maybe, the choice is equal from its point of view.
However, from the receiver point of view, decoding indefinite bytes/text
strings introduces an extra cost: it needs an additional buffer to
assemble chunks. (Think about constrained nodes.)

So, in this example, this would be preferable if the protocol
specification has a mean to restrict things to the use of definite data
serialization.

Well, from the other side, if one wants to specify serialization
constraints, the CDDL syntax may need some extensions. For instance if I
want to specify a rule where the litteral "42" must be presented into a
uint64, I would need some syntax for "casting":
  answer = (#0.27)(42)  ???

> In that vein, I’m curious what kind of application you had in mind when you wrote your message.

Here are my first tests with cbor:
https://svn.telecom-bretagne.eu/svn-public/xAAL/code/C/branches/testing_cbor/

and a global presentation of the project:
http://recherche.telecom-bretagne.eu/xaal/

Best regards
Christophe

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 19 Jun 2017 13:08:29 +1200

> Or maybe this is a bad question. A concern for dinosaurs of

I think it depends on the programming language of the recipient,
doesn't it? Suppose a field is described in CDDL as
session-id = 0..4294967295 ;up to 32 bits
(a real example)
In Python, I just don't care; cbor.loads() just returns an integer and that's
the end of it. In C, I would presumably want to store the result in
a uint32_t, regardless of how the sender happens to encode it in CBOR.
According to the Postel principle I should accept any CBOR integer
encoding as long as the result is < 4294967296. There's no type
mismatch from a high level view, so the computer scientist in me
doesn't worry.

    Brian

> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor
>

---

From: Michael Richardson <[email-masked]>
Date: Mon, 19 Jun 2017 09:43:55 -0400

Christophe Lohr <[email-masked]> wrote:
    >> it is not clear to me what you think the semantics of uint64 would be.
    >Should this be a data model level restriction?  Or are you further
    >trying to restrict the serialization?  Or, in other words, if the value
    >of a uint64 is 42, would 0x182a be an allowable serialization, or are
    >you really trying to specify that the serialization should be
    >0x1b000000000000002a?

    > 0x1b000000000000002a and reject 0x182a

I think that this is reasonable.
1) Having a consistent representative is critical if signing an object.
2) If the size of the packets changes based upon the content (particularly
   when it will encrypted), then it is often possible to derive information
   based upon the size.  It may be better to pad the result rather than
   make every item be exactly 64-bits, but that's a decision for the
   designer, not CDDL.

    > Or maybe this is a bad question. A concern for dinosaurs of

I think that it's a reasonable thing to ask.

    > assemble chunks. (Think about constrained nodes.)

It's also a possible DDoS.
Even with definite bytes/text, one can send a "here is a 1 billion byte stream"
and then just never send much.  Over UDP/CoAP it may be obvious that there is
more data coming, but with TCP or CoAP with-block-mode, the CBOR layer may
go and allocate and just sit there.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 19 Jun 2017 16:51:23 +0200

> or of an indefinite size.

Since publishing CBOR in 2013, we ran into one case so far.
IIRC, this was about embedding forward pointers into a directory of sections that are later in the data stream; you wouldn’t know from the outset what length would be required for these pointers.
Certainly not your garden variety use of CBOR…
(The solution was to embed a fixed length byte string and encode the pointer there.)

> Or maybe this is a bad question. A concern for dinosaurs of

I think the question is really what style of use of CBOR should be supported by CDDL.
Or more generally, what style of use of CBOR should we foster in the CBOR ecosystem.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 19 Jun 2017 17:02:38 +0200

(Sorry, message escaped early — one of the wonderful features of the Mac touch bar :-)

> model level. The concern of CDDL is the former. Does I understand well?

(Having worked on the original OSI "presentation layer”, I prefer not to use this term for representation, which in this case is best termed “serialization”.)
Yes, that is what I wanted to say.

> I thought that this means "up to 4 bytes”.

Yes.  I was just saying a (data model level) shortcut for “uint .size 4” or the equivalent “0..4294967295” would also be useful, and would be a plausible use of “uint32”.  I wasn’t sure yet which one you meant.

> I had wished to express "exactly 4”.

Right, I understand now.

> assemble chunks. (Think about constrained nodes.)

Yes, I can understand the need making a global restriction “do not use indefinite [byte/text] strings”.
I’m not so sure about embedding such restrictions into the structure description, switching it on here and off there.

> specification has a mean to restrict things to the use of definite data

Right now, I think the best way to do this is doing it orthogonal to CDDL.
As in (YAML syntax, sorry):

canonical: no
float: [16, 32, 64]
indefinite-containers: yes
indefinite-strings: no

(I think this is about it in terms of what actually came up in real protocols; this could even be cooked down to 6 booleans.)

> uint64, I would need some syntax for "casting":

Yes.  I’m not sure that we want this in CDDL, but if we do, we could use something like .and for attaching serialization constraints to a data type.

> http://recherche.telecom-bretagne.eu/xaal/

Will look into this at the next opportunity…

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 19 Jun 2017 17:14:42 +0200

> 1) Having a consistent representative is critical if signing an object.

We have canonical for that.  That is a single bit you just flip, instead of having to expose zillions of little serialization concerns in the model.

(More generally speaking, much of the simplicity of actually using CBOR comes from decoupling the application from serialization concerns.  You have that one six-liner that handles the variants for everything in your implementation…  So that is actually the style that I would like to support best in the CBOR ecosystem.)

>   make every item be exactly 64-bits, but that's a decision for the

We decided against putting a form of padding into CBOR — I probably should be publishing the simple design I have for that in the drawer.  Of course you can always put the padding outside CBOR.

> go and allocate and just sit there.

Yes, but that is more of a quality-of-implementation issue — your implementation should not blow up when it encounters 9b1234567812345678.  Not much for the data model designer to think about here.

Grüße, Carsten