Subject: [Cbor] CDDL spec should specify how CBOR items match types

From: Jeffrey Yasskin <[email-masked]>
Date: Thu, 6 Jul 2017 16:01:06 -0700

Apologies if I'm just reading
https://tools.ietf.org/html/draft-greevenbosch-appsawg-cbor-cddl-11
badly, but I don't see a specification of which CBOR items match which
types.

Sections like https://tools.ietf.org/html/draft-greevenbosch-appsawg-cbor-cddl-11#section-3.4
(Arrays) specify how to write an array type, but I don't see a
normative statement that '82 01 02' is an instance of [*uint] or that
'01' and '42 01 02' aren't.

I also don't see a definition of '#' in order to ground the meaning of
the prelude. "How this is used should be evident from the prelude
(Appendix E)." from 2.2.3. Representation Types isn't really
sufficient to specify it.

I think this lack of specification is at the root of Kevin Braun's set
of bugs in https://mailarchive.ietf.org/arch/msg/cbor/Z1XcZSTLbTDnqA9HP8UXjeUOspE.

I'd be inclined to fix this by specifying each CDDL type as a
recursive parser, which accepts or rejects either CBOR items or byte
strings. I think the choice of item vs bytes comes down to where we
want to put the serialization constraints (like
"indefinite-containers") described in
https://mailarchive.ietf.org/arch/msg/cbor/Pv191TJRBiG1QOJbJL8iVHjUDuE,
but it could also have some implications for how parsers return data
from unfinished byte streams.

Jeffrey

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 7 Jul 2017 08:44:10 +0200

Hi Jeffrey,

Yes, we have to replace “self-evident” etc. by actually explaining things.

However, I’m pretty sure we are on the safe side with the # constructs, as well as with [] and {}.
These may not be explained in so many words, but it is still pretty clear what they mean.

We have more of an undefined state with literals such as 10, 10.0, 1e1, and with applying range operators, in particular if they span CBOR major types (-5..10 is pretty clear, but what does 1..10.0 mean?).  When applied to floating point, the range operator also does not say what precision is needed (you can help yourself with construct such as `-10.0..10.0 .and float16`, but that is neither supported by the tool nor explained at all).

I would like to keep CDDL operating at the data model level, even with its history of toying with anchoring it at the serialization level.  Yes, the # construct is in serialization terms, but this is just a shortcut to have a single construct cover all of CBOR (well, you have to add {} and [], because these need groups).

There are a few more areas that are not fully defined, e.g., which exact PCRE variant exactly does .regexp use.
I would be inclined in *not* overspecifying this at this point.  (And we haven’t solved the “line too long” problem there; see prelude for tdate which is not using a regexp even though it could — try a `cddl -<<<x=tdate generate` for fun.)

So, lots of editorial work remaining indeed, but the technical discussion probably should focus on the numeric system underlying CDDL.

Grüße, Carsten

> On Jul 7, 2017, at 01:01, Jeffrey Yasskin <[email-masked]> wrote:
> 
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor
>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Sun, 9 Jul 2017 07:44:57 -0700

I just noticed that the current draft is informational, so it can't be
used the way ABNF is used, to help with writing other standards, as
much as I'd like to. For informational documents, I agree that "pretty
clear" is enough, but I hope we can aim for a standard instead.

Jeffrey

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 9 Jul 2017 17:46:28 +0200

> clear" is enough, but I hope we can aim for a standard instead.

Informational was where this was started as an individual effort.
Now that we have a WG, we can go beyond that.

https://datatracker.ietf.org/doc/charter-ietf-cbor/ says:

   The CBOR WG will complete the development
   of this specification by creating an Informational or Standards Track
   RFC.

So the WG has a choice here.

Grüße, Carsten

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 10 Jul 2017 08:37:08 +1200

> So the WG has a choice here.

IMHO we should go for Proposed Standard, even though it will be more
work to get it correct. It will make CDDL much more useful, and if it
succeeds at all it will be used as a de facto standard anyway.

This argues against inflating CDDL with new features at the moment.
We can always add features later.

Full disclosure: I have a draft very close to IESG approval that has
CDDL as a normative reference. If CDDL isn't aimed at the standards
track, I personally will have some extra work to do :-(.

    Brian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 10 Jul 2017 00:55:03 +0200

> We can always add features later.

Right, and it is important to have defined extension points for that.

Most of the linear growth of CDDL from -05 to the current -11 happened in “controls”.  “Controls" used to be called “annotations”, because that was the original purpose we had in mind, but the construct just turned out to be more useful than just that.  The important thing is that new “controls” can be defined pretty much without disturbing the rest of the language, so it would be easy to buffer any new requirements in such add-on specs if they fit the construct.

It is much harder to extend a language like CDDL in its foundations.  Fortunately, there are some 50 years of theory in computer grammars that can help us predict where the path might lead (and which approaches might not be so successful).

We don’t have a very good plan for making CDDL useful for specific new CBOR tags that are going to be invented (e.g., we don’t know the best way an OID tag might be best supported by CDDL).  Much can be done with “controls”, but there is also literal syntax to take care of (maybe that can also be handled with controls, just like .bits added something akin to a bit set mechanism to the capabilities of CDDL).

After having seen CDDL in use in numerous smaller and not-to-small specification projects in more than two years now, I believe -11 is pretty much feature complete.  It is likely that one future extension will want to enhance the ability of diagnostic tools to provide better error messages (again something that probably can be copied from existing grammar research; but the work of evaluating what exactly to copy hasn’t been done yet and sounds more like a research project that might take longer than the WG process).

So I don’t see much danger of CDDL bloating in the WG process leading to the first RFC, and I agree it would be good to get agreement in the WG about trying to avoid that.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Sun, 09 Jul 2017 20:21:29 -0400

Carsten Bormann <[email-masked]> wrote:
    > Most of the linear growth of CDDL from -05 to the current -11 happened
    > in “controls”.  “Controls" used to be called “annotations”, because
    > that was the original purpose we had in mind, but the construct just
    > turned out to be more useful than just that.  The important thing is
    > that new “controls” can be defined pretty much without disturbing the
    > rest of the language, so it would be easy to buffer any new
    > requirements in such add-on specs if they fit the construct.

But, since any given draft is going to point at a SPECIFIC version of CDDL,
and we can do forklift upgrades on drafts and RFCs, we don't have to worry
about the same level of "in the field" compatibility.  If we have to,
the CDDL file can start with:

"cddl 2.0"

at the top, and rotate all the operators left if we want.

(I know that you know this...)

    > After having seen CDDL in use in numerous smaller and not-to-small
    > specification projects in more than two years now, I believe -11 is
    > pretty much feature complete.  It is likely that one future extension

Wonderful!

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-