Subject: [Cbor] Self-encoded CBOR stream (tags 24, 55799)

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 12:37:50 +0200

Hey guys,

based on the discussion around the current new draft, I think it would make sense to clarify the difference between tag 24 (https://tools.ietf.org/html/rfc7049#section-2.4.4.1 <https://tools.ietf.org/html/rfc7049#section-2.4.4.1>) and 55799 (https://tools.ietf.org/html/rfc7049#section-2.4.5 <https://tools.ietf.org/html/rfc7049#section-2.4.5>) as both tags describe an CBOR element and both could be used interchangeably (at least in theory).

Thanks,
Chris

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 14 Apr 2017 13:06:57 +0200

On Apr 14, 2017, at 12:37, Christoph Engelbert <[email-masked]> wrote:
> 
> Hey guys,
> 
> based on the discussion around the current new draft, I think it would make sense to clarify the difference between tag 24 (https://tools.ietf.org/html/rfc7049#section-2.4.4.1) and 55799 (https://tools.ietf.org/html/rfc7049#section-2.4.5) as both tags describe an CBOR element and both could be used interchangeably (at least in theory).

Tag 24 contains a byte string that carries an embedded encoded CBOR item (Section 2.4.4.1).
Tag 55799 carries a data item (Section 2.4.5).

The big difference is that tag 24 contains byte string, which is represented by a head for major type 2 with the length of the embedded item, which is in that item, while tag 55799 contains that item directly.  Tag 24 is useful to have “skippable” parts within a CBOR structure, and/or parts that are meant to be conveyed literally without any need for canonicalization.  Tag 55799 is great as a magic number that you can prefix to any encoded CBOR data item to make it more obvious that it is CBOR.

“foo”:
63        # text(3)
   666f6f # "foo"

24(<<“foo”>>):
d8 18          # tag(24)
   44          # bytes(4)
      63666f6f # "cfoo"

5779(“foo”):
d9 1693      # tag(5779)
   63        # text(3)
      666f6f # "foo"

I’m using one of the proposals for extended diagnostic notation for embedded CBOR-encoded strings above, BTW.

These may look somewhat similar in this short example (s/d8184x/d91693/), but have very different semantics.

(We are missing a tag for embedding a sequence of zero or more encoded CBOR items, BTW; the 55799 equivalent here would simply use an array.)

Grüße, Carsten

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 13:16:24 +0200

Hey Carsten,

I see, ok that means my parser is to relaxed right now for tag 24 as
it just accepts any element type and just skips parsing (only
storing the offset). Since I have a lazy parser it didn't really
makes a difference to me. I'll restrict tag 24 to ByteString then.

For 55799 I understand it is designed as a magic number but could
also be used (as you said) by tagging an array to store multiple
elements that are themselves encoded in CBOR.

I'm still not sure if this distinction is obvious from the
specification, maybe it's just me though.

Chris

Am 4/14/2017 um 1:06 PM schrieb Carsten Bormann:
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

-- 

##############################
# A Digital's Life           #
##############################
Nickname: Noctarius
Location: Germany

Meet me at:
Ohloh: http://www.ohloh.net/accounts/noctarius
G+: https://plus.google.com/114622570438626215811
Web: http://www.noctarius.com
LinkedIn: http://www.linkedin.com/in/noctarius
Xing: https://www.xing.com/profile/Christoph_Engelbert
Masterbranch: https://masterbranch.com/christoph.engelbert
XMPP/Jabber: [email-masked]

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 13:33:55 +0200

Hey Carsten,

Sorry but one more question about the tag 24. What is the expected encoding of the ByteString? Just hex-represented, base64, baseXX? I think this should be specified so parsers are implemented the same way. Without a specific naming of an encoding scheme I would assume it’s hex encoding?

Chris

> On 14. Apr 2017, at 13:16, Christoph Engelbert <[email-masked]> wrote:
> 
> Hey Carsten,
>

---

From: [email-masked]
Date: Fri, 14 Apr 2017 14:50:06 +0200

On 14/04/2017 13:06, Carsten Bormann wrote:
> Tag 55799 is great as a magic number that you can prefix to any
> encoded CBOR data item to make it more obvious that it is CBOR.

Is there any magic tag to tell that this is the end of cbor data?
meaning that the rest of the stream is no more cbor, that the parser
should stop here and just has to return a pointer to the rest of
available bytes

Regards
Christophe

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 14:59:07 +0200

Hey Christophe,

See comment inline

> should stop here and just has to return a pointer to the rest of
This shouldn't be necessary since a tag is always applied to a
single element (which can be a an array or map), however you know
the exact bytesize of this following item by definition of the
specification (indefinite items have a poison byte). Therefore you
know there is no further CBOR data, when the element ends.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 14 Apr 2017 17:07:16 +0200

> Sorry but one more question about the tag 24. What is the expected encoding of the ByteString? Just hex-represented, base64, baseXX? I think this should be specified so parsers are implemented the…

No encoding needed, just a byte string.

Grüße, Carsten

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 17:12:12 +0200

> No encoding needed, just a byte string.

"Tag 24 contains a byte string that carries an embedded encoded CBOR
item (Section 2.4.4.1)."
Maybe I misunderstand the idea of "encoded CBOR" but how do I just
store binary data as a bytestring without any encoding or is
bytestring in reality a fixed size "byte-array"?

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 14 Apr 2017 17:27:25 +0200

> On Apr 14, 2017, at 17:12, Christoph Engelbert <[email-masked]> wrote:
> 
> Maybe I misunderstand the idea of "encoded CBOR”

As in, a CBOR data item that has been serialized into a sequence of bytes.

> but how do I just
> store binary data as a bytestring without any encoding

Yes, that’s what byte strings (major type 2) are good for.

> or is
> bytestring in reality a fixed size "byte-array”?

A byte string is a sequence of bytes (and each one has a certain size, not sure what “fixed” means here).

(I just noticed that we have not been using very consistent terminology for byte strings in the “extended diagnostic notation” appendix of CDDL, but I think RFC 7049 is reasonably good here.)

Grüße, Carsten

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 17:36:04 +0200

> (I just noticed that we have not been using very consistent terminology for byte strings in the “extended diagnostic notation” appendix of CDDL, but I think RFC 7049 is reasonably good here.)

Ok got it, I think bytestring is a fairly bad term in this case as
"string" implies that it contains ASCII characters. This is ok with
only the difference that byte 0 is valid in a byte array but in an
indefinite bytestring is symbolizes the end of the bytestring. I
understand a indefinite bytestring can only contain
sized-bytestrings, however byte 0 is still affected by two different
meanings. I would assume byte-stream or a term similar to stream
would make it more obvious it is also to be used for arbitrary
streams of bytes and not ASCII strings only because that is what I
understood so far.

Thanks,
Chris

---

From: Christoph Engelbert <[email-masked]>
Date: Fri, 14 Apr 2017 17:51:52 +0200

Looking through other implementations it seems I'm the only one to
assume bytestream = ASCII. Fair enough, I accept it as my fault ;-)