Subject: [apps-discuss] Defining a CBOR tag for RFC 5646 Language Tags

From: Carsten Bormann <[email-masked]>
Date: Mon, 12 May 2014 00:48:57 +0200

If you care about language tags, I hope the subject got your
attention.  If you don't, please ignore this request for assistance.

Concise Binary Object Representation (CBOR, RFC 7049) is a binary
format for structured objects.  CBOR has a number of pre-defined data
types and allows additional data types to be registered as "tags".
Among the pre-defined data types is a text string (Unicode characters,
encoded in UTF-8).  No facility is pre-defined for associating a
Language Tag with such a string.

A new CBOR tag is being proposed for combining a CBOR text string with
a Language Tag.  The (single-page) proposal is in:

http://peteroupc.github.io/CBOR/langtags.html

The proposal is almost trivially obvious (pair a language tag with an
UTF-8 string in a two-element array) and looks right to me.  But I'm
not an expert in Language Tags, and silly mistakes are being made by
non-experts all the time.

If you are an expert in Language Tags:
-- Is anything missing or could anything be done in a better way?
-- Or does this really simply look right?

Responses to me privately or to the list are appreciated.

Grüße, Carsten

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 12 May 2014 10:30:01 +0900

Just took a short look. What I noticed is that this works fine on its 
own, but as soon as this is combined with other properties/attributes of 
strings, it may not scale.

Regards,   Martin.

On 2014/05/12 07:48, Carsten Bormann wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 12 May 2014 07:32:41 -0700

On May 11, 2014, at 6:30 PM, Martin J. Dürst <[email-masked]> wrote:

> Just took a short look. What I noticed is that this works fine on its own, but as soon as this is combined with other properties/attributes of strings, it may not scale.

Martin has a good point. What Peter is proposing is, in essence:

(Tag 38)[ "en", "Hello" ]

where "Tag 38" defines the pair. If someone later wants to tag the UTF8 text with some other property, such as a statement about the Unicode property of all the characters in the string, that new tag would have to be able to either wrap a UTF8 string *or* a tagged array.

This could be replaced with an extensible map (for instance, with 0 meaning "the string" and 1 meaning "the language tag"), but then there would need to be an IANA registry for the map values. In the name of simplicity, it is probably better just to let Tag 38 be defined as it is now, and if other properties/attributes come up later, maybe then come up with a more complicated scheme.

--Paul Hoffman

---

From: "Peter Occil" <[email-masked]>
Date: Mon, 12 May 2014 10:44:17 -0400

Trying to send to the Apps-Discuss mailing list again, now that I'm 
subscribed.

-----Original Message----- 
From: Peter Occil
Sent: Sunday, May 11, 2014 10:20 PM
To: "Martin J. Dürst"
Cc: Carsten Bormann ; IETF Apps Discuss
Subject: Re: [apps-discuss] Defining a CBOR tag for RFC 5646 Language Tags

If I understand what you mean by "not scaling", I think it's because my
specification doesn't expressly allow using CBOR tags on the language tag
string or the string being tagged with a language.  If this interpretation
is right, I think this can be solved by adding the following:

"The language tag and the arbitrary string can have any number of CBOR tags
on it."

--Peter

-----Original Message----- 
From: "Martin J. Dürst"
Sent: Sunday, May 11, 2014 9:30 PM
To: Carsten Bormann ; IETF Apps Discuss
Cc: Peter Occil
Subject: Re: [apps-discuss] Defining a CBOR tag for RFC 5646 Language Tags

Regards,   Martin.

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 19 May 2014 19:30:31 +0000

Sorry, reading this list backward.

On 5/12/14, 8:32 AM, "Paul Hoffman" <[email-masked]> wrote:

>This could be replaced with an extensible map (for instance, with 0
>meaning "the string" and 1 meaning "the language tag"), but then there
>would need to be an IANA registry for the map values. In the name of
>simplicity, it is probably better just to let Tag 38 be defined as it is
>now, and if other properties/attributes come up later, maybe then come up
>with a more complicated scheme.

Why would there need to be a new IANA registry?  Doesn't RFC 5646 already
do that?

-- 
Joe Hildebrand

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 19 May 2014 13:39:28 -0700

> Why would there need to be a new IANA registry?  Doesn't RFC 5646 already

I was talking about a different IANA registry. If there are going to be many tags that could be applied to a string ("the language tag", "adult language", ...), then there would need to be an IANA registry for that. Which is why I'm not thrilled about either "this tag is over just that string" or "this is one of many tags on that string".

--Paul Hoffman

---

From: John Cowan <[email-masked]>
Date: Mon, 12 May 2014 01:37:03 -0400

> non-experts all the time.

Looks good to me.  But I would recommend requiring the encoder to do the
case folding rather than leaving it to the decoder.  This is a form of
early uniform normalization, which is generally a Good Thing if you can
get it.

The main mistake people make is trying to make the language tag fixed
length, which you have already avoided.

-- 
John Cowan          http://www.ccil.org/~cowan        [email-masked]
A: "Spiro conjectures Ex-Lax."
Q: "What does Pat Nixon frost her cakes with?"
  --"Jeopardy" for generative semanticists

---

From: "Peter Occil" <[email-masked]>
Date: Sun, 11 May 2014 22:20:01 -0400

--Peter

-----Original Message----- 
From: "Martin J. Dürst"
Sent: Sunday, May 11, 2014 9:30 PM
To: Carsten Bormann ; IETF Apps Discuss
Cc: Peter Occil
Subject: Re: [apps-discuss] Defining a CBOR tag for RFC 5646 Language Tags

Regards,   Martin.

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 19 May 2014 21:06:26 +0000

On 5/19/14, 2:39 PM, "Paul Hoffman" <[email-masked]> wrote:

>I was talking about a different IANA registry. If there are going to be
>many tags that could be applied to a string ("the language tag", "adult
>language", ...), then there would need to be an IANA registry for that.
>Which is why I'm not thrilled about either "this tag is over just that
>string" or "this is one of many tags on that string".

There's no ambiguity that I know if in xml:lang:
http://www.w3.org/TR/REC-xml/#sec-lang-tag

All I want is roughly that plus the bit that we learned about sometimes
needing alternatives, minus the bit where you have to keep context of
enclosing elements.

-- 
Joe Hildebrand