Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

From: Mykyta Yevstifeyev <[email-masked]>
Date: Thu, 07 Jul 2011 06:55:59 +0300

Hello,

I've identified the following issue in the draft.

Section 2.2 says:

>     The subtags in the 't' extension are of the following form:
>
>       +--------+-------------------------+----------------------------+
>       | Label  | ABNF                    | Comment                    |
>       +--------+-------------------------+----------------------------+
>       | t_ext= | "t"                     | Extension                  |
>       |        | ("-" lang *("-" field)  | Source + optional field(s) |
>       |        | / 1*("-" field))        | Field(s) only (no source)  |
>       | lang=  | language                | [BCP47  <http://tools.ietf.org/html/draft-davis-t-langtag-ext-01#ref-BCP47>], with restrictions |
>       |        | ["-" script]            |                            |
>       |        | ["-" region]            |                            |
>       |        | *("-" variant)          |                            |
>       | field= | sep 1*("-" 3*8alphanum) | With restrictions          |
>       | sep=   | 1ALPHA 1DIGIT           | Subtag separators          |
>       +--------+-------------------------+----------------------------+

I should note that, first of all, reference to RFC 5234 is missing; 
moreover, and this is more important, making the ABNF definition in the 
form of table makes such definition an invalid one, in terms of RFC 
5234.  Also, there are a number of ABNF nits here.  So, please consider 
changing this to:

>     The subtags in the 't' extension are of the following form, defined
>     using ABNF [RFC5234] in<t-ext>  rule:
>
>       t-ext    = "t" ("-" lang *("-" field) / 1*("-" field))
>       lang     = langtag
>       field    = sep 1*("-" 3*8alphanum)
>       sep      = ALPHA DIGIT
>       alphanum = ALPHA / DIGIT
>
>     where<langta>  rule is specified in BCP 47 [BCP47],<ALPHA>  and<DIGIT>
>     rules - in RFC 5234 [RFC5234].
Also, the minors comments on references.  Reference to BCP 47 should 
include both references to RFC 5646 and RFC 4647, like:
>     [BCP47]    Phillips, A. and M. Davis, "Matching of Language Tags",
>                BCP 47, RFC 4647, September 2006.
>
>                Phillips, A., Ed., and M. Davis, Ed., "Tags for Identifying
>                Languages", BCP 47, RFC 5646, September 2009.
...and, referencing UTS 35 you shouldn't reference specific parts of the 
document; this should be done in the text.  Finally, I don't see where 
[US-ASCII] is used in the text.

Thanks,
Mykyta Yevstifeyev

07.07.2011 2:49, Pete Resnick wrote:
> Most of the people on the ietf-languages list are probably on the 
> [email-masked] list as well, but I wanted to confirm that everyone got 
> a chance to review this before it proceeded to the IESG. Please have a 
> look at the ltru archive 
> <http://www.ietf.org/mail-archive/web/ltru/current/maillist.html> and 
> send any comments to the [email-masked] list since that's where 
> discussion seems to be taking place.
>
> Thanks.
>
> pr

---

From: Mark Davis ☕ <[email-masked]>
Date: Thu, 7 Jul 2011 07:42:38 -0700

Thanks for the feedback. We can make those corrections.

One question. The primary reason that we chose to use a BCP was primarily
because it provided a stable reference; the underlying RFCs can (and have)
changed while "BCP47" has remained the same. Listing the current RFCs
somewhat undercuts that. Note: if that is the practice we should do it, but
it seems odd.

Mark
*— Il meglio è l’inimico del bene —*

On Wed, Jul 6, 2011 at 20:55, Mykyta Yevstifeyev <[email-masked]>wrote:

> **
> moreover, and this is more important, making the ABNF definition in the form
> of table makes such definition an invalid one, in terms of RFC 5234.  Also,
> there are a number of ABNF nits here.  So, please consider changing this to:
>
>    using ABNF [RFC5234] in <t-ext> rule:
>
>    where <langta> rule is specified in BCP 47 [BCP47], <ALPHA> and <DIGIT>
> [email-masked] list as well, but I wanted to confirm that everyone got a
> chance to review this before it proceeded to the IESG. Please have a look at
> the ltru archive
> <http://www.ietf.org/mail-archive/web/ltru/current/maillist.html><http://www.ietf.org/mail-archive/web/ltru/current/maillist.html>and send any comments to the
> [email-masked] list since that's where discussion seems to be taking place.
>
> Thanks.
>
> pr
>
>
>
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ltru
>
>

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Thu, 07 Jul 2011 17:47:40 +0300

07.07.2011 17:42, Mark Davis ☕ wrote:
> One question. The primary reason that we chose to use a BCP was 
> primarily because it provided a stable reference; the underlying RFCs 
> can (and have) changed while "BCP47" has remained the same. Listing 
> the current RFCs somewhat undercuts that. Note: if that is the 
> practice we should do it, but it seems odd.
In this case the current reference is OK.  However, Addison Phillips 
isn't listed as the editor, even if he is an editor of both RFC 5646 and 
RFC 4647.

Mykyta Yevstifeyev
> Mark
> /— Il meglio è l’inimico del bene —/

---

From: Peter Constable <[email-masked]>
Date: Thu, 7 Jul 2011 15:29:08 +0000

I was not aware of the discussion on LTRU. When will it be reviewed by IESG? What is the action being requested of IESG / what’s the status of this draft?

Peter

From: [email-masked] [mailto:[email-masked]] On Behalf Of Pete Resnick
Sent: Wednesday, July 06, 2011 4:49 PM
To: [email-masked]
Subject: Fwd: draft-davis-t-langtag-ext

Most of the people on the ietf-languages list are probably on the [email-masked]<mailto:[email-masked]> list as well, but I wanted to confirm that everyone got a chance to review this before it proceeded to the IESG. Please have a look at the ltru archive <http://www.ietf.org/mail-archive/web/ltru/current/maillist.html><http://www.ietf.org/mail-archive/web/ltru/current/maillist.html> and send any comments to the [email-masked]<mailto:[email-masked]> list since that's where discussion seems to be taking place.

Thanks.

pr

-------- Original Message --------
Subject:

[Ltru] draft-davis-t-langtag-ext

Date:

Wed, 22 Jun 2011 15:00:47 -0700

From:

Mark Davis ☕ <[email-masked]><mailto:[email-masked]>

To:

Martin J. Dürst <[email-masked]><mailto:[email-masked]>

CC:

LTRU Working Group <[email-masked]><mailto:[email-masked]>, <[email-masked]><mailto:[email-masked]>

A new draft posted at http://tools.ietf.org/html/draft-davis-t-langtag-ext-01

Martin, we tried to address your concerns; please take a look and let us know what you think.

Mark
— Il meglio è l’inimico del bene —

On Tue, Jun 21, 2011 at 09:00, Mark Davis ☕ <[email-masked]<mailto:[email-masked]>> wrote:
Those are good issues; thanks for raising them and starting the discussion. Comments below.

________________________________
Mark
— Il meglio è l’inimico del bene —

On Mon, Jun 20, 2011 at 23:39, "Martin J. Dürst" <[email-masked]<mailto:[email-masked]>> wrote:
Hello Mark, others,

Overall comment:
The idea to reuse language tags to indicate transliteration/transcription source, and to add some additional tags to distinguish methods seems to be reasonable and sound.

The description of the structure of the allowed subtags and of the responsibility split between IETF (this draft) and UTC (UTS 35) looks quite messy to me, and should be cleaned up. I'd personally prefer that UTS 35 (or whatever else on the Unicode side) only define the <mechanism> part (after the m0 subtag).

That would be my preference as well (can't speak for my coauthors).

We patterned it this way following what ended up being accepted for  the -u- extension. That is, the spec is in UTS35, but there is a summary here. But of course, there are many ways to do it. And maybe this summary is too detailed, at least for the mechanism part, and we could just have it in UTS35.

We considered a number of alternatives:

  *   We could define everything after -t- to be the source language, and everything after -m- to be the mechanism. But that burns 2 extension letters, just one.
  *   We also considered having everything in the -u extension, for which we already have the structure set up. However, that would force us to have artificial source subtags like 'en0' instead of 'en', because the -u- extension wouldn't allow the 2-letter subtags (it already defines a use for them).
  *   We could also have -t- be just the source, and define the mechanism in -u-, also easy. But we felt it would be better to have everything under one extension.

Detailled comments:

"In addition, it may also be important to
  specify a particular specification for the transformation.": Too much 'spec' in one sentence.

ok

"For example, if one is transcribing the names of Italian or Russian
  cities on a map for Japanese users, each name will need to be
  transliterated into katakana using rules appropriate for the source
  language and target languages.": "source languages and target language"?

yes

BCP47 required information: The first three paragraphs should move to the introduction.

Other authors, what do you think?

"followed by a sequence of subtags that would form a language tag": Here and in general: Don't use 'would'.

Grammatically, it is that the sequence of subtags *would* form a language subtag if they *were* separated out. They are not actually a language tag, because they occur in the middle of another language subtag. How would you like that to be phrased?

  The structure of 't' subtags is determined by the Unicode CLDR
  Technical Committee, in accordance with the policies and procedures
  in http://www.unicode.org/consortium/tc-procedures.html, and subject
  to the Unicode Consortium Policies on
  http://www.unicode.org/policies/policies.html.

The following paragraph is also difficult to understand. I wouldn't know exactly what falls on what side. I think one major reason is that we are treading new ground here, it's the first time we have a singleton definition that allows reuse of language tags (with a few restrictions) as well as intends to define its own extensions.

These were both patterned after what was used for the -u- extension. We can take a look at them to try to clarify.

  Changes that can be made by successive versions of LDML [UTS35] by
  the Unicode Consortium without requiring a new RFC include the
  allocation of new subtags for use after the 't' extension.  A new RFC
  would be required for material changes to an existing 't' subtag, or
  an incompatible change to the overall syntactic structure of the 't'
  extension; however, such a change would be contrary to the policies
  of the Unicode Consortium, and thus is not anticipated.

2.1 Summary: There seems to be quite some overlap between the part of section 2 before the 2.1 heading.

One question I would have as a linguistic researcher is: How much effort and time is involved in getting a 'mechanism' approved? If such 'mechanisms' are e.g. rejected with arguments like "if we accept it, then everybody has to implement it" or so, then I would see that as a problem.

Good point. I'll propose some text.

So much for the moment.

Regards,   Martin.

On 2011/06/18 6:07, Mark Davis ☕ wrote:
Yoshito, Addison, and I had had an action for a while now from the CLDR
committee to submit a draft for a an extension. Rather than go through all
the problems in the falk draft, we put together an alternative approach,
leveraging the work we already did for the -u- extension.

It just got posted at
http://tools.ietf.org/html/draft-davis-t-langtag-ext-00

Courtney, I think this provides a superset of the functionality that you are
interested in. Perhaps you can read it over, and we can add you as an author
of the next version of this draft instead of having the two competing
proposals.

Mark

On Wed, Jun 15, 2011 at 10:50, Randy Presuhn
<[email-masked]<mailto:[email-masked]>>wrote:
Hi -

I started out with an off-list response, but I figure this is
something worth sending to the list.

Off-list, a contributor asked:

...
I'd love to see your input. I'd like to make sure I understand
all the concerns. Is there any way you could forward this to the list?

My response:

Sorry, already deleted.  As I recall, the main concerns were

 (1) there already *is* support for identifying orthographies
     (remember German?)
 (2) the I-D seems to assume that transliterations always result
     in "Latin" (previous discussion on LTRU included transliterations
     to Cyrillic and Hangul, among others)
 (3) the "original orthography" is irrelevant for the transliteration
     systems I've been able to think of.  (At the same time, some
     transliteration systems are quite "lossy" and some don't do
     "round trip" very well.)  Consider also the transliteration of
material
     which was originally in audio form...
 (4) The draft doesn't clearly distinguish "orthography" from
"transliteration".
     This may be because the boundary between the two can be fuzzy, but
even
     that is an issue that should be addressed.
 (5) How this fits in with *transcription* systems (e.g. IPA) should be
     addressed.  The boundary gets fuzzy with orthographies that are
equivalent
     to phonemic representations of the language.  (e.g., Pinyin for
Mandarin)
 (6) The proposed singleton usage appears broken and unnecessary.

Or something like that.  I may have forgotten something here, or, in the
process of reconstruction, thought of something I missed the first time.

Randy

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 16:52:13 +0100

Hi

Point of clarification only.

The draft states:

Any purely numeric subtag is a representation of a date in the

       Gregorian calendar.  It MAY occur in any mechanism field.  If it

       does occur:

       *  it MUST occur as the final subtag in the field,

Does field in this context mean the subtags after the t extension but before any further extensions?

In other words is the following allowed:  “ja-t-it-m0-xxx-v21a-2007-i-ami”

Best regards

Debbie

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis ?
Sent: 07 July 2011 15:43
To: Mykyta Yevstifeyev
Cc: Pete Resnick; [email-masked]
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

 

One question. The primary reason that we chose to use a BCP was primarily because it provided a stable reference; the underlying RFCs can (and have) changed while "BCP47" has remained the same. Listing the current RFCs somewhat undercuts that. Note: if that is the practice we should do it, but it seems odd.

Mark

On Wed, Jul 6, 2011 at 20:55, Mykyta Yevstifeyev <[email-masked]> wrote:

Hello,

Section 2.2 says:

     | lang=  | language                | [BCP47 <http://tools.ietf.org/html/draft-davis-t-langtag-ext-01#ref-BCP47> ], with restrictions |

I should note that, first of all, reference to RFC 5234 is missing; moreover, and this is more important, making the ABNF definition in the form of table makes such definition an invalid one, in terms of RFC 5234.  Also, there are a number of ABNF nits here.  So, please consider changing this to:

 

Also, the minors comments on references.  Reference to BCP 47 should include both references to RFC 5646 and RFC 4647, like:

...and, referencing UTS 35 you shouldn't reference specific parts of the document; this should be done in the text.  Finally, I don't see where [US-ASCII] is used in the text.

Thanks,
Mykyta Yevstifeyev

Most of the people on the ietf-languages list are probably on the [email-masked] list as well, but I wanted to confirm that everyone got a chance to review this before it proceeded to the IESG. Please have a look at the ltru archive  <http://www.ietf.org/mail-archive/web/ltru/current/maillist.html> <http://www.ietf.org/mail-archive/web/ltru/current/maillist.html> and send any comments to the [email-masked] list since that's where discussion seems to be taking place.

Thanks.

pr

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 17:00:56 +0100

I am also concerned about the structure of the Unicode Committee and voting rights. Perhaps someone can explain how this will work and why it is required in addition to the current structure for the registration of language tags.

Have I missed something here? (I probably have as I have been away from the list for some time)  Have Unicode already taken over some of the duties of the BCP47 registrar?

Best wishes

Debbie

 

Mark

Hello,

Section 2.2 says:

 

 

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

---

From: "Phillips, Addison" <[email-masked]>
Date: Thu, 7 Jul 2011 09:27:21 -0700

The word “field” here refers to the ABNF construct “field” on the document. So it means that the subtag must occur at the end of that part of the –t extension. The –t extension can, itself, be followed by other extensions or by private use. So, yes, that language tag would be allowed.

Addison

From: [email-masked] [mailto:[email-masked]] On Behalf Of Debbie Garside
Sent: Thursday, July 07, 2011 8:52 AM
To: 'Mark Davis ☕'; 'Mykyta Yevstifeyev'
Cc: 'Pete Resnick'; [email-masked]
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

Hi

The draft states:

Best regards

Debbie

Mark
— Il meglio è l’inimico del bene —
On Wed, Jul 6, 2011 at 20:55, Mykyta Yevstifeyev <[email-masked]<mailto:[email-masked]>> wrote:
Hello,

Section 2.2 says:

     | lang=  | language                | [BCP47<http://tools.ietf.org/html/draft-davis-t-langtag-ext-01#ref-BCP47>], with restrictions |

     |        | ["-" script]            |                            |

     |        | ["-" region]            |                            |

     |        | *("-" variant)          |                            |

     lang     = langtag

     sep      = ALPHA DIGIT

     alphanum = ALPHA / DIGIT

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

_______________________________________________
Ltru mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 17:46:21 +0100

Thank you.  I thought so but best to check.

Debbie

From: Phillips, Addison [mailto:[email-masked]] 
Sent: 07 July 2011 17:27
To: Debbie Garside; 'Mark Davis ☕'; 'Mykyta Yevstifeyev'
Cc: 'Pete Resnick'; [email-masked]
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

Addison

Hi

The draft states:

       does occur:

Best regards

Debbie

 

Mark

Hello,

Section 2.2 says:

 

 

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 18:00:30 +0100

Let me elaborate further on this.  From memory, data in CLDR may be incorporated on a vote system,  This vote system is linked to the amount paid by companies by way of subscriptions.  Or am I wrong?  I seem to remember a lot of Google data being incorporated (dumped) into CLDR against the wishes of experts in the field who had been working most diligently to make sure their data was correct.  A paid for votes system is contrary to the entire ethos of IETF where, I believe, a general consensus is required rather than paid votes and affiliations are not taken into account.  This is the basis for my concern.

Maybe I am totally wrong and if so I apologise.  Please feel free to correct me.

If I am right, there is still a way for Unicode to do this, as stipulated within the RFC, but rather by recommendation to the IETF-Languages Registrar who can then invite further discussion.

Best wishes

Debbie

Sent: 07 July 2011 17:01

Best wishes

Debbie

 

Mark

Hello,

Section 2.2 says:

 

 

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

---

From: Mark Davis ☕ <[email-masked]>
Date: Thu, 7 Jul 2011 12:58:28 -0700

It's still in discussion, so feedback is welcome. (I'd thought you were on
LTRU...)

Mark

On Thu, Jul 7, 2011 at 08:29, Peter Constable <[email-masked]>wrote:

>  I was not aware of the discussion on LTRU. When will it be reviewed by
> IESG? What is the action being requested of IESG / what’s the status of this
> draft?****
>
> ** **
>
> ** **
>
> Peter****
>
> ** **
>
> *From:* [email-masked] [mailto:
> [email-masked]] *On Behalf Of *Pete Resnick
> *Sent:* Wednesday, July 06, 2011 4:49 PM
> *To:* [email-masked]
> *Subject:* Fwd: draft-davis-t-langtag-ext****
>
> ** **
>
> -------- Original Message -------- ****
>
> *Subject: *
>
> [Ltru] draft-davis-t-langtag-ext****
>
> *Date: *
>
> Wed, 22 Jun 2011 15:00:47 -0700****
>
> *From: *
>
> Mark Davis ☕ <[email-masked]> <[email-masked]>****
>
> *To: *
>
> Martin J. Dürst <[email-masked]> <[email-masked]>****
>
> *CC: *
>
> LTRU Working Group <[email-masked]> <[email-masked]>, <[email-masked]><[email-masked]>
> ****
>
>
>
> A new draft posted at
> http://tools.ietf.org/html/draft-davis-t-langtag-ext-01
> ****
>
> ** **
>
> Martin, we tried to address your concerns; please take a look and let us
> know what you think.****
>
> ** **
>
> Mark****
>
> ****
> On Tue, Jun 21, 2011 at 09:00, Mark Davis ☕ <[email-masked]> wrote:***
> *
>
> ****
> ** **
>  ------------------------------
>
> Mark****
>
> *— Il meglio è l’inimico del bene —*****
>
>
>
> ****
>
> On Mon, Jun 20, 2011 at 23:39, "Martin J. Dürst" <[email-masked]>
> wrote:****
>
> Hello Mark, others,
>
> Overall comment:
> The idea to reuse language tags to indicate transliteration/transcription
> source, and to add some additional tags to distinguish methods seems to be
> reasonable and sound.
>
> The description of the structure of the allowed subtags and of the
> responsibility split between IETF (this draft) and UTC (UTS 35) looks quite
> messy to me, and should be cleaned up. I'd personally prefer that UTS 35 (or
> whatever else on the Unicode side) only define the <mechanism> part (after
> the m0 subtag).****
>
> ** **
>
> That would be my preference as well (can't speak for my coauthors).****
>
> ** **
>
> We patterned it this way following what ended up being accepted for  the
> -u- extension. That is, the spec is in UTS35, but there is a summary here.
> But of course, there are many ways to do it. And maybe this summary is too
> detailed, at least for the mechanism part, and we could just have it in
> UTS35.****
>
> ** **
>
> We considered a number of alternatives:****
>
>    - We could define everything after -t- to be the source language, and
>    everything after -m- to be the mechanism. But that burns 2 extension
>    letters, just one.****
>    - We also considered having everything in the -u extension, for which
>    we already have the structure set up. However, that would force us to have
>    artificial source subtags like 'en0' instead of 'en', because the -u-
>    extension wouldn't allow the 2-letter subtags (it already defines a use for
>    them).****
>    - We could also have -t- be just the source, and define the mechanism
>    in -u-, also easy. But we felt it would be better to have everything under
>    one extension.****
>
>   ** **
>
>
>
>
> Detailled comments:
>
>   specify a particular specification for the transformation.": Too much
> 'spec' in one sentence.****
>
>  ** **
>
> ok****
>
>  ****
>
>
> ****
>  ** **
> yes****
>  ****
> BCP47 required information: The first three paragraphs should move to the
> introduction.****
>
>  ** **
>
> Other authors, what do you think?****
>
>  ****
>
>
> "followed by a sequence of subtags that would form a language tag": Here
> and in general: Don't use 'would'.****
>
>  ** **
>
> Grammatically, it is that the sequence of subtags *would* form a language
> subtag if they *were* separated out. They are not actually a language tag,
> because they occur in the middle of another language subtag. How would you
> like that to be phrased?****
>
> ** **
>
> ** **
>
> ** **
>
>
> >>>>
> The following paragraph is also difficult to understand. I wouldn't know
> exactly what falls on what side. I think one major reason is that we are
> treading new ground here, it's the first time we have a singleton definition
> that allows reuse of language tags (with a few restrictions) as well as
> intends to define its own extensions.****
>
>  ** **
>
> These were both patterned after what was used for the -u- extension. We can
> take a look at them to try to clarify.****
>
> ** **
>
>  ****
>
>
> >>>>
> One question I would have as a linguistic researcher is: How much effort
> and time is involved in getting a 'mechanism' approved? If such 'mechanisms'
> are e.g. rejected with arguments like "if we accept it, then everybody has
> to implement it" or so, then I would see that as a problem.****
>
>  ** **
>
> Good point. I'll propose some text.****
>
>  ****
>
>
> So much for the moment.
>
>
> Regards,   Martin. ****
>
>
>
>
> On 2011/06/18 6:07, Mark Davis ☕ wrote:****
>
> <[email-masked]>wrote:****
>
> Hi -
>
> ...****
> all the concerns. Is there any way you could forward this to the list?****
>
>
> My response:
>
> Randy****
>    ** **

---

From: Mark Davis ☕ <[email-masked]>
Date: Thu, 7 Jul 2011 13:06:58 -0700

These are not primary language subtags or variants, as covered by BCP47.
That still remains as it has been.

This is a different area, part of an extension that is to provide a
structured approach to the specification of transliteration. The development
of extensions and their scope is covered by BCP47, and you can find the
details there.

The Unicode CLDR committee is already serves as the registrar for the -u-
extension. The committee operates under the procedures in
http://unicode.org/consortium/tc-procedures.html, but the vast majority of
decisions are taken by consensus. The consortium is already the Registration
Authority for ISO 15924 (Scripts).

Mark

On Thu, Jul 7, 2011 at 09:00, Debbie Garside <[email-masked]>wrote:

> I am also concerned about the structure of the Unicode Committee and voting
> rights. Perhaps someone can explain how this will work and why it is
> required in addition to the current structure for the registration of
> language tags.****
>
> ** **
>
> Have I missed something here? (I probably have as I have been away from the
> list for some time)  Have Unicode already taken over some of the duties of
> the BCP47 registrar?****
>
> ** **
>
> Best wishes****
>
> ** **
>
> Debbie****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Mark Davis ?
>
> *Sent:* 07 July 2011 15:43
> *To:* Mykyta Yevstifeyev
> *Cc:* Pete Resnick; [email-masked]
> *Subject:* Re: [Ltru] Fwd: draft-davis-t-langtag-ext****
>
> ** **
>
> it seems odd.****
> On Wed, Jul 6, 2011 at 20:55, Mykyta Yevstifeyev <[email-masked]>
> wrote:****
>
> Hello,
>
>    The subtags in the 't' extension are of the following form:****
>
> ** **
>
>      +--------+-------------------------+----------------------------+****
>
>      | Label  | ABNF                    | Comment                    |****
>
>      +--------+-------------------------+----------------------------+****
>
>      | t_ext= | "t"                     | Extension                  |****
>
>      |        | ("-" lang *("-" field)  | Source + optional field(s) |****
>
>      |        | / 1*("-" field))        | Field(s) only (no source)  |****
>
>      | lang=  | language                | [BCP47 <http://tools.ietf.org/html/draft-davis-t-langtag-ext-01#ref-BCP47>], with restrictions |****
>
>      |        | ["-" script]            |                            |****
>
>      |        | ["-" region]            |                            |****
>
>      |        | *("-" variant)          |                            |****
>
>      | field= | sep 1*("-" 3*8alphanum) | With restrictions          |****
>
>      | sep=   | 1ALPHA 1DIGIT           | Subtag separators          |****
>
>      +--------+-------------------------+----------------------------+****
>
>
>    The subtags in the 't' extension are of the following form, defined****
>
>    using ABNF [RFC5234] in <t-ext> rule:****
>
> ** **
>
>      t-ext    = "t" ("-" lang *("-" field) / 1*("-" field))****
>
>      lang     = langtag****
>
>      field    = sep 1*("-" 3*8alphanum)****
>
>      sep      = ALPHA DIGIT****
>
>      alphanum = ALPHA / DIGIT****
>
> ** **
>
>    where <langta> rule is specified in BCP 47 [BCP47], <ALPHA> and <DIGIT>****
>
>    rules - in RFC 5234 [RFC5234].****
>
>    [BCP47]    Phillips, A. and M. Davis, "Matching of Language Tags", ****
>
>               BCP 47, RFC 4647, September 2006.****
>
> ** **
>
>               Phillips, A., Ed., and M. Davis, Ed., "Tags for Identifying****
>
>               Languages", BCP 47, RFC 5646, September 2009.****
>
> Mykyta Yevstifeyev****
> 07.07.2011 2:49, Pete Resnick wrote: ****
>
> pr****
> https://www.ietf.org/mailman/listinfo/ltru****
>
> ** **
>

---

From: Pete Resnick <[email-masked]>
Date: Thu, 7 Jul 2011 15:22:42 -0500

On 7/7/11 10:29 AM, Peter Constable wrote:
> IESG? What is the action being requested of IESG / what's the status 
> of this draft?

It is intended to be published as an AD-sponsored individual submission 
and is currently marked as going for "Informational". I'm the AD on the 
hook. We may revisit the question of status later.

Publication of this document was requested and we decided that it needed 
to get some review on ltru and ietf-languages. As soon as the authors 
feel like they've gotten solid community review, I'll start the process 
of reviewing the document and bringing it through the IESG process.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Pete Resnick <[email-masked]>
Date: Thu, 7 Jul 2011 15:23:41 -0500

Publication of this document was requested and we decided that it needed 
to get some review on ltru and ietf-languages. (I just updated the data 
in the draft tracker to reflect that it is currently in "Expert Review", 
just as a reminder to me that it's being discussed here on the lists.) 
As soon as the authors feel like they've gotten solid community review, 
I'll start the process of reviewing the document and bringing it through 
the IESG process.

pr

---

From: Mark Davis ☕ <[email-masked]>
Date: Thu, 7 Jul 2011 13:26:18 -0700

There are two very different issues: votes of the technical committee, and
the voting process used in deriving consensus on submitted locale data for
each release.

You seem to be focusing on the later, which is not relevant for this issue.
But just to prevent further FUD on it, I'll take a minute. The voting on
data is by organization, with administrators in the organization responsible
for setting the voting level for participants. Some organizations are full
voting (and paying) members of the consortium, while others are representing
liaison (gratis) members. For more information, see
http://cldr.unicode.org/index/process.

As to the issue of "I seem to remember a lot of Google data being
incorporated (dumped) into CLDR", Google data, like others' data, has been
contributed into CLDR. There were some problems with translators'
understanding of instructions and context, but that happens all the time, so
the committee ends up going through successive data resolution phases to
mediate any differences and adjust for problems. The process isn't
perfect--but no process could be, especially when we have to handle the
volume of data with each release, with over 600 locales and up to 3,000
values per locale.

As to the IETF not being willing to work with membership organizations,
well, that would preclude the Unicode Consortium, as well as many other
organizations like the W3C. For that matter you'd have to pay a lot to be an
ISO member, since you'd have to start your own country ;-)

Mark

On Thu, Jul 7, 2011 at 10:00, Debbie Garside <[email-masked]>wrote:

> Let me elaborate further on this.  From memory, data in CLDR may be
> incorporated on a vote system,  This vote system is linked to the amount
> paid by companies by way of subscriptions.
>

> Or am I wrong?  I seem to remember a lot of Google data being incorporated
> (dumped) into CLDR against the wishes of experts in the field who had been
> working most diligently to make sure their data was correct.  A paid for
> votes system is contrary to the entire ethos of IETF where, I believe, a
> general consensus is required rather than paid votes and affiliations are
> not taken into account.  This is the basis for my concern.****
>
> ** **
>
> Maybe I am totally wrong and if so I apologise.  Please feel free to
> correct me.****
>
> ** **
>
> If I am right, there is still a way for Unicode to do this, as stipulated
> within the RFC, but rather by recommendation to the IETF-Languages Registrar
> who can then invite further discussion.****
>
> ** **
>
> Best wishes****
>
> ** **
>
> Debbie****
>
> ** **
>
> *Debbie Garside
> *Sent:* 07 July 2011 17:01
> *To:* 'Mark Davis ☕'; 'Mykyta Yevstifeyev'
>
> *Cc:* 'Pete Resnick'; [email-masked]
> include both references to RFC 5646 and RFC 4647, like:****
>

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 21:40:10 +0100

Thank you.  This confirms what I thought.  You pay your money you get a vote but individual members (such as me) get no say.

Sorry but I cannot agree with this.  It goes against the IETF ethos.  

Best regards

Debbie

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis ?
Sent: 07 July 2011 21:07
To: Debbie Garside
Cc: Mykyta Yevstifeyev; Pete Resnick; LTRU Working Group; CLDR list
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

These are not primary language subtags or variants, as covered by BCP47. That still remains as it has been.

This is a different area, part of an extension that is to provide a structured approach to the specification of transliteration. The development of extensions and their scope is covered by BCP47, and you can find the details there.

The Unicode CLDR committee is already serves as the registrar for the -u- extension. The committee operates under the procedures in http://unicode.org/consortium/tc-procedures.html, but the vast majority of decisions are taken by consensus. The consortium is already the Registration Authority for ISO 15924 (Scripts).

Mark

On Thu, Jul 7, 2011 at 09:00, Debbie Garside <[email-masked]> wrote:

Best wishes

Debbie

Sent: 07 July 2011 15:43
To: Mykyta Yevstifeyev

 

Mark

Hello,

Section 2.2 says:

 

 

 

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 22:13:43 +0100

But I still don't see why it needs to be taken out of IETF.  Where is the added value?  Where are discussions held?  Why create another list when we already have IETF-languages?

I am not trying to be obstructive and I am a supporter of Unicode but I do believe these sort of questions need answering.

I really like the IETF structure as I feel it is very inclusive whereas Unicode can be exclusive.  The damage that is done when controversial issues are raised and non-voting experts are over-ridden by those who pay for votes can actually put development back years as we lose those volunteers who work so fastidiously. I have heard from a colleague who has been instrumental in adding 100 locales to CLDR that many volunteers are disillusioned and have stopped contributing.  I would hate for IETF to find that they agree to "outsource" this work and then find that their volunteer experts disappear.

I really need to be shown the added value of outsourcing this work. I am open to being persuaded.

Best regards

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roozbeh Pournader
Sent: 07 July 2011 21:48
To: Debbie Garside
Cc: 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU Working Group'; 'CLDR list'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

Not at all.

You definitely get a say. You don't even need to be an individual member
to get a say. Then the responsible committee discusses the issue and
tries to arrive at consensus. In very few cases, there's a vote.

In my experience of being in contact with the Unicode Consortium for
more than a decade now (and I never had the right to vote), I'd say they
are very considerate of expert opinions. They are a meritocracy, not a
democracy of full members. Votes are only rarely used to decide about
very controversial issues. And there aren't many of those.

Roozbeh

On Thu, 2011-07-07 at 21:40 +0100, Debbie Garside wrote:
> Thank you.  This confirms what I thought.  You pay your money you get
> a vote but individual members (such as me) get no say.
> 
>  
> 
> These are not primary language subtags or variants, as covered by
> BCP47. That still remains as it has been.
> 
> 
>  
> 
> 
> structured approach to the specification of transliteration. The
> development of extensions and their scope is covered by BCP47, and you
> can find the details there.
> 
> 
>  
> 
> 
> The Unicode CLDR committee is already serves as the registrar for the
> -u- extension. The committee operates under the procedures in
> http://unicode.org/consortium/tc-procedures.html, but the vast
> majority of decisions are taken by consensus. The consortium is
> already the Registration Authority for ISO 15924 (Scripts).
> 
> 
>  
> 
> 
> Mark
> 
> 
> I am also concerned about the structure of the Unicode Committee and
> voting rights. Perhaps someone can explain how this will work and why
> it is required in addition to the current structure for the
> registration of language tags.
> 
>  
> 
> Have I missed something here? (I probably have as I have been away
> from the list for some time)  Have Unicode already taken over some of
> the duties of the BCP47 registrar?
> 
>  
> 
> Best wishes
> 
>  
> 
> Debbie
> 
>  
> 
> wrote:
>      | lang=  | language                | [BCP47], with restrictions |
>      |        | ["-" script]            |                            |
>      |        | ["-" region]            |                            |
>      |        | *("-" variant)          |                            |
> moreover, and this is more important, making the ABNF definition in
> the form of table makes such definition an invalid one, in terms of
> RFC 5234.  Also, there are a number of ABNF nits here.  So, please
> consider changing this to:
> 
> 
> 
> ...and, referencing UTS 35 you shouldn't reference specific parts of
> the document; this should be done in the text.  Finally, I don't see
> where [US-ASCII] is used in the text.
> 
> Thanks,
> Mykyta Yevstifeyev
> 
> 
>

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 22:23:43 +0100

Mark wrote:

>> The consortium is already the Registration Authority for ISO 15924 (Scripts).

My company (GeoLang) is RA for ISO 639-6 (language encoding) it doesn’t mean we should be the Registrar for an IETF extension subtag.

On the subject of ISO 639-6, I am ready to re-write RFC 5646 for their inclusion.  The data will be ready by the time the RFC is accepted.

Best regards

Debbi

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis ?
Sent: 07 July 2011 21:07
To: Debbie Garside
Cc: Mykyta Yevstifeyev; Pete Resnick; LTRU Working Group; CLDR list
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

 

Mark

Best wishes

Debbie

Sent: 07 July 2011 15:43
To: Mykyta Yevstifeyev

 

Mark

Hello,

Section 2.2 says:

 

 

 

Thanks,
Mykyta Yevstifeyev

Thanks.

pr

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 23:04:27 +0100

My enquiry is, why do we need Unicode/CLDR to be the registration authority for the -t extension?  We have a perfectly good registration authority for subtags that does not require any payment for votes and does not require subscribing to another list.  I have no problem with the -t extension mechanism as proposed (that I can see).

Admittedly, I haven't looked at BCP47 lately.  I was unaware of any changes to the application procedures for registration of subtags or extensions.  I obviously need to read it.

Best wishes

Debbie

-----Original Message-----
From: Roozbeh Pournader [mailto:[email-masked]] 
Sent: 07 July 2011 22:46
To: Debbie Garside
Cc: 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU Working Group'; 'CLDR list'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

On Thu, 2011-07-07 at 22:13 +0100, Debbie Garside wrote:
> But I still don't see why it needs to be taken out of IETF.  Where is
> the added value?  Where are discussions held?  Why create another list
> when we already have IETF-languages?

My understanding is that you are not objecting to the Unicode Consortium
registering authority for a BCP 47 extension subtag, but you are
objecting to BCP 47 section 3.7, right? You basically want to change BCP
47 section 3.7 to have more restrictions on the extension subtag
registering authorities, right?

Roozbeh

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 7 Jul 2011 23:13:19 +0100

I will say just say a couple of things on this and then will let it go.  I really am not about attacking either Unicode or CLDR as I believe I am still a member (and have been since they last printed a hard copy - whenever that was).

I believe that CLDR has lost experts due to a data dump from Google that overwrote their work.  I had a face to face conversation with a colleague involved whilst in Korea three weeks ago.

Saying that all the core people are still there after 20 years does not address the issue of paying for votes - they may have been paying for 20 years.  In any case, I believe CLDR was created some 7 years ago (or maybe 8).

One could ask, how many people are on the proposed CLDR committee and, of these, how many are not attached to paying organisations?

Best wishes

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roozbeh Pournader
Sent: 07 July 2011 22:58
To: Debbie Garside
Cc: 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU Working Group'; 'CLDR list'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

On Thu, 2011-07-07 at 22:13 +0100, Debbie Garside wrote:

> I have heard from a colleague who has been instrumental in adding 100
> locales to CLDR that many volunteers are disillusioned and have
> stopped contributing.

That's not because of committee voting. Maintaining and updating the
data in CLDR locales uses a vetting procedure vastly different from the
CLDR commitee itself. And I have seen the feedback from experts with
limited voting right incorporated not only in CLDR data, but also in
updates to the CLDR data vetting process. From what I've seen, most
volunteer experts objections has been about not being able to keep up
with the pace of data that comes from some full members. So, it's mostly
been volunteer contributors (myself included) trying to slow down the
process, instead of voting members. So contrary to what you think, it's
the organizational and data support from the voting members that makes
sure the process is fast enough. Not only it doesn't put development
back for years, it's usually volunteer contributors who want the process
slowed down so they can catch up.

Again, all of that is about "locale data", which is really very large
amount of data. I don't think we can use that experience to see how the
"t" extension will be maintained. A better comparison is how the Unicode
Consortium and the CLDR committee has been maintaining the
already-registered "u" extension. Do you know anybody who has had a
problem with that?

> I would hate for IETF to find that they agree to "outsource" this work
> and then find that their volunteer experts disappear.

Well, Unicode has been here for more than two decades, and I have yet to
see expertise disappear from it. Not only that, but also almost all the
core people are still contributing.

Roozbeh

---

From: Roozbeh Pournader <[email-masked]>
Date: Thu, 07 Jul 2011 13:47:32 -0700

Not at all.

Roozbeh

---

From: Roozbeh Pournader <[email-masked]>
Date: Thu, 07 Jul 2011 14:45:51 -0700

Roozbeh

---

From: Roozbeh Pournader <[email-masked]>
Date: Thu, 07 Jul 2011 14:58:04 -0700

Roozbeh

---

From: "Debbie Garside" <[email-masked]>
Date: Fri, 8 Jul 2011 09:01:51 +0100

Hi Steven

Thanks.  I really am not trying to criticise CLDR.  I understand (somewhat) the problems and the needs of industry.  As already mentioned, I am a supporter of both Unicode and CLDR.  I will ask my colleague to speak with you about his concerns.

My concern here on IETF-LTRU is that a process is being taken out of IETF unnecessarily IMHO - at least from the responses received so far, I can see no added value in CLDR functioning as the Registrar for -t extensions.

Best wishes

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Steven R. Loomis
Sent: 08 July 2011 02:28
To: Debbie Garside
Cc: 'Roozbeh Pournader'; 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU Working Group'; 'CLDR list'
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

Debbie,

I think that the concern about the data dump was due to some
 misunderstandings regarding the CLDR process. As the one who developed
 and manages major parts of the CLDR tooling (along with many others),
 I can say that the human users involved with the "data dump" (which
 was another misconception, that it was merely a one-way "dump") were
 very involved with the CLDR forum process.

We are hard at work to make the vetting process easier to use for
everyone. The sheer number of increased users and data the last time
around, brought to the forefront stability and performance issues that
were still unresolved. One of the issues is the amount of data that is
in a CLDR locale can be daunting, it would be in the tens of thousands
of data items. We've already introduced a system ('coverage') that lets
the user reduce what is shown normally to just critical items. This
system is already slated to be improved.  As well, we have some faster
hardware to run the server on that we will be testing out soon.

I realize that you are relaying a concern from a third party, but I
would invite your colleague to discuss the specific concerns with us if
they had not already.  One of the very exciting parts, for me, of this
process is that anyone regardless of other 'status' can (and does) sign
up and contribute data, and has a voice. Previous to the launch of the
CLDR project about eight years ago, this locale data existed in
multiple organization's repositories, where it would take a bug report
to cause any change.  Then I (and others in different companies,
independently) would have to look at the bug report and decide when and
if to spend time updating that data.  Now there is a process for
sorting out the data, and also a common repository and format for many
projects (both open-source and commercial) to pick up and use. It's not
a perfect process, but it's a process.

Regards,

Steven

On Thu, 7 Jul 2011 23:13:19 +0100
"Debbie Garside" <[email-masked]> wrote:

> I will say just say a couple of things on this and then will let it
> go.  I really am not about attacking either Unicode or CLDR as I
> believe I am still a member (and have been since they last printed a
> hard copy - whenever that was).
> 
> I believe that CLDR has lost experts due to a data dump from Google
> that overwrote their work.  I had a face to face conversation with a
> colleague involved whilst in Korea three weeks ago.
> 
> Saying that all the core people are still there after 20 years does
> not address the issue of paying for votes - they may have been paying
> for 20 years.  In any case, I believe CLDR was created some 7 years
> ago (or maybe 8).
> 
> One could ask, how many people are on the proposed CLDR committee
> and, of these, how many are not attached to paying organisations?
> 
> Best wishes
> 
> Debbie
> 
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Roozbeh Pournader Sent: 07 July 2011 22:58
> To: Debbie Garside
> Cc: 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU
> Working Group'; 'CLDR list' Subject: RE: [Ltru] Fwd:
> draft-davis-t-langtag-ext
> 
> > I have heard from a colleague who has been instrumental in adding
> > 100 locales to CLDR that many volunteers are disillusioned and have
> > stopped contributing.
> 
> data in CLDR locales uses a vetting procedure vastly different from
> the CLDR commitee itself. And I have seen the feedback from experts
> with limited voting right incorporated not only in CLDR data, but
> also in updates to the CLDR data vetting process. From what I've
> seen, most volunteer experts objections has been about not being able
> to keep up with the pace of data that comes from some full members.
> So, it's mostly been volunteer contributors (myself included) trying
> to slow down the process, instead of voting members. So contrary to
> what you think, it's the organizational and data support from the
> voting members that makes sure the process is fast enough. Not only
> it doesn't put development back for years, it's usually volunteer
> contributors who want the process slowed down so they can catch up.
> 
> amount of data. I don't think we can use that experience to see how
> the "t" extension will be maintained. A better comparison is how the
> Unicode Consortium and the CLDR committee has been maintaining the
> > I would hate for IETF to find that they agree to "outsource" this
> > work and then find that their volunteer experts disappear.
> 
> Well, Unicode has been here for more than two decades, and I have yet
> to see expertise disappear from it. Not only that, but also almost
> all the core people are still contributing.
> 
> Roozbeh

---

From: Peter Constable <[email-masked]>
Date: Sat, 9 Jul 2011 07:31:43 +0000

I am, but that has been inactive for so long I wasn’t paying attention to it.

Peter

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis ?
Sent: Thursday, July 07, 2011 12:58 PM
To: Peter Constable
Cc: [email-masked]; [email-masked]
Subject: Re: [Ltru] draft-davis-t-langtag-ext

It's still in discussion, so feedback is welcome. (I'd thought you were on LTRU...)

Mark

On Thu, Jul 7, 2011 at 08:29, Peter Constable <[email-masked]<mailto:[email-masked]>> wrote:
I was not aware of the discussion on LTRU. When will it be reviewed by IESG? What is the action being requested of IESG / what’s the status of this draft?

Peter

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Pete Resnick
Sent: Wednesday, July 06, 2011 4:49 PM
To: [email-masked]<mailto:[email-masked]>
Subject: Fwd: draft-davis-t-langtag-ext

Thanks.

pr

-------- Original Message --------
Subject:

Date:

From:

To:

CC:

Mark

Overall comment:

Detailled comments:

ok

yes

So much for the moment.

Regards,   Martin.

It just got posted at

Mark

...

My response:

Randy

---

From: Peter Constable <[email-masked]>
Date: Sat, 9 Jul 2011 07:34:35 +0000

I think this needs more discussion. In particular, I think there are speech scenarios that may have requirements similar to some transcription / transliteration scenarios, and that that should be considered before a proposal that covers only transcription and transliteration scenarios gets progressed.

Peter

From: Pete Resnick [mailto:[email-masked]]
Sent: Thursday, July 07, 2011 1:23 PM
To: Peter Constable
Cc: [email-masked]; [email-masked]
Subject: Re: [Ltru] draft-davis-t-langtag-ext

It is intended to be published as an AD-sponsored individual submission and is currently marked as going for "Informational". I'm the AD on the hook. We may revisit the question of status later.

Publication of this document was requested and we decided that it needed to get some review on ltru and ietf-languages. As soon as the authors feel like they've gotten solid community review, I'll start the process of reviewing the document and bringing it through the IESG process.

pr

--

Pete Resnick <http://www.qualcomm.com/~presnick/><http://www.qualcomm.com/~presnick/>

---

From: "Broome, Karen" <[email-masked]>
Date: Sat, 9 Jul 2011 17:41:43 -0700

I tend to agree with Debbie on this. I'm not sure this is best handled by an external, if tightly coupled, organization. This seems to be a useful extension to the main body of work so it seems like it should be handled in the same way.

Regards,

Karen Broome

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Debbie Garside
Sent: Friday, July 08, 2011 6:02 PM
To: 'Steven R. Loomis'
Cc: 'LTRU Working Group'; 'CLDR list'; 'Pete Resnick'; 'Roozbeh Pournader'
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

Hi Steven

Best wishes

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Steven R. Loomis
Sent: 08 July 2011 02:28
To: Debbie Garside
Cc: 'Roozbeh Pournader'; 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete Resnick'; 'LTRU Working Group'; 'CLDR list'
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

Debbie,

Regards,

Steven

---

From: "Phillips, Addison" <[email-masked]>
Date: Sat, 9 Jul 2011 18:36:48 -0700

> I tend to agree with Debbie on this. I'm not sure this is best handled by an
> external, if tightly coupled, organization. This seems to be a useful extension to
> the main body of work so it seems like it should be handled in the same way.

There is a fundamental problem with that, though.

The extension mechanism can be used to create a registry under the auspices of the IETF that is managed by IANA using the ietf-languages list. That requires Internet-Draft(s) be created laying out the process, rules, format, structure, etc. etc. for the registry. We know from experience how much effort is required to complete such work. Note that such an extension would still be a separate registry and would be managed separately (even if it were to use the same mail list, for example).

The extension mechanism also can be used (indeed, given the foregoing, is optimized for use) by standards bodies or other organizations that maintain or are willing to create and maintain language-tag-extending standards or registries. In this case, one such body (the CLDR-TC of the Unicode Consortium) has requested under the BCP 47 rules that the IESG to assign it one of the 34 remaining singletons for an extension that they will maintain.

CLDR-TC is already the maintainer of one language tag extension, so it probably meets at least meet a minimal bar for fitness as such. Given the overhead for creating a different process, isn't it reasonable to use the CLDR process and Unicode's willingness to maintain the registry for this purpose? If there are concerns about the openness of the process, etc., I believe they can be addressed in the Internet-Draft.

A separate question is whether the current proposal is adequate/appropriate for the task, including such stuff as the creation of a single registry. The other authors are open to discussing these things and making changes. Is that objectionable as an approach?

Addison

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Broome, Karen
> Sent: Saturday, July 09, 2011 5:42 PM
> To: Debbie Garside; 'Steven R. Loomis'
> Cc: 'Pete Resnick'; 'Roozbeh Pournader'; 'CLDR list'; 'LTRU Working Group'
> I tend to agree with Debbie on this. I'm not sure this is best handled by an
> external, if tightly coupled, organization. This seems to be a useful extension to
> the main body of work so it seems like it should be handled in the same way.
> 
> Regards,
> 
> Karen Broome
> 
> Thanks.  I really am not trying to criticise CLDR.  I understand (somewhat) the
> problems and the needs of industry.  As already mentioned, I am a supporter of
> both Unicode and CLDR.  I will ask my colleague to speak with you about his
> concerns.
> 
> My concern here on IETF-LTRU is that a process is being taken out of IETF
> unnecessarily IMHO - at least from the responses received so far, I can see no
> added value in CLDR functioning as the Registrar for -t extensions.
> 
> Best wishes
> 
> Debbie
> 
> 
> 
> misunderstandings regarding the CLDR process. As the one who developed  and
> manages major parts of the CLDR tooling (along with many others),  I can say
> that the human users involved with the "data dump" (which  was another
> misconception, that it was merely a one-way "dump") were  very involved with
> the CLDR forum process.
> 
> We are hard at work to make the vetting process easier to use for everyone.
> The sheer number of increased users and data the last time around, brought to
> the forefront stability and performance issues that were still unresolved. One of
> the issues is the amount of data that is in a CLDR locale can be daunting, it
> would be in the tens of thousands of data items. We've already introduced a
> system ('coverage') that lets the user reduce what is shown normally to just
> critical items. This system is already slated to be improved.  As well, we have
> some faster hardware to run the server on that we will be testing out soon.
> 
> I realize that you are relaying a concern from a third party, but I would invite
> your colleague to discuss the specific concerns with us if they had not already.
> One of the very exciting parts, for me, of this process is that anyone regardless
> of other 'status' can (and does) sign up and contribute data, and has a voice.
> Previous to the launch of the CLDR project about eight years ago, this locale
> data existed in multiple organization's repositories, where it would take a bug
> report to cause any change.  Then I (and others in different companies,
> independently) would have to look at the bug report and decide when and if to
> spend time updating that data.  Now there is a process for sorting out the data,
> and also a common repository and format for many projects (both open-source
> and commercial) to pick up and use. It's not a perfect process, but it's a process.
> 
> Regards,
> 
> Steven
> 
> 
> 
> > One could ask, how many people are on the proposed CLDR committee and,
> > of these, how many are not attached to paying organisations?
> >
> > Best wishes
> >
> > Debbie
> >
> > with limited voting right incorporated not only in CLDR data, but also
> > in updates to the CLDR data vetting process. From what I've seen, most
> > with the pace of data that comes from some full members.
> > voting members that makes sure the process is fast enough. Not only it
> > doesn't put development back for years, it's usually volunteer
> > to see expertise disappear from it. Not only that, but also almost all
> > the core people are still contributing.
> >
> > Roozbeh
> 
> 
> 
> 
>

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 10 Jul 2011 15:47:48 +0100

Hi Addison

I'm not sure that I see where the "overhead" is.  Can you elaborate?  

BCP47 states:

IANA will maintain a registry of allocated single-character
   (singleton) subtags.  This registry MUST use the record-jar format
   described by the ABNF in Section 3.1.1.  Upon publication of an
   extension as an RFC, the maintaining authority defined in the RFC
   MUST forward this registration form to <[email-masked]>, who MUST
   forward the request to <[email-masked]>.  The maintaining authority of
   the extension MUST maintain the accuracy of the record by sending an
   updated full copy of the record to <[email-masked]> with the subject
   line "LANGUAGE TAG EXTENSION UPDATE" whenever content changes.  Only
   the 'Comments', 'Contact_Email', 'Mailing_List', and 'URL' fields MAY
   be modified in these updates.

This IMHO is the overhead.  Given that we already have ietf-languages as a discussion list, and all the rules and regulations for application and registration of subtags, why create another "private" process? CLDR-TC would have to create transparent rules and regs anyway.

Keeping everything in one place/list has got to be easier for the end user.  I still cannot see any real reason for taking this out of IETF other than BCP47 allows for it.  

I can see the proposed -t extension mechanism being widely used by a number of organisations, unlike the -u extension which was specifically designed for integrating CLDR data within a subtag.  I don't think we can compare the two processes.  

Best wishes

Debbie

-----Original Message-----
From: Phillips, Addison [mailto:[email-masked]] 
Sent: 10 July 2011 02:37
To: Broome, Karen; Debbie Garside; 'Steven R. Loomis'
Cc: 'Pete Resnick'; 'Roozbeh Pournader'; 'CLDR list'; 'LTRU Working Group'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

Addison

> Debbie Garside

---

From: Kent Karlsson <[email-masked]>
Date: Sun, 10 Jul 2011 17:08:06 +0200

>    be modified in these updates.

That is for registering that there at all are certain approved extensions.
It provides *no* data about the subtags "under" any extension.

> This IMHO is the overhead.  Given that we already have ietf-languages as a
> discussion list, and all the rules and regulations for application and
> registration of subtags, why create another "private" process? CLDR-TC would
> have to create transparent rules and regs anyway.

The IANA language subtag registry has no way of registering what goes
"under" an extension. Only things preceding any extension (and separately
a list of approved singletons for extensions). Changing this would mean
revising BCP 47. That would be a major hurdle.

> Keeping everything in one place/list has got to be easier for the end user.  I
> still cannot see any real reason for taking this out of IETF other than BCP47
> allows for it.  

As Addison mentioned, one can have a separate registry for an extension that
is held by IANA. But that would mean defining lots of details on exactly
goes into the registry, its format, procedures for approval, etc. That is
NOT covered by BCP 47.

> I can see the proposed -t extension mechanism being widely used by a number of
> organisations, 

I can agree with that part.

    /Kent K

> unlike the -u extension which was specifically designed for
> integrating CLDR data within a subtag.  I don't think we can compare the two
> processes.  
> 
> Best wishes
> 
> Debbie

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 10 Jul 2011 16:17:51 +0100

Can I just clarify...

I am reading BCP47 with regard to the rules for sending IANA info on extension subtags.  The wording is not clear.  Does the paragraph (below) mean that every time there is a new part to an extension mechanism that the maintaining authority sends this to IANA? Or does it mean that just the details of (in the case) the -t singleton are sent to IANA?

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Debbie Garside
Sent: 10 July 2011 15:48
To: 'Phillips, Addison'; 'Broome, Karen'; 'Steven R. Loomis'
Cc: 'Pete Resnick'; 'Roozbeh Pournader'; 'CLDR list'; 'LTRU Working Group'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

Hi Addison

BCP47 states:

Best wishes

Debbie

-----Original Message-----
From: Phillips, Addison [mailto:[email-masked]] 
Sent: 10 July 2011 02:37
To: Broome, Karen; Debbie Garside; 'Steven R. Loomis'
Cc: 'Pete Resnick'; 'Roozbeh Pournader'; 'CLDR list'; 'LTRU Working Group'
Subject: RE: [Ltru] Fwd: draft-davis-t-langtag-ext

Addison

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 10 Jul 2011 16:31:50 +0100

Thanks Kent

I can see that I have not interpreted BCP47 correctly.

I think, taking in the comments from Karen, Doug and others, what I may be
happier with in the current scenario (Unicode as maintaining authority) is
if we could include within the RFC an obligation for Unicode to post any
requests for subtags relating to the -t extension to the ietf-languages list
for discussion, in addition to their own internal discussions, and that the
opinions of the participants on this (aforementioned) list carry the same
weight as they do within IETF. Protection for IETF participants would come
via the IESG (in that IETF participants can ask that the role of maintaining
authority for the -t extension be taken from Unicode if their views are not
considered appropriately).  Also, the resulting -t extension subtag record
should be posted to ietf-languages once agreed.

Further, the registry should be mirrored by IANA and linked to the subtag
registry.

Is this possible?

Best wishes

Debbie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Kent Karlsson
Sent: 10 July 2011 16:08
To: Debbie Garside; 'Phillips, Addison'; 'Broome, Karen'; 'Steven R. Loomis'
Cc: 'Pete Resnick'; Roozbeh Pournader; 'CLDR list'; 'LTRU Working Group'
Subject: Re: [Ltru] Fwd: draft-davis-t-langtag-ext

    /Kent K

---

From: "Phillips, Addison" <[email-masked]>
Date: Sun, 10 Jul 2011 08:50:30 -0700

The operative sentence in the paragraph you quote is the first one:

>    (singleton) subtags.  

That is, the paragraph has to do with a registry containing a list of the extensions and where to find them. 

The actual extensions are located somewhere else. 

Addison

> From: Debbie Garside [mailto:[email-masked]]
> Sent: Sunday, July 10, 2011 8:18 AM
> To: 'Debbie Garside'; Phillips, Addison; 'Broome, Karen'; 'Steven R. Loomis'
> I am reading BCP47 with regard to the rules for sending IANA info on extension
> subtags.  The wording is not clear.  Does the paragraph (below) mean that
> every time there is a new part to an extension mechanism that the maintaining
> authority sends this to IANA? Or does it mean that just the details of (in the
> case) the -t singleton are sent to IANA?
> 
> Debbie
> 
> discussion list, and all the rules and regulations for application and registration
> of subtags, why create another "private" process? CLDR-TC would have to
> create transparent rules and regs anyway.
> 
> Keeping everything in one place/list has got to be easier for the end user.  I still
> cannot see any real reason for taking this out of IETF other than BCP47 allows
> for it.
> 
> I can see the proposed -t extension mechanism being widely used by a number
> of organisations, unlike the -u extension which was specifically designed for
> > I tend to agree with Debbie on this. I'm not sure this is best handled
> > by an external, if tightly coupled, organization. This seems to be a
> > useful extension to the main body of work so it seems like it should be
> handled in the same way.
> 
> The extension mechanism can be used to create a registry under the auspices
> of the IETF that is managed by IANA using the ietf-languages list. That requires
> Internet-Draft(s) be created laying out the process, rules, format, structure, etc.
> etc. for the registry. We know from experience how much effort is required to
> complete such work. Note that such an extension would still be a separate
> registry and would be managed separately (even if it were to use the same
> mail list, for example).
> 
> The extension mechanism also can be used (indeed, given the foregoing, is
> optimized for use) by standards bodies or other organizations that maintain or
> are willing to create and maintain language-tag-extending standards or
> registries. In this case, one such body (the CLDR-TC of the Unicode Consortium)
> has requested under the BCP 47 rules that the IESG to assign it one of the 34
> remaining singletons for an extension that they will maintain.
> 
> CLDR-TC is already the maintainer of one language tag extension, so it
> probably meets at least meet a minimal bar for fitness as such. Given the
> overhead for creating a different process, isn't it reasonable to use the CLDR
> process and Unicode's willingness to maintain the registry for this purpose? If
> there are concerns about the openness of the process, etc., I believe they can
> be addressed in the Internet-Draft.
> 
> A separate question is whether the current proposal is adequate/appropriate
> for the task, including such stuff as the creation of a single registry. The other
> authors are open to discussing these things and making changes. Is that
> objectionable as an approach?
> 
> Addison
> 
> 
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Broome, Karen
> > I tend to agree with Debbie on this. I'm not sure this is best handled
> > by an external, if tightly coupled, organization. This seems to be a
> > useful extension to the main body of work so it seems like it should be
> handled in the same way.
> >
> > Regards,
> >
> > Karen Broome
> >
> > Thanks.  I really am not trying to criticise CLDR.  I understand
> > (somewhat) the problems and the needs of industry.  As already
> > mentioned, I am a supporter of both Unicode and CLDR.  I will ask my
> > colleague to speak with you about his concerns.
> >
> > My concern here on IETF-LTRU is that a process is being taken out of
> > IETF unnecessarily IMHO - at least from the responses received so far,
> > I can see no added value in CLDR functioning as the Registrar for -t
> extensions.
> >
> > Best wishes
> >
> > Debbie
> >
> >
> >
> > Behalf Of Steven R. Loomis
> > Sent: 08 July 2011 02:28
> > To: Debbie Garside
> > Cc: 'Roozbeh Pournader'; 'Mark Davis ☕'; 'Mykyta Yevstifeyev'; 'Pete
> > Resnick'; 'LTRU Working Group'; 'CLDR list'
> > The sheer number of increased users and data the last time around,
> > brought to the forefront stability and performance issues that were
> > still unresolved. One of the issues is the amount of data that is in a
> > CLDR locale can be daunting, it would be in the tens of thousands of
> > data items. We've already introduced a system ('coverage') that lets
> > would invite your colleague to discuss the specific concerns with us if they
> had not already.
> > One of the very exciting parts, for me, of this process is that anyone
> > regardless of other 'status' can (and does) sign up and contribute data, and
> has a voice.
> > Previous to the launch of the CLDR project about eight years ago, this
> > locale data existed in multiple organization's repositories, where it
> > would take a bug report to cause any change.  Then I (and others in
> > different companies,
> > independently) would have to look at the bug report and decide when
> > and if to spend time updating that data.  Now there is a process for
> > projects (both open-source and commercial) to pick up and use. It's not a
> perfect process, but it's a process.
> >
> > Regards,
> >
> > Steven
> >
> >
> >
> > > not address the issue of paying for votes - they may have been
> > > paying for 20 years.  In any case, I believe CLDR was created some 7
> > > years ago (or maybe 8).
> > >
> > > > 100 locales to CLDR that many volunteers are disillusioned and
> > > > have stopped contributing.
> > >
> > > seen, most volunteer experts objections has been about not being
> > > able to keep up with the pace of data that comes from some full members.
> > > Again, all of that is about "locale data", which is really very
> > > large amount of data. I don't think we can use that experience to
> > > see how the "t" extension will be maintained. A better comparison is
> > > how the Unicode Consortium and the CLDR committee has been
> > > maintaining the already-registered "u" extension. Do you know
> > > anybody who has had a problem with that?
> > >
> > > Well, Unicode has been here for more than two decades, and I have
> > > yet to see expertise disappear from it. Not only that, but also
> > > almost all the core people are still contributing.
> > >
> > > Roozbeh
> >
> >
> >
> >
> >

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 21 Jul 2011 19:01:53 +0900

Hello Addison,

I agree in principle with what you say. However, my impression is that 
the -u extension, by its name and nature, was much more a Unicode/CLDR 
only thing than the -t extension. In particular, my guess is that there 
might be quite a bit more interest for the -t extension than for the -u 
extension from academic and related communities. I don't want to say 
that the average IETF registry process (as e.g. used for language 
variants) is easy for outsiders, but I'd say that the Unicode process is 
quite a bit less transparent, and would probably take more time because 
lots of decisions are taken at meetings, and updates happen in big 
versioned batches.

That's why I tend to push back on general arguments of the form "it 
worked for -u, it'll work for -t". I may be convinced by specifics.

Regards,    Martin.

On 2011/07/10 10:36, Phillips, Addison wrote:
>> "Debbie Garside"<[email-masked]>  wrote:
>>