Subject: [weirds] Internationalization Issues

From: "Ning Kong" <[email-masked]>
Date: Fri, 12 Oct 2012 12:04:59 +0800

Hi all,

I'm still concerned about the i18n issues of WEIRDS. I notice that the i18n
problem is explicitly raised by the Charter, but AFAIK this problem is not
discussed too much on the mailing list or the current WG documents.

I'd like to start this thread focused on i18n issues in order to make sure
whether the text in the Internationalization Considerations Section of
using-http-00 needs to be improved. On the other hand, I hope we can find
some missing important things which should be covered within this topic
through this thread.

IMHO, indicating language/script in response may be a potential missing
thing of i18n issues. I wonder whether the 7.3 section of
draft-sheng-weirds-icann-rws-dnrd-01 is worth being reconsidered and covered
by the using-http draft.

Any thoughts or comments on this topic are appreciated.

Cheers,
Ning

---

From: Andy Newton <[email-masked]>
Date: Thu, 18 Oct 2012 17:30:45 +0000

>draft-sheng-weirds-icann-rws-dnrd-01 is worth being reconsidered and

This sounds like a good idea. But is there any reference to an explanation
of what client software is to do with these language tags?

-andy

---

From: Byron Ellacott <[email-masked]>
Date: Fri, 19 Oct 2012 10:40:40 +1000

Hi Andy,

> of what client software is to do with these language tags?

Indicate to the end user that it's not a native language?  Auto-translate?  Negotiate for native language with Accepts-Language, if indicated as possible via a Vary header?

Some RDAP services will not support multiple languages meaningfully, but there are existing whois services that provide (non-standard, varying) ways to indicate a preferred language on query, with multiple language options available for many response fields.

  Byron

---

From: Andy Newton <[email-masked]>
Date: Fri, 19 Oct 2012 11:33:59 +0000

On 10/18/12 8:40 PM, "Byron Ellacott" <[email-masked]> wrote:

>Indicate to the end user that it's not a native language?

Can you explain this further?

>  Auto-translate?

This seems to be the number 1 use case I have found while searching the
interwebs. I would say that automatic language translation of social data
is next to useless and sometimes harmful, and doing so with technical data
is inappropriate.

>  Negotiate for native language with Accepts-Language, if indicated as
>possible via a Vary header?

That's HTTP layer stuff. We're talking about embedding multiple language
tags in the response.

>Some RDAP services will not support multiple languages meaningfully, but
>there are existing whois services that provide (non-standard, varying)
>ways to indicate a preferred language on query, with multiple language
>options available for many response fields.

Can you provide an example of one of these services so we can query it?
That would go a long way in helping shape this need, I would think. Are
there registries that collect contact data in multiple languages?

-andy

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 20 Oct 2012 21:12:15 -0700

> Can you explain this further?

<participant>

I take this to mean something like an indication that "Your client is
configured for French, but the reply appears to be in Farsi," for
example.

> is next to useless and sometimes harmful, and doing so with technical data

Is this still true if the client indicates to the viewer that such
translation happened?  Or only do so when explicitly requested to do
so?

</participant>

-MSK

---

From: Byron Ellacott <[email-masked]>
Date: Mon, 22 Oct 2012 10:56:23 +1000

>> Indicate to the end user that it's not a native language?

Murray has the right sense of what I meant, for both of these.

> That's HTTP layer stuff. We're talking about embedding multiple language

Are we?  I thought draft-sheng-weirds-icann-rws-dnrd-01 sect. 7.3
suggested a single language tag for the entire response, with "possible
considerations" of multiple language tags.

But, with this point, what I'm suggesting is that the user of a particular
client likely has one or a few preferred languages, which they could
potentially indicate to the server, in the event that the server has multiple
translations.  This would be applicable for mixed language responses as
well as single language responses, since it only indicates a client preference,
not a strict requirement.

My primary perspective on this entire subject is that whatever mechanisms
or systems indicate language or language preference need to be optional,
and should support reasonable use cases for current or likely future operators.
I think there's a use case for language preference indication, as per below,
and I think Ning is suggesting a use case for tagging the language of an
entire response, inline in the response.  What are your (collective "your")
thoughts on how reasonable these use cases are?

> there registries that collect contact data in multiple languages?

$ whois -h whois.nic.ad.jp -- 'NET 113.32.19.157'
$ whois -h whois.nic.ad.jp -- 'NET 113.32.19.157 /e'

$ whois -h whois.jprs.jp -- 'jprs.jp'
$ whois -h whois.jprs.jp -- 'jprs.jp /e'

The data labels are sometimes translated, sometimes not.  In the native
language responses, there's often an English translation.  JPRS includes
an English help/info block even for the native language response.  I don't
know for sure if they collect the information in multiple languages, though
I think they do - any JPRS or JPNIC operators on the list to confirm?
Character set is ISO-2022-JP.

I don't know if there are other services with such a switch mechanism,
either - we're all aware of how hard it is to find out what's actually done
out there on port 43 :-) - but for another comparison whois.kisa.kr returns
both native and english output, at least for "kisa.kr".  Character set is
EUC-KR.

  Byron

---

From: Peter Koch <[email-masked]>
Date: Mon, 22 Oct 2012 13:44:44 +0200

> and should support reasonable use cases for current or likely future operators.

where does _language_ come into play?  The tags are going to be standardized,
the identifiers are just that. Could you elaborate?

-Peter

---

From: Byron Ellacott <[email-masked]>
Date: Tue, 23 Oct 2012 09:30:38 +1000

> the identifiers are just that. Could you elaborate?

The contents of fields such as remarks, names, and server notices may all be in languages other than English.  In some cases, the contents of fields such as address may also be in a language other than English.  The field tag names will not be localized, and identifiers aren't in any language, per se, of course.

Speaking as an operator, I will not take advantage of any language tagging features in the protocol, because here at APNIC we don't have any meta-information about language in our registration data.  CNNIC may - Ning can speak to this - and both JPRS and KISA likely may as well, judging by their port 43 services.

  Byron

---

From: Peter Koch <[email-masked]>
Date: Tue, 23 Oct 2012 06:57:58 +0200

> The contents of fields such as remarks, names, and server notices may all be in languages other than English.  In some cases, the contents of fields such as address may also be in a language other…

and we are sure that this is about language, not script or character set?
What language tagging would be appropriate for your or my name?

-Peter

---

From: Byron Ellacott <[email-masked]>
Date: Tue, 23 Oct 2012 16:59:33 +1000

Hi Peter,

> and we are sure that this is about language, not script or character set?

$ whois -h whois.jprs.jp -- 'jprs.jp'  # warning, ISO-2022-JP, not UTF-anything
…
[登録者名]                      株式会社日本レジストリサービス
[Registrant]                    Japan Registry Services Co.,Ltd.
…

These two fields are the same information.  In RDAP, the data tag would be the same, but the content might be expressed either as the Japanese text or as the English text.  I think this would be best handled with a language tag, but if I'm mistaken, please correct me.  In XML, I'd expect to see something like:

<dnrd:registrant>
	<dnrd:name xml:lang="ja">株式会社日本レジストリサービス</dnrd:name>
	<dnrd:name xml:lang="en">Japan Registry Services Co.,Ltd.</dnrd:name>
</dnrd:registrant>

I'm not sure how I'd express this in JSON, but right now I'm just wondering whether the use case exists to do it, and putting forward the things I'm personally aware of that might argue in favour of it.

I don't know when, why, or how you might signal the script used, as opposed to the language.  Character set for RDAP would normally be UTF-8, as per draft-weirds-using-http-01 s.12.2, though one might use HTTP signalling to change that.

  Byron

---

From: Andy Newton <[email-masked]>
Date: Tue, 23 Oct 2012 14:05:36 +0000

>considerations" of multiple language tags.

I was assuming multiple tags as in the uses of xml:lang.

>and should support reasonable use cases for current or likely future

I totally agree. I favor working on this but just want to understand it
better.

>From what I can tell, the /e just redacts information. Regardless, this
isn't the same thing as a language label. It is more akin to the Accept
header in HTTP.

-andy

---

From: Andy Newton <[email-masked]>
Date: Tue, 23 Oct 2012 14:06:03 +0000

>configured for French, but the reply appears to be in Farsi," for

That seems to be a reasonable use case.

-andy

---

From: Steve Sheng <[email-masked]>
Date: Tue, 23 Oct 2012 07:52:50 -0700

Hi Peter, 

  We discussed this issue extensively within ICANN community.

  The proposal is that users should be allowed to submit registration data
in his/her native language / script to the extent allowed by the
registrar.  

  In terms of which fields to internationalize, the proposed requirements
are: 

  - Domain Name: U-label and A-label
  - Nameserver: U-label and A-label
  - Registrar: US-ASCII
  - Registrant Name: Local language with language-script tags (e.g. Using
tags as defined by RFC 5646)
  - Registrant contact information: local language with language-script
tags

  One issues that is in active discussion within ICANN is in the case a
registrant submit information in his/her local language/script, whether
there should a requirement to translate and transliterate (into latin)
this information. The policy community is starting a policy development
process on that. 

 For more information, you could refer to the ICANN Final Report on
Internationalized Registration Data
(http://gnso.icann.org/en/issues/ird/final-report-ird-wg-07may12-en.pdf),
and the policy body's approval of this report and initiation of relevant
policy work (Resolution No. 5 in
http://gnso.icann.org/en/resolutions/#201210).

 Hope this helps. 

Steve

-----Original Message-----
From: Peter Koch <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: [weirds] Internationalization Issues

>> The contents of fields such as remarks, names, and server notices may
>>all be in languages other than English.  In some cases, the contents of
>>fields such as address may also be in a language other than English.
>>The field tag names will not be localized, and identifiers aren't in any
>>language, per se, of course.
>
>_______________________________________________
>weirds mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/weirds

---

From: Byron Ellacott <[email-masked]>
Date: Wed, 24 Oct 2012 09:16:03 +1000

Hi Andy,

> isn't the same thing as a language label. It is more akin to the Accept

I don't believe there's any information in the Japanese language version that's not also in the English-only version, it's just stripping the bilingual duplication.  I could see this as a case for language negotiation in the headers, or as a case for multiple language options in the response, or a mix.  Try querying nic.ad.jp for "help" and "help /e" to see a starker contrast, but we don't do help responses in RDAP.

The main point to note is that not only do the labels change (a client-side issue for RDAP) but the contents are available in more than one language.  See also whois.kisa.kr responses, which have the full response in Korean, then repeated in English.

  Byron

---

From: James Mitchell <[email-masked]>
Date: Wed, 24 Oct 2012 12:25:53 +1100

As a heads up, EPP does not provide mechanisms for clients to specify the
language of contact registration data as it appears in the <postalInfo>
element with attribute type="loc".

Any work on internationalized registration data should consider the
changes required to EPP interfaces to allow capture of language
information.

James

---

From: Kentaro Mori <[email-masked]>
Date: Wed, 24 Oct 2012 11:59:01 +0900

Byron-san and folks,

I'm Kentaro Mori from JPRS.
(sorry for late reply)

For Whois, JPRS collects English data as well as native (Japanese) one
at the time registrant applies to .JP domain name registration.
Additionally, the English data doesn't cover all of Japanese data,
e.g., it is partial.
ISO-2022-JP as character-set was a normal choice when JPRS (more
correctly, JPNIC at that time) started Whois service, though we may have
alternative choice such as UTF-8 now.

--Kentaro

---

From: MasaYUKI Okada <[email-masked]>
Date: Wed, 24 Oct 2012 14:09:43 +0900

Byron-san,

I'm Masayuki Okada at JPNIC.

Our WHOIS system(whois.nic.ad.jp:43) will response English(ASCII)
 if '/e' is put into the character of the last of an input. 

By default, it answers in Japanese(ISO-2022-JP). 

About main registration records, JPNIC are collecting both Japanese 
and English as an mandantory.

just for reference:
  - How to use JPNIC WHOIS - 
   http://www.nic.ad.jp/en/db/whois/

--
Masayuki Okada
JPNIC

---

From: Byron Ellacott <[email-masked]>
Date: Wed, 24 Oct 2012 15:12:58 +1000

Thank you both for your input.  If you are intending to implement a weirds service down the line, what would be your expectations on language support in the protocol?

  Byron

-- 
Byron Ellacott                  email:           [email-masked]
Technical Director, APNIC       sip:        [email-masked]
http://www.apnic.net            phone:         +61 7 3858 3100
________________________________________________________________________
 * Sent by email to save paper. Print only if necessary.

---

From: Kentaro Mori <[email-masked]>
Date: Wed, 24 Oct 2012 19:01:12 +0900

Byron-san;

> Thank you both for your input.  If you are intending to implement a weirds service down the line, what would be your expectations on language support in the protocol?

'My current' expectations on language support are like follows:
(Frankly saying, I haven't completely caught up on the context/concept
of the protocol, so if they won't make sense for your arguments sorry
for that, and there is a possibility that I change my mind as my
understanding develops)

- default language (or character code) specification feature
In a case client has no ability to specify which language it prefers,
it would be not so good for server to return data of all language it
supports other than to return data by default language, as I believe
vast majority of whois.jprs.jp user won't expect to obtain data by 'both
of Japanese and English' or 'English only' (so you would be a very rare
user :)
; and I wonder this is a protocol stuff or not?

- language (or character code) tag for each element
There is a possibility some element has data by multiple languages (such
as Japanese and English) and the other element has data by a single
language (such as Japanese).  If language tag is for a set of elements,
client will have no clue for determining which element of one language
corresponds to which element of other language.

- character code specification feature
In Japan, sometimes client software encounters failure in automatic
judgment of encoding character code such as between ISO-2022-JP and
Shift-JIS, so apparent indication of encoding by server will be welcomed.

Thanks for questioning,

Kentaro

---

From: MasaYUKI Okada <[email-masked]>
Date: Thu, 25 Oct 2012 15:19:22 +0900

Byron-san,

This is only my opinion.  My expectations on language support as
this time is follow line:
(There is no difference of opinion with the Kentaro-san too.)

- Specified response code at the time of inquiry

Now we are changing the response by the end of the '/e' string.

It may be sufficient to give the '/e' string to the end of the 
inquiry also the protocol under discussion(draft-ietf-weirds-rdap-query-00??)
 now.

However, anticipation of standardization, should not an adhoc approach 
like this '/e' might be good for us. 
It may be able to specify the response code when whois queried.

Thank you for the question.

-- 
Masayuki Okada
JPNIC

---

From: "Ning Kong" <[email-masked]>
Date: Thu, 25 Oct 2012 16:09:25 +0800

Hi Byron and all,

> Speaking as an operator, I will not take advantage of any language tagging
> features in the protocol, because here at APNIC we don't have any
> meta-information about language in our registration data.  CNNIC may - Ning
> can speak to this - and both JPRS and KISA likely may as well, judging by their
> port 43 services.

The current Whois system in CNNIC supports IDNs query through either U-labels or A-labels. Furthermore, the system also supports displays of U-label, A-label and variant domain names of the queried domain. As for registration information, CNNIC accepts information in either English or Chinese. The Whois system adopts UTF-8 encoding format and display the information according to users’ input, either in English or Chinese. So, a response from our Whois system may contain data in two languages (English and Chinese).

IMHO, there may be two potential use cases which may make sense.

1. Query with language tags

I propose this use case based on Byron's email [1]. If a registry supports registration information in multiple languages or has multiple translations, language tags can be added into query as language preference indication, such as http://example.com/domain/test.com?lang=zh-cn or in the HTTP header. The Whois server may try to return the response in user's preferred language. If a user queries with a language tag which is not supported by the server, the server may return the response in its default language or an alert message.
Based on the practices of JP and KR, I think this use case is reasonable to be considered by us.

2. Response with language tags

If a response from a Whois server contains i18n data, I think one or multiple language tags within the response may help the user better understand the response content, especially when the user is not familiar with the language(s) used in the response.
This use case sounds reasonable. I'm still not yet. I wonder if you guys think it's useful to indicate the language used by the response data for users. In other words, if you see "مرکز اطلاعات شبکه اینترنت چین" from one Whois reponse, do you hope there would be a corresponding language tag to tell you this info is in Farsi?
I notice that Google translation service can auto-detect languages for users. So I wonder if the second use case needs to be considered by WEIRDS.

BTW, if the language tags would be accepted by WEIRDS, I also think we should consider whether we need to correspondingly extend EPP protocols [2].

References:
[1] http://www.ietf.org/mail-archive/web/weirds/current/msg01743.html
[2] http://www.ietf.org/mail-archive/web/weirds/current/msg01765.html 

Cheers,
Ning