Subject: [apps-discuss] Fw: draft-pbryan-zyp-json-pointer: name syntax for non-ASCII

From: "t.petch" <[email-masked]>
Date: Thu, 24 Nov 2011 18:15:30 +0100

---- Original Message -----
 From: "Graham Klyne" <[email-masked]>
 To: "Martin J. Dürst" <[email-masked]>
 Cc: "Mark Nottingham" <[email-masked]>; "IETF Apps Discuss"
 <[email-masked]>
 Sent: Tuesday, November 22, 2011 12:26 PM

> I spotted this discussion and was reminded that one of the older URI specs had
> some explicit discussion of characters and octets and encoding.  I recall a
line
> that looked something like this:
>
>    URI characters -> URI octets -> URI octets %-encoded to US-ASCII
>
> but I can no longer find it (quickly).  But
http://www.ietf.org/rfc/rfc3986.txt
> sections 1.2 and section 2 (esp. intro) address some of the issues.  The point
> being that character encoding to octets is a separate concern from %-encoding
to
> URI (or IRI) on-the-wire characters.
>

 My bible for this is RFC2130 which gives

 character->coded character set->character encoding scheme->transfer encoding
 syntax

 which Unicode seemed to get spot on, but HTML and URIs ... um

 Tom Petch

<resent - where did [email-masked] go? >
>
> #g
> --
>
> On 22/11/2011 09:09, "Martin J. Dürst" wrote:
> > On 2011/11/22 8:43, Mark Nottingham wrote:
> >> The usual approach to this sort of thing is to define the "canonical" way
to
> >> do it; i.e., json pointers *are* unicode strings; then you can define
> >> encodings into various environments (like URIs).
> >>
> >> In this case, it'd probably be good enough to say that json pointers are
> >> unicode strings,
> >
> > Up to here, this makes a ton of sense.
> >
> >> but that when they need to be in ASCII environments (like URIs) they get
> >> UTF-8'ed and then percent-escaped.
> >
> > This would mean that e.g. in a Java program that for some reason is kept in
> > US-ASCII, I'd have to use %-encoding. This doesn't make sense to me at all.
> >
> > So I'd say that json pointers are escaped according to the conventions of
the
> > substrate that carries them when needed (e.g. pure ASCII, or other kinds of
> > encodings that can't handle the whole Unicode range).
> >
> > Then for json pointers as fragment identifiers, I'd mention that where
> necessary
> > (e.g. for URIs), the convention for converting from IRIs to URIs (see RFC
> 3987)
> > applies.
> >
> > By the way, I don't see a need to escape "/" at all in a fragment
identifier.
> > "/" is plain and simply allowed in fragment identifiers. Please see
> > http://tools.ietf.org/html/rfc3986#section-3.5. Of course, it's not
forbidden
> to
> > escape "/", so software that is interpreting a fragment identifier has to
make
> > sure it does the right thing.
> >
> > Regards, Martin.
> >
> >
> >> Cheers,
> >>
> >>
> >> On 22/11/2011, at 8:51 AM, Paul C. Bryan wrote:
> >>
> >>> Okay, so I'll write-up separate sections for JSON string values and URI
> >>> fragment identifiers. Any objections?
> >>>
> >>> Paul
> >>>
> >>> On Tue, 2011-11-22 at 07:55 +1100, Mark Nottingham wrote:
> >>>> +1 to Julian here -- there's no reason why non-ASCII chars need to be
> >>>> percent-encoded when they occur inside a JSON document, only when they're
> in
> >>>> a URI (or similar context).
> >>>>
> >>>> Cheers,
> >>>>
> >>>>
> >>>> On 22/11/2011, at 7:12 AM, Julian Reschke wrote:
> >>>>
> >>>>> On 2011-11-21 21:09, Paul C. Bryan wrote:
> >>>>>> The intent is to allow a JSON Pointer to be expressed as a JSON string
> >>>>>> value as well as a URI fragment identifier. The latter is the most
> >>>>>> significant driver for URI percent-encoding.
> >>>>>> ...
> >>>>>
> >>>>> Well, you could use it as fragment identifier (or otherwise URI
component)
> >>>>> by UTF-8-percent-escaping.
> >>>>>
> >>>>> The question is whether that use case requires them to be all ASCII
every
> >>>>> else, such as in a JSON patch document.
> >>>>>
> >>>>> Best regards, Julian
> >>>>> _______________________________________________
> >>>>> apps-discuss mailing list
> >>>>>
> >>>> [email-masked]
> >>>>
> >>>>>
> >>>> https://www.ietf.org/mailman/listinfo/apps-discuss
> >>>>
> >>>>
> >>>> --
> >>>> Mark Nottingham
> >>>> http://www.mnot.net/
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> apps-discuss mailing list
> >>> [email-masked]
> >>> https://www.ietf.org/mailman/listinfo/apps-discuss
> >>
> >> --
> >> Mark Nottingham http://www.mnot.net/
> >>
> >>
> >>
> >> _______________________________________________
> >> apps-discuss mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/apps-discuss
> >>
> > _______________________________________________
> > apps-discuss mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/apps-discuss
> >
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>
>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 28 Nov 2011 20:22:53 +0900

On 2011/11/25 2:15, t.petch wrote:
>   From: "Graham Klyne"<[email-masked]>

>   which Unicode seemed to get spot on,

Of course, because Unicode is a character encoding.

> but HTML and URIs ... um

They are not character encodings, but work on a higher level. HTML 
pretty well conforms to http://www.w3.org/TR/charmod/#sec-RefProcModel, 
the Reference Processing Model of the Character Model for the World Wide 
Web. See also RFC 2070.

For URIs, the situation is indeed murky, because escaping (%-encoding) 
is on the octet level, rather than on the character level, and depending 
on the component, the character->octet mapping is undefined. But for new 
stuff, such as the syntax discussed in this thread, the character->octet 
mapping just has to be fixed to UTF-8, which brings that part of an URI 
mostly in line with the above model.

Regards,    Martin.