Subject: Re: [apps-discuss] I-D Action: draft-ietf-appsawg-file-scheme-02.txt

From: Matthew Kerwin <[email-masked]>
Date: Tue, 21 Jul 2015 08:51:35 +1000

Hi Graham, my apologies for leaving it so long to reply.

On 05/06/2015 5:53 PM, "Graham Klyne" <[email-masked]> wrote:

<snip>

>
>>
>>> ...
>>>
>>> Section 3.2:
>>>
>>> [[
>>> A file URI can only be translated to a local file path if it has a
>>>     blank or no authority.  Note that this differs from the previous
>>>     specification in [RFC1738], in that previously an authority of
>>>     "localhost" was used to refer to the local file system, but in this
>>>     specification it equates to a UNC string with the host "localhost".
>>> ]]
>>>
>>> I'm uneasy with this non-backwards-compatible change of specification.
In
>>> the past, I have actually used the "localhost" convention, so that
software
>>> would be rendered incorrect by this revised spec.
>>>
>>>
>> ​This point swayed back and forth several times in the earliest
revisions.
>> It eventually settled on the current version because most of the
>> implementations I'd seen acted that way anyway. Which software supported
>> the originally-specced localhost, if I may ask?​
>>
>
> It wasn't a public product, just software I wrote.
>

I guess all I have to go with here, at the moment, are anecdotes one way or
the other. What would be the right way to move forward? A tech survey?

>
>>
>>> Further, I'm not sure what it means to say "it equates to a UNC string
>>> with the host "localhost"", expecially on systems that don't use UNC
>>> strings.  (The reference given for UNC strings, [MS-DTYP], is only
>>> informative so I guess that means this whole section is non-normative?
>>>
>>>
>> There are two points here which I'd be happy to have cleared up:
>>
>> 1. editorial: I need a better way to phrase this paragraph (in both its
>> occurrences.) If someone can suggest some text I'd be most obliged.
>>
>> 2. substantive: I'm thinking I should move all mention of UNC (apart from
>> S1.2) to the optional appendices. Is that cool with everybody? It gives
the
>> above point a lower priority, too.
>
>
> I think that moving to an appendix would go a long way to alleviating my
concern here.

That's done in my github copy, modulo a bit of editorial cleaning up.

>
>>
>>> ...
>>>
>>> Section 4:
>>>
>>> "When the file system's encoding is not known the file URI SHOULD be
>>> transported as an Internationalized Resource Identifier (IRI) [RFC3987]
to
>>> avoid ambiguity."
>>>
>>> I'm not seeing how this helps.
>>>
>>> (1) are we talking about the source system or destination system file
>>> system encoding being unknown?  I'll assume the source.
>>>
>>>
>> ​Yes, the source. The issue arises at the time of interpreting the URI,
and
>> we assume the parser/interpreter and the destination file system share
>> knowledge about things like encodings -- so that encoding is known.
>>

I should have said here: if the source filesystem's encoding is unknown to
the recipient of the URI...

I think, now, this isn't an issue: the URI author writes the URI for the
recipient, so the author's filesystem has little relevance. If the
recipient's encoding is unknown to the author, and the author is playing
nicely, there's no practical difference between an IRI and a %-encoded URI.

>>
>>> (2) If the file system name encoding is unknown, how is one to achieve a
>>> mapping to IRIs, which are defined to be a sequence of characters?
>>>
>>>
>> ​When I encode, I can (presumably) map my local encoding to Unicode, and
>> thus create an unambiguous IRI. This is (local)character ->
>> (unicode)codepoint, without needing a %-encoded intermediate.
>
>
> I was coming from a viewpoint that there was No knowledge of the filename
encoding used.  Your text mentions "Note that many modern file systems
encode directory and file names as arbitrary sequences of octets.  In those
cases, the representation as an encoded string often depends on the user's
localization settings, or defaults to UTF-8"

Ah, yeah, it hadn't occurred to me that an app creating a URI could do so
by looking at the name of a file created by some other user who happened to
have an unpredictable l10n env.

I... don't actually have a solution for this. What happens now, even in
non-URI contexts, if an application encounters a filename that isn't valid
utf-8?

>
> So yes, if the character encoding is known, it makes sense to use
Unicode/UTF-8 for transmission, and the rest works as you suggest.  On
closer reading, your text does suggest a path forward, but I think it might
be clearer ...
>
> I think part of my confusion here is that the section title (is it a verb
or a noun?) combined with the first sentence pointed me in the wrong
direction.  And the final para seems to compound my confusion.
>
> Suggest:
>
> First para of section:
>
> [[
> To avoid ambiguity, 'file:' URIs SHOULD be transported as
Internationalized Resource Identifiers (IRI) [RFC3987] or as URIs with
non-ASCII characters UTF-8 encoded and %-escaped as needed (RFC3986,
section 2.5).
> ]]
>
> Then adjust what follows to fall in line with this.  I think that's the
intent of what you're trying to say here?

I lost track at some point of what I was trying to say. I think it was Dave
Thaler who suggested I use IRIs instead of URIs wherever possible; I asked:

"[...] are “file URLs” in Windows in fact not IRIs, but something even more
general (allowing non-Unicode code pages)?"

And Dave answered:

"URIs are not necessarily %HH-encoded UTF-8.   They can be %HH-encoded
<something else>.   Sometimes the charset (code page) is known.  Sometimes
it isn’t known at the point you get a URI, and you cannot simply assume
it’s in the system code page, since it may have come from outside the local
system."

So I guess Dave's argument was: %-encoded URIs aren't always UTF-8/Unicode,
but IRIs are always Unicode, so use IRIs.

If we could guarantee that all %-encoded URIs were UTF-8 I don't think
there would be a problem.

Cheers

> #g
> --

---

From: Graham Klyne <[email-masked]>
Date: Tue, 21 Jul 2015 11:26:24 +0100

> Hi Graham, my apologies for leaving it so long to reply.

np

> the other. What would be the right way to move forward? A tech survey?

My suggestion would be to start with a presumption of no incompatible changes, 
but to allow the change is there's a fairly strong consensus to apply it (e.g. 
if the change itself represents a significant body of actual use).

> That's done in my github copy, modulo a bit of editorial cleaning up.

Ack. Thanks.

> nicely, there's no practical difference between an IRI and a %-encoded URI.

I agree, and I _think_ (it's been a while) that's why I raised the point.

> non-URI contexts, if an application encounters a filename that isn't valid

I think that's a matter for the application, if the spec focuses on what 
actually gets exchanged.

> but IRIs are always Unicode, so use IRIs.

Ah, interesting point.  Dave's right (cf RFC3986 sect 2).  But my sense is that 
folks are trying to move away from that position; e.g. RFC3986 also says:

[[
    The URI producer will transform the
    local encoding to one that is suitable for a public interface and
    then transform the public interface encoding into the restricted set
    of URI characters (reserved, unreserved, and percent-encodings).
]]

and

[[
    A system that internally provides identifiers in the form of a
    different character encoding, such as EBCDIC, will generally perform
    character translation of textual identifiers to UTF-8 [STD63] (or
    some other superset of the US-ASCII character encoding) at an
    internal interface, thereby providing more meaningful identifiers
    than those resulting from simply percent-encoding the original
    octets.
]]

and

[[
    When a new URI scheme defines a component that represents textual
    data consisting of characters from the Universal Character Set [UCS],
    the data should first be encoded as octets according to the UTF-8
    character encoding [STD63]; then only those octets that do not
    correspond to characters in the unreserved set should be percent-
    encoded.
]]

-- http://tools.ietf.org/html/rfc3986#section-2.5

I think my suggested text is in the spirit of what you're trying to achieve, 
just more explicit that it's OK to %-encode the non-US-ASCII characters of an IRI.

> there would be a problem.

Yes indeed!  (We maybe can't do that, but it seems reasonable to try and nudge 
people in that direction.)

#g

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 23 Jul 2015 15:39:43 +1000

On 21 July 2015 at 20:26, Graham Klyne <[email-masked]> wrote:

> My suggestion would be to start with a presumption of no incompatible
> changes, but to allow the change is there's a fairly strong consensus to
> apply it (e.g. if the change itself represents a significant body of actual
> use).
>
>
Actually, looking into it more, it seems Chrome on Windows is the only one
that interprets "localhost" as a samba server, and then only sometimes.
(IE11 doesn't seem to like the name "localhost" even in constructs like
"file:////localhost/" ...)

In that case I think I really should revert the change, and make
"localhost" mean what it used to. I'll do that in the next revision.

> Ah, interesting point.  Dave's right (cf RFC3986 sect 2).  But my sense is
> that folks are trying to move away from that position; e.g. RFC3986 also
> says:
>
> [[
> I think my suggested text is in the spirit of what you're trying to
> achieve, just more explicit that it's OK to %-encode the non-US-ASCII
> characters of an IRI.
>
>
​Okay, fair enough, I'll take your text. Maybe with a friendly warning
along the lines of what Dave was saying.

> Yes indeed!  (We maybe can't do that, but it seems reasonable to try and
> nudge people in that direction.)
>
> #g
>
>
​Cheers

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/