Subject: Re: [apps-discuss] I-D Action: draft-ietf-appsawg-file-scheme-02.txt

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 4 Jun 2015 06:24:57 -0700

On Thu, May 28, 2015 at 12:18 AM, <[email-masked]> wrote:

>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>  This draft is a work item of the Applications Area Working Group Working
> Group of the IETF.
>
>         Title           : The file URI Scheme
>         Author          : Matthew Kerwin
>         Filename        : draft-ietf-appsawg-file-scheme-02.txt
>         Pages           : 19
>         Date            : 2015-05-28
>
> Abstract:
>    This document specifies the "file" Uniform Resource Identifier (URI)
>    scheme, obsoleting the definition in RFC 1738.
>
>    It attemps to define a common core which is intended to interoperate
>    across the broad spectrum of existing implementations, while at the
>    same time documenting other current practices.
>

Colleagues,

There have been no comments posted on this version of the draft.  The
previous version appeared to address Tom Petch's earlier concerns.  Have
any others looked at this one?  Does it need more work or are we ready to
go to WGLC here?

Dave Crocker has offered to act as document shepherd.

-MSK, APPSAWG co-chair

---

From: Graham Klyne <[email-masked]>
Date: Thu, 04 Jun 2015 17:25:56 +0100

Apologies for responding late in the day.  I had intended to review this but it 
fell out of sight.  I am duly reminded and offer some comments below.

...

Section 1 vs Section 3 appear to be in conflict:

(sect 1): "... there is no well-defined set of operations that can be performed 
on file URIs..."

and

(sect 3): "Implementations SHOULD, at a minimum, provide a read-like operation 
to return the contents of a file located by a file URI."

I expect the intent was to say different things, but I find the result is 
confusing.  Suggest drop the confusing bit in section 1, para 2, hence:

[[
The file URI scheme is not coupled with a specific protocol, nor with a specific 
media type.  See section 3 for a discussion of operations that can be performed.
]]

...

Section 3.2:

[[
A file URI can only be translated to a local file path if it has a
    blank or no authority.  Note that this differs from the previous
    specification in [RFC1738], in that previously an authority of
    "localhost" was used to refer to the local file system, but in this
    specification it equates to a UNC string with the host "localhost".
]]

I'm uneasy with this non-backwards-compatible change of specification.  In the 
past, I have actually used the "localhost" convention, so that software would be 
rendered incorrect by this revised spec.

Further, I'm not sure what it means to say "it equates to a UNC string with the 
host "localhost"", expecially on systems that don't use UNC strings.  (The 
reference given for UNC strings, [MS-DTYP], is only informative so I guess that 
means this whole section is non-normative?

Also, this incorporation of UNC string processing appears to be at odds with 
"This specification neither defines nor forbids a mechanism for accessing 
non-local files.  See SMB [MS-SMB], NFS [RFC3530], NCP [NOVELL] for examples of 
protocols that can be used to access files over a network." (which I think is fine).

...

Section 4:

"When the file system's encoding is not known the file URI SHOULD be transported 
as an Internationalized Resource Identifier (IRI) [RFC3987] to avoid ambiguity."

I'm not seeing how this helps.

(1) are we talking about the source system or destination system file system 
encoding being unknown?  I'll assume the source.

(2) If the file system name encoding is unknown, how is one to achieve a mapping 
to IRIs, which are defined to be a sequence of characters?

(In practice, I have found the main utility of file: URIs has not been for 
transportation of complete URIs between systems, but providing a mechanism for 
resolving relative references to the local file system, whatever that may be. 
In such a scenario, the base file: URI is determined locally and used to 
generate a full URI from a supplied URI reference, which can in turn be used to 
access the local file system.  Here it is the relative references that are 
transported.  Encoding considerations still apply, but I don't see that there's 
any reliable way to make this work if the encoding of source file names is not 
known.  My approach has been to %-encode anything that is provided by the file 
system that is not a known valid URI (as opposed to IRI) character.  How this is 
interpreted at the target system is somewhat dependent on the nature of the 
system and application concerned.)

I'm not claiming my approach solves the compatibility/ambiguity problem, but I 
also don't think that just saying "send IRIs" does either.  I think we have to 
acknowledge that there may be problems when transferring filenames whose 
original encoding is unknown.

...

That's all for now.

#g
--

On 04/06/2015 14:24, Murray S. Kucherawy wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 5 Jun 2015 10:20:30 +1000

Thanks for the review Graham (and thanks for the reminder Murray, I'd
forgotten to post a follow-up.)

On 5 June 2015 at 02:25, Graham Klyne <[email-masked]> wrote:

> Apologies for responding late in the day.  I had intended to review this
> but it fell out of sight.  I am duly reminded and offer some comments below.
>
> ...
>
> (sect 1): "... there is no well-defined set of operations that can be
> performed on file URIs..."
>
> and
>
> (sect 3): "Implementations SHOULD, at a minimum, provide a read-like
> operation to return the contents of a file located by a file URI."
>
> The file URI scheme is not coupled with a specific protocol, nor with a
> specific media type.  See section 3 for a discussion of operations that can
> be performed.
> ]]
>
>
​Yes, I think that's the right thing to do. Partly the text is still in two
minds about the original "the only operation is translation to/from a file
path" and the new "file URIs are CRUDable."

My personal problem is with the phrase "well-defined"... I don't know if "a
read-like operation..." counts as well-defined or not. Irrespective, I'll
try to write something more accurate in section 1.

> I'm uneasy with this non-backwards-compatible change of specification.  In
> the past, I have actually used the "localhost" convention, so that software
> would be rendered incorrect by this revised spec.
>
>
​This point swayed back and forth several times in the earliest revisions.
It eventually settled on the current version because most of the
implementations I'd seen acted that way anyway. Which software supported
the originally-specced localhost, if I may ask?​

> Further, I'm not sure what it means to say "it equates to a UNC string
> with the host "localhost"", expecially on systems that don't use UNC
> strings.  (The reference given for UNC strings, [MS-DTYP], is only
> informative so I guess that means this whole section is non-normative?
>
>
There are two points here which I'd be happy to have cleared up:

1. editorial: I need a better way to phrase this paragraph (in both its
occurrences.) If someone can suggest some text I'd be most obliged.

2. substantive: I'm thinking I should move all mention of UNC (apart from
S1.2) to the optional appendices. Is that cool with everybody? It gives the
above point a lower priority, too.

> Also, this incorporation of UNC string processing appears to be at odds
> with "This specification neither defines nor forbids a mechanism for
> accessing non-local files.  See SMB [MS-SMB], NFS [RFC3530], NCP [NOVELL]
> for examples of protocols that can be used to access files over a network."
> (which I think is fine).
>
>
​As I read it (which might be wrong) parsing or translating UNC strings
doesn't necessarily mean accessing network shares.​ However I think that
moving UNC to an optional appendix reaffirms the written assertion
("...neither defines nor forbids..."), rather than confusing it (as the
current text does.)

> ...
>
> Section 4:
>
> "When the file system's encoding is not known the file URI SHOULD be
> transported as an Internationalized Resource Identifier (IRI) [RFC3987] to
> avoid ambiguity."
>
> (1) are we talking about the source system or destination system file
> system encoding being unknown?  I'll assume the source.
>
>
​Yes, the source. The issue arises at the time of interpreting the URI, and
we assume the parser/interpreter and the destination file system share
knowledge about things like encodings -- so that encoding is known.

> (2) If the file system name encoding is unknown, how is one to achieve a
> mapping to IRIs, which are defined to be a sequence of characters?
>
>
​When I encode, I can (presumably) map my local encoding to Unicode, and
thus create an unambiguous IRI. This is (local)character ->
(unicode)codepoint, without needing a %-encoded intermediate.

When I decode, I can (presumably) map Unicode to my local encoding. Again,
this is (unicode)codepoint -> (local)character, without a %-encoded
interiedmate. There might be codepoints in the IRI that can't be mapped to
my local encoding, in which case there's a problem - but it's a known
problem. That's no different, really, from including bad %XX sequences in a
URI.

As the example in Appendix D says, if I'm an NTFS system (for example) and
the URI is "file:///%E3%81%A1" and I don't know anything about who encoded
it, I don't know if that's a single UTF-8 character, or three CP437
characters, or three EBCDIC characters, any of which could have been a
valid filename at the source. The IRIs "file://ち", "file://πüí" and
"file://Ta~" are mostly unambiguous.

> transportation of complete URIs between systems, but providing a mechanism
> for resolving relative references to the local file system, whatever that
> may be. In such a scenario, the base file: URI is determined locally and
> used to generate a full URI from a supplied URI reference, which can in
> turn be used to access the local file system.  Here it is the relative
> references that are transported.  Encoding considerations still apply, but
> I don't see that there's any reliable way to make this work if the encoding
> of source file names is not known.  My approach has been to %-encode
> anything that is provided by the file system that is not a known valid URI
> (as opposed to IRI) character.  How this is interpreted at the target
> system is somewhat dependent on the nature of the system and application
> concerned.)
>
>
​The example is the same as above, bu​t without the "file://" part. If, for
argument's sake, you have two hypertext documents with "interesting" file
names, that reference each other relatively, and you're going to copy them
around between FAT, NTFS, HFS and Ext devices, how can you be sure those
relative references can be properly resolved at any given time (on any
given device)?

> I'm not claiming my approach solves the compatibility/ambiguity problem,
> but I also don't think that just saying "send IRIs" does either.  I think
> we have to acknowledge that there may be problems when transferring
> filenames whose original encoding is unknown.
>
>
​Only if you want to reconstruct the filename as it was on the source file
system. But I can't think of a time we'd want to do that.

> #g
​Thanks!​

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Graham Klyne <[email-masked]>
Date: Fri, 05 Jun 2015 08:53:36 +0100

Responses inline...

(I'll probably be out of contact for a day or so after this, so I may not be 
able to respond promptly.  I think we're converging :) )

> try to write something more accurate in section 1.

OK.  Here, in the intro, maybe less is more?

> the originally-specced localhost, if I may ask?​

It wasn't a public product, just software I wrote.

> above point a lower priority, too.

I think that moving to an appendix would go a long way to alleviating my concern 
here.

> ("...neither defines nor forbids..."), rather than confusing it (as the

Ack.

> (unicode)codepoint, without needing a %-encoded intermediate.

I was coming from a viewpoint that there was No knowledge of the filename 
encoding used.  Your text mentions "Note that many modern file systems encode 
directory and file names as arbitrary sequences of octets.  In those cases, the 
representation as an encoded string often depends on the user's localization 
settings, or defaults to UTF-8"

So yes, if the character encoding is known, it makes sense to use Unicode/UTF-8 
for transmission, and the rest works as you suggest.  On closer reading, your 
text does suggest a path forward, but I think it might be clearer ...

I think part of my confusion here is that the section title (is it a verb or a 
noun?) combined with the first sentence pointed me in the wrong direction.  And 
the final para seems to compound my confusion.

Suggest:

First para of section:

[[
To avoid ambiguity, 'file:' URIs SHOULD be transported as Internationalized 
Resource Identifiers (IRI) [RFC3987] or as URIs with non-ASCII characters UTF-8 
encoded and %-escaped as needed (RFC3986, section 2.5).
]]

Then adjust what follows to fall in line with this.  I think that's the intent 
of what you're trying to say here?

#g
--