Subject: Re: [apps-discuss] "local convention" in draft-kerwin-file-scheme

From: Graham Klyne <[email-masked]>
Date: Tue, 14 Oct 2014 09:10:59 +0100

Hi Sean,

Something I make a lot of use of in code I've written over the past few years is 
using relative URI references that can be resolved with respect to either file: 
or http: base URIs.  I often seem to find myself creating cross-linked resources 
in a local file system which are destined for web publication.

Using Python, the problem of  "/" vs "\" is largely resolved for me - Python 
allows "/" for file path construction, even on Windows.  Non-US-ASCII characters 
can be problematic, but assuming UTF-8 filenames and applying %-escaping often 
seems to work.  Windows device names often don't arise in this context, but when 
they do treating (say) "C:\file" as "file:///C:/file" often works, though the 
original looks to a URI resolver like a relative URI reference, which can cause 
problems.

The high order bit of my comment here is to say that local resources and 
"Internet" resources are not disjoint classes - there is also some need for 
mapping between them.  So the suggestion "do whatever at the endpoints, it 
doesn't affect the network" doesn't entirely work.  And for my work, I 
absolutely do need to be able to parse a file: URI (often without code knowing 
it's actually a filename.)

One thing that has not proven to be an issue for me is contextualization file 
file: URIs - i.e. URIs file file://computername/file - in my work, the context 
is implicitly the localhost (I sometimes resolve against file://localhost/) - 
that's part of what the file: scheme gives me.

I'm not sure how well this can all sit with what you're proposing, but its a 
datum for your consideration.

I'd love to see a regularization of the file: scheme that put my practices on a 
firmer foundation, but past attempts have floundered on the details.  I'm 
sympathetic to the idea of focusing on the common cases and treating the rest as 
extensions (as someone here suggested) if that helps get an updated file: scheme 
specification through the gate.

#g
--

On 29/09/2014 19:35, Sean Leonard wrote:
> Colleagues:
>
> We ought to step back for a moment and ask ourselves what exactly the goals are
> of getting a common file URI scheme through the IETF process.
>
> Ultimately we are all here to promote "interoperability". There is a lot of
> software out there that needs to identify local resources in the OS file system
> in the same data type (URI) as Internet resources. The way to distinguish these
> local resources in the URI way is--at a minimum--to tack on "file:".
>
> File system resources are normally addressed by the operating system's
> convention. We can come up with all sorts of notations, but at the end of the
> day the OS system call (POSIX open, Win32 CreateFile, etc.) is what matters. The
> OS needs to be consistent with itself and with applications on it; it's also
> important that two instances of the same OS (or OS families) agree on things, so
> that applications and data written with that OS (family) in mind can work on
> different machines.
>
> But from the perspective of Internet architecture, all those things are a "local
> convention"--that magical escape that in IETF-speak means "do whatever at the
> endpoints, it doesn't affect the network". draft-kerwin-file-scheme talks about
> local conventions but the overall gist is trying to stuff local conventions into
> a common format, and flailing about by addressing these disparate use cases
> (mostly Windows, but some OpenVMS) that have cropped up.
>
> It is not surprising, by the way, that Windows causes difficulties while
> Unix/POSIX is less problematic, since the URI convention of "/" for path
> separators and "/" as the root path directly descends from Unix/POSIX (and from
> FTP [RFC 959]).
>
> I would like to suggest a totally different approach:
> 1. file: identifies local filesystem resources (this includes filesystem
> resources accessible via network paths such as UNC, since the filesystem
> facility of the OS accommodates them).
> 2. # is the fragment.
> 3. everything else (the RFC 3986 <hier-part> and <query> productions) is a local
> convention.
>
> The draft can still include all of the current content, but move it into
> informative appendices.
>
> If an application encounters a file URI for a local filesystem resource, it
> should deal with it using the local convention. If an application encounters a
> file URI for some non-local filesystem resource, it should *leave it alone*. The
> end.
>
> In other words: why bother parsing a file URI if you know it's not going to
> refer to a resource that you can make use of? If you're on Ubuntu 14.04, why do
> you care about interpreting a file: URI that's meant for Windows 7? The only
> thing this does is invite security problems, because attackers will tell you
> (via the network) to perform operations on local resources that they don't have
> the authority to say anything about.
>
> Another issue is the matter of non-ASCII character encoding, which has always
> been dicey. [POSIX] is very clear that a paths as a sequence of *bytes*, not
> characters. Other than "/", <NUL>, and the relative pathnames "." and "..", it's
> anything-goes. The mapping between bytes and characters depends on the locale.
> In Windows NT, the kernel treats pathnames as Unicode (specifically
> UCS-2/UTF-16); translations are done in user-space. In Mac OS X, the HFS Plus
> file system converts all file names to decomposed Unicode, i.e., Normalization
> Form D [HFSPLUSREF].
>
> It is kind of seen as a moral imperative in the IETF to use UTF-8 (and
> Normalization Form C), possibly because of [BCP18] and [RFC5198]. But here, the
> only thing common is diversity--and each operating system (vendor) has its own
> good reasons for its individual decisions. Since file: URIs are always stored
> *in context*, context (the operating system documentation, the LC_ALL
> environment variable, the declared encoding of the text document, the identity
> of the local machine, etc.) provides all the info that you need. Just say that
> the file: URI (outside of the fragment component) encodes with local
> conventions--which are provided by context--and be done with it.
>
> If you are taking things like file: URIs out of context...three suggestions:
>
> 1. Translate the file URI into the destination context. For example: if you are
> in an HTML editor and you really want to reference <file:///D:/pics/catpic.jpg>,
> but you are saving the HTML document in a workgroup share, change the reference
> to <file://mycomputername/pics/catpic.jpg>, and ensure that D:\pics is shared as
> \\mycomputername\pics with appropriate access. Better yet: upload the resource
> to an Internet-accessible location such as <http://i.imgur.com/ziZNc2s.jpg>,
> which for purposes of Internet architecture, provides a more universal context.
>
> 2. Dereference the file URI and put whatever content you need into the context.
> For example: in an HTML e-mail, don't use <img src="file:///D:/catpic.jpg">; use
> <img src="cid:catpic93298@myemail"> and put the cat pic in the e-mail per
> [RFC2392] and [RFC2387].
>
> 3. Don't do that. It doesn't make sense.
>
> Best regards,
>
> Sean
>
> [RFC959]: https://tools.ietf.org/html/rfc959
> [POSIX]: http://pubs.opengroup.org/onlinepubs/9699919799/
> [BCP18]: http://tools.ietf.org/html/bcp18
> [RFC5198]: http://tools.ietf.org/html/rfc5198
> [HFSPLUSREF]: https://developer.apple.com/library/mac/qa/qa1235/_index.html
> [RFC2392]: http://tools.ietf.org/html/rfc2392
> [RFC2387]: http://tools.ietf.org/html/rfc2387
>
>
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Eric Burger <[email-masked]>
Date: Tue, 14 Oct 2014 05:52:17 -0400

Rather than trying to force fit new semantics on file:, is there any reason not to create a new, different scheme, that does what people want?

Today’s file: is pretty much a local matter.

Perhaps there is justification for a ‘nfile:’ scheme, with a network-normalized-but-local-OS access method?

On Oct 14, 2014, at 4:10 AM, Graham Klyne <[email-masked]> wrote:

> Hi Sean,
> 
> Something I make a lot of use of in code I've written over the past few years is using relative URI references that can be resolved with respect to either file: or http: base URIs.  I often seem to find myself creating cross-linked resources in a local file system which are destined for web publication.
> 
> Using Python, the problem of  "/" vs "\" is largely resolved for me - Python allows "/" for file path construction, even on Windows.  Non-US-ASCII characters can be problematic, but assuming UTF-8 filenames and applying %-escaping often seems to work.  Windows device names often don't arise in this context, but when they do treating (say) "C:\file" as "file:///C:/file" often works, though the original looks to a URI resolver like a relative URI reference, which can cause problems.
> 
> The high order bit of my comment here is to say that local resources and "Internet" resources are not disjoint classes - there is also some need for mapping between them.  So the suggestion "do whatever at the endpoints, it doesn't affect the network" doesn't entirely work.  And for my work, I absolutely do need to be able to parse a file: URI (often without code knowing it's actually a filename.)
> 
> One thing that has not proven to be an issue for me is contextualization file file: URIs - i.e. URIs file file://computername/file - in my work, the context is implicitly the localhost (I sometimes resolve against file://localhost/) - that's part of what the file: scheme gives me.
> 
> I'm not sure how well this can all sit with what you're proposing, but its a datum for your consideration.
> 
> I'd love to see a regularization of the file: scheme that put my practices on a firmer foundation, but past attempts have floundered on the details.  I'm sympathetic to the idea of focusing on the common cases and treating the rest as extensions (as someone here suggested) if that helps get an updated file: scheme specification through the gate.
> 
> #g
> --
>

---

From: Graham Klyne <[email-masked]>
Date: Tue, 14 Oct 2014 12:43:13 +0100

> Rather than trying to force fit new semantics on file:, is there any reason not to create a new, different scheme, that does what people want?
I don't think anyone is trying to force fit new semantics, just reflect existing 
semantics (and syntax) more consistently. I think use of file: is pretty deeply 
embedded in a number of existing applications, so IMO making a replacement would 
be more confusing than helpful.
> Today’s file: is pretty much a local matter.
The point I was making in my comments is that it's not just a local matter, but 
that file: also mediates some interactions between local storage and web access.

#g
--

---

From: Dave Cridland <[email-masked]>
Date: Tue, 14 Oct 2014 12:48:22 +0100

On 14 October 2014 09:10, Graham Klyne <[email-masked]> wrote:

> Using Python, the problem of  "/" vs "\" is largely resolved for me -
> Python allows "/" for file path construction, even on Windows.

Windows, since NT I think, allows "/" as a directory separator, even though
the canonical one is "\".

More info is here:

http://superuser.com/questions/176388/why-does-windows-use-backslashes-for-paths-and-unix-forward-slashes

But the TL;DR is that this isn't a Python thing. Even if a particular OS
allows "/" in file or directory names, the solution is straightforward -
encode them in URIs and (optionally) replace the separator. IMAP already
has to do this at times.

Dave.