Subject: Re: [apps-discuss] I-D Action: draft-ietf-appsawg-file-scheme-05.txt (relative URIs)

From: Stephan Bosch <[email-masked]>
Date: Wed, 16 Mar 2016 15:24:21 +0100

Hi,

I have a question about relative URI resolution with respect to the file 
URI. I know I am a bit late to the party, but this work hasn't caught my 
eye before; I only read this draft in a moment of boredom in my coffee 
break. I quickly skimmed the mailing list, but I haven't seen any recent 
discussions on this topic. There is only a very specific Windows case 
described in Appendix C of the draft.

I am not an expert, but I find RFC 3986 Section 5 rather confusing. When 
is reference resolution applied? According to Section 5.2.2, absolute 
URIs can be subject to the resolution algorithm as well. The main effect 
of that is the application of the remove_dot_segments() algorithm, which 
removes all "../" and "./" instances from the path. So, is this always 
applied before a URI is interpreted? Does it apply to an absolute file 
URI? This leads to the following concrete questions:

- Does the OS file system path extracted from an absolute file URI like 
"file:/frop/./friep/../frml" include the dot segments? Or is the 
remove_dot_segments() normalization always applied? So, is "/frop/frml" 
or "/frop/./friep/../frml" going to be passed to e.g. the open() system 
call on a POSIX system?

Does that matter? It might: when the path contains symbolic links on a 
POSIX system, the use of ".." can cause semantic differences between the 
URI-based normalization and what POSIX systems do. Here's the GLibC 
implementation of the stdlib realpath() function that performs the POSIX 
equivalent of remove_dot_segments():

https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/canonicalize.c;h=58bb8de949bc8cc705de7c0840c859fed7b06c01;hb=HEAD#l43

- In that regard, how are relative URIs handled when the base is a file URI?

In any case, I think these concerns need to be addressed explicitly in 
the document, so that the semantics are always as clear as possible.

Regards,

Stephan.

Op 1-12-2015 om 1:56 schreef Matthew Kerwin:
> Hi folks,
>
> I've pushed out a new version of the file URI draft which I believe 
> addresses most of the feedback received over the past couple of weeks. 
> Looking at the rfcdiff[1] the biggest changes were: being more clear 
> about how we're using/overriding components like "authority" from RFC 
> 3986, and rearranging the examples throughout the appendices to be 
> "description - example" (from "example - description".)
>
> I also misspelled "ooperation."  :\
>
> Cheers
>
> [1]: 
> https://tools.ietf.org/rfcdiff?url2=draft-ietf-appsawg-file-scheme-05.txt
>
>
> On 1 December 2015 at 10:43, <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     A New Internet-Draft is available from the on-line Internet-Drafts
>     directories.
>      This draft is a work item of the ART Area General Applications
>     Working Group Working Group of the IETF.
>
>             Title           : The file URI Scheme
>             Author          : Matthew Kerwin
>             Filename        : draft-ietf-appsawg-file-scheme-05.txt
>             Pages           : 20
>             Date            : 2015-11-30
>
>     Abstract:
>        This document specifies the "file" Uniform Resource Identifier
>     (URI)
>        scheme, obsoleting the definition in RFC 1738.
>
>        It attempts to define a common core which is intended to
>     interoperate
>        across the broad spectrum of existing implementations, while at the
>        same time documenting other current practices.
>
>     Note to Readers (To be removed by the RFC Editor)
>
>        This draft should be discussed on the IETF Applications Area
>     Working
>        Group discussion list <[email-masked]
>     <mailto:[email-masked]>>.
>
>
>     The IETF datatracker status page for this draft is:
>     https://datatracker.ietf.org/doc/draft-ietf-appsawg-file-scheme/
>
>     There's also a htmlized version available at:
>     https://tools.ietf.org/html/draft-ietf-appsawg-file-scheme-05
>
>     A diff from the previous version is available at:
>     https://www.ietf.org/rfcdiff?url2=draft-ietf-appsawg-file-scheme-05
>
>
>     Please note that it may take a couple of minutes from the time of
>     submission
>     until the htmlized version and diff are available at
>     tools.ietf.org <http://tools.ietf.org>.
>
>     Internet-Drafts are also available by anonymous FTP at:
>     ftp://ftp.ietf.org/internet-drafts/
>
>     _______________________________________________
>     apps-discuss mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/listinfo/apps-discuss
>
>
>
>
> -- 
>   Matthew Kerwin
> http://matthew.kerwin.net.au/
>
>
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 17 Mar 2016 08:04:51 +1000

> I am not an expert, but I find RFC 3986 Section 5 rather confusing. When
is reference resolution applied? According to Section 5.2.2, absolute URIs
can be subject to the resolution algorithm as well. The main effect of that
is the application of the remove_dot_segments() algorithm, which removes
all "../" and "./" instances from the path. So, is this always applied
before a URI is interpreted? Does it apply to an absolute file URI?

Quoting from RFC 3986:

§4.1 "A URI-reference is either a URI or a relative reference." This is a
confusing and somewhat isolated statement, but it suggests to me that all
URIs can be subject to normalisation as part of resolution.

---8<---
5.2.4.  Remove Dot Segments

   The pseudocode also refers to a "remove_dot_segments" routine for
   interpreting and removing the special "." and ".." complete path
   segments from a referenced path.  This is done after the path is
   extracted from a reference, **whether or not the path was relative**, in
   order to remove any invalid or extraneous dot-segments prior to
   forming the target URI.
--->8---

(My emphasis added.) Note that it says "from a reference", which includes
'URIs' per §4.1.

---8<---
6.2.2.3.  Path Segment Normalization

   The complete path segments "." and ".." are intended only for use
   within relative references (Section 4.1) and are removed as part of
   the reference resolution process (Section 5.2).  However, some
   deployed implementations incorrectly assume that reference resolution
   is not necessary when the reference is already a URI and thus fail to
   remove dot-segments when they occur in non-relative paths.  URI
   normalizers should remove dot-segments by applying the
   remove_dot_segments algorithm to the path, as described in
   Section 5.2.4.
--->8---

This is the bit that gets me. A relative reference is clearly defined in
the spec as: not starting with a scheme. The term 'URI' is fuzzy, but seems
to be the complement (i.e. it does start with a scheme). So perhaps what we
think of as an "absolute URI" is actually just a "URI" until it's been
through remove_dot_segments and co.

The quoted section also says "intended only for use within relative
references", but doesn't outright forbid use elsewhere (i.e. in "URIs").

> This leads to the following concrete questions:
>
"file:/frop/./friep/../frml" include the dot segments? Or is the
remove_dot_segments() normalization always applied? So, is "/frop/frml" or
"/frop/./friep/../frml" going to be passed to e.g. the open() system call
on a POSIX system?

According to §6.2.2.3, if you normalise the URI then the dots are removed
before the URI is converted to a file system path. It doesn't mandate said
normalisation, so I guess that's between you and the OS.

> [snip]

No different from any other URI scheme, except that in Windows/DOS you have
this weird sub-authority/drive letter issue (C:/a/../../b => C:/b). Which
is quirky and hard to reconcile, hence the appendix.

Maybe. I think I might be missing some text in the appendix saying that a
non-normalised URI passed to a DOS-based OS is likely to treat dot-segments
a particular way.

Maybe it's time to update 3986 itself; although I don't know that there's
enough energy in the community to undertake such a feat.

Cheers