Subject: [apps-discuss] Review of draft-ietf-appsawg-file-scheme

From: Dave Crocker <[email-masked]>
Date: Tue, 12 Apr 2016 12:29:29 -0700

Review of: The file URI Scheme
I-D: draft-ietf-appsawg-file-scheme-05

      (I see that a -06 version was released today.  Based on a quick
      scan of differences from the -05 version, I do not see their
      affecting the review.)

Reviewer:  D. Crocker
Date:      12 April 2016

This review was performed in my role as document shepherd.

The review was originally written last November but unfortunately I lost 
track of it and apologize for the delay.

I should also comment that I've just consulted a number of others who 
have been involved in discussions about the draft and hope that they 
will response to this review, on the mailing list.

Summary:

The specification seeks to detail the file: URI scheme, which 
"identifies a file on a particular file system", and replaces RFC 1738.

The document has been housed within the Apps Area WG, but has received 
little substantive comment -- somewhere in the range of 5-8 people, with 
small bursts of activity. Although at least some have tended to be 
supportive, there have been notable exceptions. Worse, significant 
issues that were raised were not obviously resolved on the list.  (I 
looked at postings by others, immediately after the author's postings, 
to see whether there were statements of agreement that an issue had been 
resolved, and did not generally see them, concerning those substantive 
issues.)

The draft began as an individual effort, in June 2013, went through many 
revisions, and was adopted by AppsAWG in January 2015, and has had a 
number of revisions.  I note a 9/26/2014 (pre-AppsAWG) comment from 
Daniel Stenberg:

      "I would rather have a new spec straighten up and tighten the
      language somewhat so that we can get a stricter interpretation of
      how a file:// is supposed to work."

which, in spite of the many document revisions, unfortunately matches my 
own assessment of the current draft.

I also note a 9/15/2015 (post-AppsAWG adoption) comment from Mark 
Nottingham, on behalf of the W3C TAG:

      Regarding the use of `file://` URIs on the web, there are a few
      issues that need to be resolved for interoperability:

      1. How does `file://` fit into the web's origin model?

      2. How does retrieval of `file://` URIs fit into
         [Fetch](https://fetch.spec.whatwg.org/) and/or the [URL
         standard](https://url.spec.whatwg.org)?

      This document does not address any of these issues, so we
      encourage the APPSAWG to consider addressing these.

One point of process discussion about the draft was to distinguish 
between documenting current practice, versus specifying enhancements. 
Another was to distinguish between use of the construct locally (within 
a user's own system) versus globally, across the public Web.  Again, 
neither matter appears to be resolved clearly within the draft, and I 
can't tell tell clearly which goals the draft targets, even with the 
Abstract text:

      "It attempts to define a common core which is intended to
      interoperate across the broad spectrum of existing
      implementations, while at the same time documenting other current
      practices."

These two goals entail a classic choice in development of a 
specification that deals with existing practice.  Clarity between the 
two goals and careful precision about which is being served, is essential.

A file: construct should be of significant utility to the Internet 
community.  So it warrants careful community review and extensive 
indication of active support.  That is, there ought to be a basis for 
assessing the likelihood of implementation and use.  As of now, this is 
not possible.

In technical terms, document seems to suffer some confusion about its 
role as a format specification, versus as a protocol specification.  I 
believe this issue is basic and important.  It needs to be resolved.

For a specification involving such a potentially and presumably 
important capability, I think significant community support should be 
required... unless the spec is to be offered as Experimental, which is 
the most I'm inclined to recommend at this point...

Details:

> Applications Area Working Group                                M. Kerwin
> Internet-Draft                                                       QUT
> Obsoletes: 1738 (if approved)                           December 1, 2015

I believe it merely updates it.  It essentially replaces only Section
3.10 of that RFC.

> Intended status: Standards Track
> Expires: June 3, 2016
>
>
>                           The file URI Scheme
>                    draft-ietf-appsawg-file-scheme-05
>
> Abstract
>
>    This document specifies the "file" Uniform Resource Identifier (URI)
>    scheme, obsoleting the definition in RFC 1738.
>
>    It attempts to define a common core which is intended to interoperate

attempts to define -> defines

also: common core of what? I think the answer is a common core of object
storage naming convention, but whatever is correct, it should be made
explicit here.

>    across the broad spectrum of existing implementations, while at the
>    same time documenting other current practices.

implementations of ... URI-based file system accessing mechanisms?

>
> Note to Readers (To be removed by the RFC Editor)
>
>    This draft should be discussed on the IETF Applications Area Working
>    Group discussion list <[email-masked]>.
>
> Status of This Memo
>
>    This Internet-Draft is submitted in full conformance with the
>    provisions of BCP 78 and BCP 79.
>
>    Internet-Drafts are working documents of the Internet Engineering
>    Task Force (IETF).  Note that other groups may also distribute
>    working documents as Internet-Drafts.  The list of current Internet-
>    Drafts is at http://datatracker.ietf.org/drafts/current/.
>
>    Internet-Drafts are draft documents valid for a maximum of six months
>    and may be updated, replaced, or obsoleted by other documents at any
>    time.  It is inappropriate to use Internet-Drafts as reference
>    material or to cite them other than as "work in progress."
>
>    This Internet-Draft will expire on June 3, 2016.
>
> Copyright Notice
>
>    Copyright (c) 2015 IETF Trust and the persons identified as the
>    document authors.  All rights reserved.
>
>    This document is subject to BCP 78 and the IETF Trust's Legal
>    Provisions Relating to IETF Documents
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 1]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    (http://trustee.ietf.org/license-info) in effect on the date of
>    publication of this document.  Please review these documents
>    carefully, as they describe your rights and restrictions with respect
>    to this document.  Code Components extracted from this document must
>    include Simplified BSD License text as described in Section 4.e of
>    the Trust Legal Provisions and are provided without warranty as
>    described in the Simplified BSD License.
>
> Table of Contents
>
>    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
>      1.1.  History . . . . . . . . . . . . . . . . . . . . . . . . .   3
>      1.2.  Similar Technologies  . . . . . . . . . . . . . . . . . .   3
>      1.3.  Notational Conventions  . . . . . . . . . . . . . . . . .   4
>    2.  Syntax  . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
>    3.  Operations on file URIs . . . . . . . . . . . . . . . . . . .   5
>      3.1.  Translating Local File Path to file URI . . . . . . . . .   5
>      3.2.  Translating Non-local File Path to file URI . . . . . . .   6
>      3.3.  Incompatible File Paths . . . . . . . . . . . . . . . . .   7
>        3.3.1.  Win32 Namespaces  . . . . . . . . . . . . . . . . . .   7
>    4.  Encoding  . . . . . . . . . . . . . . . . . . . . . . . . . .   7
>    5.  Origins . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
>    6.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
>    7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   8
>    8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
>    9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
>      9.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
>      9.2.  Informative References  . . . . . . . . . . . . . . . . .  10
>    Appendix A.  Example URIs . . . . . . . . . . . . . . . . . . . .  12
>    Appendix B.  System-specific Operations . . . . . . . . . . . . .  12
>      B.1.  POSIX Systems . . . . . . . . . . . . . . . . . . . . . .  12
>      B.2.  DOS- and Windows-Like Systems . . . . . . . . . . . . . .  12
>      B.3.  Mac OS X Systems  . . . . . . . . . . . . . . . . . . . .  13
>      B.4.  OpenVMS Files-11 Systems  . . . . . . . . . . . . . . . .  13
>    Appendix C.  Nonstandard Syntax Variations  . . . . . . . . . . .  13
>      C.1.  DOS and Windows Drive Letters . . . . . . . . . . . . . .  13
>        C.1.1.  Relative Paths  . . . . . . . . . . . . . . . . . . .  14
>        C.1.2.  Vertical Bar Character  . . . . . . . . . . . . . . .  14
>      C.2.  UNC Strings . . . . . . . . . . . . . . . . . . . . . . .  15
>      C.3.  UNC Paths . . . . . . . . . . . . . . . . . . . . . . . .  15
>      C.4.  Backslash as Separator  . . . . . . . . . . . . . . . . .  16
>    Appendix D.  Example of IRI vs Percent-Encoded URI  . . . . . . .  17
>    Appendix E.  UNC Syntax . . . . . . . . . . . . . . . . . . . . .  18
>    Appendix F.  Collected Rules  . . . . . . . . . . . . . . . . . .  19
>    Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  20
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 2]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> 1.  Introduction
>
>    A file URI identifies a file on a particular file system.  It can be
>    used in discussions about the file, and if other conditions are met
>    it can be dereferenced to directly access the file.

Since this is a specification, and it has intended for wide use, there
should be some sort of basic definition of what a file system is.
Nothing fancy. And by way of priming the pump:

      ...on a particular file system, which is an object stored in a
structured naming-and-accessing environment on a host. The URI can be
used...

>
>    The file URI scheme is not coupled with a specific protocol, nor with
>    a specific media type.  See Section 3 for a discussion of operations
>    that can be performed on a file URI.

I think these defined operations are not really performed 'on' the
file URI, but rather are used to create or apply the URI.

Also:

    media type. -> media type [rfc6838].

>    This document defines a syntax that is compatible with most extant
>    implementations, while attempting to push towards a stricter subset
>    of "ideal" constructs.  In many cases it simultaneously acknowledges
>    and deprecates some less common or outdated constructs.

*** ideal?

How does it 'acknowledge' such constructs?

>
> 1.1.  History
>
>    The file URI scheme was first defined in [RFC1630], which, being an
>    informational RFC, does not specify an Internet standard.  The

My personal preference is for documents to refrain from referring to
their standards status or the status of other documents. "Status" is an
ephemeral attribute that should be external to the details of a
specification, IMO.

That is, make the discussion be in terms of the technical and
operational issues, not the standards status.

>    definition was standardised in [RFC1738], and the scheme was
>    registered with the Internet Assigned Numbers Authority (IANA);

IANA registration is a side-effect of the specs and typically isn't
called out this way, I believe.

>    however that definition omitted certain language included by the
>    former that clarified aspects such as:

    ... by former that...   missing word?

>
>    o  the use of slashes to denote boundaries between directory levels
>       of a hierarchical file system; and
>
>    o  the requirement that client software convert the file URI into a
>       file name in the local file name conventions.

*** Hmmm. A requirement like that moves this from being a URI
specification to being a file protocol specification...

>
>    The Internet draft [I-D.hoffman-file-uri] was written in an effort to
>    keep the file URI scheme on standards track when [RFC1738] was made
>    obsolete, but that draft expired in 2005.  It enumerated concerns
>    arising from the various, often conflicting implementations of the
>    scheme.  It serves as the spiritual predecessor of this document.
>
>    Additionally the WHATWG defines a living URL standard [WHATWG-URL],
>    which includes algorithms for interpreting file URIs (as URLs).

How does it relate to the current draft?  At the least, doesn't it
instead belong in the next 'Similar Technologies' section?

>
> 1.2.  Similar Technologies
>
>    The Universal Naming Convention (UNC) [MS-DTYP] defines a string
>    format that can perform a similar role to the file URI scheme in
>    describing the location of files.  A UNC filespace selector string
>    has three parts: host, share, and path; see Appendix E.  This
>    document describes but does not specify a means of translating
>    between UNC filespace selector strings and file URIs in Appendix C.2.

"describes but does not specify" seems an odd distinction.  Not exactly
sure what it means.

>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 3]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> 1.3.  Notational Conventions
>
>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>    document are to be interpreted as described in [RFC2119].
>
>    Throughout this document the term "local" is used to describe files
>    that can be accessed directly through the local file system.  It is
>    important to note that a local file may not be physically located on
>    the local machine, for example if a networked file system is
>    transparently mounted into the local file system.

I'm not exactly sure what 'accessed directly' means, in the modern world
of file systems.  Unfortunately, this is cast as a major point of
concern in the document.  How is 'directly' different from 'no authority
value is specified'?

> 2.  Syntax
>
>    The file URI syntax is defined here in Augmented Backus-Naur Form
>    (ABNF) [RFC5234], including the core ABNF syntax rule "ALPHA" defined
>    by that specification, and importing the "userinfo", "host" and
>    "path-absolute" rules from [RFC3986] (as updated by [RFC6874].)

Possibly cleaner:

      [RFC5234]. Imported rules:

         From [RFC5234]: ALPHA

         From [RFC3986], [RFC6874]: userinfo, host, authority,
                                   path-absolute

>    The core syntax in [RFC3986] includes "path" and "authority"
>    components, for each of which only a subset is used in the definition
>    of the file URI scheme.  The relevant subset of "path" is "path-
>    absolute", and the subset of "authority" is "file-auth", given below.
>
>    Please note Appendix C that lists some other commonly seen but
>    nonstandard variations.
>
>       file-URI       = file-scheme ":" file-hier-part
>
>       file-scheme    = "file"
>
>       file-hier-part = "//" auth-path
>                      / local-path
>
>       auth-path      = [ file-auth ] path-absolute
>
>       local-path     = path-absolute
>
>       file-auth      = [ userinfo "@" ] host

Syntax problem:

    If auth-path has no file-auth, since it's optional, then both the
auth-path and local-path of file-hier-part reduce to path-absolute.
Methinks that's creates a parsing ambiguity?

Hmmm. I'm also not sure whether stylistic nuance and exact compatibility
with the meta-specification documents this is based on are worth
worrying about, but...

I believe this could derive more smoothly from section 3 of RFC3986,
with something like:

    URI        = scheme ":" hier-part    ; from RFC 3986

    scheme     = "file"

    hier-part  = "//"
                 ( authority
                 / local-path )

(The parens are to make the binding of the alternatives visually
unambiguous. Relying on formal, implicit binding rules can create tricky
usability errors. The hier-part rule in RFC 3986 might be an example:
Concatenation is has tighter binding than alternatives, in ABNF, but the
labeling in RFC3986 could imply a different parsing. Presumably the
authors of RFC 3986 got the BNF correct, but I'm guessing a random
reader could easily get it wrong... )

>    The syntax definition above is different from those given in
>    [RFC1630] and [RFC1738] as it is derived from the generic syntax of
>    [RFC3986], which post-dates all previous specifications.

  all previous -> the previous file URI

>    As a special case, the "file-auth" rule can match the string
>    "localhost" or the empty string; either value is interpreted as "the
>    machine from which the URI is being interpreted," exactly as if no
>    authority was present.  To maximise compatibility with previous

'as if no authority" was present.'  Probably should be 'were' present,
but more significantly 'authority' is not defined.

This is a reason to re-use the ABNF rulenames from RFC 3986, when you
are importing them, but modify the <element> portion to suit the file
scheme.  (Note that Ned Freed offers extended discussion about this,
including going down a different line of resolution on this issue, in
his 2 November 1:42pm posting. However the issue here is resolved, the
main point is that the text needs to be consistent and clear in its
language. It seems that the current text does not fully resolve the
points he raised.)

Since file-auth is shown as optional, that handles the empty string
already and doesn't need a comment.

Offhand, I wonder whether the details of the "As special case" paragraph
can (and should) instead be reflected directly in the ABNF?

> Kerwin                    Expires June 3, 2016                  [Page 4]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    specifications, implementations MAY choose to include an empty "file-
>    auth".
>
>    Systems exhibit different levels of case-sensitivity.  Unless the
>    file system is known to be case-insensitive, implementations MUST
>    maintain the case of file and directory names when translating file
>    URIs to and from the local system's representation of file paths, and
>    any systems or devices that transport file URIs MUST NOT alter the
>    case of file URIs they transport.

This is protocol language, not URI format/semantics language.

I believe that the needs of this spec are served by something along the
lines of:

     Some systems have case-sensitive file naming and some do not.  Hence
the  file scheme supports case sensitivity, in order to retain the case
as given. Any transport-related handling of the file URI scheme MUST
retain the case as given. Any mapping to or from a case-insensitive form
is soley the responsibility of the implementation processing the file
URI on behalf of the referenced file system.

>
> 3.  Operations on file URIs

Use of normative language in this section is inappropriate.  It does not
provide protocol details and provides essentially no semantics.

>    Implementations that provide dereferencing ooperations on file URIs

    ooperations -> operations

>    SHOULD, at a minimum, provide a read-like operation to return the
>    contents of a file located by a file URI.  Additional operations MAY
>    be provided, such as writing to, creating, and deleting files.  See
>    the POSIX file and directory operations [POSIX] for examples of
>    standardized operations that can be performed on files.

What is the practical benefit of giving the reader superficial tutorial
information about file system operations?

>    File URIs can also be translated to and from other, similar
>    constructs, such as local file paths or UNC strings.

I don't know what this means.  And how is this sentence useful?

>    A file URI can be dependably dereferenced or translated to a local
>    file path only if it is local.  A file URI is considered "local" if
>    it has a blank or no authority, or the authority is the special
>    string "localhost".
>
>    This specification neither defines nor forbids a mechanism for
>    accessing non-local files.  See SMB [MS-SMB], NFS [RFC7530], NCP
>    [NOVELL] for examples of protocols that can be used to access files
>    over a network.  Also see Appendix C.2 for a discussion on
>    translating non-local file URIs to and from UNC stings.
>
> 3.1.  Translating Local File Path to file URI
>
>    Below is an algorithmic description of the process used to convert a
>    file path to a URI; see Section 4.
>
>    1.  Resolve the file path to its fully qualified absolute form.

What does this mean?  Where is it defined?

>
>    2.  Initialise the URI with the "file:" scheme identifier.
>
>    3.  If including an empty authority field, append the "//" sigil to
>        the URI.

sigil ???  pretty stylized vocabulary...

What about the alternative of including a /non-empty/ authority field?

>    4.  Append a slash character "/" to the URI, to signify the path
>        root.>

>    5.  For each directory in the path after the root:
>
>        1.  Transform the directory name to a path segment ([RFC3986],
>            Section 3.3) as per Section 2 of [RFC3986].

I think that Section 2 does not specify how to do a transform, although
yes, it does make reference to doing one.  Rather, it specifies encoding
rules.  The details of how to do a transform are left to the implementer.

Hence I believe the above should be something like:

      1. Transform the directory name to a path segment (RFC3986],
Section 3.3]) to conform to the encoding rules of Section 2 of
[RFC3986].  The specific rules for mapping between a file system name
and a file scheme URI are outside the scope of this specification.

>        2.  Append the transformed segment and a delimiting slash
>            character "/" to the URI.
>
>    6.  If the path includes a file name:
>
>        1.  Transform the file name to a path segment as above.
>
>        2.  Append the transformed segment to the URI.

A slash is required at the end of a directory, even if there is no file
name?

> Differences from RFC 1738

Seems like this belongs elsewhere in the document, like maybe an
appendix.  It is irrelevant except as an historical note.

>    In [RFC1738] a file URL always started with the token "file://",
>    followed by an (optionally blank) authority and a "/".  That "/" was
>    not considered part of the path.  This implies that the correct
>    encoding for a file path in a UNIX-like environment would have been:
>
>         token     + authority + slash + path
>       = "file://" + ""        + "/"   + "/path/to/file.txt"
>       = "file:////path/to/file.txt"
>
>    However that construct was never observed in practice, and in fact
>    would have collided with the eventual encoding of UNC strings in URIs
>    described in Appendix C.3.
>
> 3.2.  Translating Non-local File Path to file URI
>
>    Translating a non-local file path, including a UNC string, to a file
>    URI follows the same basic algorithm as for local files, above,
>    except that the authority MUST refer to the network-accesible node
>    that hosts the file.

    accesible  -> accessible

>    For example, in a clustered OpenVMS Files-11 system the authority
>    would contain the node name.  Where the original node reference
>    includes a username and password in an access control string, they
>    MAY be transcribed into the userinfo field of the authority
>    ([RFC3986], Section 3.2.1), security considerations (Section 6)
>    notwithstanding.
>
>    See Appendix C.2 for an explicit handling of UNC strings.
>
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 6]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> 3.3.  Incompatible File Paths
>
>    Some conventional file path formats are known to be incompatible with
>    the file URI scheme.
>
> 3.3.1.  Win32 Namespaces
>
>    The Microsoft Windows API defines Win32 Namespaces [Win32-Namespaces]
>    for interacting with files and devices using Windows API functions.
>    These namespaced paths are prefixed by "\\?\" for Win32 File
>    Namespaces and "\\.\" for Win32 Device Namespaces.  There is also a
>    special case for UNC file paths in Win32 File Namespaces, referred to
>    as "Long UNC", using the prefix "\\?\UNC\".
>
>    This specification does not define a mechanism for translating
>    namespaced paths to or from file URIs.

No it doesn't, although it contains some language that almost seems to.
  The language needs to be removed, in favor of the above simple sentence.

Further, language about 'incompatibility' is a commentary that belongs
in the Introduction or some other place outside the main spec.  It's an
important discussion point, but it's not part of the spec.

>
> 4.  Encoding
>
>    To avoid ambiguity, a file URI SHOULD be transported as an
>    Internationalized Resource Identifier (IRI) [RFC3987], or as a URI
>    with non-ASCII characters encoded according to the UTF-8 character
>    encoding [STD63] and percent-encoded as needed ([RFC3986],
>    Section 2.5).
>
>    The encoding of a file URI depends on the file system that stores the

I'm not sure this sentence is correct.  It's a natural assumption but
arguably one can lay any encoding convention on top of any data store.

>    identified file.  If the file system uses a known non-Unicode
>    character encoding, the path SHOULD be converted to a sequence of
>    characters from the Universal Character Set [ISO10646] normalized
>    according to Normalization Form C (NFC) [UTR15], before being
>    translated to a file URI, and conversely a file URI SHOULD be
>    converted back to the file system's native encoding when
>    dereferencing or translating to a file path.
>
>       Note that many modern file systems encode directory and file names
>       as arbitrary sequences of octets.  In those cases, the
>       representation as an encoded string often depends on the user's
>       localization settings, or defaults to UTF-8 [STD63].
>
>    When the file system's encoding is not known the file URI SHOULD be
>    transported as an Internationalized Resource Identifier (IRI)
>    [RFC3987] to avoid ambiguity.  See Appendix D for examples.

I'm inclined to think that this section either needs to be far more
complete -- and I'm not recommending it do that -- or it merely needs to
caution implementers to make sure that file scheme URI storage needs to
be idempotent with the original, interoperable form.

>
> 5.  Origins
>
>    As per [RFC6454], Section 4, when determining the origin of a file
>    URI implementations MAY return an implementation-defined value.

Now we are back to protocol, rather than basic representation. But it
seems minimal in detail and utility.
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 7]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    Historically, user agents have granted content from the file URI
>    scheme a tremendous amount of privilege.  However, granting all local
>    files such wide privileges can lead to privilege escalation attacks.
>    Some user agents have had success granting local files directory-
>    based privileges, but this approach has not been widely adopted.
>    Other user agents use globally unique identifiers for each file URI,
>    which is the most secure option.

This paragraph belongs in the next, Security Considerations section.
Without the protocol-ish paragraph before it.

>
> 6.  Security Considerations
>
>    There are many security considerations for URI schemes discussed in
>    [RFC3986].
>
>    File access and the granting of privileges for specific operations
>    are complex topics, and the use of file URIs can complicate the
>    security model in effect for file privileges.  Software using file
>    URIs MUST NOT grant greater access than would be available for other
>    file access methods.

This sort of normative statement has no real meaning, and certainly none
without explanation.  At base, the reader cannot tell was satisfies the
normative statement and what does not.

>    File systems typically assign an operational meaning to special
>    characters, such as the "/", "\", ":", "[", and "]" characters, and
>    to special device names like ".", "..", "...", "aux", "lpt", etc.  In
>    some cases, merely testing for the existence of such a name will
>    cause the operating system to pause or invoke unrelated system calls,
>    leading to significant security concerns regarding denial of service
>    and unintended data transfer.  It would be impossible for this
>    specification to list all such significant characters and device
>    names.  Implementers MUST research the reserved names and characters
>    for the types of storage device that may be attached to their
>    application and restrict the use of data obtained from URI components
>    accordingly.
>
>    Additionally, as discussed in the HP OpenVMS Systems Documentation
>    <http://h71000.www7.hp.com/doc/84final/ba554_90015/ch03s09.html>
>    "access control strings include sufficient information to allow
>    someone to break in to the remote account, [therefore] they create
>    serious security exposure."  In a similar vein, the presence of a
>    password in a "user:password" userinfo field is deprecated by
>    [RFC3986].  As such, the userinfo field of a file URI, if present,
>    MUST NOT contain a password.

Is there really no stable, published document that says something similar?

> 7.  IANA Considerations
>
>    This document defines the following URI scheme, so the "Permanent URI
>    Schemes" registry has been updated accordingly.  This registration
>    complies with [BCP35].
>
>    Scheme name:
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 8]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>       file
>
>    Status:
>       permanent
>
>    Applications/protocols that use this scheme name:
>       Commonly used in hypertext documents to refer to files without
>       depending on network access.  Supported by major browsers.
>
>       Windows API (PathCreateFromUrl, UrlCreateFromPath).
>
>       Perl LWP.
>
>    Contact:
>       Matthew Kerwin <[email-masked]>
>
>    Change Controller:
>       This scheme is registered under the IETF tree.  As such, the IETF
>       maintains change control.
>
> 8.  Acknowledgements
>
>    This specification is derived from [RFC1738], [RFC3986], and
>    [I-D.hoffman-file-uri] (expired); the acknowledgements in those
>    documents still apply.
>
>    Additional thanks to Dave Risney, author of the informative IE Blog
>    article <http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-
>    windows.aspx>, and Dave Thaler for their comments and suggestions.
>
> 9.  References
>
> 9.1.  Normative References
>
>    [BCP35]    Thaler, D., Ed., Hansen, T., and T. Hardie, "Guidelines
>               and Registration Procedures for URI Schemes", BCP 35,
>               RFC 7595, DOI 10.17487/RFC7595, June 2015,
>               <http://www.rfc-editor.org/info/bcp35>.
>
>    [ISO10646]
>               International Organization for Standardization,
>               "Information Technology - Universal Multiple-Octet Coded
>               Character Set (UCS)", ISO/IEC 10646:2003, December 2003.
>
>    [RFC1035]  Mockapetris, P., "Domain names - implementation and
>               specification", STD 13, RFC 1035, DOI 10.17487/RFC1035,
>               November 1987, <http://www.rfc-editor.org/info/rfc1035>.
>
>
>
>
> Kerwin                    Expires June 3, 2016                  [Page 9]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    [RFC1123]  Braden, R., Ed., "Requirements for Internet Hosts -
>               Application and Support", STD 3, RFC 1123,
>               DOI 10.17487/RFC1123, October 1989,
>               <http://www.rfc-editor.org/info/rfc1123>.
>
>    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>               Requirement Levels", BCP 14, RFC 2119,
>               DOI 10.17487/RFC2119, March 1997,
>               <http://www.rfc-editor.org/info/rfc2119>.
>
>    [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
>               Resource Identifier (URI): Generic Syntax", STD 66,
>               RFC 3986, DOI 10.17487/RFC3986, January 2005,
>               <http://www.rfc-editor.org/info/rfc3986>.
>
>    [RFC3987]  Duerst, M. and M. Suignard, "Internationalized Resource
>               Identifiers (IRIs)", RFC 3987, DOI 10.17487/RFC3987,
>               January 2005, <http://www.rfc-editor.org/info/rfc3987>.
>
>    [RFC4291]  Hinden, R. and S. Deering, "IP Version 6 Addressing
>               Architecture", RFC 4291, DOI 10.17487/RFC4291, February
>               2006, <http://www.rfc-editor.org/info/rfc4291>.
>
>    [RFC5234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
>               Specifications: ABNF", STD 68, RFC 5234,
>               DOI 10.17487/RFC5234, January 2008,
>               <http://www.rfc-editor.org/info/rfc5234>.
>
>    [RFC6874]  Carpenter, B., Cheshire, S., and R. Hinden, "Representing
>               IPv6 Zone Identifiers in Address Literals and Uniform
>               Resource Identifiers", RFC 6874, DOI 10.17487/RFC6874,
>               February 2013, <http://www.rfc-editor.org/info/rfc6874>.
>
>    [UTR15]    Davis, M. and K. Whistler, "Unicode Normalization Forms",
>               August 2012,
>               <http://unicode.org/reports/tr15/tr15-18.html>.
>
> 9.2.  Informative References
>
>    [Bug107540]
>               Bugzilla@Mozilla, "Bug 107540", October 2007,
>               <https://bugzilla.mozilla.org/show_bug.cgi?id=107540>.
>
>    [I-D.hoffman-file-uri]
>               Hoffman, P., "The file URI Scheme", draft-hoffman-file-
>               uri-03 (work in progress), January 2005.
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 10]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    [MS-DTYP]  Microsoft Open Specifications, "Windows Data Types, 2.2.56
>               UNC", January 2013,
>               <http://msdn.microsoft.com/en-us/library/gg465305.aspx>.
>
>    [MS-NBTE]  Microsoft Open Specifications, "NetBIOS over TCP (NBT)
>               Extensions", May 2014,
>               <http://msdn.microsoft.com/en-us/library/dd891412.aspx>.
>
>    [MS-SMB]   Microsoft Open Specifications, "Server Message Block (SMB)
>               Protocol", January 2013,
>               <http://msdn.microsoft.com/en-us/library/cc246231.aspx>.
>
>    [NOVELL]   Novell, "NetWare Core Protocols", 2013,
>               <http://www.novell.com/developer/ndk/
>               netware_core_protocols.html>.
>
>    [POSIX]    IEEE, "IEEE Std 1003.1, 2013 Edition", 2013.
>
>    [RFC1630]  Berners-Lee, T., "Universal Resource Identifiers in WWW: A
>               Unifying Syntax for the Expression of Names and Addresses
>               of Objects on the Network as used in the World-Wide Web",
>               RFC 1630, DOI 10.17487/RFC1630, June 1994,
>               <http://www.rfc-editor.org/info/rfc1630>.
>
>    [RFC1738]  Berners-Lee, T., Masinter, L., and M. McCahill, "Uniform
>               Resource Locators (URL)", RFC 1738, DOI 10.17487/RFC1738,
>               December 1994, <http://www.rfc-editor.org/info/rfc1738>.
>
>    [RFC6454]  Barth, A., "The Web Origin Concept", RFC 6454,
>               DOI 10.17487/RFC6454, December 2011,
>               <http://www.rfc-editor.org/info/rfc6454>.
>
>    [RFC7530]  Haynes, T., Ed. and D. Noveck, Ed., "Network File System
>               (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530,
>               March 2015, <http://www.rfc-editor.org/info/rfc7530>.
>
>    [STD63]    Yergeau, F., "UTF-8, a transformation format of ISO
>               10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November
>               2003, <http://www.rfc-editor.org/info/std63>.
>
>    [WHATWG-URL]
>               WHATWG, "URL Living Standard", May 2013,
>               <http://url.spec.whatwg.org/>.
>
>    [Win32-Namespaces]
>               Microsoft Developer Network, "Naming Files, Paths, and
>               Namespaces", June 2013, <https://msdn.microsoft.com/en-
>               au/library/windows/desktop/aa365247(v=vs.85).aspx>.
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 11]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> Appendix A.  Example URIs
>
>    The syntax in Section 2 is intended to support file URIs that take
>    the following forms:
>
>    Local files:
>
>    o  A traditional file URI for a local file, with an empty authority.
>       This is the most common format in use today.  E.g.:
>
>       *  "file:///path/to/file"
>
>    o  The minimal representation of a local file, with no authority
>       field and an absolute path that begins with a slash "/".  E.g.:
>
>       *  "file:/path/to/file"
>
>    Non-local files:
>
>    o  A non-local file, with an explicit authority.  E.g.:
>
>       *  "file://host.example.com/path/to/file"
>
> Appendix B.  System-specific Operations
>
>    This appendix is not normative; it highlights some observed
>    behaviours and provides system-specific guidance for interacting with
>    file URIs and paths.
>
> B.1.  POSIX Systems
>
>    There is little to say about POSIX file systems; the file URI
>    structure already closely resembles POSIX file paths.
>
> B.2.  DOS- and Windows-Like Systems
>
>    When mapping a DOS- or Windows-like file path to a file URI,
>    implementations typically map the drive letter (e.g. "c:") into the
>    first path segment.
>
>    See Appendix C.1 for explicit (but non-normative and strictly
>    optional) rules for interacting with DOS- or Windows-like file paths
>    and URIs.
>
>
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 12]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> B.3.  Mac OS X Systems
>
>    The HFS+ file system uses a non-standard normalization form, similar
>    to Normalization Form D.  Take care when transforming HFS+ file paths
>    to and from URIs using Normalization Form C Section 4.
>
> B.4.  OpenVMS Files-11 Systems
>
>    When mapping a VMS file path to a file URI, map the device name into
>    the first path segment.  Note that the dollars sign "$" is a reserved
>    character per the definition in [RFC3986], Section 2.2, so should be
>    percent-encoded if present in the device name.
>
>    If the VMS file path includes a node reference, use that as the
>    authority.  Where the original node reference includes a username and
>    password in an access control string, they can be transcribed into
>    the userinfo field of the authority ([RFC3986], Section 3.2.1),
>    security considerations (Section 6) notwithstanding.
>
> Appendix C.  Nonstandard Syntax Variations
>
>    These variations may be encountered for historical reasons, but are
>    not supported by the normative syntax of Section 2.
>
>    This appendix is not normative.
>
> C.1.  DOS and Windows Drive Letters
>
>    On Windows- or DOS-based file systems a absolute file path can begin
>    with a drive letter.  To facilitate this, the "local-path" rule in
>    Section 2 can be replaced with the following:
>
>       local-path     = [ drive-letter ] path-absolute
>
>       drive-letter   = ALPHA ":"
>
>    This is intended to support the minimal representation of a local
>    file in a DOS- or Windows-based environment, with no authority field
>    and an absolute path that begins with a drive letter.  E.g.:
>
>    o  "file:c:/path/to/file"
>
>    URIs of the form "file:///c:/path/to/file" are already supported by
>    the "path-absolute" rule.
>
>    Note that comparison of drive letters in DOS or Windows file paths is
>    case-insensitive.  Some implementations therefore canonicalize drive
>    letters in file URIs by converting them to uppercase.
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 13]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> C.1.1.  Relative Paths
>
>    In DOS- or Windows-based file systems, relative paths beginning with
>    a slash "/" should be resolved relative to the drive letter, and
>    resolution of ".." dot segments (per Section 5.2.4 of [RFC3986])
>    should not ever overwrite the drive letter.
>
>    e.g.:
>
>       base:       file:///c:/path/to/file.txt
>       rel. URI:   /some/other/thing.bmp
>       resolved:   file:///c:/some/other/thing.bmp
>
>       base:       file:///c:/foo.txt
>       rel. URI:   ../../bar.txt
>       resolved:   file:///c:/bar.txt
>
>    Relative paths with a drive letter followed by a character other than
>    a slash (e.g. "c:bar/baz.txt" or "c:../foo.txt") should not be
>    accepted as dereferenceable URIs in DOS or Windows systems.
>
> C.1.2.  Vertical Bar Character
>
>    Historically some implementations have used a vertical line character
>    "|" instead of a colon ":" in the drive letter construct.  [RFC3986]
>    forbids the use of the vertical line, however it may be necessary to
>    interpret or update old URIs.
>
>    For interpreting such URIs, the "auth-path" and "local-path" rules in
>    Section 2 and the "drive-letter" rule above are replaced with the
>    following:
>
>       auth-path      = [ file-auth ] path-absolute
>                      / [ file-auth ] file-absolute
>
>       local-path     = [ drive-letter ] path-absolute
>                      / file-absolute
>
>       file-absolute  = "/" drive-letter path-absolute
>
>       drive-letter   = ALPHA ":"
>                      / ALPHA "|"
>
>    This is intended to support regular DOS or Windows file URIs with
>    vertical line characters in the drive letter construct.  E.g.:
>
>    o  "file:///c|/path/to/file"
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 14]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    o  "file:/c|/path/to/file"
>
>    o  "file:c|/path/to/file"
>
>    To update such an old URI, replace the vertical line "|" with a colon
>    ":".
>
> C.2.  UNC Strings
>
>    A UNC filespace selector string can be directly translated to a URI;
>    see Section 4.  The following is an algorithmic description of the
>    process of translating a UNC string to a file URI:
>
>    1.  Initialise the URI with the "file:" scheme identifier.
>
>    2.  Append the authority:
>
>        1.  Append the "//" authority sigil to the URI.
>
>        2.  Append the hostname field of the UNC string to the URI.
>
>    3.  Append the sharename:
>
>        1.  Transform the sharename to a path segment ([RFC3986],
>            Section 3.3) as per Section 2 of [RFC3986].
>
>        2.  Append a delimiting slash character "/" and the transformed
>            segment to the URI.
>
>    4.  For each objectname:
>
>        1.  Transform the objectname to a path segment ([RFC3986],
>            Section 3.3) as per Section 2 of [RFC3986].
>
>        2.  Append a delimiting slash character "/" and the transformed
>            segment to the URI.
>
>    Example:
>
>       UNC String:   \\host.example.com\Share\path\to\file.txt
>       URI:          file://host.example.com/Share/path/to/file.txt
>
> C.3.  UNC Paths
>
>    It is common to encounter file URIs that encode entire UNC strings in
>    the path, usually with all backslash "\" characters replaced with
>    slashes "/".
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 15]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    To interpret such URIs, the "auth-path" rule in Section 2 is replaced
>    with the following:
>
>       auth-path      = [ file-auth ] path-absolute
>                      / unc-authority path-absolute
>
>       unc-authority  = 2*3"/" [ userinfo "@" ] file-host
>
>       file-host      = inline-IP / IPv4address / reg-name
>
>       inline-IP      = "%5B" ( IPv6address / IPvFuture ) "%5D"
>
>    This syntax uses the "userinfo", "IPv4address, "IPv6address",
>    "IPvFuture", and "reg-name` rules from [RFC3986].
>
>       Note that the "file-host" rule is the same as "host" but with
>       percent-encoding applied to "[" and "]" characters.
>
>    This extended syntax is intended to support URIs that take the
>    following forms, in addition to those in Appendix A:
>
>    Non-local files:
>
>    o  The "traditional" representation of a non-local file, with an
>       empty authority and a complete (transformed) UNC string in the
>       path.  E.g.:
>
>       *  "file:////host.example.com/path/to/file"
>
>    o  As above, with an extra slash between the empty authority and the
>       transformed UNC string, conformant with the definition from
>       [RFC1738].  E.g.:
>
>       *  "file://///host.example.com/path/to/file"
>
>       This representation is notably used by the Firefox web browser.
>       See Bugzilla#107540 [Bug107540].
>
>    It also further limits the set of file URIs that can be translated to
>    a local file path to those with a path that does not encode a UNC
>    string.
>
> C.4.  Backslash as Separator
>
>    Historically some implementations have copied entire file paths into
>    the path components of file URIs.  Where DOS or Windows file paths
>    were copied thus, resulting URI strings contained unencoded backslash
>    "\" characters, which are forbidden by both [RFC1738] and [RFC3986].
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 16]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    It may be possible to translate or update such an invalid file URI by
>    replacing all backslashes "\" with slashes "/", if it can be
>    determined with reasonable certainty that the backslashes are
>    intended as path separators.
>
> Appendix D.  Example of IRI vs Percent-Encoded URI
>
>    The following examples demonstrate the advantage of encoding file
>    URIs as IRIs to avoid ambiguity (see Section 4).
>
>    Example: file IRI:
>
>    | Bytes of file IRI in a UTF-8 document:
>    |    66 69 6c 65 3a 43 3a 2f 72 65 c3 a7 75 2e 74 78 74
>    |    f  i  l  e  :  c  :  /  r  e  ( c ) u  .  t  x  t
>    |
>    | Interpretation:
>    |    A file named "recu.txt" with a cedilla on the "c", in the
>    |    directory "C:\" of a DOS or Windows file system.
>    |
>    | Character value sequences of file paths, for various file system
>    | encodings:
>    |
>    |  o UTF-16 (e.g. NTFS):
>    |       0043 003a 005c 0072 0065 00e7 0075 002e 0074 0078 0074
>    |
>    |  o Codepage 437 (e.g. MS-DOS):
>    |       43   3a   5c   72   65   87   75   2e   74   78   74
>
>    Counter-example: ambiguous file URI:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 17]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    | Percent-encoded file URI, in any ASCII-compatible document:
>    |    "file:///%E3%81%A1"
>    |
>    | Possible interpretations of the file name, depending on the
>    | encoding of the file system:
>    |
>    |  o UTF-8:
>    |       <HIRAGANA LETTER TI (U+3061)>
>    |
>    |  o Codepage 437:
>    |       <GREEK SMALL LETTER PI (U+03C0)> +
>    |       <LATIN SMALL LETTER U WITH DIAERESIS (U+00FC)> +
>    |       <LATIN SMALL LETTER I WITH ACUTE (U+00ED)>
>    |
>    |  o EBCDIC:
>    |       "Ta~"
>    |
>    | etc.
>
> Appendix E.  UNC Syntax
>
>    The UNC filespace selector string is a null-terminated sequence of
>    characters from the Universal Character Set [ISO10646].
>
>    The syntax of a UNC filespace selector string, as defined by
>    [MS-DTYP], is given here in Augmented Backus-Naur Form (ABNF)
>    [RFC5234] for convenience.  Note that this definition is informative
>    only; the normative description is in [MS-DTYP].
>
>       UNC = "\\" hostname "\" sharename *( "\" objectname )
>       hostname   = netbios-name / fqdn / ip-address
>       sharename  = <name of share or resource to be accessed>
>       objectname = <depends on resource being accessed>
>
>    o  "netbios-name" from [MS-NBTE], Section 2.2.1.
>
>    o  "fqdn" from [RFC1035] or [RFC1123]
>
>    o  "ip-address" from Section 2.1 of [RFC1123], or Section 2.2 of
>       [RFC4291].
>
>    The precise format of "sharename" depends on the protocol; see: SMB
>    [MS-SMB], NFS [RFC7530], NCP [NOVELL].
>
>
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 18]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
> Appendix F.  Collected Rules
>
>    Here are the collected syntax rules for all optional appendices,
>    presented for convenience.  This collected syntax is not normative.
>
>       file-URI       = file-scheme ":" file-hier-part
>
>       file-scheme    = "file"
>
>       file-hier-part = "//" auth-path
>                      / local-path
>
>       auth-path      = [ file-auth ] path-absolute
>                      / [ file-auth ] file-absolute
>                      / unc-authority path-absolute
>
>       local-path     = [ drive-letter ] path-absolute
>                      / file-absolute
>
>       file-auth      = [ userinfo "@" ] host
>
>       unc-authority  = 2*3"/" [ userinfo "@" ] file-host
>
>       file-host      = inline-IP / IPv4address / reg-name
>
>       inline-IP      = "%5B" ( IPv6address / IPvFuture ) "%5D"
>
>       file-absolute  = "/" drive-letter path-absolute
>
>       drive-letter   = ALPHA ":"
>                      / ALPHA "|"
>
>    This collected syntax is intended to support file URIs that take the
>    following forms:
>
>    Local files:
>
>    o  A traditional file URI for a local file, with an empty authority.
>       E.g.:
>
>       *  "file:///path/to/file"
>
>    o  The minimal representation of a local file, with no authority
>       field and an absolute path that begins with a slash "/".  E.g.:
>
>       *  "file:/path/to/file"
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 19]
> 
> Internet-Draft                 file-scheme                 December 2015
>
>
>    o  The minimal representation of a local file in a DOS- or Windows-
>       based environment, with no authority field and an absolute path
>       that begins with a drive letter.  E.g.:
>
>       *  "file:c:/path/to/file"
>
>    o  Regular DOS or Windows file URIs, with vertical line characters in
>       the drive letter construct.  E.g.:
>
>       *  "file:///c|/path/to/file"
>
>       *  "file:/c|/path/to/file"
>
>       *  "file:c|/path/to/file"
>
>    Non-local files:
>
>    o  The representation of a non-local file, with an explicit
>       authority.  E.g.:
>
>       *  "file://host.example.com/path/to/file"
>
>    o  The "traditional" representation of a non-local file, with an
>       empty authority and a complete (transformed) UNC string in the
>       path.  E.g.:
>
>       *  "file:////host.example.com/path/to/file"
>
>    o  As above, with an extra slash between the empty authority and the
>       transformed UNC string.  E.g.:
>
>       *  "file://///host.example.com/path/to/file"
>
> Author's Address
>
>    Matthew Kerwin
>    Queensland University of Technology
>    Victoria Park Road
>    Kelvin Grove, QLD  4059
>    Australia
>
>    Email: [email-masked]
>
>
>
>
>
>
>
>
>
> Kerwin                    Expires June 3, 2016                 [Page 20]

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 12 Apr 2016 20:21:32 -0300

On Tue, Apr 12, 2016 at 4:29 PM, Dave Crocker <[email-masked]> wrote:

> [...]
> In technical terms, document seems to suffer some confusion about its role
> as a format specification, versus as a protocol specification.  I believe
> this issue is basic and important.  It needs to be resolved.
>
> For a specification involving such a potentially and presumably important
> capability, I think significant community support should be required...
> unless the spec is to be offered as Experimental, which is the most I'm
> inclined to recommend at this point...
>

Thanks, Dave.  I've consulted with our ADs and they're fine with keeping
APPSAWG open to process this document so long as we see continued work on
its development.  If it goes stale again for too long, we'll likely shut
down without completing this final milestone and pass the document over to
DISPATCH for disposition under their handling model.  If we're sure we want
Standards Track, that will mean sponsorship by an AD, or a (likely
small/short) working group to finish it off.

Matthew, do you have the time to put into closing this out?

-MSK

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 13 Apr 2016 09:32:39 +1000

> Matthew, do you have the time to put into closing this out?

Yes, I don't foresee any big obstacles in the coming weeks/month. I'm
working my way through Dave's feedback right now, in fact.

Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 13 Apr 2016 18:28:26 +1000

Hi Dave,

Thanks for this review; it's nice and hefty. I'll work my way through it,
commenting inline where relevant, and will endeavour to incorporate your
suggestions into my working copy.

On 13 April 2016 at 05:29, Dave Crocker <[email-masked]> wrote:

> I should also comment that I've just consulted a number of others who have
> been involved in discussions about the draft and hope that they will
> response to this review, on the mailing list.
>
>
>
> Summary:
>
> The specification seeks to detail the file: URI scheme, which "identifies
> a file on a particular file system", and replaces RFC 1738.
>
> supportive, there have been notable exceptions. Worse, significant issues
> that were raised were not obviously resolved on the list.  (I looked at
> postings by others, immediately after the author's postings, to see whether
> there were statements of agreement that an issue had been resolved, and did
> not generally see them, concerning those substantive issues.)
>
>
> revisions, and was adopted by AppsAWG in January 2015, and has had a number
> of revisions.  I note a 9/26/2014 (pre-AppsAWG) comment from Daniel
> Stenberg:
>
This is a surprise, since this comment from Daniel was what triggered the
complete restructure of the draft into its current form (with a relatively
short normative body, and a bunch of informational stuff in appendices.)

>      encourage the APPSAWG to consider addressing these.
After some discussions with W3 and WHAT folk, I did include mention of
'origin' in the draft.

When I received the comments, fetch was very protocol-oriented, and 'file:'
doesn't belong to any particular protocol, so there was no obvious fit
there; and the draft as it was designed was a close match for the WHAT URL
standard as it was at the time (and from what I can see, still is now) so I
don't know what more need be said. If someone could tell me what sort of
thing they think I should be saying, then I could probably say it.

>
> One point of process discussion about the draft was to distinguish between
> documenting current practice, versus specifying enhancements. Another was
> to distinguish between use of the construct locally (within a user's own
> system) versus globally, across the public Web.  Again, neither matter
> appears to be resolved clearly within the draft, and I can't tell tell
> clearly which goals the draft targets, even with the Abstract text:
>
> These two goals entail a classic choice in development of a specification
> that deals with existing practice.  Clarity between the two goals and
> careful precision about which is being served, is essential.
>
>
Yes, this could really do with rewriting and clarification. More on this
below, in your inline breakdown.

> assessing the likelihood of implementation and use.  As of now, this is not
> possible.
>
>
Technically it's already implemented in a bunch of places, not always
entirely interoperably. Most of the non-interoperable parts are in the edge
cases, but the "common core" is pretty universal. Since there's no
(non-obsolete) spec that defines it, what I want to achieve with this draft
-- if nothing else -- is to put that "common core" in a (non-obsolete) spec.

If that just meant de-obsoleting RFC 1738 that would almost be enough
(although an update might be in order.) However the discussion that started
with that question has lead us to where we are now, with this draft.

For what it's worth, at least one potential implementation is likely: the
Ruby standard library OpenURI module, which was the original reason I
started pursuing this in the first place (
https://bugs.ruby-lang.org/issues/8544)

> this issue is basic and important.  It needs to be resolved.
I don't disagree. Guidance on how to clarify this would be appreciated. How
many URI specs don't also define a protocol? If this is the only one, then
I probably don't know how to set the precedent. (Some of this is sorted
below, with your suggestions.)

> inclined to recommend at this point...
To be honest, if that's what you think is most suitable and if we can't
improve it, I don't have a problem with Experimental. We're fighting
against decades of stagnation and ennui; if an Experimental spec can kick
some life into it and stir up the muck that's a good start. If something
resolves out of it, then there's nothing wrong with redoing it "properly"
in future, with more engagement and recent experience to call on.

But to reiterate, it's already implemented pretty widely so I don't think
it can count as an "experimental scheme" -- rather, this would be an
experiment in restandardising a diverged and stagnant scheme.

> I believe it merely updates it.  It essentially replaces only Section
Yeah, just like 4248 and 4266 did. I'll follow your guidance here. (If I
update an obsolete spec, do I inherit that obsoletion?)

> attempts to define -> defines
ACK

> storage naming convention, but whatever is correct, it should be made
A common core of what current file URI implementations support and do. At
the very least, a common core of syntax.

> implementations of ... URI-based file system accessing mechanisms?
Um, yes, I guess that's a way of putting it. Implementations of things that
use file URIs. Should I not write that is "implementations of this scheme"
or just "implementations"?

>> ​<snip>
Hmm, ok. Are parenthetical definitions like this alright?

> A file URI identified an object (a "file") stored in a structured
> object naming and accessing environment on a host (a "file system.")
> The URI can be used ...

> file URI, but rather are used to create or apply the URI.
Translating to a local file path is one you perform on the URI itself. How
about "...operations that can be performed on a file URI or the object it
identifies"? Or just "...on the object identified by a file URI" if you
don't like the transformation.

>    media type. -> media type [rfc6838].
Sure. Is that an informative reference, or normative? We're mentioning it
as something that doesn't apply here, so I'm not sure.

I will replace this whole paragraph, it's no longer correct. How about:

> This document specifies a syntax that is compatible with most extant
> implementations.  It also documents other less common or outdated
> constructs.

> operational issues, not the standards status.
Ok. How about I just remove "History" altogether? It doesn't add anything
technical or operational.

>    ... by former that...   missing word?
It says "...by the former that..."

> specification to being a file protocol specification...
Thank you for saying that, you've triggered a bit of a light-bulb moment
for me about why I've had so much trouble getting this draft straight in my
head -- maybe it is actually a protocol spec. That said, I'd rather cut it
back to be a URI scheme spec. If we need to define the protocol in future,
then that's a future issue.

> instead belong in the next 'Similar Technologies' section?
I was just listing other specs that roughly do what this one does. UNC
isn't the same as file:, but file: URLs are the same as file: URIs. That
said I could just wipe out the whole History section, then I'd probably
move the WHATWG reference to Similar Technologies.

> "describes but does not specify" seems an odd distinction.  Not exactly
Normative vs informative; the appendices are non-normative descriptions of
things that are currently done, or that could be done -- but not things
that MUST be done (this comes back to Daniel's comment from 2014.)

I will work on the language, and work on clarifying the document structure,
since I don't want this confusion to be an issue.

> concern in the document.  How is 'directly' different from 'no authority
Um, I mean something like "...can be access through the local file system
API without explicitly establishing network connections or engaging network
protocols." I don't know if that's any better.

I should also clarify that I'm talking about local files, not local URIs --
although they're equivalent. The definition of a local file URI is given in
section 3.

I'll try to rework it.

​I think we might have an operator precedence issue; I mean:

> file-hier-part = ( "//" auth-path ) / local-path​

​not:

> file-hier-part = "//" ( auth-path / local-path )

So auth-path and local-path can look the same, but are clearly
distinguished by the preceding double-slashes. I'll add parens to clarify
this (per guidance in https://tools.ietf.org/html/rfc2234#section-3.5 )

> ​
See above.

>  all previous -> the previous file URI
ACK

> language. It seems that the current text does not fully resolve the
I defined the authority using a reference to RFC3986, and then said that
the relevant syntactic subset is given as 'file-auth'. Is that not enough?
Also, "authority" is a word in its own right, not just a syntactic element
from RFC 3986. I use the word "path" throughout the text without defining
it, but nobody complains that there's no `path` in my ABNF.

> already and doesn't need a comment.
Hmm, yes, I suppose so. I was thinking that an element of the ABNF not
matched because it is [optional] is distinct from one not matched because
it's in an ( unused / alternative ) branch. I can fix it up easily enough,
I think, just by deleting a phrase.

I could change file-auth thus:

> file-auth = "localhost"
>​           / [ userinfo "@" ] host

(do I need parens again?)

Then the "special case" paragraph would just be a bit of text explaining
why "localhost" is its own special token. That said, I'm pretty sure I
already had that written at some point, and somebody complained that
"localhost" was redundant because it was already syntactically covered by
'host'. Should the ABNF lean towards a pure syntactic parser, or include
semantic chunks as well? (Asking for AD guidance here.)

> ​​
Sure thing, I'll work with that suggestion.

> provide protocol details and provides essentially no semantics.
​So, should the section even exist? Or should it just say: we're just
defining a scheme here, operations are protocol business.

BCP35 (http://tools.ietf.org/html/rfc7595#section-3.4) says that the scheme
definitions should define a default dereference operation. It doesn't seem
to account for schemes that aren't protocol-bound, though.

>    ooperations -> operations
Yep, already noted and fixed.

> information about file system operations?
Well, I had to define what a file is earlier.

Does it not follow from the previous two sentences? I can remove it if it's
too bad, and point out the analogy between file operations and file URI
operations.

> I don't know what this means.  And how is this sentence useful?
A file URI is a structured string, a file path is a structured string, and
a UNC string is a structured string. You can translate the former to and
from (one of) the latter.

In early version of the draft this translation was the only operation that
could be "performed on a file URI". We can remove it, but then should we
keep 3.1?

> What does this mean?  Where is it defined?
I guess it's short-hand for much more text about how some file systems have
concepts of relative vs. absolute paths and we have to use the absolute one
when such a thing exists. I'll think more about this one.

> sigil ???  pretty stylized vocabulary...
Well, it's a magical rune that doesn't mean anything outside of this
context. I've spent too much time in D&D (or possibly perl), sorry. Is
"token" better?

> What about the alternative of including a /non-empty/ authority field?
Hmm, yes, that's a point. If we want to keep this section, I should add
"localhost".

> and a file scheme URI are outside the scope of this specification.
That works for me.

> A slash is required at the end of a directory, even if there is no file
If you're using it as a directory then yes. If you're using it as the
ultimate object (the "file") then no. We defined "file" as an "object" in
the file system earlier, which (going with the UNIX interpretation that
everything is a file) can include directories. As far as I know most
non-UNIXy systems around today can deal with this interpretation too.
Should I spell it out, or leave it up to interpretation?

> appendix.  It is irrelevant except as an historical note.
I agree. I will look at moving it to an appendix.

>>    except that the authority MUST refer to the network-accesible node
ACK

> important discussion point, but it's not part of the spec.
Intro, or yet another appendix? I don't mind either way, it depends what
you think is best.

> arguably one can lay any encoding convention on top of any data store.
I think this sentence came out of a discussion with Dave Thaler, although I
can't recall for sure (it appeared some time around September 2014.) What
you say is true, but it's only useful if that encoding is compatible with
(or can be translated to another encoding that is compatible with) the file
system's, so there's still a dependence. I mean, the whole point of this
paragraph is to say that you ought to encode it in UCS(+UTF8), even if the
file system doesn't use that.

Do you have a suggestion for something better to write, that isn't flirting
with untruth?

> ​​
A lot of this is just quoting or paraphrasing RFC 3986. I could probably
replace​ a bunch of it with a reference, but that would take some editorial
effort on my part, so I won't attempt it immediately. This goes back to
Dave Thaler's comments about encoding file URIs and how he wants them to go
away completely because of it. (The IRI bit definitely comes from him,
filtered through my interpretation.)

> seems minimal in detail and utility.

mnot asked how 'file:' fits into the web's origin model; this sentence is
pretty much the answer. If it doesn't belong here, then I'm not sure how to
answer the question.

> Without the protocol-ish paragraph before it.

This same thought occurred to me yesterday. I'll move it. It means the
final sentence need to be fixed, though, since that's talking about
"origin" identifiers.

> normative statement and what does not.
Yes, I agree. This is really old language (possibly predating my draft, or
maybe suggested to me early on, I can't quite remember). Is it good enough
to get rid of this sentence, and move the paragraph from above to here
(since they address the same thing, modulo the term "user agents")?

> Is there really no stable, published document that says something similar?
​Not that I could discover.​

> ​<snip>
I've just spent most of a day not working on my day job and my family is
wondering where I am, so I'll leave the comments at that for now. Some of
the incorporated/draft changes to my working copy are online now:
http://phluid61.github.io/internet-drafts/file-scheme/

Cheers, and thanks again.
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 10:54:51 +0100

Hi Dave, Matthew,

On 13/04/2016 09:28, Matthew Kerwin wrote:
>> >      how afile://  is supposed to work."
>> >

This is my perception too.  My take is that the main body of the document does 
now cover core material that can be followed quite strictly.

But because of file:'s long history and accretions, it seemed helpful to have 
*informative* material describing variations that might be in the wild - I think 
moving these to the appendixes and clearly marking these as informative keeps 
them clearly separated from the core material about how it should be used.

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:19:38 +0100

Dave,

(writing as a s/w dev who uses URIs a lot...)

I think the role of file: URIs is slightly unusual as a protocol element.  It 
appears in contexts where a network address may be used, but (in my experience) 
is mainly used to trigger local access.  So in an application that uses a URI 
like "http://example.org/somedata" to address some data on the web, it can also 
be really useful to be able to use "file:///somelocaldata" to tell the 
application to read a local file.  What I never use is a file URI to actually 
access data from another system (*).  So I don't think that interop of 
cross-system data access is the primary concern here.

BUT, there is one case where common interpretation and interop is really 
important, and that is relative URI reference resolution.  Something I commonly 
do is use relative references (e.g. "someplace/somedata") in my data, which may 
be hosted on a web server (e.g. "http://www.example.org/") or in a local file 
system ("file:///home/user/exampledata/").  What is important to me is that the 
relative reference resolves consistently in these situations.  E.g., for the 
examples given, to yield absolute addresses respectively of:

   http://www.example.org/someplace/somedata
and
   file:///home/user/exampledata/someplace/somedata

In the past, the problem has been that the local system quirks cause deviant 
behaviour (e.g. what happens if the base URI used is based on a Windows path, 
e.g. "C:/someuser/") - it is "edge cases" like this that have caused me 
difficulties in the past running software on different systems.

I believe that Matthew's draft, as currently structured, does usefully navigate 
this space between desired and widely implemented core functionality and 
likely-to-be-encountered quirks.  I would hope that by publishing this as a 
standard document, we will see programming language URI-handling library 
functions converge to provide the defined core behaviour.

#g
--

(*) other than a mounted network file system.

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:34:59 +0100

> below, with your suggestions.)

A URI is a structured data element used in protocols (e.g. HTTP) and in data 
formats (e.g. HTML).  As such, I don't think it makes sense to call it a "protocol".

The expected interpretation of some URI schemes will depend on a protocol 
specification (e.g. HTTP), and its definition may appear in the protocol spec. 
E.g. https://tools.ietf.org/html/rfc7230#section-2.7.1, but the URI itself is a 
string that conforms to indicated syntax that can be used predictably with 
common URI handling operations such as resolution 
(https://tools.ietf.org/html/rfc3986#section-5).

As for other URI specs that don't define a protocol...

     urn:
     acct:
     about:
     cid:
     mid:
     ni:
     (etc)

(That's from a very brief look at 
http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml)

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:41:31 +0100

> experiment in restandardising a diverged and stagnant scheme.

+1

file:// URIs are *very* widely used - I don't think it makes sense to call them 
"experimental".  I think it should be standards-track, but if the community 
can't agree then I'd suggest "informational" with permanent registration.

(I don't anticipate any spec needing to normatively reference file: URI 
specifically, as opposed to URIs generically, so I don't see "informational" 
causing any problems.)

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:46:11 +0100

> then that's a future issue.

Yes, I think staying clear of "protocol" is probably best.

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:53:04 +0100

> Sure thing, I'll work with that suggestion.

I think this is pretty much redundant.  URIs are case sensitive per RFC3986.

(Local interpretation may not be, but I'd suggest that's a local implementation 
issue.  At most, make a note as a local handling issue in the appendices? 
Hmmm... are there any security considerations here that should be flagged - 
relating to possible unexpected aliasing from case-only differences between files.)

#g

---

From: Graham Klyne <[email-masked]>
Date: Wed, 13 Apr 2016 11:59:06 +0100

> Should I spell it out, or leave it up to interpretation?

I'd also say "yes".

This is an area where URI resolution works differently to file path resolution 
(on some systems), so it's not just an academic point.

If you don't include the trailing "/" on a directory used as a base URI, then 
the final directory segment gets dropped when performing URI resolution.

Developers working in this space will know this anyway, but I think it's important.

Also, "file:///home/user" and "file:///home/user/" are different URIs per 
RFC3986.  It's usially a good idea to avoid gratuitous URI aliasing, which 
suggests it would be good practice to always include the trailing "/".

#g
--

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 07:12:59 -0700

On 4/13/2016 3:41 AM, Graham Klyne wrote:
> file:// URIs are *very* widely used - I don't think it makes sense to
> call them "experimental".  I think it should be standards-track, but if
> the community can't agree then I'd suggest "informational" with
> permanent registration.

I need to clarify the basis for suggesting Experimental status.  (And I 
had to explain this a few times before posting the review, so it's no 
surprise I need to do it on-list.)

The motivation is not concern whether 'file:' is deployed and used.  Of 
course it is and very widely and very heavily.

The concern is with the specification itself.  If it is seeking to 
document existing practice, the question is whether it does that well 
and usefully.

So the idea behind Experimental -- and I see this as especially strong 
/because/ there already is very wide use of the file: construct -- is to 
publish the draft and see whether the community adopts use of it, to 
cover that existing practise.

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 07:18:38 -0700

> Yes, I think staying clear of "protocol" is probably best.

One of the challenges in writing Internet technical specifications, is 
to make sure that the focus is on bits over the wire, rather than code 
on a machine.  The former is system-independent.  The latter always 
suffers the risk of being implementation-dependent.

Even the primary use of file: for "local" environments needs this 
distinction, in terms of talking only about abstractions.

The writing style requirements for making this work well have a variety 
of characteristics and common practices.  I think there's no guide for 
this, though.  So it's more clinical art than mechanical science.

One, trivial suggestion that occurs to me is to not use the word 
implementation -- unless specification reviewing implementation history 
or the like -- and instead use the word 'use'.  It's a gimmick, but it 
might help.

d/

-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 07:21:02 -0700

On 4/13/2016 3:59 AM, Graham Klyne wrote:
> This is an area where URI resolution works differently to file path
> resolution (on some systems), so it's not just an academic point.

There is enough history with yes-vs-no about terminating syntax 
characters -- cf, trailing dot in a domain name -- that it might help to 
include an emphasizing/clarifying comment exactly along the lines of 
Grahams text, just above.

d/

-- 

   Dave Crocker

---

From: t.petch <[email-masked]>
Date: Wed, 13 Apr 2016 17:42:45 +0100

----- Original Message -----
From: "Graham Klyne" <[email-masked]>
Sent: Wednesday, April 13, 2016 11:41 AM

> > For a specification involving such a potentially and presumably
important
> file:// URIs are *very* widely used - I don't think it makes sense to
call them
> can't agree then I'd suggest "informational" with permanent
registration.

RFC2026

"   The "Experimental" designation typically denotes a specification
that
   is part of some research or development effort. "

No, I do not think that this I-D comes into that category.

Tom Petch

> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: t.petch <[email-masked]>
Date: Wed, 13 Apr 2016 17:39:58 +0100

----- Original Message -----
From: "Graham Klyne" <[email-masked]>
To: "Matthew Kerwin" <[email-masked]>; "Dave Crocker"
<[email-masked]>
Cc: <[email-masked]>; "Apps Discuss"
<[email-masked]>
Sent: Wednesday, April 13, 2016 11:19 AM

> functions converge to provide the defined core behaviour.

So do I.

I note that RFC3986 starts by saying
"   This document specifies an Internet standards track protocol .."
which I think a load of rubbish.  It then goes on to say
" A Uniform Resource Identifier (URI) is a compact sequence of
   characters that identifies an abstract or physical resource.  "
which I find spot on.  But in the days of RFC3986, the 'rubbish' had to
appear first:-(

I think that the IETF is expert at specifying protocols and may struggle
to find the right language to specify entities that are not protocols,
as commonly understood, such as DDLs, identifiers, meta-syntactic
languages and so on.  By which I mean that I disagree with Dave's
comments on this I-D relating to protocols; were we specifying a
protocol sensu stricto, they would have merit - but we are not.

Tom Petch

---

From: t.petch <[email-masked]>
Date: Wed, 13 Apr 2016 17:51:58 +0100

----- Original Message -----
From: "Graham Klyne" <[email-masked]>
Sent: Wednesday, April 13, 2016 11:59 AM

> This is an area where URI resolution works differently to file path
resolution
> suggests it would be good practice to always include the trailing "/".

I agree.  I am used to Windows where a trailing slash on a URI makes a
substantive difference.

Tom Petch

---

From: "John Levine" <[email-masked]>
Date: 13 Apr 2016 20:08:25 -0000

>cover that existing practise.

I coulda sworn that's what Proposed Standard means.

R's,
John

PS: More concretely, what's the timeline for the experiment and how
will we tell what the results are?

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 14 Apr 2016 07:08:54 +1000

> http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml)
Urgh, of course you're right, and I knew about those. In my mind I was
still hung up on the difference between a purely naming scheme (like urn)
and a very dereferenceable one (like mid).

As far as precedent, mid: and cid: are both dereferenceable URLs (that is,
they support an operation like HTTP's GET), but they're really old so don't
have the formalisms we expect in a modern spec. (In fact, they seem to
assume that the fact that they can get 'got' at all is basically implied by
the fact that they're URLs. (I'm sorry for writing that sentence.))

acct: and ni: are nice and modern, but don't define any methods per se
(although they do point to webfinger and/or define a mapping to HTTP URIs).

Given the precedent, and Dave's protocol-vs-format comment, I'm wondering
if it's even appropriate to include methods in the file: spec at all (BCP
35 only says the definition SHOULD include operations.)

Cheers

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 14 Apr 2016 07:15:38 +1000

> protocols." I don't know if that's any better.
Actually that's not precisely what I mean either. I think what I mean is:
can be accessed using only the information included in the file path (i.e.
not relying on other information such as a network address.)

That definition is a bit hairy for file naming schemes that include network
addresses (like UNC, or VMS with a node reference.)

Cheers

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 15:41:06 -0700

> I coulda sworn that's what Proposed Standard means.

Thought my text explained why not.

I'll try again:

    When a specification carries risk of affecting existing operations, 
there often is a requirement for it to demonstrate acceptable safety, 
prior to classing it as Proposed.

    Given that this specification relates to existing use of an 
extremely important construct -- and especially given how little 
community review it has received -- I'm suggesting invoking the safety path.

d/

-- 

   Dave Crocker

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Apr 2016 10:02:24 +1000

The draft states:

"""
This document defines a syntax that is compatible with most extant implementations, while attempting to push towards a stricter subset of "ideal" constructs.  In many cases it simultaneously acknowledges and deprecates some less common or outdated constructs.
"""

I'm concerned about how well this draft aligns with current and future behaviour of common clients (especially, but not exclusively, Web browsers), especially since it's making decisions about deprecation and subsetting.

Have we had any input or statements of support from those communities?

Is there a test suite or other data to validate the decisions made?

Absent those things, I question whether it should be published at all. Experimental doesn't seem like the correct path, because the experiment would effectively be to "see if this sticks." It's one thing to define a new protocol extension with the hope that it will get traction in time; it's another to do hope-based definition of existing protocol constructs.

Regards,

> On 14 Apr 2016, at 6:08 AM, John Levine <[email-masked]> wrote:
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 14 Apr 2016 12:11:05 +1000

On 14 April 2016 at 10:02, Mark Nottingham <[email-masked]> wrote:

> implementations, while attempting to push towards a stricter subset of
> "ideal" constructs.  In many cases it simultaneously acknowledges and
> deprecates some less common or outdated constructs.
> """
>
>
Actually since Dave's review the working copy now says:

"""
​This document specifies a syntax that is compatible with most extant
implementations. It also documents other less common or outdated constructs.
"""

​That is, the idealism and deprecation are both removed. Essentially it's
an update of the syntax from RFC 1738 to fit better with 3986, and some
information regarding unspecified edge cases and implementation-specific
divergences.

As I mentioned in my reply to Dave's review, I want to work out how to
clarify the intention and structure of the document; I think this helps
inform that.

I'm concerned about how well this draft aligns with current and future
> behaviour of common clients (especially, but not exclusively, Web
> browsers), especially since it's making decisions about deprecation and
> subsetting.
>
A little, mostly tangential or out of band.

Is it typical for lots of entities to do a similar thing in different ways,
and block standardisation by not engaging? It sounds snarky, but I'm
genuinely asking, since I'm still quite new to this whole open
standardisation thing and don't know the politics of it.

If it's my fault for not appropriately seeking that engagement, I'll
happily go seeking it (even if it's too late for this draft.)

Nope. How would that look? Would a list of URIs and the expected outcomes
of parsing/dereferencing them (alongside metrics of known implementations
that pass/fail) be enough?

Absent those things, I question whether it should be published at all.
> Experimental doesn't seem like the correct path, because the experiment
> would effectively be to "see if this sticks." It's one thing to define a
> new protocol extension with the hope that it will get traction in time;
> it's another to do hope-based definition of existing protocol constructs.
>
>
Since everyone "does" file URIs, but they all do it differently, and the
spec that defines it is ancient, underspecified, and officially obsolete,
would you suggest instead I write an informational RFC describing the
situation and what everyone is doing right now? Because I really don't want
to walk away and leave the situation the way it currently is.

Cheers

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 19:28:39 -0700

On 4/13/2016 7:11 PM, Matthew Kerwin wrote:
> On 14 April 2016 at 10:02, Mark Nottingham <[email-masked]
> <mailto:[email-masked]>>wrote:

just to lighten things up with a bit of superficial nit-picking about 
wording:

    http://grammarist.com/usage/existent-extant/

> ​That is, the idealism and deprecation are both removed. Essentially
>

>     I'm concerned about how well this draft aligns with current and
>     future behaviour of common clients (especially, but not exclusively,
>     Web browsers), especially since it's making decisions about
>     deprecation and subsetting.
>
> Is it typical for lots of entities to do a similar thing in different
> ways, and block standardisation by not engaging? It sounds snarky, but
> I'm genuinely asking, since I'm still quite new to this whole open

This is why focusing on bits over the wire is better than talking about 
software implementation details.  If the 'different ways' mean different 
bits over the wire, then they are using different formats or different 
protocols.  And they won't interoperate.

If they generate/parse the same bits and same semantics over the wire, 
this we don't care how the built the software to do it, because they 
/do/ interoperate.

> Nope. How would that look? Would a list of URIs and the expected
> outcomes of parsing/dereferencing them (alongside metrics of known
> implementations that pass/fail) be enough?

There has long been a part of the world that seeks to test network 
software with comformance suites, and the like.  The historical 
alternative in the IETF world is interoperability test between real 
implementations, rather than having independent implementations bang 
against an artificial, third-party 'test suite'.

(On the other hand, test suites can be very efficient for aiding basic 
debugging...)

d/

-- 

   Dave Crocker

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 14 Apr 2016 12:55:07 +1000

>    http://grammarist.com/usage/existent-extant/
Fair enough. I think I did mean 'extant', but I'm not super invested in the
word.

> this we don't care how the built the software to do it, because they /do/
> interoperate.
>
>
The 'different ways' are actually different bits over the wire. Mostly
those are the bits that weren't part of the original spec, but were widely
deemed useful/necessary. I've tried to sidestep too much controversy by
continuing not to specify them, but I did write down some of the ways some
folk have decided to represent them.

> software with comformance suites, and the like.  The historical alternative
> in the IETF world is interoperability test between real implementations,
> rather than having independent implementations bang against an artificial,
> third-party 'test suite'.
>
Mostly I just looked at the various browsers and APIs and worked out what
they had in common. It's hard to define an interoperability test for file
URIs, since they really only "interoperate" by being parsed/dereferenced
from shared hypertext documents -- i.e. basically a 'test suite.'

Cheers

---

From: Dave Crocker <[email-masked]>
Date: Wed, 13 Apr 2016 20:06:16 -0700

On 4/13/2016 7:55 PM, Matthew Kerwin wrote:
>     This is why focusing on bits over the wire is better than talking
>     about software implementation details.  If the 'different ways' mean
>     different bits over the wire, then they are using different formats
>     or different protocols.  And they won't interoperate.
>
>     If they generate/parse the same bits and same semantics over the
>     wire, this we don't care how the built the software to do it,
>     because they /do/ interoperate.
>
>
> those are the bits that weren't part of the original spec, but were
> widely deemed useful/necessary. I've tried to sidestep too much
> controversy by continuing not to specify them, but I did write down some
> of the ways some folk have decided to represent them.

OK.  My advice:

      If there is a common core of bits over the wire that they all do 
do, but then some /additional/ bits over the wire that are different, 
then write the spec for the common parts and note (but do not document) 
that there are various independent extensions.

      Treat any effort to document the variations as completely separate 
from the common core.

d/

-- 

   Dave Crocker

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Apr 2016 15:35:16 +1000

What do you consider "the wire" for the purposes of the file:// URL scheme?

> On 14 Apr 2016, at 1:06 PM, Dave Crocker <[email-masked]> wrote:
> 
>     If there is a common core of bits over the wire that they all do do, but then some /additional/ bits over the wire that are different, then write the spec for the common parts and note (but do not document) that there are various independent extensions.
> 
>     Treat any effort to document the variations as completely separate from the common core.
> 
> 
> d/
> 
> -- 
> 
>  Dave Crocker

---

From: Graham Klyne <[email-masked]>
Date: Thu, 14 Apr 2016 11:17:22 +0100

I'm not sure if this will help, or just add more noise, but...

TL;DR:  I don't think the "over the wire" issue should be elevated to the level 
of dogma.  Systems are more complicated than that.

...

For me, as a developer, the value of a file:// URI is not what gets sent "over 
the wire", but what does *not* get sent over the wire.

I mainly work on applications for "small data" (but very heterogeneous). I 
create data that can be served on the web, or locally from a file system.  The 
data itself is location agnostic (i.e. portable), and internally uses relative 
URI references.  The existence of file: URIs is important to this mechanism 
because they can be used locally as a base URI to access data that is stored 
locally.

So why do I feel the file: is an important specification that impacts 
interoperability in a network protocol suite?

1. it can be used uniformly as part of a structure that does operate 
predominantly over the network.

2. It can be used to provide a local context for interpreting data that *has* 
been sent "over the wire".

This is why I have said that (for me at least), the most important part of the 
file: URI spec is that it behaves consistently with respect to URI resolution.

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Thu, 14 Apr 2016 11:20:43 +0100

> Is there a test suite or other data to validate the decisions made?

Not specifically that I know of, but there are test suites for URIs in general 
that describe resolution behaviours that file:// URIs should conform to by 
virtue of following rules set out by RFC3986.

#g
--

---

From: Dave Crocker <[email-masked]>
Date: Thu, 14 Apr 2016 06:49:24 -0700

On 4/13/2016 10:35 PM, Mark Nottingham wrote:
> What do you consider "the wire" for the purposes of thefile://  URL scheme?

the bits that are exchanged 'between' producer and consumer.  by 
whatever mechanism does that.

d/

-- 

   Dave Crocker

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 15 Apr 2016 16:32:52 +1000

On 13 April 2016 at 20:59, Graham Klyne <[email-masked]> wrote:

> If you don't include the trailing "/" on a directory used as a base URI,
> then the final directory segment gets dropped when performing URI
> resolution.
>
​That's a good point. In my working copy I've added this as a footnote to
the algorithm:

""
Some file systems allow directory objects to be treated as files
in some cases.  This can be reflected in a file URI by using the
name of the ultimate directory as the file name (such that the URI does
not have a trailing slash "/").  Be aware that merging a relative URI
reference to such a base URI as per Section 5.2 of [RFC3986] could
remove the directory name from the resulting target URI.
""

Editorially a footnote might not be the best way to do it, but it gets the
words in there. Does that cover it?

> suggests it would be good practice to always include the trailing "/".
It's true that they are different URIs; in the same way
https://example.com/foo and https://example.com/foo/ are different URIs. In
that case it's the webserver itself that redirects a request for the former
to the latter. I don't know if we really need to say anything more about it
here, especially since we've pointed out an actual functional difference.

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 15 Apr 2016 17:10:14 +1000

On 13 April 2016 at 20:53, Graham Klyne <[email-masked]> wrote:

> ​​
>
> (Local interpretation may not be, but I'd suggest that's a local
> implementation issue.  At most, make a note as a local handling issue in
> the appendices? Hmmm... are there any security considerations here that
> should be flagged - relating to possible unexpected aliasing from case-only
> differences between files.)
>
>
I think the talk about case-sensitivity is worthwhile keeping inline, even
if it's touching on system-specific issues.

Regarding security considerations, I've added some tentative hand-waving:

""
Some file systems have case-sensitive file naming and some do not.
Care must (?) be taken to avoid issues resulting from possibly
unexpected aliasing from case-only differences between file paths or
URIs.
""

I'm open to suggestions for improvement (or deletion.)

Cheers

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Apr 2016 09:16:14 +0200

On 2016-04-15 09:10, Matthew Kerwin wrote:
> On 13 April 2016 at 20:53, Graham Klyne <[email-masked]
> <mailto:[email-masked]>>wrote:
>
>     ​​
>
>     implementation issue.  At most, make a note as a local handling
>     issue in the appendices? Hmmm... are there any security
>     considerations here that should be flagged - relating to possible
>     unexpected aliasing from case-only differences between files.)
>
>
> I think the talk about case-sensitivity is worthwhile keeping inline,
> even if it's touching on system-specific issues.
>

case-insensitive (DOS?) < case-preserving (NTFS) < case-sensitive (others)

...so maybe mention all the three cases.

Best regards, Julian

PS: it might also be good to touch Unicode normalization forms...

---

From: Graham Klyne <[email-masked]>
Date: Fri, 15 Apr 2016 12:14:53 +0100

Yesterday, in discussion with one of the original authors of RFC 1738, I learned 
something about file:// URIs that I had not previously realized.

The upshot of this for me, in particular with references to Mark Nottingham's 
comment 
(https://mailarchive.ietf.org/arch/msg/apps-discuss/pZG6iMoerYpa5qq6BkHa8j0Nfjo), is 
that there may be parts of this spec that go beyond the clarification exercise 
required to maintain file:// URI scheme as a standard-track specification.

I offer a new (not detailed) review with suggestions for elements that might be 
dropped, in the hope that what remains is truly just a clarification rather than 
a modification of the RFC1732 description of file:// URIs.

(My own experience and use of file:// URIs is not affected by what I learned, 
and in my previous reviews have assumed that contributions relating to UNC files 
in particular were based on appropriate knowledge or experience.)

...

I have learned that the intent of including a hostname component was *not* to 
allow some kind of cross-host file access to be invoked, but to act as a signal 
to software using the URI that was not running on the specified host that the 
corresponding resource was not dereferencable.

This is supported by, or at least consistent with, a close reading of the 
relevant text in https://tools.ietf.org/html/rfc1738#section-3.10:

[[
    A file URL takes the form:

        file://<host>/<path>

    where <host> is the fully qualified domain name of the system on
    which the <path> is accessible, and <path> is a hierarchical
    directory path of the form <directory>/<directory>/.../<name>.
]]

Also, RFC 1630 is more explicit that the file scheme is for *local* file access. 
  The following from https://tools.ietf.org/html/rfc1630 is much clearer about 
this than RFC1738:

[[
file

    The other URI schemes (except nntp) share the property that they are
    equally valid at any geographical place.

    There is however a real practical requirement to be able to generate
    a URL for an object in a machine's local file system.

    The syntax is similar to the ftp syntax, but in this case the slash
    is used to donate boundaries between directory levels of a
    hierarchical file system is used.  The "client" software converts the
    file URL into a file name in the local file name conventions.  This
    allows local files to be treated just as network objects without any
    necessity to use a network server for access.  This may be used for
    example for defining a user's "home" document in WWW.

    There is clearly a danger of confusion that a link made to a local
    file should be followed by someone on a different system, with
    unexpected and possibly harmful results.  Therefore, the convention
    is that even a "file" URL is provided with a host part.  This allows
    a client on another system to know that it cannot access the file
    system, or perhaps to use some other local mecahnism to access the
    file.

    The special value "localhost" is used in the host field to indicate
    that the filename should really be used on whatever host one is.
    This for example allows links to be made to files which are
    distribted on many machines, or to "your unix local password file"
    subject of course to consistency across the users of the data.

    A void host field is equivalent to "localhost".
]]

Reviewing https://tools.ietf.org/html/draft-ietf-appsawg-file-scheme-06 again in 
light of this, I have the following comments:

...

Section 1:

[[
]]

I don't think "deprecates" is right here, as it doesn't discourage any 
behaviours specifically allowed by RFC1732.  (cf. Mark's comment.)

...

Section 1.2:

It now seems to me that the role of a host name in UNC name is quite different 
to its (original) role in a file:// URI.  In light of this, should this section 
be dropped?

...

Section 2:

[[
]]

The inclusion of "userinfo @" is an extension to previous *specifications* of 
the file URI.  As such, I question whether this change should be included. 
Dropping it doesn't affect compatibility with RFC3986.

...

Section 3:

[[
    This specification neither defines nor forbids a mechanism for
    accessing non-local files.  See SMB [MS-SMB], NFS [RFC7530], NCP
    [NOVELL] for examples of protocols that can be used to access files
    over a network.  Also see Appendix C.2 for a non-normative discussion
    on translating non-local file URIs to and from UNC strings.
]]

Given the new information noted, this seems extraneous to me.  Suggest dropping.

...

Section 3.1:

There is an option to include the host name even for local files, as an 
indication that the same file should not be expected to exist on other hosts.

I think the default position, in practice, is to not specify a host name.  But 
if the applications expects that the full absolute URI may be passed to another 
system it may make sense to include it to avoid dereferencing the value in an 
inappropriate context.

...

Section 3.2:

Given the new information noted, this section seems extraneous to me.  Suggest 
dropping.

...

Section 4:

Given the new information, the discussion of encoding when sending a file URI to 
a different system seems less relevant.  I would be inclined to drop all but the 
first paragraph of this section.

...

Section 6:

If the userinfo option is removed (see above), the final paragraph becomes moot. 
  Suggest drop.

...

Appendix A:

The characterization of "local" and "non-local" files isn't really germane. 
Suggest drop the sub-categorization and just list the examples.

...

(I'm not reviewing Appendices B and C at this time, as they aren't affected by 
my new perspective.)

...

#g
--

---

From: Matthew Kerwin <[email-masked]>
Date: Mon, 18 Apr 2016 15:13:01 +1000

Hi,

Thanks for some Monday reading to keep my mind occupied :)

On 15 April 2016 at 21:14, Graham Klyne <[email-masked]> wrote:

> Yesterday, in discussion with one of the original authors of RFC 1738, I
> learned something about file:// URIs that I had not previously realized.
>
> The upshot of this for me, in particular with references to Mark
> Nottingham's comment (
> https://mailarchive.ietf.org/arch/msg/apps-discuss/pZG6iMoerYpa5qq6BkHa8j0Nfjo),
> is that there may be parts of this spec that go beyond the clarification
> exercise required to maintain file:// URI scheme as a standard-track
> specification.
>
> I offer a new (not detailed) review with suggestions for elements that
> might be dropped, in the hope that what remains is truly just a
> clarification rather than a modification of the RFC1732 description of
> file:// URIs.
>
> (My own experience and use of file:// URIs is not affected by what I
> learned, and in my previous reviews have assumed that contributions
> relating to UNC files in particular were based on appropriate knowledge or
> experience.)
>
>
This is toeing the edge of a catch-22; when I first came up with a draft it
was very minimal update from RFC 1738's ABNF to match RFC 3986. The very
strong suggestion then was that this wouldn't be enough, and I should
address all the other cruft that's built up over the decades.

Part of that cruft is in handling file URIs with a non-localhost hostname
like UNC paths.

...
>
> I have learned that the intent of including a hostname component was *not*
> to allow some kind of cross-host file access to be invoked, but to act as a
> signal to software using the URI that was not running on the specified host
> that the corresponding resource was not dereferencable.

>    directory path of the form <directory>/<directory>/.../<name>.
I entirely agree. That's why I have this text: "A file URI can be
dependably dereferenced or translated to a local file path only if it is
local. A file URI is considered “local” if it has no file-auth, or the
file-auth is the special string “localhost”."

I should note that there is some historical baggage here. If I'm on a
machine called "fred", can I dereference this URL? "file://fred/tmp/foo"

Surely, according to the spec, the answer is "yes". Otherwise putting
anything other than "localhost" in the 'host' part of the URL makes it
undereferenceable. However as I understand it most implementations actually
went with "no" (the heuristic presumably being: only a URL with a blank or
"localhost" host can be dereferenced).

Then on top of that some (notably Windows and IE) extended the heuristic to
be "if the host is blank, it's local; otherwise it's a network share." So
in Windows "file://localhost/tmp/foo" is accessed via SMB. On this final
point, after much discussion and back and forth, I went with RFC 1738's
specification of "localhost", leaving Windows and IE noncompliant.

So I'm not as far from 1738 as I could be.

Also, RFC 1630 is more explicit that the file scheme is for *local* file
> access.  The following from https://tools.ietf.org/html/rfc1630 is much
> clearer about this than RFC1738:
>
> [[
> file
>
This text was dropped when it was effectively replaced by RFC 1738. It also
says "...or perhaps to use some other local mechanism to access the file."

In either case, I don't believe I've violated the intent (or at least the
letter) of 1738 too badly. It may be my failure to communicate -- we'll see
as I address the comments below.

> Reviewing https://tools.ietf.org/html/draft-ietf-appsawg-file-scheme-06
> again in light of this, I have the following comments:
>
> ...
>
> Section 1:
>
> [[
As per my replies to Dave and Mark, this wording is already changed.

...
>
> Section 1.2:
>
> It now seems to me that the role of a host name in UNC name is quite
> different to its (original) role in a file:// URI.  In light of this,
> should this section be dropped?
>
>
They both name files, and potentially allow files to be dereferenced.
That's what I mean by "similar technologies." If it was "identical but for
syntax" I wouldn't bother working on the file URI scheme.

...
> The inclusion of "userinfo @" is an extension to previous *specifications*
> of the file URI.  As such, I question whether this change should be
> included. Dropping it doesn't affect compatibility with RFC3986.
>
>
Hmm, possibly. I'm trying to recall what discussions and ideas lead to it
looking the way it does in the draft. It may just be because that's what is
currently allowed/expected by those who do "use some other local mechanism
to access the file."

It could be removed to the relevant appendices, I think, without any issues.

...
> Given the new information noted, this seems extraneous to me.  Suggest
I admit it might come across that having the references here are a weaselly
way of saying, "I won't tell you what to do, but you should do this." I
think they belong in the document somewhere, though; perhaps in a relevant
appendix.

...
> I think the default position, in practice, is to not specify a host name.
> But if the applications expects that the full absolute URI may be passed to
> another system it may make sense to include it to avoid dereferencing the
> value in an inappropriate context.
>
>
I might just remove section 3.1 altogether; it should be enough to have the
ABNF syntax and then describe in prose the way the segments map to their
equivalents in a file path. Lazily, that also puts the burden of
identifying and dealing with all these edge cases on not-me.

All I really wanted to emphases was that you can't use backslashes, even in
Windows.

...
>
> Section 3.2:
>
> Given the new information noted, this section seems extraneous to me.
> Suggest dropping.
>
>
...or moving to an appendix, since in my experience lots of file URL
minters expect this to be a thing that works.

...
>
> Section 4:
>
> Given the new information, the discussion of encoding when sending a file
> URI to a different system seems less relevant.  I would be inclined to drop
> all but the first paragraph of this section.
>
>
​Except that there's no rule that says "you MUST NOT send a file URI to
another computer".

Imagine you're into modding your favourite obscure video game, but most of
the cool developers are Ukrainian. You unzip a self-contained bundle of
libraries and some very beautiful hand-crafted HTML-based documentation
where all the relative URLs look like "%E3%B3%E4/123.jpg". A clever person
knows the percent-encoded bit is in Windows-1251 and matches the directory
called "гід", but a dumb computer doesn't.

So we do still need to document cross-system encoding. That said, on
re-reading I do think the first sentence isn't so much an introductory
statement for the other two as a condensed and less-specified restatement.
I should work on it.

...
>
> Section 6:
>
> If the userinfo option is removed (see above), the final paragraph becomes
> moot.  Suggest drop.
>
> ...
>
> Appendix A:
>
> The characterization of "local" and "non-local" files isn't really
> germane. Suggest drop the sub-categorization and just list the examples.
>
> ...
>
> (I'm not reviewing Appendices B and C at this time, as they aren't
> affected by my new perspective.)
>
> ...
>
>
So in summary you're suggesting:

1. if the hostname is given and resolves to the local machine, it's the
same as "localhost"

2. remove the userinfo from the authority

3. don't mention UNC or network shares or "non-local" files in the main text

Is that right (if glib)?

Regarding #1, we might have issues with Windows, IE, and Chrome, among
others.

For #2 I'll need to ask around and see what is expected here.

And as to #3, I don't know. I might be able to clean it up much more as
part of my effort to make the whole thing more readable and clear of
purpose. I'll give it some solid work this afternoon and tomorrow.

Cheers

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Apr 2016 15:07:14 +0900

> case-insensitive (DOS?) < case-preserving (NTFS) < case-sensitive (others)

DOS, at least to the extent I know it from modern Windows, is also 
case-preserving.

> PS: it might also be good to touch Unicode normalization forms...

Yes, this is relevant, because Apple is essentially NFD, but Windows and 
Linux don't care and usually are NFC.

Regards,   Martin.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 18 Apr 2016 00:00:59 -0700

On a purely procedural note:

Given the renewed activity here and the interest in making changes, I
presume this will be reworked enough now to warrant an eventual second
Working Group Last Call.  Unless there's objection to doing so, I'll move
the document back to that state.

-MSK

---

From: Graham Klyne <[email-masked]>
Date: Mon, 18 Apr 2016 08:55:27 +0100

Matthew,

(FWIW, I think you've done sterling work on this spec, and I'd hate to see it 
die like previous efforts did - my suggestions are offered in the hope of a 
route to sufficient consensus to allow it to proceed.  I've spoken to a few 
developers last week at the WWW conference about this, and the general feeling I 
get is that they'd like to see file:// URIs "back in the fold", primarily for 
their use as a unifying mechanism for local and web access.  My suggested 
approach to achieve this has been to stick very close to the intent of earlier 
file:// specs, while bringing it inline with RFC3986.  To the extent that 
documented extensions have community consensus I see no cause to pull them; my 
comments have come from a place of uncertainty about this.)

On 18/04/2016 06:13, Matthew Kerwin wrote:
> On 15 April 2016 at 21:14, Graham Klyne <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Nottingham's comment
>     (https://mailarchive.ietf.org/arch/msg/apps-discuss/pZG6iMoerYpa5qq6BkHa8j0Nfjo),
>     I offer a new (not detailed) review with suggestions for elements that might
>     be dropped, in the hope that what remains is truly just a clarification
>     rather than a modification of the RFC1732 description of file:// URIs.
>
>     learned, and in my previous reviews have assumed that contributions relating
>     to UNC files in particular were based on appropriate knowledge or experience.)
>
>
> This is toeing the edge of a catch-22; when I first came up with a draft it was
> very minimal update from RFC 1738's ABNF to match RFC 3986. The very strong
> suggestion then was that this wouldn't be enough, and I should address all the
> other cruft that's built up over the decades.
>
> Part of that cruft is in handling file URIs with a non-localhost hostname like
> UNC paths.

Yeah. These discussions have been running a long while now, and some of the rest 
of the world is moving on. That seemed reasonable to me at the time as long as 
it wasn't controversial - but if it now looks like divergence from current 
practice then I'd suggest it be more clearly discursive, not normative.

> I entirely agree. That's why I have this text: "A file URI can be dependably
> dereferenced or translated to a local file path only if it is local. A file URI
> is considered “local” if it has no file-auth, or the file-auth is the special
> string “localhost”."

Yes.  For me, the new understanding calls for something something stronger, 
maybe along the lines of:

"A file URI is intended to be dereferenced for local file access only.  The 
presence of a host component in the URI is intended to be used as a signal to 
implementations that are running on hosts other than the one named that the file 
is not accessible to them."

and maybe add (possibly in an appendix):

"Some implementations allow a host component in file URIs to access files on 
non-local hosts - such behaviour is beyond the scope of this specification."

>
> I should note that there is some historical baggage here. If I'm on a machine
> called "fred", can I dereference this URL? "file://fred/tmp/foo"
>
> Surely, according to the spec, the answer is "yes". Otherwise putting anything
> other than "localhost" in the 'host' part of the URL makes it undereferenceable.

Agreed. I think that's clear in the original RFC1630 text (but not so clear in 
RFC1732).

> However as I understand it most implementations actually went with "no" (the
> heuristic presumably being: only a URL with a blank or "localhost" host can be
> dereferenced).

Maybe so - I think that is something that got obscured in RFC1732.  For myself, 
and developers I've spoken to, this is somewhat moot as the file:// URI is 
mostly used as a relative reference resolved against a locally generated base 
URI, or just specified locally.

>
> Then on top of that some (notably Windows and IE) extended the heuristic to be
> "if the host is blank, it's local; otherwise it's a network share." So in
> Windows "file://localhost/tmp/foo" is accessed via SMB. On this final point,
> after much discussion and back and forth, I went with RFC 1738's specification
> of "localhost", leaving Windows and IE noncompliant.

I recall that, and I agree with the outcome re "localhost".  (Implementations 
are always free to be non-compliant - it can hurt interop when they are, but in 
this case I think maybe not so much.)

> So I'm not as far from 1738 as I could be.

Sure.  I think it's the adaptations to bless remote file access that may be 
problematic - the rest looks fine to me.

> This text was dropped when it was effectively replaced by RFC 1738. It also says
> "...or perhaps to use some other local mechanism to access the file."
>
> letter) of 1738 too badly. It may be my failure to communicate -- we'll see as I
> address the comments below.
>
>
>

Ack.  (I haven't seen the new text yet.)

>     different to its (original) role in a file:// URI.  In light of this, should
>     this section be dropped?
>
>
> They both name files, and potentially allow files to be dereferenced. That's
> what I mean by "similar technologies." If it was "identical but for syntax" I
> wouldn't bother working on the file URI scheme.

But the UNC form, to my perspective, is specifically about accessing files on 
different hosts - something which was not part of the original intent of file:.

This is why I suggest it might be dropped (or moved to an informative 
appendix?).  In the face of Mark's push-back, I think that in the absence of 
hard evidence of widespread implementation we need to focus on clarifying what 
is already stated in RFC1732, and bringing it in line with RFC3986.

My understanding is that the primary purpose of this spec is to bring file:// 
URIs back onto the standards track following obsoleting of RFC1732, and that is 
something I'd really like to see myself.  My suggestions are attempting to 
remove the obstacles to this.

> currently allowed/expected by those who do "use some other local mechanism to
> access the file."
>

I could live with that.  I have never personally come across an implementation 
of file:// that uses that form (or if I did I didn't notice it).

(See also comments at the end of this message.)

> I admit it might come across that having the references here are a weaselly way
> of saying, "I won't tell you what to do, but you should do this." I think they
> belong in the document somewhere, though; perhaps in a relevant appendix.
>

Now it's clearer to me that file:// was never intended to allow remote access, I 
think an appendix that positions it more clearly as a non-standard extension 
would be more appropriate if the text is to remain.

> I might just remove section 3.1 altogether; it should be enough to have the ABNF
> syntax and then describe in prose the way the segments map to their equivalents
> in a file path. Lazily, that also puts the burden of identifying and dealing
> with all these edge cases on not-me.
>
> All I really wanted to emphases was that you can't use backslashes, even in Windows.

Ack.

> ...or moving to an appendix, since in my experience lots of file URL minters
> expect this to be a thing that works.
>

Ack.

> ​Except that there's no rule that says "you MUST NOT send a file URI to another
> computer".

Sure, but the intent of the original file:// spec is that if you do this, it not 
be dereferencable on that other computer.

But I take your point, and agree, that encoding is still relevant for relative 
references.

>
> Imagine you're into modding your favourite obscure video game, but most of the
> cool developers are Ukrainian. You unzip a self-contained bundle of libraries
> and some very beautiful hand-crafted HTML-based documentation where all the
> relative URLs look like "%E3%B3%E4/123.jpg". A clever person knows the
> percent-encoded bit is in Windows-1251 and matches the directory called "гід",
> but a dumb computer doesn't.
>
> So we do still need to document cross-system encoding. That said, on re-reading
> I do think the first sentence isn't so much an introductory statement for the
> other two as a condensed and less-specified restatement. I should work on it.
>
>
>     ...
>
>     Section 6:
>
>     (I'm not reviewing Appendices B and C at this time, as they aren't affected
>     by my new perspective.)
>
>     ...
>
>
> 1. if the hostname is given and resolves to the local machine, it's the same as
> "localhost"

Yes.  (I believe that's the intent of the original spec.)

> 2. remove the userinfo from the authority

I was, but see below.

> 3. don't mention UNC or network shares or "non-local" files in the main text

Yes.

> Is that right (if glib)?
>
> Regarding #1, we might have issues with Windows, IE, and Chrome, among others.

The point here, IMO, is not to *specify* all extension behaviours.  Not 
including extension behaviours in the spec doesn't prohibit them.

> For #2 I'll need to ask around and see what is expected here.

Hmmm, I can see your point here.  This is an incompatible extension to the 
original spec, in the sense that including a userinfo would be a syntax 
violation.  In light of my comment above, I see that may be too proscriptive.

Maybe keeping it in the syntax (for compatibility), but being clear that its 
effect is not defined by this specification?

>
> And as to #3, I don't know. I might be able to clean it up much more as part of
> my effort to make the whole thing more readable and clear of purpose. I'll give
> it some solid work this afternoon and tomorrow.

I think the key test here is whether it's seen as diverging from current 
practice - if there's consensus that this material does reflect current practice 
then I think there's no cause to pull it.  I was responding mainly to the 
concern that the spec was defining extension behaviours that do not reflect 
current widespread practice.

Thanks for your efforts!

#g
--