Subject: Re: [apps-discuss] [Uri-review] URI scheme registration request - dchub

From: Graham Klyne <[email-masked]>
Date: Thu, 21 Feb 2013 12:14:49 +0000

Fredrik,

[I'm cc'ing [email-masked] on this response, as I feel there may be 
application protocol issues that list-members there may be able to comment upon 
with regard to specification, interoperability and security considerations issues.]

On 20/02/2013 20:26, Fredrik Ullner wrote:
> Hi,
>
> This is a request for registration of an URI scheme, dchub. The scheme can
> be viewed at <http://nmdc.sourceforge.net/nmdc-uri-scheme.txt>.
>
> The URI scheme should meet the necessary requirements for a Permanent URI
> scheme, as indicating in RFC 4395. The scheme specifies encoding,
> interoperability and security aspects (and more). If the scheme should not
> provide enough information for a Permanent scheme, please specify what is
> lacking or at least consider a Provisional request instead.

NOTE: I am IANA's designated reviewer for URI scheme registrations, but this is 
a personal response, and does not represent any formal position at this time.

You'll need to send your request to IANA (cf. RFC 4395, section 5.2, item 5) for 
it to be considered for registration.  Requesting review here is the appropriate 
first step - the Right Thing to do :)

I would allow that the scheme is appropriate for provisional registration.

But the lack of a public specification (according to 
http://en.wikipedia.org/wiki/Direct_Connect_(file_sharing), which is cited by 
your registration template) causes me to question whether it's appropriate for 
permanent registration at this time.  On the other hand, the diversity of 
implementations suggests permanent registration may be appropriate.

I see that there is a specification at 
http://nmdc.sourceforge.net/Versions/NMDC-1.1.html, linked via the other 
reference in your registration template.  The question, then, is the extent to 
which this can be regarded as definitive for the protocol.

Of the criteria for permanent registration given in RFC4395:
      2.1.  Demonstratable, New, Long-Lived Utility  . . . . . . . . .  4
      2.2.  Syntactic Compatibility  . . . . . . . . . . . . . . . . .  5
      2.3.  Well-Defined . . . . . . . . . . . . . . . . . . . . . . .  5
      2.4.  Definition of Operations . . . . . . . . . . . . . . . . .  6
      2.5.  Context of Use . . . . . . . . . . . . . . . . . . . . . .  6
      2.6.  Internationalization and Character Encoding  . . . . . . .  7
      2.7.  Clear Security Considerations  . . . . . . . . . . . . . .  7
      2.8.  Scheme Name Considerations . . . . . . . . . . . . . . . .  7

I would judge that requirements 2.1, 2.2 (not checked in detail), 2.5, 2.6 and 
2.8 are satisfied.  My comments noted above are concerns with respect to 2.3 and 
2.4.  I would also be concerned that 2.7 security concerns have not been given 
sufficient consideration and review for permanent registration.

To recommend permanent registration, I would like to see:

(a) some clear indication that there is good consensus in the relevant developer 
community about what constitutes a suitable specification of the protocol 
(preferably with indications that implementations conforming to aid 
specification are interoperable, but I wouldn't see that as a must-have 
requirement here).

(b) some review within the IETF, or a community with corresponding security 
expertise, of the security considerations.  As they stand, they look very thin 
to me.

I hope this helps.

#g
--

---

From: Graham Klyne <[email-masked]>
Date: Fri, 22 Feb 2013 15:13:20 +0000

On 21/02/2013 21:07, Fredrik Ullner wrote:
> On Thu, Feb 21, 2013 at 7:13 PM, Eric Johnson<[email-masked]>  wrote:
>> >
>> >The protocol itself ought to be filed as an RFC. That would pretty much
>> >guarantee permanence of the reference.

> It has actually been discussed previously. I have a semi-complete document
> written, but it's far from complete. I am also unsure of how one would
> manage messages that are now not used by any implementation. If people
> would feel that that would be the best approach, then I'm all for it.

"Ought to be" maybe too strong, but if dchub is widely used and supported that's 
an option to consider.  But if you go down that route, be aware that in can be a 
lot of work preparing and herding a spec through full IETF review.  As Eric 
says, that would almost certainly justify making the URI scheme registration 
permanent.

> I would like to invite people for discussion to the DC community. There is
> a development forum athttp://dcbase.org  as well as the development talk
> hub at adcs://hub.dcbase.org:16591 (note that this is another URI scheme
> that is intended to be registered in the future). (Download the application
> DC++, press Ctrl+Q and enter the adcs:// address to join.) Is there perhaps
> an IRC channel otherwise where one can discuss these things?

I took a quick look.  It appears to be a vibrant developer community there, but 
I got a bit lost looking, e.g., for possible clues about interoperability of 
different implementations.

The adcs: reference wasn't useful to me, as my browser doesn't know what to do 
with it - this is a perennial problem with new URI schemes, and is one reason 
why there is some reluctance to allow new permanent schemes - it's extremely 
difficult and expensive to get new schemes widely deployed.

#g
--

---

From: Fredrik Ullner <[email-masked]>
Date: Thu, 21 Feb 2013 22:07:55 +0100

I have uploaded the document to the SVN, so people can track changes that
are made to the document. See it here; <
http://nmdc.svn.sourceforge.net/viewvc/nmdc/trunk/nmdc-uri-scheme.txt?view=log
>.

The index page of nmdc.sf.net was incorrect, it did not have a news item
for 1.2, which contain more content. Note that the document is descriptive
of the protocol and its implementations and not prescriptive. The protocol
was built in 1999 (i.e., shortly after Napster and before BitTorrent) but
it has been expanded upon many times, but the original version was never
made public, hence reverse engineering efforts.

The protocol is very much still used today as it has been used in the past.
There are still many users using the protocol daily.

Note that the URI was constructed by Jonathan Hess for the original NMDC
client and this is an attempt at a formalization of that scheme. The
phrasing of the scheme is such that some current implementations only
support a partial of the syntax listed. There haven't been any innovation
(to the best of my knowledge) in the URI scheme in the past years.

I appreciate all suggestions and criticism. I didn't expect responses so
quickly. :)

On Thu, Feb 21, 2013 at 12:09 PM, Bjoern Hoehrmann <[email-masked]>
 wrote:
>
> Link "http://en.wikipedia.org/wiki/Direct_Connect(file_sharing)" should
> be "http://en.wikipedia.org/wiki/Direct_Connect_(file_sharing)".

Ah, yeah, of course.

On Thu, Feb 21, 2013 at 1:14 PM, Graham Klyne <[email-masked]> wrote:
>
> (a) some clear indication that there is good consensus in the relevant
> developer community about what constitutes a suitable specification of the
> protocol (preferably with indications that implementations conforming to
> aid specification are interoperable, but I wouldn't see that as a must-have
> requirement here).
>
I understand the concern, but I am unsure of the type of necessary explicit
consensus here. There are multiple implementations that are interoperable -
isn't that a de facto concensus? The protocol document does not specify
what implementations support what, but I'm not sure if that'd be applicable
in that type of document. (Another document perhaps that could be
interesting.) It wouldn't be difficult to construct such an implementation
matrix, it would simply require some time. (But obviously if that's
necessary to be considered, then that is something that must be done.)

On Thu, Feb 21, 2013 at 1:14 PM, Graham Klyne <[email-masked]> wrote:

> (b) some review within the IETF, or a community with corresponding
> security expertise, of the security considerations.  As they stand, they
> look very thin to me.
>
Noted, and I will discuss with others on potential reviews and revising of
this section (see also below).

On Thu, Feb 21, 2013 at 7:13 PM, Eric Johnson <[email-masked]> wrote:
>
> The syntax section:
>
>  * [a] discusses semantics. For example: "The client should download the
>    user's file list if path is omitted".
>  * [b] doesn't actually reference the syntax components of RFC 3986. It
>    doesn't explicitly exclude possible portions of the syntax, such as
>    fragment identifiers and queries. Am I to assume that those are
>    allowed, but not specified in their use, or disallowed?
>  * [c] seems to allow "dchub://" as a valid URI, however, if I follow RFC
>    3986 properly, that is not a valid URI, and at least a "host" is
>    required.
>  * [d] Is RFC 3986 "userinfo" allowed in the URI? If so, how does that
>    interact with the protocol?

a) Ah, ok. It could definitely move to the 'scheme' semantics section.
b/d) The syntax presented is what implementations support today. Anything
else apart from the explicit syntax listed should be considered as 'not
allowed' (save for an extension, which there hasn't been any).
c) Ah, "dchub://" would indeed be wrong; a host is required. Doesn't the
current syntax require it? "dchub://<host>" As far as I can tell, that
makes the host required.

On Thu, Feb 21, 2013 at 7:13 PM, Eric Johnson <[email-masked]> wrote:
>
> The semantics section:
>
>  * [e] should steal the portions from the syntax section that address the
>    meaning of a particular form, and further ought to tie that to
>    specific portions of the spec. As in "download the user's file list"
>    - which NMDC messages does this correspond to?
>  * [f] doesn't discuss at all how one might compare two of these URIs. Is
>    the user name case-sensitive? If characters occur that require
>    encoding via %xx, should those be normalized first before comparison?
>  * [g] Since the URI contains a path, does the path need to be normalized
>    before comparison?

e) Regarding protocol message correspondence: I wasn't aware that that was
required/desirable. Wouldn't the protocol specification specify what
commands were necessary for that? I mean, in the event that the download
command changes (a portion of it has actually done that), one wouldn't want
the URI scheme to be too narrow. (Admittedly, there haven't been changes in
this aspect in years, so there wouldn't be that much of a problem. I just
find it strange to have this type of information in an URI scheme.)
f/g) Nicks are not case-sensitive, as it is up to the hub to enforce this,
although this is not specified anywhere in the spec. I'll make sure to add
that as a note.   Addresses should be treated likewise as case-insensitive,
and I'd say that any such normalization should be done before comparison.
How do other URI schemes specify this in a more succinct manner?

On Thu, Feb 21, 2013 at 7:13 PM, Eric Johnson <[email-masked]> wrote:
>
> Under the "use" section:
>
>   * [h] An interoperability point is raised here that should be in the
> "interoperability" section.

h) I presume you mean "Not all implementations support the user and path
syntax". All right.

On Thu, Feb 21, 2013 at 7:13 PM, Eric Johnson <[email-masked]> wrote:
>
> The security section:
> * [i] Future implementations may care about using TLS to secure
>    communications. Will that need the use of "dchubs://", or is the
>    intent that the "dchub:" scheme can serve both purposes?
>  * [j] Might one want to actually include some sort of indicator (such as a
>    hash) in the URI (as part of a query) so that a client can request a
>    file, and if the user or path has changed, and now points to a
>    completely different file, that the provider and consumer of said
>    URI will be able to detect the discrepancy?
>  * [k] What of using weird constructs like "../../../foo/bar", as in
>    "dc-hub://host/myname/../../..**/foo" - does this need to make sure
>    that cannot be used to retrieve a file that wasn't shared, or wasn't
>    intended to be shared with the specific user.
>  * [l] DOS attacks, or DDOS attacks?
>  * [m] Homographs?
>
i) There are indeed implementations that support TLS, and they use the
syntax "nmdcs" (nmdcs://example.com). There is, as far as I'm aware, an
identical syntax and management otherwise. Should that be part of this URI
scheme or should it be filed as a completely separate URI scheme?
j) NMDC uses tiger Tiger Tree Hashes (TTHs), so in theory your suggestion
would be fine. However, no implementation currently supports that type of
syntax.
k) Most (all) implementations have safe guards for the ../ behaviour. I
suppose including a note about in the scheme is fine, although isn't it
more general to have it in the protocol specification ("implementations
should safe guard against relative paths when downloading")?
l) Ah, yes, this was identified (and to some extent managed) years ago.
Clients connecting to a hub will now send a "referrer" during protocol
negotiation (search for 'ref' in the spec) (similar to an HTTP referrer). A
note in the URI scheme sounds good.
m) I wasn't aware that that was a security concern here?

a development forum at http://dcbase.org as well as the development talk

-- 
Fredrik Ullner