Subject: Re: I-D Action:draft-reschke-webdav-post-00.txt

From: Julian Reschke <[email-masked]>
Date: Mon, 22 Sep 2008 17:42:54 +0200

FYI -- this is an early draft of what I proposed in July 
(<http://lists.w3.org/Archives/Public/w3c-dist-auth/2008JulSep/0011.html>).

Feedback appreciated,

Julian

[email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 	Title           : Using POST to add to Web Distributed Authoring and Versioning (WebDAV) Collections
> 	Author(s)       : J. Reschke
> 	Filename        : draft-reschke-webdav-post-00.txt
> 	Pages           : 11
> 	Date            : 2008-09-22
> 
> The Hypertext Transfer Protocol (HTTP) Extensions for the Web
> Distributed Authoring and Versioning (WebDAV) do not define the
> behavior for the "POST" method when applied to collections, as the
> base specification (HTTP) leaves implementers lots of freedom for the
> semantics of "POST".
> 
> This has lead to a situation where many WebDAV servers do not
> implement POST for collections at all, although it is well suited to
> be used for the purpose of adding new members to a collection, where
> the server remains in control of the newly assigned URL.  As a matter
> of fact, the Atom Publishing Protocol (AtomPub) uses POST exactly for
> that purpose.  On the other hand, WebDAV-based protocols such as the
> Calendar Extensions to WebDAV (CalDAV) frequently require clients to
> pick a unique URL, although the server could easily perform that
> task.
> 
> This specification defines a discovery mechanism through which
> servers can advertise support for POST requests with the
> aforementioned "add collection member" semantics.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-reschke-webdav-post-00.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 22 Sep 2008 12:26:22 -0400

Hi Julian,

--On September 22, 2008 5:42:54 PM +0200 Julian Reschke 
<[email-masked]> wrote:

> F

Looks good overall. Some comments:

1) When I first read the title I somehow thought this was talking about 
using POST to "create" collections. Perhaps change it to:

Using POST to add members to Web Distributed Authoring and Versioning 
(WebDAV)
                              Collections

2) I think this is a sufficiently worthy addition to WebDAV that using DAV: 
for the namespace is OK (just as we did with the current-user-principal 
property).

3) What about COPY/MOVE behavior? Is it OK to use the add-member property 
value as the Destination for COPY or MOVE when creating a new member in the 
destination? Or does a client instead have to user POST to create an empty 
resource first, then target that with the COPY/MOVE?

4) Something needs to be stated about error handling: e.g., "If there are 
pre-conditions related to creating a resource in the collection using a PUT 
request, then those same pre-conditions apply to the new POST request 
behavior, and the same DAV:error response body returned on failure". This 
would be a "catch-all" to allow protocols such as CalDAV, which restrict 
certain aspects of the data stored in collections, to simply return the 
same pre-condition failure information for POST as for PUT.

5) Is the server allowed to re-use new member URIs? i.e. /a/b.txt exists at 
some point, then is deleted. Is the server then allowed to use b.txt as a 
new member of /a/, or does the new member path segment have to be unique 
for the entire existence of that collection? If the member path segment is 
expected to be unique there should be some guidance to servers on how they 
might implement that (uuid's for instance).

6) It is also worth adding a note to client implementors stating that the 
server-generated member path segment is likely to not be suitable for 
direct presentation to end users, and instead clients should rely on the 
DAV:displayname property for presentation purposes.

7) Security consideration: "Servers MUST NOT derive the member path segment 
from the data being stored in the resource". This is important because you 
don't want server's leaking information in the URI that would not otherwise 
be visible (e.g. a user can PROPFIND the members but cannot read the 
content of the members - leaking content in the URI would be bad). Note 
this impacts how the server generates the member path segment. e.g. an md5 
hash of the data only may not be appropriate.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Mon, 22 Sep 2008 19:19:30 +0200

Hi Cyrus,

that may have been the world record for the fastest feedback after an 
Internet Draft being published!

> Using POST to add members to Web Distributed Authoring and Versioning 

Actually, it was supposed to say that. Thanks for pointing this out (as 
usually, current edits are at 
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-latest.html>).

> 2) I think this is a sufficiently worthy addition to WebDAV that using 
> DAV: for the namespace is OK (just as we did with the 
> current-user-principal property).

Thanks.

> 3) What about COPY/MOVE behavior? Is it OK to use the add-member 
> property value as the Destination for COPY or MOVE when creating a new 
> member in the destination? Or does a client instead have to user POST to 
> create an empty resource first, then target that with the COPY/MOVE?

The former would only work when the server actually has minted a 
separate URI (which I think will be the less frequent case).

That being said, that COPY/MOVE functionality could of course be exposed 
using *another* URI.

> 4) Something needs to be stated about error handling: e.g., "If there 
> are pre-conditions related to creating a resource in the collection 
> using a PUT request, then those same pre-conditions apply to the new 
> POST request behavior, and the same DAV:error response body returned on 
> failure". This would be a "catch-all" to allow protocols such as CalDAV, 
> which restrict certain aspects of the data stored in collections, to 
> simply return the same pre-condition failure information for POST as for 
> PUT.

Sounds good.

> 5) Is the server allowed to re-use new member URIs? i.e. /a/b.txt exists 
> at some point, then is deleted. Is the server then allowed to use b.txt 
> as a new member of /a/, or does the new member path segment have to be 
> unique for the entire existence of that collection? If the member path 
> segment is expected to be unique there should be some guidance to 
> servers on how they might implement that (uuid's for instance).

I don't expect it to be unique. Of course a collection *could* have that 
property, but that would need to be exposed differently (DAV:resourcetype?).

> 6) It is also worth adding a note to client implementors stating that 
> the server-generated member path segment is likely to not be suitable 
> for direct presentation to end users, and instead clients should rely on 
> the DAV:displayname property for presentation purposes.

That depends. For instance, if the client specified a Slug, and the 
server used that, the resulting URI actually is expected to work ok for 
display purposes.

> 7) Security consideration: "Servers MUST NOT derive the member path 
> segment from the data being stored in the resource". This is important 
> because you don't want server's leaking information in the URI that 
> would not otherwise be visible (e.g. a user can PROPFIND the members but 
> cannot read the content of the members - leaking content in the URI 
> would be bad). Note this impacts how the server generates the member 

Good catch.

> path segment. e.g. an md5 hash of the data only may not be appropriate.

...how could that one leak information?

BR, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 22 Sep 2008 14:53:20 -0400

Hi Julian,

--On September 22, 2008 7:19:30 PM +0200 Julian Reschke 
<[email-masked]> wrote:

> The former would only work when the server actually has minted a separate
> URI (which I think will be the less frequent case).
>

So p:copy-member and p:move-member properties?

One thing this reminds me of: another reason why this POST may be needed is 
if the server enforces a particular naming scheme on members. e.g., a 
CalDAV server may require that all member path segments match the UID in 
the calendar data, or match a record-id in its data store etc. In this case 
the add-member behavior is required. So its not just the case of "the 
client doesn't care to name members itself", but also this other one. 
Probably worth adding a comment on this.

> property, but that would need to be exposed differently

If they are not guaranteed to be unique then we could run into client 
synchronizations issues if a resource is deleted and then a new one added, 
but created using the same name as the old one. That new resource is really 
meant to be totally different from the old, but any clients that had cached 
the old one and attempt to re-synchronize will end up doing so with the new 
resource. That could cause all kinds of nastiness when two ways sync'ing 
might be expected (e.g, disconnected CalDAV clients).

So, it might be worth pointing out that there are some use cases (two way 
sync) where it is best for a server to generate unique member names (or 
rather not re-use old, now deleted, member names).

> ...how could that one leak information?

Well, knowing that users X, Y and Z all had the same member name in their 
different collections would imply that they all had received copies of the 
same resource. In calendaring that could mean they were all invited to the 
same event. Being able to infer that might not be ideal. But then again 
giving someone the ability to PROPFIND a collection but not read the 
contents of the members is arguably leaking too much anyway (though I have 
had several arguments about this with colleagues who disagree).

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Mon, 22 Sep 2008 21:09:09 +0200

> So p:copy-member and p:move-member properties?

Potentially.

My primary concern was to allow WebDAV collections to do something 
sensible with POST which is compatible with AtomPub, and which also 
addresses a real need that we've seen for instance with CalDAV. This 
turned out to be trivial.

I'm concerned to add something that's not as easy to achieve, and where 
I'm not sure about what the use case is.

Note that Microsoft does define an extension header for things like 
that, essentially allowing the server to "rename" the Destination URI. 
Maybe it's worthwhile looking at that for COPY/MOVE.

> One thing this reminds me of: another reason why this POST may be needed 
> is if the server enforces a particular naming scheme on members. e.g., a 
> the calendar data, or match a record-id in its data store etc. In this 
> case the add-member behavior is required. So its not just the case of 
> "the client doesn't care to name members itself", but also this other 
> one. Probably worth adding a comment on this.

Of course, that's a major reason to do that. I'll add that to the 
Introduction.

> synchronizations issues if a resource is deleted and then a new one 
> added, but created using the same name as the old one. That new resource 
> is really meant to be totally different from the old, but any clients 
> that had cached the old one and attempt to re-synchronize will end up 
> doing so with the new resource. That could cause all kinds of nastiness 
> when two ways sync'ing might be expected (e.g, disconnected CalDAV 
> clients).

You could use DAV:resource-id to distinguish them.

> So, it might be worth pointing out that there are some use cases (two 
> way sync) where it is best for a server to generate unique member names 
> (or rather not re-use old, now deleted, member names).

Agreed, but doesn't that consideration belong into the specification 
that defines that specific collection's behavior? (such as vcarddav?)

> ...
> Well, knowing that users X, Y and Z all had the same member name in 
> their different collections would imply that they all had received 
> copies of the same resource. In calendaring that could mean they were 
> all invited to the same event. Being able to infer that might not be 
> ideal. But then again giving someone the ability to PROPFIND a 
> collection but not read the contents of the members is arguably leaking 
> too much anyway (though I have had several arguments about this with 
> colleagues who disagree).

Thanks for the example.

And yes, whether non-accessible members of a collection should show up 
upon PROPFIND is an interesting question, and we're not going to solve 
that here.

(I personally believe that if the presence of a collection member's name 
already leaks out information, you really should restrict the read 
access to the parent collection...)

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 23 Sep 2008 17:09:48 +0200

> one. Probably worth adding a comment on this.

Well, if the server enforces a particular naming scheme, and does not 
support POST, then the client will need out-of-band information about 
the naming scheme, right? How would that work in practice?

I think what's more common is that the server would *like* to enforce a 
naming scheme, but can't, as clients assume PUT will just work. Isn't 
that the situation for many CalDAV servers?

What I've added is 
(<http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-issues.html#issue.member-uri-content>):

"As a matter of fact, letting the server choose the member URI not only 
is a simplification for certain types of clients, but can also reduce 
the complexity of the server (in that it doesn't need to persist an 
additional client-supplied identifier where the it already has an 
internal one like a UUID or a primary key)."

That said, I think I'm ready to submit a new draft soon; comments on 
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-latest.html> 
appreciated.

BR, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 23 Sep 2008 11:29:57 -0400

Hi Julian,

--On September 23, 2008 5:09:48 PM +0200 Julian Reschke 
<[email-masked]> wrote:

> support POST, then the client will need out-of-band information about the
> naming scheme, right? How would that work in practice?
>

Yes.

> is a simplification for certain types of clients, but can also reduce the
> complexity of the server (in that it doesn't need to persist an

                                              ^^^ remove "the"

> <http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-latest.html>

So one option for the server to enforce its naming scheme would be to 
require POST by the client to create new resources rather than allowing 
PUT/COPY/MOVE to do so. However there is no way fort a client to discover 
that such a restriction might be in place other than getting a 403. How 
about a new pre-condition code for that so that the server can indicate 
"DAV:only-post-allowed-to-create-new-members" to the client? With perhaps a 
more compact name!

This brings up another question: presumably the DAV:bind privilege is 
required on the collection for the new POST ;add-member behavior to be 
allowed (just as it would be required for PUT creating a new member). I 
think we therefore need a statement in Security Considerations:

"When a server supports WebDAV ACL [RFC3744], the DAV:bind privilege is 
required to be granted on the collection resource in which the new member 
resource is being created. If this privilege is denied or not present, the 
POST request MUST fail."

Another question: there is no restriction on what p:add-member URI can be? 
e.g. if I have the collection "/a/b/" can the p:add-member be another 
resource entirely, e.g. "/a/use-c-to-create-in-b/"? If this is possible it 
should be called out, as the behavior might be somewhat unexpected for 
clients. It might even be the case that the p:add-member URI is on a 
different server (e.g. new member items in a collection need "approval" 
from some other service). The interaction with WebDAV ACL in this case 
would need to be clear - i.e. what privileges are required on the 
p:add-member URI?

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Tue, 23 Sep 2008 17:46:12 +0200

>> additional client-supplied identifier where the it already has an

Fixed (thanks!).

> PUT/COPY/MOVE to do so. However there is no way fort a client to 
> discover that such a restriction might be in place other than getting a 
> 403. How about a new pre-condition code for that so that the server can 
> indicate "DAV:only-post-allowed-to-create-new-members" to the client? 
> With perhaps a more compact name!

Actually, it would be discoverable through OPTIONS.

When /collection does not allow PUT to add new members, an OPTIONS 
request on /collection/a should not return "PUT" in the Allow header.

That being said, stating that these kinds of collections could exist, 
and defining a precondition name certainly is a good idea.

> required to be granted on the collection resource in which the new 
> member resource is being created. If this privilege is denied or not 
> present, the POST request MUST fail."

Yes, although I'd move that into a "Relation to WebDAV ACL" section.

> Another question: there is no restriction on what p:add-member URI can 
> be? e.g. if I have the collection "/a/b/" can the p:add-member be 
> another resource entirely, e.g. "/a/use-c-to-create-in-b/"? If this is

I thought that was already clear; maybe I should have chosen a different 
URI :-)

> possible it should be called out, as the behavior might be somewhat 
> unexpected for clients. It might even be the case that the p:add-member 
> URI is on a different server (e.g. new member items in a collection need 
> "approval" from some other service). The interaction with WebDAV ACL in 

It seems that that kind of redirection would need a different mechanism. 
Let's not make things more complicated than they need to be.

> this case would need to be clear - i.e. what privileges are required on 
> the p:add-member URI?

I think it would be sufficient to state the ACL requirements in terms of 
DAV:bind on the "real" collection. The add-member URI in general could 
me a non-WebDAV resource, so talking about WebDAV ACLs really doesn't 
make sense here.

BR, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 23 Sep 2008 12:04:47 -0400

Hi Julian,

> I thought that was already clear; maybe I should have chosen a different

A couple of different examples would help make it clear.

> Let's not make things more complicated than they need to be.

OK, then let's explicitly state that the add-member URI must not be on a 
different server.

> DAV:bind on the "real" collection. The add-member URI in general could me
> a non-WebDAV resource, so talking about WebDAV ACLs really doesn't make
> sense here.

OK, makes sense.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Tue, 30 Sep 2008 18:57:45 +0200

> A couple of different examples would help make it clear.

<http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-latest.html#rfc.issue.uri-format>:

       Note: the "Add-Member" URI can be the identical to the
       collection's URI (in which case the server just advertises the
       fact that POST to the WebDAV collection's URI is supported as
       defined within this specification).  But it can also be different
       from it, in which case it doesn't need to have any relation to the
       collection's URI.

       Given a collection URI of

        /docs/collection/

       all of the following URIs might occur as "Add-Member" URIs:

        /docs/collection/
        /docs/collection/;post
        /docs/collection;post/
        /docs/collection/&post
        /post-service?path=/collection/

       The remainder of the document uses the same format just for
       reasons of consistency; any other HTTP URI would do as well.

> OK, then let's explicitly state that the add-member URI must not be on a 

Speaking of which, it could also be considered a security problem (the 
ability of server A to cause clients to post to server B != A).

BR, Julian