Subject: Re: 304 or 412 for If-Modified-Since?

From: Julian Reschke <[email-masked]>
Date: Thu, 01 Jul 2010 09:43:46 +0200

On 01.07.2010 01:37, Elias Sinderson wrote:
> ...
> With some further analysis, my earlier statement obviously does not
> hold. Here are my current thoughts --
>
> For If-Modified-Since, depth requests can only really be evaluated
> correctly by iterating over the entire set of resources in scope. Given
> the unfortunate diversity in how timestamps are managed by different
> systems, the only reliable way to support this would be to specify that
> the server must consider the entire set of resources in scope and treat
> If-Modified-Since as if it applies to everything. That is, we can't
> indicate any optimizations in the spec wrt evaluation of a collections
> children / descendants. Given that, however, this is a rather
> straightforward feature to support since a single date is used across
> multiple comparisons and there may be some obvious performance
> optimizations to be made for a specific architecture.
> ...

I'm not completely sure I understand.

Are you saying that the overall response code (not the multistatus body) 
for a PROPFIND with Depth:infinity and If-Modified must take the 
last-modified dates for all resources in scope into account?

I don't think this is what the spec says, nor that it should (it would 
be extremely expensive to implement, and to work reliably would require 
internal atomicity that usually can't be guaranteed).

> If-None-Match is more interesting. For this, we need something akin to
> CTags. I thought this had been discussed on the dist-auth list

I'm not convinced we need those. We really should make all of this more 
compatible with GET, and rely on ETags.

> previously but a quick search of the archives came up empty. Cyrus has
> written up a quickie draft proposal on this idea for CalDAV [1], so
> maybe it was only discussed on that list? A critical behavior to address
> is whether CTag changes 'bubble up' to parent containers. The existing
> writeup doesn't specify this, but it is necessary if CTags are to
> support the semantics necessary for efficient evaluation of PROPFIND /
> REPORT type methods at depth. While obviously a performance hit for
> servers to maintain, it would provide some (perhaps more?) performance
> benefit downstream when evaluating requests with depth 1 or infinity.
>
> Have you considered this at any length before?

I'm currently not in the business of writing servers. That being said, 
change tags "bubbling up" will be very expensive to implement in many 
servers.

It would probably be a good idea to look at the problem that needs to be 
solved first. Maybe defining a related resource that aggregates all this 
hierarchy state (and which as a proper ETag) is all that's needed.

Best regards, Julian

---

From: Elias Sinderson <[email-masked]>
Date: Thu, 01 Jul 2010 18:53:08 +1000

Julian Reschke wrote:
>> [...] For If-Modified-Since, depth requests can only really be evaluated
>> correctly by iterating over the entire set of resources in scope. [...]
> Are you saying that the overall response code (not the multistatus 
> body) for a PROPFIND with Depth:infinity and If-Modified must take the 
Sorry, I should have been more specific -- I would expect to see 412's 
in the <D:status> elements of the multistatus response, without any 
corresponding propertied returned. Assuming, of course, that the server 
would evaluate the entire resource set at all... This would additionally 
require some mechanism for the client to indicate that it wanted the 
header applied beyond the target resource. (Along the lines of the DAV 
header, as you previously indicated, or otherwise.)

Yes, expensive, but perhaps not much more than the evaluation of the 
PROPFIND itself, without any If- headers.

>> If-None-Match is more interesting. For this, we need something akin 
>> to CTags. I thought this had been discussed on the dist-auth list [...]
> I'm not convinced we need those. We really should make all of this 
> more compatible with GET, and rely on ETags.
That would certainly simplify things, if possible.   :)

>> [...] Have you considered this at any length before?
Yes, agreed, especially where there is a 1:1 correspondence between 
[E/C]Tag updates and file I/O operations. I'm generally more willing to 
take this hit on the authoring side if it translates into comparable 
savings on the read side, and doubly so in domains that have better than 
an order of magnitude more reads than writes. IMO, this covers a the 
lions share of web apps.

> It would probably be a good idea to look at the problem that needs to 
> be solved first. Maybe defining a related resource that aggregates all 
> this hierarchy state (and which as a proper ETag) is all that's needed.
Possibly -- do you think such a resource could be generated more cheaply 
than maintaining hierarchies of [E/C]Tags? I think this depends largely 
on the implementation details of a given server architecture.

Regards,
Elias

---

From: Julian Reschke <[email-masked]>
Date: Thu, 01 Jul 2010 11:07:09 +0200

> header, as you previously indicated, or otherwise.)

The Depth header? I think this is what the definition of the Depth 
header already requires.

> PROPFIND itself, without any If- headers.

PROPFIND/Depth infinity is already expensive (and thus many servers do 
not allow it anyway). When they do, they better *stream* the result to 
the client. So, if there was a requirement to test all resources in 
scope against a condition *before* starting to send the multistatus this 
would be bad.

> on the implementation details of a given server architecture.

I'd think of it as a different way to put things on the wire, not really 
changing what the server needs to do in the end.

It would be great if we can avoid adding even more stuff to WebDAV, and 
re-use more of plain HTTP instead.

See, for instance: 
<http://greenbytes.de/tech/webdav/draft-reschke-http-get-location-latest.html>

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 01 Jul 2010 10:10:33 -0400

Hi Elias,

> on the implementation details of a given server architecture.

So the WebDAV collection Sync REPORT is meant to take over the role of a 
ctag and provide a true "what changed" synchronization capability. Please 
review past messages on this mailing list.

The one open issue on that REPORT (that I need to follow up on) is whether 
a true Depth:infinity capability should be added to the current Depth:1 
behavior.

-- 
Cyrus Daboo