Subject: Re: Thoughts on relation to WebDAV

From: Petr Tomasek <[email-masked]>
Date: Thu, 22 May 2008 21:55:36 +0200

On Sat, Mar 15, 2008 at 11:55:44AM +0100, Julian Reschke wrote:
> 
> Hi,
> 
> here are a few thoughts about how to handle the WebDAV dependencies of 
> CardDAV (mostly based on what was discussed in the VCARDDAV WG meeting 
> in Philadelphia):
> 
> 1) WebDAV base requirements
> 
> I think CardDAV should require WebDAV level 3 support, as defined by RFC 
> 4918. It seems there was some confusion about what level 3 actually is: 

I think this is a big error, requiring all that stuff. Consider if one
would want to implement read-only CardDAV implementation.
Why should one worry about locking, MKCOL and the other stuff?

-- 
Petr Tomasek <http://www.etf.cuni.cz/~tomasek>
Jabber: [email-masked]
SIP: [email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 May 2008 11:40:44 +0200

> Why should one worry about locking, MKCOL and the other stuff?

In that case you would simply return 403 Forbidden.

Anyway, what does this have to do with the question whether CardDAV 
should require level 1 or level 3?

BR, Julian

---

From: Simon Perreault <[email-masked]>
Date: Fri, 23 May 2008 08:00:11 -0400

> Why should one worry about locking, MKCOL and the other stuff?

I think you're confused (as I was) about what class 3 implies. It does *not* 
imply locking. This is the very confusion that Julian is talking about.

I wonder where you got the strange idea that class 3 would include MKCOL. :)

---

From: Simon Perreault <[email-masked]>
Date: Fri, 23 May 2008 09:21:06 -0400

> I wonder where you got the strange idea that class 3 would include MKCOL.

Woah, that didn't make a lot of sense. Let's try again:

I wonder where you got the strange idea that class *1* would *not* include 
MKCOL. :)

...sleepy...

---

From: Henrik Holst <[email-masked]>
Date: Fri, 23 May 2008 15:36:46 +0200

> I wonder where you got the strange idea that class *1* would *not* include 
There is no MUST in RFC4918 for the MKCOL Method so class 1 does not
include MKCOL as far as I can see.

/Henrik

---

From: Simon Perreault <[email-masked]>
Date: Fri, 23 May 2008 09:50:14 -0400

> include MKCOL as far as I can see.

Is it intentional? I mean, RFC2518 says "All DAV compliant resources MUST 
support the MKCOL method." And advertising class 3 would not imply support 
for MKCOL, if I read the definition of class 3 correctly.

---

From: Henrik Holst <[email-masked]>
Date: Fri, 23 May 2008 15:57:23 +0200

> for MKCOL, if I read the definition of class 3 correctly.
Well that is interesting, class 1 would thus be slightly redefined in
4918 and supporting class 3 tells the client that your server follows
this redefinition I guess.

Hopefully we can get some info from the people involved with 4918 how
the ideas where when MKCOL was changed from MUST.

/Henrik Holst

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 May 2008 16:08:30 +0200

> the ideas where when MKCOL was changed from MUST.

It wasn't intentional. As far as I recall, we lost the first paragraph 
without considering compliance.

In practice I think it doesn't make any difference at all.

If a server vendor can't implement support for creating collections, 
he/she simply won't. A 403 always was possible, RFC4918 now also allows 405.

What difference does this make in practice?

BR, Julian

---

From: Werner Baumann <[email-masked]>
Date: Fri, 23 May 2008 17:55:20 +0200

Julian Reschke wrote:
>>> Is it intentional? I mean, RFC2518 says "All DAV compliant resources 
>>> MUST support the MKCOL method." And advertising class 3 would not 
>>> imply support for MKCOL, if I read the definition of class 3 correctly.
>>>

If it was not intentional, it should go in the Errata document to avoid 
confusion.

> What difference does this make in practice?

405 was allowed in RFC 2518 too. HTTP/1.1 also knows 501 Not Implemented.

RFC 2616:

   10.5.2 501 Not Implemented
   The server does not support the functionality required to fulfill
   the request. This is the appropriate response when the server does
   not recognize the request method and is not capable of supporting
   it for any resource.

RFC 2518/4918:

   403 (Forbidden) - This indicates at least one of two
   conditions: 1) the server does not allow the creation of collections
   at the given location in its namespace, or 2) the parent collection
   of the Request-URI exists but cannot accept members.

   405 (Method Not Allowed) - MKCOL can only be executed on a
   deleted/non-existent resource.

No difference in practice? Are clients supposed not to care about these 
different status codes?

A server, that does not implement MKCOL, is not compliant according RFC 
2616, but is compliant according RFC 4918. No difference in practice?

I have to concede: I never saw a compliant WebDAV-server. So does it 
make a difference in practice, whether there is a standard at all?

Werner

---

From: Julian Reschke <[email-masked]>
Date: Fri, 23 May 2008 18:34:43 +0200

> If it was not intentional, it should go in the Errata document to avoid 

Well, go on and report the erratum.

> No difference in practice? Are clients supposed not to care about these 

Again, what difference does it make *in practice*???

> 2616, but is compliant according RFC 4918. No difference in practice?

Yes, exactly.

> make a difference in practice, whether there is a standard at all?

Not sure what you're trying to say.

There are many ways how a server can refuse to do MKCOL. It can always 
claim "forbidden", it can say "not implemented" etc.

In practice it means the client won't be able to create collections.

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Tue, 27 May 2008 22:01:14 +0200

On 15.03.2008, at 11:55, Julian Reschke wrote:
> There are several things where CardDAV would benefit from generic  
> WebDAV extensions, most of which are covered by Cyrus' proposals:
>
> - using MKCOL to create "special" collections (instead of inventing  
> new methods all the time) (<http://tools.ietf.org/html/draft-daboo-webdav-mkcol-00.html 
> >)

I like this one.

> - potentially extract the REPORT method definition from RFC3253, and  
> move it into a separate spec, including clarifications

Something I would love to see is a generic WebDAV/HTTP mechanism to do  
the bulk resource GET, aka the CALDAV:calendar-multiget and  
CARDDAV:addressbook-multiget REPORTs.
Those are not really representation format specific, except that the  
XML result somehow implies a text entity.

I really would love to see this in a generic way instead of inventing  
new REPORTs for each protocol (just like with MKCOL).

Note: I'm not convinced that a REPORT is the proper way to do this. I  
think some kind of nested request would be nice (maybe a multipart?).

Probably this was discussed before? Was the conclusion that pipelining  
is a sufficient replacement?

Thanks,
   Helge
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 09:32:16 +0200

Helge Hess wrote:
> ...
> ...

Any kind of a batched GET defeats caching. So far I haven't seen 
evidence that doing it will work better than just issuing many GET requests.

So, no, there's certainly no consensus for or against it, but it would 
be really useful to have scenarios described and benchmarked...

BR, Julian

---

From: Jack Cleaver <[email-masked]>
Date: Wed, 28 May 2008 09:15:48 +0100

Julian Reschke wrote:
> 
> Helge Hess wrote:
>> ... Probably this was discussed before? Was the conclusion that
>> pipelining is a sufficient replacement? ...
> 

Should caching really be such a big issue for protocol designers in this
modern world? My ISP has just announced that it is abandoning its
20-year-old web-cache, on the grounds that:

- In a broadband world, end-users don't perceive much benefit
- In a world of SSL, web-apps and sessions, caches don't work anyway
- You can't seriously expect to cache any useful propertion of
   multimedia content

The impact of web-caches on global bandwidth usage is presumably
marginal, nowadays; HTTP page requests must be a pretty small fraction
of general bandwidth usage. [No stats to hand - ed.]

-- 
Jack.

---

From: Werner Donné <[email-masked]>
Date: Wed, 28 May 2008 10:30:13 +0200

I agree that most of the general bandwidth goes to YouTube, but
caches take off load from origin servers. Anyway, if caches have
become less relevant than a multiget doesn't represent much gain
either. The only overhead on a persistent connection is a few request
and response headers.

A multiget is a simple client-side aggregation. You can obtain the
complete tree with one request and then perform individual gets.
I do this for synchronisation purposes and the performance is just
fine.

Werner.

--
Werner Donné  --  Re                                     http://www.pincette.biz
Engelbeekstraat 8                                               http://www.re.be
BE-3300 Tienen
tel: (+32) 486 425803	e-mail: [email-masked]

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 11:14:09 +0200

On 28.05.2008, at 09:32, Julian Reschke wrote:
> Helge Hess wrote:
>> ...
>> Probably this was discussed before? Was the conclusion that  
> Any kind of a batched GET defeats caching.

Could you elaborate? An MGET capable cache could still disassemble the  
MGET and serve the individual resources from its cache (which is in  
fact why I would prefer a real MGET instead of a REPORT). It would  
really just be a nested request.

> So far I haven't seen evidence that doing it will work better than  
> just issuing many GET requests.

The overhead in database based servers is magnitudes higher (at  
various points, record lookup, flushes, etc).
In theory they could coalesce pipelined requests and then perform them  
in bulk, but this is very hard to do and not viable in practice.  
Besides that few clients do pipelining.

There are many backends which can do set operations magnitudes faster  
than individual ones. (but I think everyone is aware of that :-)

> So, no, there's certainly no consensus for or against it, but it  
> would be really useful to have scenarios described and benchmarked...

Its not a really good example because its not a hugely compliant  
WebDAV server nor implemented that well, but the OpenGroupware.org  
ZideStore serves iCal and vCard files via GroupDAV.
An individual GET with all the authentication, authorization and vCard  
building takes ~50ms which hurts a lot when you retrieve 10.000  
records, its ~8mins in the initial sync! We do a lot of caching etc to  
speed things up, but still its pretty slow.
Now a multi-GET on moderatly sized batches takes almost the same time,  
eg for 100 records its still ~70ms.

I think thats the reason why CalDAV added those multiget REPORTs, on  
request of RDBMS based server vendors.

Thanks,
   Helge

PS: I'm not suggesting to do this in the core HTTP protocol for  
general usage! Just as a well defined replacement for the REPORTs so  
that CalDAV, CardDAV, etc don't invent their own.

PS2: Are there servers which actually do pipelining properly? W/o it  
MGET is also pretty much required to avoid latency on mobile networks.
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 11:21:07 +0200

On 28.05.2008, at 10:15, Jack Cleaver wrote:
> Should caching really be such a big issue for protocol designers in  
> this modern world?

Yes, its quite important even in my setup.

> - In a world of SSL, web-apps and sessions, caches don't work anyway

I'm not so much concerned about proxy caches for clients, but reverse  
proxies for backends. Having a cache in front of appservers is really  
nice for scaling.

Also I'm looking for a more consistent replacement for CalDAV/ 
CardDAV:multiget, not necessarily a generic HTTP/1.x mechanism.

Thanks,
   Helge
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 12:05:00 +0200

Helge Hess wrote:
> MGET and serve the individual resources from its cache (which is in fact 
> why I would prefer a real MGET instead of a REPORT). It would really 
> just be a nested request.

OK, that would work, but will be tricky to deploy. *Existing* caches 
will not know how to do it.

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 13:08:36 +0200

On 28.05.2008, at 12:05, Julian Reschke wrote:
> Helge Hess wrote:
>> Could you elaborate? An MGET capable cache could still disassemble  
>> the MGET and serve the individual resources from its cache (which  
>> is in fact why I would prefer a real MGET instead of a REPORT). It  
>> would really just be a nested request.

Si. But they don't know how to deal with MKCOL or REPORT either, so I  
think thats not really relevant to my question :-)
Point is: MGET method would be cachable according to the rules which  
apply to the embedded requests.

So, does it make sense to do a 'mget-ext' proposal? What about caldav/ 
carddav vendors, would you implement it? Cyrus?

Maybe this should actually be a 'BATCH' method, not an 'MGET' (GET  
specific).

---snip:rq---
BATCH /calendar HTTP/1.1
content-type: x-http/batch-rq
content-len: xyz
accept-encoding: gzip

GET /calendar/123.ics
accept: text/calendar; charset-utf-8
if-none-match: "myetag"
content-length: 0

GET 345.ics # allow relative URLs? would be nice
accept: text/calendar; charset-utf-8
if-none-match: "myetag"
content-length: 0

POST /calendar HTTP/1.0
content-type: text/calendar; charset=utf-8
content-length: uujk

BEGIN:VCALENDAR
...
---snap---

---snip:res---
HTTP/1.1 200 OK
content-type: x-http/batch-res
content-len: xyz
content-encoding: gzip # the whole batch could be encoded, w/o TE

HTTP/1.1 304 Not Modified
content-length: 0
content-location: /calendar/123.ics

HTTP/1.1 200 OK
content-length: opq
content-type: text/calendar; charset=utf-8
content-location: /calendar/345.ics

BEGIN:VCALENDAR
...

HTTP/1.1 201 Created
content-length: 0
content-type: text/calendar; charset=utf-8
content-location: /calendar/999.ics
location: /calendar/999.ics

---snap---

Makes sense? I admit, its very similiar to pipelining, but much easier  
to implement using existing infrastructure.

Thanks,
   Helge
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Petr Tomasek <[email-masked]>
Date: Wed, 28 May 2008 13:15:55 +0200

> content-type: x-http/batch-rq

It would be sane if the BATCH method specified the number of consequent requests to be sent...

P.T.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 13:21:21 +0200

Helge Hess wrote:
>>> the MGET and serve the individual resources from its cache (which is 
>>> in fact why I would prefer a real MGET instead of a REPORT). It would 

> So, does it make sense to do a 'mget-ext' proposal? What about 
> caldav/carddav vendors, would you implement it? Cyrus?
> 
> 

The responses to REPORT *could* be made cacheable, if we adopt one of 
the proposals outlined in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-get-location-01.html>.

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 14:47:38 +0200

> consequent requests to be sent...

The content-length must be specified (as shown), but why do you need  
the number of requests?

Thanks,
   Helge

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 28 May 2008 10:02:23 -0400

Hi Helge,

> proxies for backends. Having a cache in front of appservers is really

Right, and in a lot of cases the web service itself will setup its own 
cache which will have knowledge of the backend servers. i.e. the need for 
some third-party intermediary to run a web cache may get less and less as 
the origin servers themselves use caches for scalability.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 28 May 2008 10:08:33 -0400

Hi Helge,

--On May 28, 2008 11:14:09 AM +0200 Helge Hess 
<[email-masked]> wrote:

> Its not a really good example because its not a hugely compliant WebDAV
> server nor implemented that well, but the OpenGroupware.org ZideStore
> serves iCal and vCard files via GroupDAV.
> building takes ~50ms which hurts a lot when you retrieve 10.000 records,
> its ~8mins in the initial sync! We do a lot of caching etc to speed
> things up, but still its pretty slow.
> Now a multi-GET on moderatly sized batches takes almost the same time, eg
> for 100 records its still ~70ms.
>

Well you don't need an RDBMS backed server to see the benefit. The Apple 
CalDAV server uses flat files to store each calendar resource plus an index 
in each calendar collection to optimize queries. multiget is a big win for 
us. One of the biggest factors is being able to aggregate the ACL check of 
all the resources. There are several optimizations that can be done, and in 
the best case only one privilege check is needed for all the resources 
being fetched.

The other key thing to remember is that multiget allows the client to fetch 
properties as well as data. i.e. its a combination of multi-PROPFIND as 
well as multi-GET. Obviously DAV:getetag is the one key property clients 
will fetch along with the data. That might not be enough to fully justify 
this approach, but it does keep it flexible.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 28 May 2008 10:10:19 -0400

Hi Julian,

--On May 28, 2008 12:05:00 PM +0200 Julian Reschke <[email-masked]> 
wrote:

> OK, that would work, but will be tricky to deploy. *Existing* caches will
> not know how to do it.

I could quite easily see specialized CalDAV/CardDAV caches being used that 
do have knowledge of multiget behavior and likely would be supplied by the 
CalDAV/CardDAV server vendors themselves for their own scalability reasons.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 28 May 2008 10:17:20 -0400

Hi Helge,

--On May 28, 2008 1:08:36 PM +0200 Helge Hess 
<[email-masked]> wrote:

> Makes sense? I admit, its very similiar to pipelining, but much easier to
> implement using existing infrastructure.

So I guess one question is whether you prefer to use serialized responses 
to wrap all this up or XML. I think in the WebDAV world XML makes sense as 
we already have the multistatus response.

-- 
Cyrus Daboo

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 16:46:27 +0200

On 28.05.2008, at 16:17, Cyrus Daboo wrote:
> --On May 28, 2008 1:08:36 PM +0200 Helge Hess <[email-masked] 
> > wrote:
>> Makes sense? I admit, its very similiar to pipelining, but much  
>> easier to implement using existing infrastructure.
> So I guess one question is whether you prefer to use serialized  
> responses to wrap all this up or XML. I think in the WebDAV world  
> XML makes sense as we already have the multistatus response.

Its not really WebDAV specific in any way, it could be used for  
arbitrary HTTP based protocols.

The problem with XML is that HTTP content is quite often binary :-)  
Makes sense for properties (eg inside a batch), but IMHO not for HTTP  
resource contents.

BATCH would be a simple concat of the requests/responses as if they  
where issued separately. (but giving the server the chance to group  
them for more efficient processing).

Thanks,
   Helge
-- 
http://helgehess.eu/

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 16:15:47 +0200

On 28.05.2008, at 16:08, Cyrus Daboo wrote:
> The other key thing to remember is that multiget allows the client  
> to fetch properties as well as data. i.e. its a combination of multi- 
> PROPFIND as well as multi-GET. Obviously DAV:getetag is the one key  
> property clients will fetch along with the data. That might not be  
> enough to fully justify this approach, but it does keep it flexible.

In that case one would just use GET with the 'etag' HTTP header in my  
'BATCH' method suggestion. Though you could also use BATCH to  
transport multiple PROPFINDs requests in one step.

Thanks,
   Helge
-- 
http://helgehess.eu

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 17:02:30 +0200

Helge Hess wrote:
> where issued separately. (but giving the server the chance to group them 
> for more efficient processing).

The problem with that is that it's extremely hard to deploy.

The right way would be to have it deep down in the stack, for instance 
the servlet container. But that doesn't happen easily.

If you implement on top, you basically have to re-implement most of the 
server-side HTTP stack to support it.

So it's kind of elegant to specify, but has its shortcomings :-)

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Wed, 28 May 2008 17:46:59 +0200

On 28.05.2008, at 17:02, Julian Reschke wrote:
> The right way would be to have it deep down in the stack, for  
> instance the servlet container. But that doesn't happen easily.
>
> If you implement on top, you basically have to re-implement most of  
> the server-side HTTP stack to support it.

Good point. After all the idea *is* to reimplement it on top, because  
the endpoint could aggregate common payloads in set operations.

But then the server always has the option to reject the BATCH with  
501, which would then make the client fallback.
Well, and a servlet which does WebDAV, needs to do most of the HTTP  
stack itself anyways ;-)

Thanks,
   Helge
-- 
Helge Hess
http://helgehess.eu

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 17:58:05 +0200

Helge Hess wrote:
> But then the server always has the option to reject the BATCH with 501, 
> which would then make the client fallback.

Well, not entirely. I'd say the stuff that is defined in Part1 of 
httpbis 
(<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-latest.html>) 
probably is done by the servlet container.

BR, Julian

---

From: Jack Cleaver <[email-masked]>
Date: Wed, 28 May 2008 17:24:57 +0100

Julian Reschke wrote:
>>  501, which would then make the client fallback. Well, and a
>> servlet which does WebDAV, needs to do most of the HTTP stack
>> itself anyways ;-)
> 

I found both Tomcat's and Jetty's HTTP stack to be unsuitable as a basis
for building a DAV stack. Certainly it *should* be as you suggest; but
these programs weren't designed with this kind of extension in mind.
Really, it is much easier to throw it all away and start again.

Jetty is much better than Tomcat in this respect; but even Jetty has
some limitations. This isn't really criticism; designing software to be
extensible without having much of a clue what extensions people are
going to try and build is very difficult.

-- 
Jack.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 28 May 2008 18:33:49 +0200

> going to try and build is very difficult.

Out of curiosity: what kinds of problems did you experience?

As far as I can tell, the most compliant and complete WebDAV servers are 
currently implemented as servlets...

BR, Julian

---

From: Jack Cleaver <[email-masked]>
Date: Wed, 28 May 2008 19:43:43 +0100

Julian Reschke wrote:
> Jack Cleaver wrote:
>> ... I found both Tomcat's and Jetty's HTTP stack to be unsuitable
>> as a basis for building a DAV stack. Certainly it *should* be as
>> you suggest; but these programs weren't designed with this kind of
>> extension in mind. Really, it is much easier to throw it all away
>> and start again.
>> 
>> Jetty is much better than Tomcat in this respect; but even Jetty
>> has some limitations. This isn't really criticism; designing
>> software to be extensible without having much of a clue what
>> extensions people are going to try and build is very difficult. ...
>> 
> 

As I recall, I couldn't override the built-in Tomcat HTTP method
handlers without overriding the HTTP servlet itself.

This was over a year ago, and I haven't touched the code for a while.

-- 
Jack.

---

From: Werner Donné <[email-masked]>
Date: Wed, 28 May 2008 21:01:20 +0200

Indeed, you have to override the HttpServlet class in order to
deal with the new methods, but I fail to see why that should be
a problem. My concerns regarding an application server are:

- that it complies with RFC 2616;
- that it complies with the relevant J2EE specifications;
- that it provides a robust infrastructure.

Werner.

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 29 May 2008 09:41:23 +0200

This BATCH/multi-GET topic comes up quite regularly every year or so.  
It's a pony discussion.

For all use cases I have seen so far either
1. pipelining was adequate for it
2. or examples where as you described with optimizations for multiple  
operations (security checks, dbms lookup, etc.)

Case 2 could be betterh andled (e.g. without changing the worldwide  
HTTP implementation) by optimizing the URL namespace. What I mean is  
that one adds URLs to the application which represent specific views  
on multiple resources. Those you can then retrieve with a single GET.

Look how google does it. They have given the first 20 search results  
a single URL.

//Stefan

Am 28.05.2008 um 16:08 schrieb Cyrus Daboo:

>> WebDAV
>> records,
> Well you don't need an RDBMS backed server to see the benefit. The  
> Apple CalDAV server uses flat files to store each calendar resource  
> plus an index in each calendar collection to optimize queries.  
> multiget is a big win for us. One of the biggest factors is being  
> able to aggregate the ACL check of all the resources. There are  
> several optimizations that can be done, and in the best case only  
> one privilege check is needed for all the resources being fetched.
>
> to fetch properties as well as data. i.e. its a combination of  
> multi-PROPFIND as well as multi-GET. Obviously DAV:getetag is the  
> one key property clients will fetch along with the data. That might  
> not be enough to fully justify this approach, but it does keep it  
> flexible.
>
> -- 
> Cyrus Daboo
>
>

--
<green/>bytes GmbH, Hafenweg 16, D-48155 Münster, Germany
Amtsgericht Münster: HRB5782

---

From: Julian Reschke <[email-masked]>
Date: Thu, 29 May 2008 10:35:40 +0200

Stefan Eissing wrote:
> Case 2 could be betterh andled (e.g. without changing the worldwide HTTP 
> implementation) by optimizing the URL namespace. What I mean is that one 
> adds URLs to the application which represent specific views on multiple 
> resources. Those you can then retrieve with a single GET.
> ...

Right.

Actually, my proposal allows a server to assign a GETtable URI to any 
REPORT response, so a client would have an extremely simply way to find 
out about the URI.

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Thu, 29 May 2008 11:07:36 +0200

On 29.05.2008, at 09:41, Stefan Eissing wrote:
> This BATCH/multi-GET topic comes up quite regularly every year or  
> so. It's a pony discussion.

I guessed so ...

> 2. or examples where as you described with optimizations for  
> multiple operations (security checks, dbms lookup, etc.)
>

Yes, maybe. However, my question/motiviation is quite specific and not  
strictly related to plain HTTP. (extending HTTP using hardcoded  
namespaces is not done by any WebDAV protocol, good or bad, thats the  
'style' of those).

Again, my basic motivation is to consolidate the CalDAV/CardDAV/xyzDAV  
multiget REPORTs, not necessarily to provide a generic BATCH  
capability (though it would be very nice if we can usefully catch this  
with one memo).
Those specialized batch ops already exist and won't get removed.

Now I would prefer to do the batch thing in a generic way, but if the  
outcome is that this is too hard (eg because Servlets can't easily  
hook into the HTTP stack), we could also do a specialized REPORT  
(instead of a new, plain-HTTP, non WebDAV-specific BATCH method).

On 29.05.2008, at 10:35, Julian Reschke wrote:
> Actually, my proposal allows a server to assign a GETtable URI to  
> any REPORT response, so a client would have an extremely simply way  
> to find out about the URI.

Maybe I'm reading your memo wrong, but to me it does not look like a  
BATCH or multiget REPORT replacement. You would need to encode all  
requested URIs in the URL or assign a stateful token.

Its not that useful to cache changeset information either (because it  
constantly changes and is client-cache specific). Its more important  
to cache the contained, individual contents. In fact for the latter a  
BATCH looks to be optimal, since the actual content caching is exactly  
the same like for individual contents, whereas a REPORT needs custom  
infrastructure.

I can see that your proposal is useful for other kinds of reports  
(more in the GData feed-like category).

Thanks,
   Helge

PS: I think BATCH would be relatively easy to implement in the Apache2  
infrastructure.
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 29 May 2008 12:10:24 +0200

Helge Hess wrote:
>> REPORT response, so a client would have an extremely simply way to 
>> find out about the URI.
> 

It's not meant to be a replacement. But it can make a specific report 
GETtable, and thus cacheable (not the individual parts).

> constantly changes and is client-cache specific). Its more important to 

It can be. If a client polls for changes, it's totally useful if the 
server can say "304".

Also, it's not necessarily client specific.

> cache the contained, individual contents. In fact for the latter a BATCH 
> looks to be optimal, since the actual content caching is exactly the 
> same like for individual contents, whereas a REPORT needs custom 
> infrastructure.

The assumption seems to be: we need multi-GET because of initial sync, 
and thus, if we have it, we'll use it for everything.

Another approach would be to have a special GETtable resource that can 
be used for the initial sync, and then use regular, conditional GET 
requests for everything else....

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Thu, 29 May 2008 12:37:27 +0200

On 29.05.2008, at 12:10, Julian Reschke wrote:
> The assumption seems to be: we need multi-GET because of initial  
> sync, and thus, if we have it, we'll use it for everything.

Well, not for 'everything', for fetching changed resources from a  
server into a cache. This is a very common and useful operation. And  
its *not* just the initial sync, initial sync is just the worst case.  
Its not that unusual that hundreds of resources are changed while I'm  
out of office for a day ...

But sure, since pipelining has no good realworld adoption, it would  
have the potential to actually replace it. (just look at CSS image  
sprites!)

> Another approach would be to have a special GETtable resource that  
> can be used for the initial sync, and then use regular, conditional  
> GET requests for everything else....

Yes, that would be an OK workaround. But it somehow has a hackish  
feeling to me, since the obvious solution for the actual problem is  
indeed a BATCH :-)

Anyways, can we drop the generic-BATCH discussion and focus on options  
to generalize the multiget-REPORTs?

Thanks,
   Helge
-- 
Helge Hess
http://www.helgehess.eu/

---

From: Geoffrey M Clemm <[email-masked]>
Date: Thu, 29 May 2008 09:23:43 -0400

+1 for Stefan's comments.

Cheers,
Geoff

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 29 May 2008 12:29:00 -0400

Hi Julian,

--On May 29, 2008 12:10:24 PM +0200 Julian Reschke <[email-masked]> 
wrote:

> Another approach would be to have a special GETtable resource that can be
> used for the initial sync, and then use regular, conditional GET requests
> for everything else....

But I think that will end up looking exactly like multiget for the most 
part. Some things to note: clients will typically need to use several 
multigets for the initial sync if the collection is large. I believe iCal 
broke down multigets into 50 or 100 resource fetches at a time. Our server 
won't allow more than 1000 to be returned in a report. So your special GET 
will need to have a way to select which ranges to fetch (just as multiget 
does) and it will need a way to expose, at the very least, the getetag 
property of each of those (again multiget can do that but also allows more 
properties to be returned). So I think you just end up with pretty much the 
same feature set.

-- 
Cyrus Daboo