Subject: RE: getting WebDAV SEARCH ready for the IESG

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 08:24:31 -0700

Some comments:

In section 2.3.1 Result Set Truncation 

- Would be nice to indicate what the search limit is (after what number
of results was the query truncated)

- Partial results: I read this to mean whatever partial results you send
back, they must be ordered (within themselves)
as the client requested.  In many cases the client wants the full list
ordered, and then send back the partial results.
Any way to indicate this in the request; i.e. if you have to send back
partial results (a 507 condition) I want them fully ordered, not just
within themselves?  Perhaps the server can send back a 507 response for
the arbiter URI and no results, if it can't comply with ordering the
full result set, and sending back partial results.

In section 5.17.1 Relationship to Result Ordering

- I read this to mean that the full results should first be ordered by
the server, and then send back the requested limit.  This seems to
contradict what's specified in section 2.3.1, where the results are
limited and then ordered (if I'm reading it correctly).  I think these 2
sections should be consistent with each other.

Regards,

-John

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Julian Reschke
Sent: Wednesday, July 02, 2008 9:12 AM
To: [email-masked]
Cc: WebDAV
Subject: getting WebDAV SEARCH ready for the IESG

Hi,

we recently made some progress on getting WebDAV SEARCH ready for
publication.

We received some feedback from Chris Newman, and the latest edits on the
draft
(<http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest.ht
ml>,
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest-fro
m-previous.diff.html>)
take those into account.

Unless there's new feedback, I'm planning to submit this as draft 16,
which, if all goes well, will then by last called.

Feedback appreciated,

Julian

This email and any attachments may contain confidential and proprietary
information of Blackboard that is for the sole use of the intended
recipient.  If you are not the intended recipient, disclosure, copying,
re-distribution or other use of any of this information is strictly
prohibited.  Please immediately notify the sender and delete this
transmission if you received this email in error.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 17:47:12 +0200

Hi John,

thanks for the comments.

The server I worked on a long time ago never truncated results, so I 
don't have any preference.

However, it seems to me that the text in 2.3.1 was phrased this way on 
purpose -- there may be cases where it's not possible to first sort, 
then truncate. For instance, when searching can be delegated to an 
underlying SQL store, but ordering can not, how would you implement 
that? Thus, I'm hesitant doing any change over here.

If you feel strongly about that, we *could* add a statement into the 
"future extensions" appendix.

And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm really 
not sure we can change this at this point of time.

BR, Julian

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 08:57:35 -0700

> However, it seems to me that the text in 2.3.1 was phrased this way on
purpose
> -- there may be cases where it's not possible to first sort, 
underlying 
> SQL store, but ordering can not, how would you implement that? 
> Thus, I'm hesitant doing any change over here.

Completely understood.  I'm just saying a client may not want results
that aren't ordered over the entire result set.  It might be preferred
to get no results (and have to further refine the search) than to get
truncated results that aren't "globaly" ordered.

I don't feel that strongly about this, just a nice-to-have for some
clients.

> And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm
really not
> sure we can change this at this point of time.

This I think is a bigger deal.  If you acknowledge that some servers
cannot (at least easily) order a global result set and then limit the
results returned, then how can this be a MUST?  Seems like the same
issue to me.

-John

John Barone wrote:
> - Would be nice to indicate what the search limit is (after what 
> number of results was the query truncated)
> 
> - Partial results: I read this to mean whatever partial results you 
> send back, they must be ordered (within themselves) as the client 
> requested.  In many cases the client wants the full list ordered, and 
> then send back the partial results.

> within themselves?  Perhaps the server can send back a 507 response 
> for the arbiter URI and no results, if it can't comply with ordering 
> the full result set, and sending back partial results.
> 
> 

> limited and then ordered (if I'm reading it correctly).  I think these

> 2 sections should be consistent with each other.
> 
> 
> Regards,
> 
> -John
> 
> 
> We received some feedback from Chris Newman, and the latest edits on 
> the draft 
> This email and any attachments may contain confidential and 
> proprietary information of Blackboard that is for the sole use of the 
> intended recipient.  If you are not the intended recipient, 
> disclosure, copying, re-distribution or other use of any of this 
> information is strictly prohibited.  Please immediately notify the 
> sender and delete this transmission if you received this email in
error.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 18:23:40 +0200

> truncated results that aren't "globaly" ordered.

I do agree that this may be more useful. I'm just skeptic about making 
this change many years after people have written implementations.

> results returned, then how can this be a MUST?  Seems like the same

I just checked the document's history, and that particular requirement 
was added in 2003, see the thread around 
<http://lists.w3.org/Archives/Public/www-webdav-dasl/2002OctDec/0033.html>. 
Back then we probably did not realize that we're introducing an 
inconsistency between truncation (server enforced) and limiting (on 
behalf of the client).

If this is a minor problem, we should just state it somewhere. If it's a 
major problem, we could try to fix it. The server I worked on didn't 
truncate, so I don't have a strong preference. That being said, it would 
be interesting to know how the other servers (Xythos, Catacomb, 
Slide...?) behave...

BR, Julian

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 09:54:54 -0700

Another comment on this; there doesn't seem to be any discussion about
issuing a SEARCH and limiting on multi-valued properties, in the WHERE.
Is this beyond the scope of this particular draft, or is this treated in
some other specification/draft?

Regards,

-John 

-----Original Message-----

Hi,

Feedback appreciated,

Julian

information of Xythos that is for the sole use of the intended

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 09:39:37 -0700

The Xythos server currently doesn't implement the limit feature.  The
server does truncate results based on a server setting.  Making sure the
truncated results are globally ordered is difficult, for the reasons you
outlined and particularly when the search spans multiple data stores.
Implementing the limit feature would pose the same ordering challenges.
I think making 5.17.1 a MUST places a heavy burden on the server
implementation.

-John

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, August 04, 2008 9:24 AM
To: John Barone
Cc: [email-masked]; [email-masked]
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:
>> -- there may be cases where it's not possible to first sort, then 
>> truncate. For instance, when searching can be delegated to an
> underlying

<http://lists.w3.org/Archives/Public/www-webdav-dasl/2002OctDec/0033.htm
l>. 

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 19:39:29 +0200

> outlined and particularly when the search spans multiple data stores.

...another good point I forgot.

> I think making 5.17.1 a MUST places a heavy burden on the server

One alternative would be "SHOULD", another one would be just stating 
that DAV:limit is optional, and servers that can't do the MUST level 
requirement should reject the query.

Any preference?

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 19:44:18 +0200

> some other specification/draft?

Short answer: out of scope.

Long answer: this requires a common understanding of what multivalued 
properties are, and how they are serialized.

A proposal was contained in a draft of what later was published as RFC 
4316, see 
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-property-datatypes-06.html#rfc.section.A>. 
Back when we discussed this, there was a preference of keeping things 
simple, and so it was removed from subsequent drafts.

That being said, I know that the proposal was implemented both by SAP 
and Xythos (although likely in different namespaces). I would be happy 
to resurrect that part of the spec as a separate activity, but today I 
wouldn't be able to update the SAP implementation anymore.

BR, Julian

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 11:07:05 -0700

I think that would be worthwhile, but need not muddy these discussions.

-John 

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, August 04, 2008 10:44 AM
To: John Barone
Cc: [email-masked]
Subject: Re: getting WebDAV SEARCH ready for the IESG

> Is this beyond the scope of this particular draft, or is this treated 
> in some other specification/draft?
> 
> Regards,
> 
> -John

A proposal was contained in a draft of what later was published as RFC
4316, see
<http://greenbytes.de/tech/webdav/draft-reschke-webdav-property-datatype
s-06.html#rfc.section.A>. 
Back when we discussed this, there was a preference of keeping things
simple, and so it was removed from subsequent drafts.

BR, Julian

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 11:19:22 -0700

First and foremost I would be in favor of wording that is consistent
with what's outlined in section 2.3.1, for truncation.  From a client
perspective, I would think that the MUST wording in section 5.17.1 is
most desirable.  However, from a practical (and admittedly self-serving)
point of view, simply stating that the results MUST ordered as the
client directed, would be preferred.  Section 2.3.1 goes on to say:

"... the partial results returned MAY be any subset of the result set
that would have satisfied the original query".

Perhaps in section 5.17.1 the additional sentence could be phrased:

"... the results that are included in the response document SHOULD be
those that order highest"

Regards,

-John

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, August 04, 2008 10:39 AM
To: John Barone
Cc: [email-masked]; [email-masked]
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:
> server does truncate results based on a server setting.  Making sure 
> the truncated results are globally ordered is difficult, for the 
> reasons you outlined and particularly when the search spans multiple
data stores.

Any preference?

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 20:34:44 +0200

> those that order highest"

So, to be precise, the single change you're proposing is to relax the 
"must" to a "should"?

I'd be ok with that.

BR, Julian

---

From: "John Barone" <[email-masked]>
Date: Mon, 4 Aug 2008 11:39:07 -0700

Yes, that would be fine.

I wasn't privy to the original discussion that resulted in the MUST.
So, I don't know if there are some client requirements/concerns that
would be stepped on by changing it to a SHOULD.

Regards,

-John 

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, August 04, 2008 11:35 AM
To: John Barone
Cc: [email-masked]; [email-masked]
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:
> most desirable.  However, from a practical (and admittedly 
> self-serving) point of view, simply stating that the results MUST 
> ordered as the client directed, would be preferred.  Section 2.3.1

I'd be ok with that.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 20:49:38 +0200

John Barone wrote:
> ...

I would assume that any query that DAV:limits itself, or results in 
server-enforced truncation, is unlikely to be useful in a programmatic 
client. So I wouldn't expect this to be a problem.

BR, Julian