Subject: Re: Thoughts on relation to WebDAV

From: Werner Baumann <[email-masked]>
Date: Sun, 25 May 2008 20:48:54 +0200

Julian Reschke wrote:
 > Well, sorry. It seems we live in different worlds then.

Agreed, and I'm not sorry.

 > The problem arises when clients use in-band information for the wrong
 > purpose; for instance refuse to use PROPFIND, just because OPTIONS
 > doesn't return a DAV header.

They use the information just for the purpose it is intended for. A 
WebDAV-client checks the DAV-header to see, whether the server is a 
WebDAV-server (at what are the capabilities).

Your idea of partial implementation of WebDAV isn't mentioned in the 
spec and it was probably unknown to the implementers of WebDAV-clients. 
When you come up with a new idea, you will have to ask implementers for 
support.
What you propose is: fool the clients by sending wrong information in 
the DAV-header and then misuse status code 403 to reject what the 
clients can expect to succeed. If this really is done, one element of 
RFC 4918, the DAV-header is rendered useless.

I can see, that you think it useless anyway. But RFC 4918 was released 
just one year ago and you are one of the 4 creators of this document. 
Undermining this specification the way you do, is what does not fit in 
my world and what I call "not taking standards seriously".

Werner

---

From: Helge Hess <[email-masked]>
Date: Sun, 25 May 2008 21:10:06 +0200

On 25.05.2008, at 20:48, Werner Baumann wrote:
> then misuse status code 403 to reject what the clients can expect to  
> succeed

The clients cannot expect the operation to succeed (200 OK), whether  
its misused or not. They need to deal with 403 even in the context of  
fully conforming servers.
I guess thats why Julian was saying that the 'misuse' has no practical  
consequences.

Thanks,
   Helge

---

From: Julian Reschke <[email-masked]>
Date: Sun, 25 May 2008 21:20:19 +0200

> WebDAV-server (at what are the capabilities).

But most of the time, a client doesn't need to know that. For instance, 
when displaying folder contents. Either PROPFIND is there or not. What 
other parts of WebDAV are there is just irrelevant for that.

> RFC 4918, the DAV-header is rendered useless.

I happen to believe that it is not really useful, yes.

> just one year ago and you are one of the 4 creators of this document. 

No, I'm not. The "creator" is the working group in total, or, if you 
wish, the Editor acting on behalf of the working group.

But even if I was, I wouldn't have removed or changed it; maybe I would 
have pointed client implementors to smarter ways to detect functionality.

What the WG did was looking at all issues that had been raised against 
RFC2518, and trying to resolve as many as possible in RFC2518bis.

Usage patterns for compliance levels were IMHO never discussed, so 
changing this was never on the table, at least not as far as I recall.

What was discussed and fortunately rejected was a requirement to return 
the DAV header upon "OPTIONS *", btw.

And, for the record; I tried to stop the addition of another compliance 
level (L3), because, after all, RFC4918 is meant to be 
backwards-compatible, and all the extensions we did can be detected in a 
smarter way anyway.

> my world and what I call "not taking standards seriously".

I think you read much more into the compliance levels than it was intended.

Getting back to what got us here (I think): are you saying that a server 
that exposes a flat collection of items (such a set of vcards) is not 
allowed to say "I'm doing WebDAV" because it doesn't allow creating 
child collections?

Do you have evidence of a server that actually *does* support 
arbitrarily deeply nested collections? What am I missing here?

BR, Julian

PS: I note that you still didn't followup on 
<http://lists.w3.org/Archives/Public/w3c-dist-auth/2008AprJun/0046.html>:

> In practice, there are many things that RFC2518 and RFC4918 do not even 
> talk about, but will affect conformance and interoperability:
> 
> - the set of characters allowed in path segments (think Unicode 
> normalization)
> - the maximum length of a path segment
> - whether or not path segments are case-sensitive
> - the maximum number of path segments
> - the maximum total lenth
> - size constraints for property values
> - ...
> 
> One key feature of HTTP and protocols based on HTTP is that messages and 
> responses are self-descriptive, and that close coupling (as in WS-*) is 
> avoided. Yes, it's always nice to know upfront if a server is going to 
> allow something (being it out-of-band by spec, by introspection 
> (OPTIONS), whatever...). But in practice, a client will have to deal 
> with unexpected errors (and yes, good status codes help with that).

Let me add a few more:

- whether path segments are stored as is (using URI escaping), byte 
sequences (Unix/Posix file IO) or character sequences (Windows, HFS+)

- how trailing blanks are handled (ignored vs stored)

- how trailing dots are handled

- whether or not content written with PUT can be rewritten by the server

and so on...

PS2: Again, for the record: I totally agree with you that a server needs 
to implement things in a consistent, meaningful way. Such as, HEAD and 
GET need to work as required by HTTP/1.1, and PROPFIND responses should 
be consistent with HEAD/GET responses.

---

From: Werner Baumann <[email-masked]>
Date: Sun, 25 May 2008 23:20:27 +0200

> other parts of WebDAV are there is just irrelevant for that.
Whether the server believes it is irrelevant does not matter. When the 
server announces full support for WebDAV, using the DAV-header, the 
client is allowed to interpret it according to the specification. When 
this causes trouble, the responsibility is on the side of the server 
that intentionally sent wrong information.

> have pointed client implementors to smarter ways to detect functionality.
Now, as you would not have removed the DAV-header anyway, you ought to 
respect it, as defined in the specification. If you want to deprecate 
it, say it loud and *clear*. What I oppose is your suggestion to 
intentionally misuse it (Send DAV: 1 and not support required methods) 
and this way render it useless.

> allowed to say "I'm doing WebDAV" because it doesn't allow creating 
Looks like a CardDav (?) problem, and they really seem to be in trouble 
here. If they really want to restrict to flat collections, which means 
no MKCOL, they are not compliant to WebDAV and should not use the 
DAV-header (you don't like it anyway). But as they intend to create an 
extension protocol, there surely is way to handle this properly.

> arbitrarily deeply nested collections? What am I missing here?
Not allowing arbitrarily deeply nested collections is not the same as 
not allowing the creation of collections at all. This looks like a use 
case for 403, according to the specs.

>> In practice, there are many things that RFC2518 and RFC4918 do not 
>> even talk about, but will affect conformance and interoperability:
>>
As far as I know unicode is allowed in pathsegments and there is a rule, 
how to escape them. The practical issue seem to me different reserved 
characters on different operating systems. I have know solution, but 
education. Perhaps servers should reject pathnames that cry for trouble.

>> - whether or not path segments are case-sensitive
Neither servers nor clients should change the case.

>> - size constraints for property values
Are these real problems today. As long as people have a minimum of good 
taste, there should not be a problem. In most other cases servers are 
allowed to simply reject such request.

>> One key feature of HTTP and protocols based on HTTP is that messages 
>> and responses are self-descriptive, and that close coupling (as in 
>> WS-*) is avoided. Yes, it's always nice to know upfront if a server is 
>> going to allow something (being it out-of-band by spec, by 
>> introspection (OPTIONS), whatever...). But in practice, a client will 
>> have to deal with unexpected errors (and yes, good status codes help 
>> with that).
>
This could be arguments, when you start an *open* discussion to remove 
the DAV-header from the spec.

> - how trailing dots are handled
I think "store" or not "store" is not in the scope of the spec. But 
neither a server nor a client is allowed to silently change the URL of a 
resource (and expect it to work) (with the exception when the server 
forgot about the trailing slash of a collection-URL). If a server does 
not like a new URL created by a client, it should reject it.

> - whether or not content written with PUT can be rewritten by the server
It is out of the spec what a server is allowed to do with its resources. 
But in my opinion: regarding distributed authoring, co- and 
anti-authoring by the server is the last think we need.

BTW: Is anybody, who refuses to accept clear misuse of protocol 
elements, obliged to answer this - quite unrelated - questions? Is this 
a test, whether I am allowed to open my mouth when important people are 
talking about standards?

Werner

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 May 2008 07:59:05 +0200

Werner Baumann wrote:
> Julian Reschke wrote:
>> But most of the time, a client doesn't need to know that. For 
>> instance, when displaying folder contents. Either PROPFIND is there or 
>> not. What other parts of WebDAV are there is just irrelevant for that.
>>

Yes.

> and this way render it useless.

I disagree that it renders it useless. It isn't very useful in the first 
place.

And the only reason to deprecate it would be because it may harm usage 
WebDAV for novel things, such as Vcarddav.

>> Getting back to what got us here (I think): are you saying that a 
>> server that exposes a flat collection of items (such a set of vcards) 
>> is not allowed to say "I'm doing WebDAV" because it doesn't allow 
>> creating child collections?
>>

Yes, maybe it's better to not to use the header for them. The problem is 
that it'll make it *much* harder to specify the protocol, with little 
real-world advantage.

> case for 403, according to the specs.

So, at what nesting level do you draw the line? Any spec text to support 
that?

> education. Perhaps servers should reject pathnames that cry for trouble.

Well. Servers do.

A server that stores path segments as unicode will likely reject a path 
segment of "%80", because it's not a legal UTF-8 octet sequence.

So, what do you do in a Unix setup where the local filename contains a 
single 0x80 octet, and the user wants to copy the file to that WebDAV 
server?

> Neither servers nor clients should change the case.

So what about case-preserving vs case-sensitive? Many servers do 
preserve the case, but can't store things that only differ in case, such 
as "Makefile" vs "makefile", to name a popular example.

Compliant?

> allowed to simply reject such request.

Yes, but this is something RFC4918 is totally silent about. So where do 
you draw the line between where servers a free to have restrictions, and 
where not?

>>> WS-*) is avoided. Yes, it's always nice to know upfront if a server 
>>> is going to allow something (being it out-of-band by spec, by 

OK, there's go IIS and iDisk (I think).

> a test, whether I am allowed to open my mouth when important people are 

No, I'm just trying to understand where you draw the line about areas 
where a server may have restrictions that aren't mentioned in the spec.

Not sure about what you mean when you say "important people". Maybe you 
may want to read an FAQ about the IETF process...

BR, Julian

---

From: Werner Baumann <[email-masked]>
Date: Mon, 26 May 2008 10:12:17 +0200

Julian Reschke wrote:
(concerning the DAV-header, Werner)

> that it'll make it *much* harder to specify the protocol, with little 
That is exactly the point I wanted to make:
They intend to make their new protocol an extension of WebDAV. It turns 
out that WebDAV does not really suit their needs. What they could do:
- rethink, whether WebDAV was a good choice, and maybe make
   it an extension to HTTP, or create a completely separate protocol
- propose to chance the WebDAV-specification. This will need consensus
   and will take a lot of time.
What must not be done: break elements of the WebDAV-protocol 
(DAV-header, 403 status code) because it is easier for them.
While you only see little real-world advantage in going the "*much* 
harder" way, I see the disadvantage of breaking protocols.

(arbitrarily deeply nested collections)
> So, at what nesting level do you draw the line? Any spec text to support 
I do not draw a line and the spec should not do it either. Everybody 
knows that the resources of any server are limited. There is no chance 
to draw a line that meets the needs of the most restricted servers and 
at the same time does not put unnecessary and annoying restrictions on 
others. When a server reaches its limits, it is allowed to reject the 
request with an appropriate status code. Including a more informative 
error message in the body would be good behaviour.

> single 0x80 octet, and the user wants to copy the file to that WebDAV 
%80 is the escaped form. Clients and servers must always use this form 
on the wire. How they are going to remember it, and how they like to 
present it to the user (we don't want to be file system fixated) is out 
of the spec.
> as "Makefile" vs "makefile", to name a popular example.
Changing the case, changes the URL and is not compliant.
All these file name issues cannot be solved by HTTP or WebDAV. They are 
indeed a problem and they are an heritage. There are different operating 
systems with different restrictions and different character sets. Now 
they have to cooperate over WebDAV. It will not be possible to map these 
specialities seamlessly from one OS to another; and there is no use in 
trying. I only see two "solutions"
- hope for universal use of utf-8
- educate the users; don't suggest them everything can be used as a file 
name (e.g. the first two paragraphs of a Word document); for the time 
being, only use lower case ascii, never use characters with special 
meaning in any operating systems file system. WebDAV-applications can 
support this, but rejecting bad file names and informing the user about it.

>> neither a server nor a client is allowed to silently change the URL of 
>> a resource (and expect it to work) (with the exception when the server 
Bad for IIS and iDisk and annoying for clients that may have to do 
another workaround for broken, but widely used servers; but no reason to 
change the spec. But as said before: nobody should willingly create 
idiotic file names, that are known to ask for trouble.

Werner

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 May 2008 12:52:18 +0200

Werner Baumann wrote:
>> So, at what nesting level do you draw the line? Any spec text to 
>> support that?
>>

+1!

>> A server that stores path segments as unicode will likely reject a 
>> path segment of "%80", because it's not a legal UTF-8 octet sequence.
>>

So, you expect "%80" to work (meaning, be stored and roundtripped) with 
any compliant server?

Well, then the set of compliant servers just got much smaller :-)

>> preserve the case, but can't store things that only differ in case, 
>> such as "Makefile" vs "makefile", to name a popular example.
>>
>> Compliant?
>>

It doesn't change the case, it would just disallow both of them to be 
present in the same collection.

> support this, but rejecting bad file names and informing the user about it.

Well, only using lower case ASCII is something that's not sufficient 
once you leave the experimental space. Users expect that they can assign 
names in their own languages.

>  ...

BR, Julian

---

From: Werner Baumann <[email-masked]>
Date: Mon, 26 May 2008 18:11:06 +0200

> names in their own languages.
I know that the situation is not satisfying at all. But even with 
servers that can handle any character from any character-set: if users 
access the same WebDAV-repository with clients that run on different 
operating systems with different default character sets they will be 
disappointed, unless they learn to restrict characters in file names to 
a secure set. This is an operating system issue.
Happily, things are getting better with increasing use of utf-8. But as 
long as clients want to map URLs into local file-system names, there 
will stay the problem of different sets of characters with special 
meaning in the context of file-systems. The spec could help with a list 
of characters that client implementations should reject when entered by 
the user.

Servers that rely for storing pathnames from URLs as file names on their 
local file system will have trouble to get really compliant, especially 
when the local file system can not distinguish cases. But of course, 
they could store pathnames in a different way, without loss, and 
maintain a mapping to the stored resources.

My estimation is less than 10% of compliant WebDAV-servers (but there 
are many servers, I did not test). Unluckily, this number seems to 
decrease with every new server. (My suspicion: beginners courses in 
programming have replaced the "Hello World"-exercise with 
"WebDAV-server".) Here is the latest real-world example, I had to deal 
with on the Help-forum, just to give an impression what clients have to 
fight with. It are not the highly sophisticated border cases. Please 
look at Date, Last-Modified and the status code returned on PUT:

GET /test.txt HTTP/1.1
Host: example.com
User-Agent: davfs2/1.3.1 neon/0.26.3

HTTP/1.1 200 OK
Date: Mon, 19 May 2008 22:03:34 GMT
Server: Cpanel::Httpd like Apache
Content-Length: 0
Last-Modified: Mon, 19 May 2008 21:59:07 GMT

HEAD /test.txt HTTP/1.1
Host: example.com
User-Agent: davfs2/1.3.1 neon/0.26.3

HTTP/1.1 200 OK
Date: Mon, 19 May 2008 22:03:45 GMT
Server: Cpanel::Httpd like Apache
Content-Length: 0
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT

PUT /test.txt HTTP/1.1
Host: example.com
User-Agent: davfs2/1.3.1 neon/0.26.3

HTTP/1.1 201 CREATED
Date: Mon, 19 May 2008 22:03:45 GMT
Server: Cpanel::Httpd like Apache
Content-Length: 0

HEAD /test.txt HTTP/1.1
Host: example.com
User-Agent: davfs2/1.3.1 neon/0.26.3

HTTP/1.1 200 OK
Date: Mon, 19 May 2008 22:03:45 GMT
Server: Cpanel::Httpd like Apache
Content-Length: 0
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT

Werner