Subject: punctuation and path names

From: "Phillips, Mark" <[email-masked]>
Date: Tue, 27 May 2008 12:04:39 -0600

Hello,

I am new to this list, so please point me in the right direction if, and when, I post an off topic question. For the record, I am new to DAV, but have read the RFC's and the book by Lisa Dusseault. Thus I have a modest grip on the terminology. I did succeed in building a working client so I have a small amount of "hands on" experience. Still, the level of discourse on this list easily exceeds my depth.

I have a question regarding embedded punctuation, e.g. forward and backward slashes, colons, question marks, etc.. The source data and end user habits seem to require performing a substitution for some components of the URI. Say, "Client 16_Red Documents" in lieu of "Client 16/Red Documents". That is, I have reached the gap between correct URI and human language.

I assume others have considered this problem before. I would like to know what strategies and techniques others have employed to resolve or mitigate it. 

Thank you,

 - Mark Phillips

---

From: Tim Olsen <[email-masked]>
Date: Tue, 27 May 2008 14:52:13 -0400

One option is to percent-encode the URI.  I recommend taking a look at
RFC 3986 on URIs.  In particular, see the following section on
percent-encoding:

http://greenbytes.de/tech/webdav/rfc3986.html#percent-encoding

Cheers,
Tim

On Tue, 27 May 2008 12:04:39 -0600
"Phillips, Mark" <[email-masked]> wrote:

> 
> Hello,
> 
> I am new to this list, so please point me in the right direction if,
> and when, I post an off topic question. For the record, I am new to
> DAV, but have read the RFC's and the book by Lisa Dusseault. Thus I
> have a modest grip on the terminology. I did succeed in building a
> working client so I have a small amount of "hands on" experience.
> Still, the level of discourse on this list easily exceeds my depth.
> 
> I have a question regarding embedded punctuation, e.g. forward and
> backward slashes, colons, question marks, etc.. The source data and
> end user habits seem to require performing a substitution for some
> components of the URI. Say, "Client 16_Red Documents" in lieu of
> "Client 16/Red Documents". That is, I have reached the gap between
> correct URI and human language.
> 
> I assume others have considered this problem before. I would like to
> know what strategies and techniques others have employed to resolve
> or mitigate it. 
> 
> Thank you,
> 
>  - Mark Phillips
> 
>

---

From: Simon Perreault <[email-masked]>
Date: Tue, 27 May 2008 14:54:00 -0400

On Tuesday 27 May 2008 14:04:39 Phillips, Mark wrote:
> I have a question regarding embedded punctuation, e.g. forward and backward
> slashes, colons, question marks, etc.. The source data and end user habits
> seem to require performing a substitution for some components of the URI.
> Say, "Client 16_Red Documents" in lieu of "Client 16/Red Documents". That
> is, I have reached the gap between correct URI and human language.

Take a look at Section 2 of RFC 3986 
(http://tools.ietf.org/html/rfc3986#section-2). If you still have questions 
after reading that, please don't be afraid to ask again.

---

From: "Phillips, Mark" <[email-masked]>
Date: Tue, 27 May 2008 13:30:17 -0600

Thank you, Simon and Tim, for your replies.

I reviewed the links as suggested. I am using Apache 2.x, and last week I had run a few tests of MKCOL. Each test was run against Apache, one instance running on CentOS 4, and another on Windows XP.

In the first effort, given a source name of "Client16/Red Docments", where the desire outcome is a directory, or node, named as such, what I ended up with was two directories.

I then enabled the "AllowEncodedSlashes" directive. This did allow the server to accept encoded slashes in the URI, but the result was not as desired. Given the example above, the server returned a failure on MKCOL if the encoded node name contained a slash.

That said, I am a uncertain about this. What is hoped for is a method to accept these names, as bad as they are from the point of view of URI, and respect the end users intention. 

I was reading about the DAV property "displayName" and wondered if it might play a role in this case. 

 - Mark

---

From: Werner Baumann <[email-masked]>
Date: Wed, 28 May 2008 00:13:56 +0200

The problem, you encountered, is due the the fact, that clients running 
on different operating systems cooperate over WebDAV. Whenever the names 
(or last path-segments) of WebDAV-resources are used as file names on 
the local operating system of the client, you will have to cope with this:
Each operating system has a set of restricted characters, that must not 
be used in file names. Additionally there are still files systems that 
are case insensitive.

In your example:
"Client 16/Red Documents" is a perfectly reasonable (though perfectly 
ugly) file name on windows systems. But "/" is the path separator on 
Unix-like systems (like "\" on windows) and must not be used in file 
names. As there are a lot of operating systems with different 
restrictions, I think it is not possible to to something like an 
automatic translation between these reserved characters.

What you can do:

1. Tell your users, to restrict file names to a secure set, that will 
not cause trouble on any OS. My suggestion is lower-case-ascii, dash, 
underscore, colon and tilde.

2. When a user wants to create a new resource and enters a name, your 
client could check for problematic characters. If it finds one, it 
should prompt the user and explain the problem. Most problematic are 
"/", "\" and ":". But there are probably some more.

3. When your client gets URLs from the server (via PROPFIND Depth 1 for 
example) and creates file names from the last path segment, check for 
characters that are not allowed in file names on the client's OS. It may 
replace them by something ugly. davfs2, running on GNU/Linux, replaces 
all "/" by "slash-", "-slash-" or "-slash". But of course, in 
HTTP-requests, it must use the original URL. So the client has to 
remember the URL.

Will "displayname" help?
No. It is useless nonsense. The WebDAV-specification could never make up 
its mind whether it is a name or a description. Last decision is, that 
it is not required to be unique within one collection and must not be 
used to identify a resource. Why would you present to a user a list of 
resource names, when she cannot identify the resources by these names?

Werner

---

From: Jim Luther <[email-masked]>
Date: Wed, 28 May 2008 09:56:41 -0700

On May 27, 2008, at 3:13 PM, Werner Baumann wrote:

> Each operating system has a set of restricted characters, that must  
> not be used in file names. Additionally there are still files  
> systems that are case insensitive.

Another problem to note is with UTF precomposed vs UTF decomposed --  
some servers don't support both.

You'll also find servers that restrict the characters allowed even  
though the file system used by the server does not have those  
restrictions. For example <http://support.microsoft.com/default.aspx?scid=kb;en-us;905231 
 >.

For customers of Apple's .Mac service's iDisk (a WebDAV file server),  
we have the following recommendations <http://help.apple.com/mac2/1/help/homepage/pgs/hpg21.html 
 > for file names.

In Apple's WebDAV file system client, we don't allow "/" characters or  
NUL characters. Anything other characters we get are properly percent- 
encoded and sent to the server. It might work on the server, it might  
not.

Going back to the original question...

> correct URI and human language.

Before Mac OS X, our file system APIs used the ":" character as a path  
separator. So with Mac OS X those older APIs convert ":" to "/" and  
convert "/" to ":" when building a path to send to the Mac OS X file  
system APIs. For names coming back from the Mac OS X APIs, we do the  
opposite conversion.

- Jim