Subject: Re: [Simple] XCAP proposal on conditional inserts

From: Jari Urpalainen <[email-masked]>
Date: Mon, 09 Jan 2006 17:06:15 +0200

On Thu, 2005-12-29 at 16:37 -0500, ext Jonathan Rosenberg wrote:
> At the last IETF meeting, I raised the possibility of fixing the current
> conditional insert problem as another change to make to the spec while
> we are editing it. The problem, to refresh everyones memory, is that
> HTTP (and thus XCAP) allows for conditional operations on a resource
> only if the resource being operated on exists. Thats because if it
> doesn't exist, it doesn't have an etag. So, you can't insert an element
> conditioned on the etag of the entire document and all of its components.
> 
> So, if you really mean to insert this thing ONLY if the document is
> unchanged, you'll need to do the insert "blindly", and then after its
> done, you'll find out whether or not the insert was done against the
> version you intended. If so, great. If not, you need to somehow undo the
> change you just made.
> 
> This is far from ideal, but a known issue, and documented in the spec
> today. I've been increasingly concerned that this is just not going to
> work well, as it substantially raises the bar for getting the system to
> work reliably.
> 
> I made a proposal at IETF to fix it, and some good comments were
> provided. Based on that, here is my modified proposal for how to fix
> this issue.
> 
> An XCAP resource representing an XML element in a document is said to
> exist if the element appears in the document, OR its parent appears in
> the document. If the XCAP resource exists but doesn't appear in the
> document, it is said to be empty. So, for example:
> 
> <test>
>    <el1/>
> </test>
> 
> test/el1 exists and isn't empty. test/el1/el2 exists but is empty.
> test/el1/blah-blah exists but is empty. In fact, every single potential
> child element of el1 exists - they're just all empty. Because they exist,
> they all have etags, which are all equal to the document etag. Based on
> this definition, I can now go and insert a new element, which isn't
> technically an insertion, but rather a modification: I'm modifying the
> element to change it from empty to a specific value.
> 
> This does not apply to attributes, however. An attribute exists only if
> its physically present in the document (the difficulty in getting this
> mechanism to work with attributes was one of the issues raised at ietf).
> 
> Now, there are some actual consequences to this change in definition:
> 
> * A GET against one of these empty resources will not return a 404 as it
> does today. Rather, it would return a 200 OK, with a Content-Type of
> application/xcap-el+xml and a Content-Length of zero. This indicates
> that the resource is empty.
> 
> So, based on the above example, GET test/el1/foobar would return a 200
> OK with Content-Length zero.
> 
> * A GET against an element which doesnt exist WOULD return a 404. So,
> for example, a GET of test/el1/foobar/nothere would return 404.
> 
> * A DELETE of an element would be forbidden, and always return a 405. To
> delete an element, you need to modify it to be empty. So, you'd do a PUT
> with a Content-Length of 0.
> 
> * We would no longer have a way to say, "insert this element into the
> document if the element is not currently present in the document". In
> the current spec, this is done by a PUT with If-None-Match: *. However,
> with the proposed change, this condition would always fail, since the
> element will now always exist. What we gain instead is the ability to
> insert conditionally based on whether the document on the server matches
> the one cached by the client.
> 
> * There would no longer be a need for the xcap-diff document that is
> returned in a 202 Created response. The reason this xcap-diff document
> gets sent is to let the client know what the etag was prior to
> insertion, so if it didn't match the cached copy, the client could try
> to repair the error. Eliminating the need for this would remove the
> dependency on xcap-diff, xml-patch-ops, and so on.
> 
I'd truly love to see conditional inserts, However, having "empty"
elements from xml pov is really weird. Also after an element delete, the
response should contain the doc ETag (or the parent element ETag) so
that conditional operations could be used easily and consistently (and
dependency to xcap-diff, etc. can be dropped). 

> 
> It is tempting to take a slightly different approach. Call the approach 
> above "approach 1". The idea with approach 2 is to declare that, even 
> though a resource doesn't exist, it has an etag, and that etag is equal 
> to the document's etag, assuming the document does exist. Approach 2 
> would work much like approach 1, except that:
> 
> * A GET against a non-existent resource would be a 404, rather than a 
> 200 OK with CL of zero
> 
> * A DELETE would be allowed
> 
> Note however, that with approach 2, you still cannot support the 
> operation that says, "insert this but only if this element doesn't 
> already exist in the document". Approach 2 has the problem that it 
> seriously bends (and possibly breaks), an underlying http assumption 
> about the scope and meaning of etags. That might have odd and 
> unpredicatable interactions with caching systems and other http 
> intermediaries. For this reason, I wouldn't recommend it.
> 
approach 2 is what I implemented first. If-None-Match: * was not in a
usage scope at all. Certainly there are (some) twisting of http ETags in
this case (or using ETags of different resources). However, as said,
approach 1 does the same thing to xml which is imo worse than approach
2. The need for caching of XCAP resources is imo rare (tls used or
authenticated requests mostly) and error prone (interdependencies).
Unpredictable interactions might appear with delete, but only with
broken proxies with put, I'd assume ? 

> So, I would like to make the concrete proposal of moving to approach 1. 
> The main drawback I see is the loss of the ability to insert an element 
> if an element with that same name/attribute already exist.
> 
> Comments?
> 
> -Jonathan R.
> 
> 
So this issue is all about what's an xcap resource ?. As e.g. Lisa has
noted that moving all selections to the query parameter would be more
compatible with WebDAV (also locking). However, rfc3986 says that query
component contains non-hierarchical data. So all in all some
bending/twisting seems to be needed unless the current approach is used.
So unfortunately there are several, far from prefect options, but I
don't like approach 1.
br,
Jari

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Aki Niemi <[email-masked]>
Date: Mon, 09 Jan 2006 18:03:05 +0200

Hi Jonathan,

ext Jonathan Rosenberg wrote:
> already exist in the document". 

I think that's OK. The most important thing is the ability to be safe 
from modifying an element someone else PUT in the meantime, by doing an 
insert. I think clients will in most cases work with cached copies 
anyway. And it's enough to be able to condition changes to the entire 
document using the etags, not so much of single elements.

> Approach 2 has the problem that it 

But XCAP already instructs clients to use the no-cache directive, and 
probably PUTs are anyway authenticated. These should be quite strong 
indicators to any HTTP intermediaries that they should not cache these 
XCAP requests.

> if an element with that same name/attribute already exist.

I like approach 2. In an ideal world, I'd prefer the node-selector as a 
query string (as discussed back in Nov 04), but I can live with approach 
2 since it demonstrates similar good properties like returning a 404 for 
a non-existent node.

Cheers,
Aki

---

From: [email-masked]
Date: Tue, 10 Jan 2006 15:14:03 -0800

Hi All,

Without adding further support for one or the other proposed approach, I
just want to point out the urgent need to solve this (and other
outstanding) issue(s) related to XCAP. Specifications in other foras
(like OMA) are dependent on the simple-xcap draft and waiting for xcap
to change back to RFC Ed queue status.

Thanks,
Krisztian

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] 
>On Behalf Of ext Aki Niemi
>Sent: Monday, January 09, 2006 8:03 AM
>To: ext Jonathan Rosenberg
>Cc: Simple WG
>Subject: Re: [Simple] XCAP proposal on conditional inserts
>
>Hi Jonathan,
>
>> It is tempting to take a slightly different approach. Call the 
>> approach above "approach 1". The idea with approach 2 is to declare 
>> that, even though a resource doesn't exist, it has an etag, and that 
>> etag is equal to the document's etag, assuming the document does 
>> exist. Approach 2 would work much like approach 1, except that:
>> 
>I think that's OK. The most important thing is the ability to 
>be safe from modifying an element someone else PUT in the 
>meantime, by doing an insert. I think clients will in most 
>cases work with cached copies anyway. And it's enough to be 
>able to condition changes to the entire document using the 
>etags, not so much of single elements.
>
>But XCAP already instructs clients to use the no-cache 
>directive, and probably PUTs are anyway authenticated. These 
>should be quite strong indicators to any HTTP intermediaries 
>that they should not cache these XCAP requests.
>
>> The main drawback I see is the loss of the ability to insert an 
>> element if an element with that same name/attribute already exist.
>> 
>> Comments?
>
>I like approach 2. In an ideal world, I'd prefer the 
>node-selector as a query string (as discussed back in Nov 04), 
>but I can live with approach
>2 since it demonstrates similar good properties like returning 
>a 404 for a non-existent node.
>
>Cheers,
>Aki
>

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 16 Jan 2006 11:49:39 -0500

inline.

> document using the etags, not so much of single elements.

Since posting this, it has occurred to me that you can still do "insert
this but only if this element doesn't already exist in the document" for
most cases. Thats because the selector would not be just on the element
name, but on some attribute too. So for example:

<list>
   <entry uri="sip:[email-masked]"/>
   <entry uri="sip:[email-masked]"/>
</list>

then if you do:

PUT .../list/entry[@uri="sip:aki@domain1/com"]
If-None-Match:*

this insertion would only be done if that specific entry wasn't there.

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Cisco Fellow                                   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 16 Jan 2006 17:28:54 -0500

Unless I am completely confused, this does not work.
The semantics of if-none-match * was that the PUT operation is only done if 
nothing matches any etag value.
With the old semantics, this worked for PUT because if the element was new, 
there was no etag, so "non-match *".
With the new semantics, there is an etag for the non-existant element.
Given that there is an etag, the if-none-match * will fail.

Hence, there would be no way, as far as I understand the proposed change, 
to do an insert that fails if the element is already there.  It seemed to 
me that this was important.
For many changes to a document, it does not matter what else has been done 
to the document as long as the given entry is not there.
For example, if I have a buddy list, with information about each buddy, I 
would not want to overwrite a buddy that is already there by accident.  But 
as long as the given buddy is not in the given list, I don't care what else 
I (or someone else) might have done to that same list, or other lists in 
the document.

Maybe I'm confused (I am not building cell phone applications) but it seems 
to me to be of comparable or higher frequency for me to want to know if 
this element is present rather than wanting to know if I have the most 
current document.  [This does assume that the phone does not automatically 
check the etag when it starts, and fetch the newest document all the 
time.  Such fetching would be a potential waste of bandwidth.  However, if 
such fetching is assumed, then one comes to different conclusions about the 
likelihood of needing a different check.]

Yours,
Joel

---

From: Aki Niemi <[email-masked]>
Date: Tue, 17 Jan 2006 01:03:40 +0200

Inline.

ext Joel M. Halpern wrote:
> The semantics of if-none-match * was that the PUT operation is only done 
> if nothing matches any etag value.
> With the old semantics, this worked for PUT because if the element was 
> new, there was no etag, so "non-match *".

Right. The way I see things is that very rarely can you actually do 
these "blind" conditional inserts. Often the data you insert has 
dependencies on other data elsewhere in the document, which means that 
clients updating it need the latest, full document copy, and need to 
condition the request for this full document being unchanged.

As an example, you simply cannot add an action to a common policy 
document rule without being fully aware of what the conditions part of 
that rule contains, or what other rules there are in the ruleset.

Also, if the client is inserting an element into a strictly flat list, 
like Jonathan's example below, I don't think you necessarily need to 
make the PUT conditional at all. So what if the element existed already; 
the element would simply be re-written, but what is so bad about that?

All in all, I view the ability to condition an insert over the changes 
of the complete document more important than the ability to do this 
"blind" conditional insert.

> Hence, there would be no way, as far as I understand the proposed 
> change, to do an insert that fails if the element is already there.  It 
> seemed to me that this was important.
> For many changes to a document, it does not matter what else has been 
> done to the document as long as the given entry is not there.
> For example, if I have a buddy list, with information about each buddy, 
> I would not want to overwrite a buddy that is already there by 
> accident.  But as long as the given buddy is not in the given list, I 
> don't care what else I (or someone else) might have done to that same 
> list, or other lists in the document.

Perhaps a way to avoid this is to split a potentially huge AU so that 
the various parts of the document are actually separate documents. Then 
whenever a GET is required, at least it is only to small portion of the 
document.

To generalize, any parts of the application data that have no 
interdependency can be specified as different documents, with different 
etags scope.

> Maybe I'm confused (I am not building cell phone applications) but it 
> seems to me to be of comparable or higher frequency for me to want to 
> know if this element is present rather than wanting to know if I have 
> the most current document.  [This does assume that the phone does not 
> automatically check the etag when it starts, and fetch the newest 
> document all the time.  Such fetching would be a potential waste of 
> bandwidth.  However, if such fetching is assumed, then one comes to 
> different conclusions about the likelihood of needing a different check.]

I'm sure some amount of fetching is required in any case. It is then a 
question of the relative frequency and size of the fetches. I'm inclined 
to say that for buddy list management either the data is fairly static 
(I don't generally muck with my buddy list more than a few times per 
year, after an initial bootstrapping phase), or has a high level of 
interdependency of the data elements (authorization policy being an 
example of this).

Cheers,
Aki