Subject: Re: [Simple] WGLC: XCAP Base

From: Jonathan Rosenberg <[email-masked]>
Date: Sun, 01 Aug 2004 11:25:15 -0400

Thanks for your comments, Jari. Responses inline.

Jari Urpalainen wrote:

> A few comments/questions.
> 
> 8.5 Managing Etags
> 
> As Jonathan has interpreted (complying with rfc2616) that a node or even 
> an attribute of an xml-document can be regarded as a resource is it 
> allowable to send the document ETag after partial delete which is then 
> actually an ETag of a different resource ?

I don't know. I fear that the answer is no; after a DELETE, the resource 
is deleted, so by definition THAT resource doesn't have an etag anymore.

> I would certainly prefer that 
> it is still being sent as otherwise you introduce a second deficiency 
> into safe updates of the document (the first one being partial append). 

I'm not sure what ability we have to mandate this. Since we're using 
HTTP, its going to depend on what HTTP says. I looked around for any 
text which talks about etags in DELETE responses, and didnt see anything.

> This would mean that only partial modifies are safe. I can think only 
> ugly solutions to this IMO annoying problem which is mostly due to the 
> fact that partial updates were (naturally) not thought when http was 
> created.

Well, we have what we have. I do think we need to clarify the etag 
behavior in DELETE. If someone can confirm that a DELETE response would 
never contain an etag, that would be helpful.

An alternative that I debated (but discarded) for this whole mess was to 
think of resources in a document as not getting deleted, but rather, 
becoming empty. That is, if element <foo> exists in a document, than any 
child of that element is said to exist, but be empty. Thus, if you did a 
GET against an element whose direct parent existed, you would not get a 
404; you'd get a 204 (No Content). That would include an entity tag, 
since 204 explicitly permits meta-data.

Thus, you would never actually DELETE. To remove an element, you would 
do a PUT against it, and the body of the PUT would be empty.

This has the benefit that an insertion operations morphs into a 
modification. An element gets modified from empty to having the value in 
the PUT.

As a result, entity tags would become meaningful for insertions, 
deletions and modifications as we know them today.

The drawback to this approach is that its clearly a hack. It will 
require the server to be able to create and manage meta-data for an 
infinite number of resources. Clearly this would be done on-the-fly, but 
its ugly. It also adds some complexity, and produces behavior which is 
non-obvious. Furthermore, it might have ugly interactions with other 
mechanisms we might consider down the road, but I'm not sure.

So, I discarded this approach.

> 
> 6.3 Node selector
> 
> Although I have already commented on the default namespace issue I'll 
> repeat it here: XPath 1.0 has the interpretation that namespace uri is 
> null if the location step prefix is empty. The upcoming XPath 2.0 seems 
> to use the element context which is the only sane way in XCAP, too. So 
> just clearly state that in the text (otherwise compliant with XPath 1.0).
> 

This is flagged as "todo" in the document. I understand that the default 
namespace for xpath expressions is null (something I have never 
understood the rationale for). I agree this seemed horribly broken. In 
XPath 2.0, help me understand how this works. Lets say the namespace is 
defined as the one from the current element context in which the Xpath 
expression is being evaluated. I have a doc that is like this:

<foo>
   <test:bar xmlns:test="urn:testing:namespace"/>
   <other:bar xmlns:other="urn:testing:other"/>
</foo>

Lets say I want to select the test:bar element, and I'm evaluating an 
expression within the foo context. I can't figure out how to construct 
an xpath expression to do that. The reason is that, if the evaluation 
context is foo, I can't name test:bar or other:bar because the 
namespaces test and other aren't defined in the foo context.

I'm hoping that I've just misunderstood how to construct the namespace 
context for xpath expression evaluations....

> 7 Client Operations
> 
> I am sorry to repeat but I still don't understand idempotent delete. 
> What should the server do during "a subsequent DELETE to the same URI 
> should FIND the resource deleted..." ? I don't understand find here.

The server wouldn't do anything different. WHen the second DELETE 
arrives, it would look at the URI, and find that the resource doesn't 
exist, and thus reject the DELETE with a 404.

I reworded the text thusly:

look the same. Similarly, when a client performs a DELETE, if it
succeeds, a subsequent DELETE to the same URI will generate a 404;
the resource no longer exists on the server since it was deleted by
the previous DELETE operation. To maintain this property, the client

Is that better?

> ---------------
> I still miss non-default namespace examples. 

Once I figure out how this works, I will add them. The norm for us will 
include non-default namespaces, so, in fact, I will change all of them 
to use non-defaults.

> Also in all examples where 
> location steps exist there should be more predicates to imply that each 
> step has to point to a single element.

OK, I will do that.

> 
> Also I havn't noticed any discussions in the mailing lists about caching 
> proxies as they will certainly try to cache responses unless 
> Cache-Control or Expires headers are used. So some recommendations are 
> needed in the text.

Good point. Caching responses is OK, I think. If a client is GETing lots 
of different elements within the document, the caching won't be very 
effective. A cache won't know how to relate the URI for the document 
with the URIs for the elements and attributes within the document, but 
thats OK. The caching isn't broken; its just not taking advantage of 
xcap URI interdependencies to improve performance. Putting that aside, 
subsequent requests for the same URI/resource could fetch the content 
from a cache. That would be OK. The server probably wants to set the 
max-age such that the content expires relatively quickly, but this 
probably depends on the application usage.

As long as all of that sounds fine, I will add text explaining this in a 
new section on caching.

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                600 Lanidex Plaza
Chief Technology Officer                    Parsippany, NJ 07054-2711
dynamicsoft
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Jari Urpalainen <[email-masked]>
Date: Mon, 02 Aug 2004 13:27:04 +0300

> context for xpath expression evaluations....
As you can't send namespace prefix and corresponding namespace uris with
queries (unless you use namepace-uri() and local-name() functions) you
have to use in-scope definitions that you find from the document. Given
your example I don't see a problem: <foo> does not know test or other
namespace definitions only <bar>'s have in-scope definitions. So you can
do relative selections like "test:bar[condition xxx]" within <foo>
context as <bar> (which is a child node of <foo> -> also a new location
step) clearly belongs to test:.. namespace. So when doing
implementations location step parsers have to be aware of all in-scope
namespace definitions. (and if you use fully xpath 1.0 compatible
libraries they'll most likely have some "hooks" where namepaces can be
"registered" when evaluating xpath expressions). I'll also remind that
with XCAP it was agreed that namespace definitions can not be added like
attributes, so during insert you can not just insert <test:bar> (as it
has namespace definition), instead you have to insert <foo>.
> the previous DELETE operation. To maintain this property, the client
definitely, though idempotency issue still remains, btw. has anybody
contacts to the http authors ?
> include non-default namespaces, so, in fact, I will change all of them 
as both models are allowed it would imo be best to have also both
examples.
> As long as all of that sounds fine, I will add text explaining this in a 
What worries me here mostly is that after partial delete/put caches may
send incorrect content (e.g. when getting the whole document). This is a
similar problem as what we have with ETags, because a single document
contains many resources which are somehow related.

Thanks,
Jari

---

From: Miguel Garcia <[email-masked]>
Date: Mon, 02 Aug 2004 20:27:06 +0300

Jonathan Rosenberg wrote:

>> As Jonathan has interpreted (complying with rfc2616) that a node or 
>> even an attribute of an xml-document can be regarded as a resource is 
>> it allowable to send the document ETag after partial delete which is 
>> then actually an ETag of a different resource ?
> 
> 

Although I agree, one can think of a few tricks that would help to 
transport the new etag of the new version of the document. For instance, 
  you can think of first allocating a new etag to the resource to be 
deleted, then executing the delete operation. The 200 OK for the DELETE 
contains the etag of the resource right before its deletion. This 
happens to be the etag of the new remaining document.

I also found that RFC 2616 says:

"In order to be legal, a strong entity tag MUST change whenever the 
associated entity value changes in any way."

Does "in any way" include "when the resource is deleted? I am trying to 
find if it is fine to create a new entity value for a deleted resource.

> 
>> I would certainly prefer that it is still being sent as otherwise you 
>> introduce a second deficiency into safe updates of the document (the 
>> first one being partial append). 
> 
> 

I looked around and I didn't see that is forbidden. On the other hand, I 
suspect that in an XCAP server modeled as a layer of an HTTP stack and 
an XCAP application, the XCAP application will choose the etag value and 
pass it through an internal API to the HTTP stack; the HTTP stack will 
encod it in a Etag header according to HTTP. In other words, since the 
value of the Etag is selected by the XCAP application and pass to the 
HTTP stack, we don't break HTTP by including it in a 200 OK for a DELETE 
response. Comments?

Regards,

      Miguel
-- 
Miguel A. Garcia           tel:+358-50-4804586
Nokia Research Center      Helsinki, Finland

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 02 Aug 2004 14:29:00 -0400

Jari Urpalainen wrote:

> context as <bar> (which is a child node of <foo> 

Here is where I don't follow. Within foo, the namespace "test" is NOT 
defined. So, how can I possibly canonicalize it in order to do an 
equality check? Remember, equality checks with namespaces require 
canonicalization to the namespace URI; that is, the prefix is always an 
alias to the namepsace URI. Thus, the following two elements are equal:

<foo xmlns="namespace">

<ns:foo xmlns:ns="namespace">

-> also a new location
> namespace definitions.

Right, I agree. But the two namespaces I need to select the children are 
not in scope!

  (and if you use fully xpath 1.0 compatible
> has namespace definition), instead you have to insert <foo>.

No, xcap does not say this. It says you cannot SELECT based on a 
namespace attribute. That does NOT mean that an inserted element (i.e., 
the thing in the  body of the PUT) cannot contain a namespace 
declaration attribute.

> as both models are allowed it would imo be best to have also both

OK.

> contains many resources which are somehow related.

Ahh, interesting. You are correct. A cache knows to invalidate when it 
gets a PUT to a resource, but not when it sees a PUT to another resource 
which invalidates the cache.

Of course, with ANY caching scheme, the resource could change by a PUT 
to that resource that doesn't go through the cache. Thats a consequence 
of caching. I don't think XCAP changes anything there. The same caveats 
apply; if you cache, the document may be stale.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 02 Aug 2004 14:32:09 -0400

inline.

Miguel Garcia wrote:

>> I don't know. I fear that the answer is no; after a DELETE, the 
>> resource is deleted, so by definition THAT resource doesn't have an 
>> etag anymore.
> 
> 

Thats not a valid usage of etag.

> find if it is fine to create a new entity value for a deleted resource.

I doubt it, but again, will ask around while here.

> HTTP stack, we don't break HTTP by including it in a 200 OK for a DELETE 

Just because an API might allow it, doesnt mean its right or a good idea.

-Jonathan R.

---

From: Jari Urpalainen <[email-masked]>
Date: Tue, 03 Aug 2004 12:02:40 +0300

> <ns:foo xmlns:ns="namespace">

right.

> Right, I agree. But the two namespaces I need to select the children are 

right, not in-scope of <foo> but instead in-scope of <bar> the child of
<foo>. So if you select from foo's parent context with relative request
foo/test:bar you'll first find <foo> (lets's say no default namespace
defined) as no prefix no namespace expansion. Then you take the next
step, the bar node which happens to have prefix and in-scope namespace
definition so you have to compare the namespace uri and element name in
<bar> context. 

Similarly, if <foo> happens to be a root node of the document and you
have namespace definition there, then you also have "in-scope" namespace
definitions that has to be followed:

<foo xmlns="namespace">
</foo>

selecting with absolute query /foo will fail with xpath 1.0 compatible
query because <foo> has in-scope namespace definition. For the request
to succeed you should provide a query like /ns:foo where ns="namespace".
With xpath 1.0 you can have
/*[local-name()="foo"][namespace-uri()="namespace"]
This syntax is of course "not so nice" and it is desirable to use the
in-scope namespace definition which is defined in this case within the
child element. So the confusion seems to be where the in-scope namespace
is defined and how the query is interpreted.

> the thing in the  body of the PUT) cannot contain a namespace 

ok, that's better than my incorrect interpretation. This is definitely
worth mentioning more clearly in the draft, because it needs some
handling during implementation. Even an example would not be bad...

> apply; if you cache, the document may be stale.

that's right that caches may miss requests (e.g. different route), but 
in order for them to be efficient they should also know xcap semantics
the requirement of which is not desirable.

> -Jonathan R.

br,
Jari

---

From: Aki Niemi <[email-masked]>
Date: Wed, 04 Aug 2004 00:48:55 +0300

> is deleted, so by definition THAT resource doesn't have an etag anymore.

There is something with Etags here that keeps worrying me. Because of
the way that etags are distributed over an XCAP document (i.e., all
elements/resources share the same etag value), it seems that we lose
some important property for some XCAP applications.

Let's say we have an XML document that has two sub trees. In one subtree
(called <bar>) you have a flat list of properly indexed, simple
elements, like 

<foo name="x">TRUE</foo>

Naturally updating this list doesn't require safe updates at all. One
could simply PUT/GET/DELETE blindly using the name attribute as
selection key.

Now let's say that the other subtree (called <baz>) is a much more
elaborate object, where safe updates are required. SOmething akin to
common policy <rules> for example.

Because of the way we have currently defined etags usage, I believe any
blind update to the <bar> subtree automatically forces also getting the
<baz> subtree, because the etags have changed.

I think this is really suboptimal, and it would be great if it could be
somehow avoided. For example, having the possibility for an application
to define etag "ranges". In the above example, there would be three
ranges defined: One for the root element of the document, second for the
bar subtree and the third for the baz tree. A change would only
propagate within a single range.

The application could still GET the whole document by selecting the root
element, but it would need to do separate GETs for each of the subtrees
ot get their respective etags. A HEAD to each subtree might suffice as
well, at least to some extent.

Thoughts?

Cheers,
Aki

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 03 Aug 2004 20:30:10 -0400

inline.

> propagate within a single range.

We've been through this, Aki.

I proposed extensive text onto the list (on 6/24/04) on a framework for 
Etag scopes that we *could* support in order to allow what you are 
talking about. As I wrote up the details, the complexity just grew and 
grew, and it hit the point that I didnt have any good ideas on how to 
practically implement it. As such, an open issue was raised (Issue 8), 
and the list consensus was to simplify, and keep etags scoped to a document.

We also heard from Scott Lawrence today that having the etag scope to a 
document was really an essential feature for using HTTP as well.

As such, I am disinclined to revisit this open issue, as we've been 
through it a few times already. Unless you can bring a new workable 
proposal to the table, I think we should stay with the current solution.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 03 Aug 2004 20:52:55 -0400

inline.

Jari Urpalainen wrote:

> <foo>. 

Right; but we are selecting based on foo's context!

> So if you select from foo's parent context with relative request

Your assumption here is that the context is modified each time we do an 
equality comparison. That is, when I check to see if the location step 
matches a particular child, I adopt the namespace context in scope for 
that child. This seems circular to me, and not what the Xpath spec is 
saying (I thought).

> is defined and how the query is interpreted.

Right. So, are you saying that XPath 2.0 says that, when evaluating 
steps along a child axes, you define the scope as the scope of that 
child when performing the evaluation?

> handling during implementation. Even an example would not be bad...

OK, I will clarify.

> the requirement of which is not desirable.

Well, we can't (and shouldn't) ask for caches to change.

-Jonathan R.

---

From: Jari Urpalainen <[email-masked]>
Date: Wed, 04 Aug 2004 16:10:24 +0300

> that child. This seems circular to me, and not what the Xpath spec is 

I have just glanced at the spec but I can't imagine any other way how
expression evaluation could work, because in xml you can/will attach the
namespace definition to the same node where it is defined.

Also XPath 1.0 spec does not say from where you do find the namespace
definitions (prefix and uri), for that e.g. XPointer (look 5.2) is
needed (i.e. if you don't use local-name() and namespace-uri()). If (and
when) you have to rely on the document context I don't see any other
options unless you have better ideas ?

> child when performing the evaluation?

if you look at the query /foo, / means document root and <foo> is the
child of /, i.e. the document root. So if we select from the document
root we'll find the child the <foo> node and yes the context in <foo> is
different from / as it is another step, right ? And if you have a query
like /ns:foo it is clear to me that you start searching for definition
for ns from <foo> context given the xml namespace model. 

The bogus default namespace behavior in xpath 1.0 is a well known
feature (deficiency) and I just assume that 2.0 will have that fixed by
this in-scope namespace definition. Also I assume that declaration of
namespaces in queries are left to XQuery so that XPath 2.0 don't either
have them (I might be wrong here). However, imo it doesn't matter that
much how it will be handled in the new xpath 2.0 spec because XCAP is
having an own bnf which is otherwise compatible with xpath 1.0, but
default namespace handling just needs to be different (i.e. sensible)
and that search for non-default namespace definitions should also be
defined in the draft. 

I am aware that using document context is far from optimal (or elegant)
but do we really have any good options ?

Furthermore, all of the libraries that I am aware of and claim to be
XPath 1.0 compatible leave this non-default namespace definitions search
out of the query, so you have to "register" namespace definitions
separately.

This is certainly a problem (as you have said) but imo it is up to us to
find a proper search model. 

br,
Jari

---

From: Aki Niemi <[email-masked]>
Date: Wed, 04 Aug 2004 21:14:27 +0300

> We've been through this, Aki.

Sorry about that, Jonathan. I now remember seeing the issue on the list
but with vacations and all it had simply slipped out of the memory
banks...

> and the list consensus was to simplify, and keep etags scoped to a document.

I agree with the complexity this would bring. In fact, reading the
concrete proposal text made my stomach turn ;) 

Since the scope markers would simply have the same effect as splitting
the document would have, why bother.

> proposal to the table, I think we should stay with the current solution.

Agreed, so let's stick to option 1 (scope is always the full document).

That in mind, perhaps we could add text to the chapter that contains
guidelines to application usage that highlights this issue, and explains
how one might go about avoiding this. 

Namely, that splitting the document into several documents, where it is
important to have etags apply to a certain place only. (That combined
with naming conventions for the "starting place" will do exactly what I
was actually looking for).

Again sorry for totally missing the discussion of this issue the first
time around.

Cheers,
Aki