Subject: [Simple] XCAP Issue 1: Schema extensibility

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 10 May 2004 01:03:03 -0400

One of the issues we've been back and forth on a few times is handling 
schema extensibility. That is, does the xcap server need to understand 
the namespaces and schemas in all documents it handles? Initially, in 
early versions of xcap, the answer was yes. Then, in the current 
approach, the answer is no, but it requires an ugly Require-like set of 
elements within the actual documents themselves to negotiate what is 
supported. Worse yet, it came out during the meeting in Seoul that this 
simply wouldnt work. The problem is that the current xcap rules specify 
that the server performs an insertion of an element in a document such 
that, after insertion, the R-URI selects the element, and the resulting 
document is schema compliant.

As a result of this, the server has to actually use awareness of the 
schema to determine where to insert a new element. That means that the 
server will need to understand the schemas, and it also introduces 
complexity into the implementation. It also introduces a substantial 
piece of "magic" into the server processing of PUT which arguably bends 
the rules on what PUT can do.

So, I would propose the following.

Servers do not need to understand the schemas of the documents they 
handle. When inserting a new element, the server inserts the element 
such that, after insertion, the R-URI would select that element. When 
there are many such insertion points with this property, the server puts 
it at the "bottom" - that is, in the location whereby there are the 
fewest number of siblings following it.

This insertion rule is schema-unaware, which is a good thing. It puts 
the burden on the client to construct the R-URI when inserting an 
element, such that it gets put in a place whereby the resulting document 
remains schema compliant. For example, lets say a schema for a document 
said that, within the foo element, you had zero or one bar elements 
followed by zero or one baz elements. If we start with this document:

<foo>
   <baz/>
</foo>

If I insert bar, bar *has* to be inserted such that it appears before 
baz. Thats what the schema says. Doing this:

PUT http://server/xcap-root/document/foo/bar

<bar/>

according to the above rules, will create a document that is not schema 
compliant.

So, to allow the client to manipulate the document so that the resulting 
documents are schema compliant, we need to be able to support positional 
insertions better than we do today. Please see a separate email on that 
topic, which is an open issue in and of itself.

My proposal is to allow for positional insertions and, consequently, 
define the server insertion rules to be schema-unaware, thus allowing 
for the server to operate without understanding the schema.

Thanks,
Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                600 Lanidex Plaza
Chief Technology Officer                    Parsippany, NJ 07054-2711
dynamicsoft
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 10 May 2004 08:30:28 -0400

While this rule may be a good thing on its own, I believe that the driver / 
motivation given is wrong, and that perpetuating this driver will get us 
into more trouble.

Let me state a few premises.  If these are wrong, please correct them.
1) The information stored on the server must be correct / consistent / valid.
2) The server must not discard user changes without notifying the user.
3) The only time we have to notify a user of an error is in the response to 
the PUT.

 From these premises, it seems to follow that the XCAP server must perform 
full validation on the input it is provided so as to ensure that the 
resulting document is valid.  This validation is actually more 
comprehensive than schema validation, in that it may reference semantic 
integrity constraints that con not be represented in the schema.
I realize that the model calls for allowing a separated front end XCAP 
server and a back end information server.  I am not asking that such a 
separation be removed.  However, the front end process must perform all 
validation.

Given that the XCAP server must be fully semantically aware, this change 
may nonetheless be useful in that it reduces the complexity of the location 
identifying portion of the engine.  But it should not be justified by 
saying that the XCAP server itself is schema unaware.  The server is more 
than the location resolution logic.

Yours,
Joel M. Halpern

At 01:03 AM 5/10/2004 -0400, Jonathan Rosenberg wrote:
>schema extensibility. That is, does the xcap server need to understand the 
>namespaces and schemas in all documents it handles? Initially, in early 
>versions of xcap, the answer was yes. Then, in the current approach, the 
>answer is no, but it requires an ugly Require-like set of elements within 
>the actual documents themselves to negotiate what is supported. Worse yet, 
>it came out during the meeting in Seoul that this simply wouldnt work. The 
>problem is that the current xcap rules specify that the server performs an 
>insertion of an element in a document such that, after insertion, the 
>R-URI selects the element, and the resulting document is schema compliant.
>
>complexity into the implementation. It also introduces a substantial piece 
>of "magic" into the server processing of PUT which arguably bends the 
>rules on what PUT can do.
>
>handle. When inserting a new element, the server inserts the element such 
>that, after insertion, the R-URI would select that element. When there are 
>many such insertion points with this property, the server puts it at the 
>"bottom" - that is, in the location whereby there are the fewest number of 
>siblings following it.
>
>This insertion rule is schema-unaware, which is a good thing. It puts the 
>burden on the client to construct the R-URI when inserting an element, 
>such that it gets put in a place whereby the resulting document remains 
>schema compliant. For example, lets say a schema for a document said that, 
>within the foo element, you had zero or one bar elements followed by zero 
>or one baz elements. If we start with this document:
>
><foo>
>   <baz/>
></foo>
>
>If I insert bar, bar *has* to be inserted such that it appears before baz. 
>Thats what the schema says. Doing this:
>
>define the server insertion rules to be schema-unaware, thus allowing for 
>the server to operate without understanding the schema.
>
>Thanks,
>Jonathan R.
>
>
>--

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 10 May 2004 08:30:28 -0400

Yours,
Joel M. Halpern

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 11 May 2004 11:45:16 -0400

inline.

Joel M. Halpern wrote:

> While this rule may be a good thing on its own, I believe that the 
> driver / motivation given is wrong, and that perpetuating this driver 
> will get us into more trouble.
> 

Agreed.

> 2) The server must not discard user changes without notifying the user.

Agreed.

> 3) The only time we have to notify a user of an error is in the response 
> to the PUT.

Right for the most part; there can of course be errors in GET if the URI 
  is wrong, but in terms of data validation, those errors would only 
occur in resposne to PUT.

> 
>  From these premises, it seems to follow that the XCAP server must 
> perform full validation on the input it is provided so as to ensure that 
> the resulting document is valid. 

Right.

> This validation is actually more 

Agreed. Thats why xcap allows application usages to define additional 
constraints not expressible in schema.

> may nonetheless be useful in that it reduces the complexity of the 
> location identifying portion of the engine. 

Well, to be clear, it may be that, for some extension to an existing 
piece of data, no validation is required per se. By removing the need 
for schema awareness from the "location identifying portion", xcap can 
still *work*, and if you want validation, it can be added separately at 
a later time. Thats the concern that has been expressed, which I was 
trying to address - the desire to want to manage data for which no 
validation is required, or is not needed initially.

The driving case, in fact, is pidf-manipulation, where we want the user 
to be able to manage their default presence document in xcap. It should 
be possible for that default presence document to have presence 
extensions in it that the server does not understand.

> But it should not be 
> justified by saying that the XCAP server itself is schema unaware.  The 
> server is more than the location resolution logic.

Thats a good point. The specific proposal, as you say, is to make it 
such that schema awareness is only needed for validation operations, not 
for location resolution or other normative parts of the specification.

Hope that clarifies,
Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 11 May 2004 11:45:16 -0400

inline.

Agreed.

Agreed.

Right.

Hope that clarifies,
Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 17 May 2004 03:43:21 -0400

I believe we have reached conclusion on this thread, and consensus. We 
will go with the approach for schema extensibility I described in my 
note, appropriately clarified as suggested in the thread.

Thanks,
Jonathan R.

Jonathan Rosenberg wrote:

>> 3) The only time we have to notify a user of an error is in the 
>> response to the PUT.
> 
> 
>> perform full validation on the input it is provided so as to ensure 
>> that the resulting document is valid. 
> 
> 
> Right.
> 
>> This validation is actually more comprehensive than schema validation, 
>> in that it may reference semantic integrity constraints that con not 
>> be represented in the schema.
> 
> 
>> separation be removed.  However, the front end process must perform 
>> all validation.
>>
>> Given that the XCAP server must be fully semantically aware, this 
>> change may nonetheless be useful in that it reduces the complexity of 
>> the location identifying portion of the engine. 
> 
> 
>> But it should not be justified by saying that the XCAP server itself 
>> is schema unaware.  The server is more than the location resolution 
>> logic.
> 
>

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 17 May 2004 03:43:21 -0400

Thanks,
Jonathan R.