Subject: [Simple] PUT vs POST in XCAP

From: Dean Willis <[email-masked]>
Date: Wed, 03 Mar 2004 21:56:42 -0600

This became a heated discussion in today's XCON meeting.

I believe that the heart of the discussion around PUT vs POST hinges on 
the following:

Is the operation:

1) Storage of a resource into a repository

or

2) Invocation of server-side logic on a parameter set being passed to 
that logic.

Given that at least some of the CPCP logic has the server making a 
transformation to the parameter set (adding the conference URI element), 
I believe that we fall into case 2. Certainly it would be much easier to 
build an XCAP implementation on Apache's HTTPD using a POST interface 
than it would using a PUT interface.

For reference, it should be noted that both XML-RPC and SOAP use POST to 
invoke server-side logic.

For example, see:

http://www.w3.org/TR/soap12-part0/#L26866

which says:

4.1.2 SOAP HTTP POST Usage
Using the HTTP binding with the SOAP Request-Response message exchange 
pattern is restricted to the HTTP POST method. Note that the use of this 
message exchange pattern in the SOAP HTTP binding is available to all 
applications, whether they involve the exchange of general XML data or 
RPCs (as in the following examples) encapsulated in SOAP messages.

Examples 9 and 10 show an example of a HTTP binding using the SOAP 
Request-Response message exchange pattern, using the same scenario as 
that for Example 4 and Example 5a, respectively, namely conveying an RPC 
and its return in the body of a SOAP message. The examples and 
discussion in this section only concentrate on the HTTP headers and 
their role.

Example 9
POST /Reservations HTTP/1.1
Host: travelcompany.example.org
Content-Type: application/soap+xml; charset="utf-8"
Content-Length: nnnn

<?xml version='1.0' ?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" >
  <env:Header>
    <t:transaction
            xmlns:t="http://thirdparty.example.org/transaction"
            env:encodingStyle="http://example.com/encoding"
            env:mustUnderstand="true" >5</t:transaction>
  </env:Header>
  <env:Body>
   <m:chargeReservation
      env:encodingStyle="http://www.w3.org/2003/05/soap-encoding"
           xmlns:m="http://travelcompany.example.org/">
    <m:reservation xmlns:m="http://travelcompany.example.org/reservation">
     <m:code>FT35ZBQ</m:code>
    </m:reservation>
     <o:creditCard xmlns:o="http://mycompany.example.com/financial">
      <n:name xmlns:n="http://mycompany.example.com/employees">
            Åke Jógvan Øyvind
      </n:name>
      <o:number>123456789099999</o:number>
      <o:expiration>2005-02</o:expiration>
     </o:creditCard>
    </m:chargeReservation>
   </env:Body>
</env:Envelope>RPC in Example 4 carried in an HTTP POST Request

This usage smells a whole lot like XCAP's to me.

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: "Niemi Aki (Nokia-M/Espoo)" <[email-masked]>
Date: Thu, 04 Mar 2004 08:53:31 +0200

ext Dean Willis wrote:
> I believe that the heart of the discussion around PUT vs POST hinges 
> on the following:
> 
> Is the operation:
> 
> transformation to the parameter set (adding the conference URI 
> element), I believe that we fall into case 2.

I disagree.

As was seen in today's excellent XCAP tutorial by Jonathan, you can very
well think of the system as consisting of two XCAP clients: a CPCP
client that PUTs the policy of a conference, and a CPCP client
(colocated with the conference server) that fills in the missing
details, like the conference URI.

Although I'm all for having discussion on this topic, I'd like to see
the counterproposal. What goes in the POST body? What then? It's really
hard to argue against imagineware...

> Certainly it would be much easier to build an XCAP implementation on 
> Apache's HTTPD using a POST interface than it would using a PUT 
> interface.

Could you elaborate? For example, does the CGI interface not give access
to the body of a PUT? What's the problem exactly?

Cheers,
Aki

<snip />

---

From: "Niemi Aki (Nokia-M/Espoo)" <[email-masked]>
Date: Thu, 04 Mar 2004 08:53:31 +0200

I disagree.

Cheers,
Aki

<snip />

---

From: Dean Willis <[email-masked]>
Date: Thu, 04 Mar 2004 05:24:40 -0600

> hard to argue against imagineware...

Basically, the same thing that goes in a PUT body now, but framed out as 
  an argument set. One approach might be to make it a text field in a form.

> to the body of a PUT? What's the problem exactly?

I'm not an expert on Apache internals, although I have been building web 
sites with it for ten years or so. As I understand it, PUT requests are 
interpreted by the WebDAV module (or by the basic web posting module if 
you don't have the WebDAV module) and literally results in writing a 
file to the file system. Nothing gets "triggered" when this happens, no 
magic application gets invoked to go do things like assign a conference 
number and update the etag, or anything like that. The data just sits 
there in a file. To implement the current XCAP model, you'd have to 
write some magic filesystem-monitoring daemon that notices the file has 
changed, locks it, reads it, and writes it back with new etags. This 
makes for 1) a lot of polling the filesystem for changes, and 2) really 
wicked locking problems between writes-from-apache and 
writes-from-the-magic-daemon to the data. There's probably a more clever 
way to do it, but it's going to be moderately painful in any case.

Compare this to the POST interface: the server hands the request off to 
the appropriate script-handler, which executes program logic on the 
request. Typically, the body of the request is passed into the handler, 
which parses, validates, and does something useful with that data.

In the most-like today's XCAP case, the handler would parse the request 
to determine the correct XCAP resource, parse out the specifier to get 
to the right place in the XML document, lock and rewrite the XML 
document (remember, this is a sequential text file), return a copy of 
the XML document with a new etag, then IPC notify the list server daemon 
about the change so that it could re-read, validate, and revise (for 
example, assign a conference URI) the resource and notify subscribers of 
changes.

If one can abandon the current convoluted XCAP update model (which 
writes the data BEFORE validating it), I think a much cleaner process 
happens if the script-handler validates the input, locks the resource, 
applies any needed transformation (like adding a conference URI), writes 
the resource, and returns the revised resource and its new etag to the 
POSTer, then IPC-notifies the list server.

This approach would appear relatively easy to implement in a Perl or PHP 
script invoked by the cgi-bin interface in Apache.  In fact, other than 
the xml processing, it's no harder than the scripts I use to run the 
softarmor web site. This is substantially easier to deal with (using my 
antquated skills, at least) than something that requires me to break out 
the compiler and write a new Apache dynamically loaded module that 
intercepts PUT requests.

--
dean

---

From: Dean Willis <[email-masked]>
Date: Thu, 04 Mar 2004 05:24:40 -0600

--
dean

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 09 Mar 2004 03:20:56 -0500

This is a debate we went through last June (I think) during the initial 
design phases of xcap. There were a few arguments against a POST mechanism:

1. is a tell-tale sign that you are really not using http, but rather, 
tunnelling some other protocol through http, which is a no-no 
(ftp://ftp.rfc-editor.org/in-notes/rfc3205.txt)

2. The vast majority of the cases, the xcap operations really WERE GET 
and PUT, tunneling these over POST would just result in effectively 
tunneling http over http.

To a large degree, the perspective depends on how much you think that 
server computed data is going to be a piece of the xcap puzzle. My view 
is that this was really an exception to the general xcap case, which had 
client managing data resident on a server. Right now, across all of the 
applications we have discussed, the one, and only one case for server 
computed data is to compute a URI associated with a resource, whether 
thats a conference (in the case of CPCP), or its a buddy list (in the 
case of resource lists). As such, xcap is optimized for the case where 
the vast majority of the protocol is worrying about direct modification 
of data (put/delete). I do not think we want to turn xcap into a general 
protocol for having servers fill in arbitrary data for different apps.

That said, assuming Dean's assessment of apache operation is correct 
(can someone verify that PUT works this way? I always thought one could 
write a servlet that handles PUT...), this would pose a problem. A key 
goal of XCAP is that it was easily built on existing servers and 
existing wireless clients. If people know of reasons why this goal has 
not been met, we should explore that.

A possible solution to this issue is to cleanly separate the URI 
allocation from the manipulation. We could define a separate piece of 
the protocol (POST based, perhaps), which has the server allocate the 
URI to the client. Once allocated, the client would PUT the entire 
document with that URI filled in. In this way, the data manipulation is 
using PUT/DELETE and is separated from the server computation.

This still requires two round trips to create a list. That seemed to 
give people a LOT of heartburn, but I dont think its an optimization we 
need to worry about that much. None of these operations are latency 
sensitive; at least, not to the point where one round trip vs. two 
really makes a difference. Furthermore, we are talkign only about the 
creation of the list/conference, which is not likely to be that frequent 
of an event. Thus, I dont see a bandwidth issue per se either.

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                600 Lanidex Plaza
Chief Technology Officer                    Parsippany, NJ 07054-2711
dynamicsoft
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 09 Mar 2004 03:20:56 -0500

-Jonathan R.

---

From: Jari Urpalainen <[email-masked]>
Date: Tue, 09 Mar 2004 12:58:37 +0200

> not been met, we should explore that.
Right, you can easily write apache modules that handle all requests e.g.
so that within some defined directory all requests/responses are handled
with that module (more to follow below). 

> of an event. Thus, I dont see a bandwidth issue per se either.
This is probably the best way to go, at least it is better than the
current one.
> > way to do it, but it's going to be moderately painful in any case.
I have been writing an apache module to handle basic xcap-requests. In
practice, an apache module will get parsed requests (+payload) and the
module may respond however it wishes. It is not necessary that any
file-io will ever happen, however in xcap, it is of course desirable.
There are also other implementation difficulties that could be
elaborated here, but from the modules point of view, apache only
provides basic http transport and module has to take care of the all
"real" handling.

> > the compiler and write a new Apache dynamically loaded module that 
While I'll admit that POST handling might be easier to implement in some
cases, but the much more difficult issue in implementation is
XML-manipulation and other issues, which are the same regardless of the
http method used. However, I'll agree that semantics in XCON may not fit
that well with XCAP but it still gives you a good baseline to start
with.

BR,
Jari

---

From: Jari Urpalainen <[email-masked]>
Date: Tue, 09 Mar 2004 12:58:37 +0200

BR,
Jari

---

From: Ted Hardie <[email-masked]>
Date: Tue, 9 Mar 2004 11:57:04 -0800

At 3:20 AM -0500 03/09/2004, Jonathan Rosenberg wrote:
>This is a debate we went through last June (I think) during the 
>initial design phases of xcap. There were a few arguments against a 
>POST mechanism:
>
>1. is a tell-tale sign that you are really not using http, but 
>rather, tunnelling some other protocol through http, which is a 
>no-no (ftp://ftp.rfc-editor.org/in-notes/rfc3205.txt)
>
>2. The vast majority of the cases, the xcap operations really WERE 
>GET and PUT, tunneling these over POST would just result in 
>effectively tunneling http over http.
>

I agree with Jonathan here, but I would put it slightly differently. 
We started
from a point where the easy method for updating a buddy list was downloading
the buddy list to the client, making the changes, and uploading the
new version to the server.  This is very much core GET and PUT stuff (or
WEBDAV stuff, if you want the locking elements of WEBDAV).

We wanted to optimize this transaction so that small changes did not
require expensive over-the-air downloads and uploads; to do that, we
considered using something like XPATH to reference document internals.
Given the complexity of XPATH, we agreed instead to limit the
use cases for XCAP to data which could be structured hierarchically.
That allowed us to treat sub-elements as independent resources, and
to use GET and PUT to update single sub-elements without incurring
the costs of the over the air download.  As a side benefit, it allowed
us to use well-understood, existing methods and gave an upgrade
path to the use of full WebDAV semantics if that became desirable
or necessary (e.g. using locks or PROPPATCH).

I still believe that is the core of the XCAP work.  We are beginning to
see a few stresses, however, as new use cases are found.  Use cases
which are very closely related to the configuration work done by
XCAP look like a natural fit, but we have to be careful that we don't
stretch XCAP too far.  As originally conceived, it's more-or-less a description
of how to use base HTTP methods with a particular set of resources; as we add
semantics beyond that, we risk needing new HTTP methods or
other handling mechanisms.  Those can be added if needed, but they
reduce the interoperability with existing HTTP implementations.
Rather than heading in that direction, it may be better to reduce
the scope of XCAP to its core functionality and let other extensions
create those semantics.

Creating mechanisms that are essentially new methods but implementing
them through POST is a bad long-term idea.  POST is designed to be
specific to a particular context (a form being filled in, essentially)
and trying to ensure that it has long term context turns out to be very
hard.  One of the HTTP wg members (sorry, forgotten who) used
to call it the "magic happens" method, in honor of an old cartoon
which showed carefully drawn flow charts for a set of actions, with
one box labelled "magic happens" that covered all the icky bits.
As tempting as it is, it turns out to be very hard to ensure that the
same magic happens every time, everywhere.

Just my two cents,

				regards,
					Ted Hardie

>To a large degree, the perspective depends on how much you think 
>that server computed data is going to be a piece of the xcap puzzle. 
>My view is that this was really an exception to the general xcap 
>case, which had client managing data resident on a server. Right 
>now, across all of the applications we have discussed, the one, and 
>only one case for server computed data is to compute a URI 
>associated with a resource, whether thats a conference (in the case 
>of CPCP), or its a buddy list (in the case of resource lists). As 
>such, xcap is optimized for the case where the vast majority of the 
>protocol is worrying about direct modification of data (put/delete). 
>I do not think we want to turn xcap into a general protocol for 
>having servers fill in arbitrary data for different apps.
>
>(can someone verify that PUT works this way? I always thought one 
>could write a servlet that handles PUT...), this would pose a 
>problem. A key goal of XCAP is that it was easily built on existing 
>servers and existing wireless clients. If people know of reasons why 
>this goal has not been met, we should explore that.
>
>allocation from the manipulation. We could define a separate piece 
>of the protocol (POST based, perhaps), which has the server allocate 
>the URI to the client. Once allocated, the client would PUT the 
>entire document with that URI filled in. In this way, the data 
>manipulation is using PUT/DELETE and is separated from the server 
>computation.
>
>give people a LOT of heartburn, but I dont think its an optimization 
>we need to worry about that much. None of these operations are 
>latency sensitive; at least, not to the point where one round trip 
>vs. two really makes a difference. Furthermore, we are talkign only 
>about the creation of the list/conference, which is not likely to be 
>that frequent of an event. Thus, I dont see a bandwidth issue per se 
>either.
>
>-Jonathan R.
>
>
>
>Dean Willis wrote:
>
>>Basically, the same thing that goes in a PUT body now, but framed 
>>out as  an argument set. One approach might be to make it a text 
>>field in a form.
>>
>>>
>>>>Certainly it would be much easier to build an XCAP implementation 
>>>>on Apache's HTTPD using a POST interface than it would using a 
>>>>PUT interface.
>>>
>>>
>>>
>>I'm not an expert on Apache internals, although I have been 
>>building web sites with it for ten years or so. As I understand it, 
>>PUT requests are interpreted by the WebDAV module (or by the basic 
>>web posting module if you don't have the WebDAV module) and 
>>literally results in writing a file to the file system. Nothing 
>>gets "triggered" when this happens, no magic application gets 
>>invoked to go do things like assign a conference number and update 
>>the etag, or anything like that. The data just sits there in a 
>>file. To implement the current XCAP model, you'd have to write some 
>>magic filesystem-monitoring daemon that notices the file has 
>>changed, locks it, reads it, and writes it back with new etags. 
>>This makes for 1) a lot of polling the filesystem for changes, and 
>>2) really wicked locking problems between writes-from-apache and 
>>writes-from-the-magic-daemon to the data. There's probably a more 
>>clever way to do it, but it's going to be moderately painful in any 
>>case.
>>
>>Compare this to the POST interface: the server hands the request 
>>off to the appropriate script-handler, which executes program logic 
>>on the request. Typically, the body of the request is passed into 
>>the handler, which parses, validates, and does something useful 
>>with that data.
>>
>>In the most-like today's XCAP case, the handler would parse the 
>>request to determine the correct XCAP resource, parse out the 
>>specifier to get to the right place in the XML document, lock and 
>>rewrite the XML document (remember, this is a sequential text 
>>file), return a copy of the XML document with a new etag, then IPC 
>>notify the list server daemon about the change so that it could 
>>re-read, validate, and revise (for example, assign a conference 
>>URI) the resource and notify subscribers of changes.
>>
>>writes the data BEFORE validating it), I think a much cleaner 
>>process happens if the script-handler validates the input, locks 
>>the resource, applies any needed transformation (like adding a 
>>conference URI), writes the resource, and returns the revised 
>>resource and its new etag to the POSTer, then IPC-notifies the list 
>>server.
>>
>>This approach would appear relatively easy to implement in a Perl 
>>or PHP script invoked by the cgi-bin interface in Apache.  In fact, 
>>other than the xml processing, it's no harder than the scripts I 
>>use to run the softarmor web site. This is substantially easier to 
>>deal with (using my antquated skills, at least) than something that 
>>requires me to break out the compiler and write a new Apache 
>>dynamically loaded module that intercepts PUT requests.
>>
>>--
>>dean
>>

---

From: Ted Hardie <[email-masked]>
Date: Tue, 9 Mar 2004 11:57:04 -0800

Just my two cents,

				regards,
					Ted Hardie

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 11 Mar 2004 00:03:20 -0500

inline.

> This is probably the best way to go, at least it is better than the

Any other comments?

> provides basic http transport and module has to take care of the all

Can you comment on those implementation difficulties? If there are 
aspects of the protocol that make it hard to implement, and some change 
would help, I'd like to know that.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 11 Mar 2004 00:03:20 -0500

inline.

Any other comments?

-Jonathan R.

---

From: Jari Urpalainen <[email-masked]>
Date: Thu, 11 Mar 2004 14:44:54 +0200

> > This is probably the best way to go, at least it is better than the
One could also do this post? after the resource has been created so that
you'll reference to the resource and request for uri creation. The
response will include created uri's with corresponding xpath-references
(so the server will add them and will tell all the changes to client)

> would help, I'd like to know that.

Well, that's my goal...

I'll try to shortly sum up the current state of the implementation
status. First, the purpose is just to do a proof-of-concept reference
implementation. Secondly, implementation is done in linux with C with
the aid of libxml2-library aiming at a standalone server or an Apache
module. All the XML and XPATH handling is being done with libxml2. It
has also a schema checker which isn't adequate at the moment so Xerces
C++ library has been used instead.

In general with this background, I would say that the basic server
implementation with all XPATH interpretation is fairly easy to implement
and fairly performance-wise. What is not so easy and where not too much
attention (or at all?) has yet been, is this XCAP http-uri versus
sip-uri mapping in resource-lists concerned with list-subscriptions.
While XCAP server knows both endpoints it IMO should also keep track
that user won't define separate lists with the same sip-uri. Adding this
logic to the server was somewhat more complex. I have also some concerns
related to performance of schema validation which seems to easily drop
the throughput of requests. Secondly, all the au's will need some
interpretation of their own and I haven't yet started with presence
authorization.

The client side of implementation is/was probably even more challenging
as you have to interfere with http-client library and have to keep some
internal state information when doing updates to resources (and when
doing just partial updates which we are eagerly promoting).

Writing this reminded me of the thing that the base XCAP could have a
few words about If-None-Match -header with conditional GET so that a
client can get a simple sync with a single file. We'll also need the
directory-list still. Btw. I recently implemented this as a recursive
directory list so that you can get e.g. all the files at once for a user
simply by a query get http://xcap/resource-lists/users/joe and of course
going one path up you could get all files for all users (still wondering
whether separate au is needed...). The package will be needed still, I
think.

BR,
Jari

---

From: Jari Urpalainen <[email-masked]>
Date: Thu, 11 Mar 2004 14:44:54 +0200

Well, that's my goal...

BR,
Jari

---

From: "Niemi Aki (Nokia-M/Espoo)" <[email-masked]>
Date: Thu, 11 Mar 2004 15:17:19 +0200

> using PUT/DELETE and is separated from the server computation.

This sounds like a good approach to solving this issue.

> of an event. Thus, I dont see a bandwidth issue per se either.

I agree.

Cheers,
Aki

---

From: "Niemi Aki (Nokia-M/Espoo)" <[email-masked]>
Date: Thu, 11 Mar 2004 15:17:19 +0200

I agree.

Cheers,
Aki

---

From: Kawaguti Ginga <[email-masked]>
Date: Fri, 12 Mar 2004 12:47:04 +0900

In Tue, Mar 09, 2004 at 11:57:04AM -0800,
> same magic happens every time, everywhere.

The current XCAP design for policy handling itself implies quite much
of "magic happening" behind this protocol, which are not explicitly
specified.

XCAP itself seems to be a "file editing" protocol.
However, XCAP's PUT action in policy-controlling context
is not just "putting a file (fragment)".
It implicitly...

(1) edits a policy document file,
    (not nessesarily a FILE, but it requires EDITing anyway)
(2) checks if it is a valid document,
    (well-formed? valid? acceptable for site-policy? etc.
     Some of these decisions can be made outside the XCAP server,
     as indicated in Jonathan's XCAP tutorial.)
(3) informs the presence server of the policy-updates.
    (kill -HUP or whatever.)

To make policy handling work robust and fail-safe,
the above points cannot be ignored.
As an interface/protocol definition, these actions should be taken
into consideration, although some poor-implementation may 
not act exactly as above.

IMHO, XCAP's PUT has lot more to do than an ordinally HTTP's PUT.

Regards,
Ginga Kawaguti

-- 
Office:  NTT Communications Innovative IP Architecture Center 1PT 1P
Address: KAWAGUTI Ginga <[email-masked]>
Phone:   voice=+81-3-6800-3032; fax=+81-3-5388-0550

---

From: Kawaguti Ginga <[email-masked]>
Date: Fri, 12 Mar 2004 12:47:04 +0900

Regards,
Ginga Kawaguti

---

From: Dean Willis <[email-masked]>
Date: Mon, 15 Mar 2004 09:49:32 -0600

> not been met, we should explore that.

One can indeed write modules that intercerpt PUT operations. There seem 
to be limitations -- for example, much of the literature asserts that 
people have difficulty making mod-perl handle PUT, and that implementers 
have had to fall back to much-less-efficient external CGI invocations.

The other problem is that if other modules that handle PUT (such as 
mod-dav, the DAV module) are active in the same pseudoroot there is no 
mechanism of which I am aware to specify which module handles the PUT 
request.

POST is much more specific -- it indicates not only which module, but 
(through the URL) which server-side program is to handle the data.

--
dean

---

From: Dean Willis <[email-masked]>
Date: Mon, 15 Mar 2004 09:49:32 -0600

--
dean

---

From: Kawaguti Ginga <[email-masked]>
Date: Thu, 11 Mar 2004 20:09:34 +0900

(1) edits a policy document file
    (not nessesarily a FILE, but it requires EDITing anyway)
(2) checks if it is a valid document
    (well-formed? valid? acceptable for site-policy? etc.
     These decisions can be made outside the XCAP server,
     as indicated in Jonathan's XCAP tutorial.)
(3) informs the presence server of the policy-updates.
    (kill -HUP or whatever.)

As an interface/protocol definition, these points should be taken

Regards,
Ginga Kawaguti

---

From: Kawaguti Ginga <[email-masked]>
Date: Thu, 11 Mar 2004 20:09:34 +0900

Regards,
Ginga Kawaguti