Subject: RE: Proposal: WebDAV and transactions

From: "Babich, Alan" <[email-masked]>
Date: Mon, 9 Sep 2002 12:36:26 -0700

The database vendors faced the issue of transactions in a client-server
network when they distributed their databases some ago. The naïve approach
of just remoting each database call was a performance disaster on large
networks. That is why they invented database script programming languages
and stored procedures. Stored procedures are scripts (in a database
programming language) that allow fairly general database transactions to be
programmed. These scripts are stored in the database. Clients invoke the
scripts by passing them parameters. 

Besides performance, one of the advantages of stored procedures is
standardization of the database processing. The system doesn't have to
depend upon every client getting the update processing correct. The stored
procedures can theoretically enforce all the checking and standardization
required. If one limits who can write stored procedures to a small trusted
group, the integrity of the system is protected.

As you pointed out, it is sometimes the case that the processing involved in
WebDAV transactions as you envision them is conditional on the result of
previous processing steps.

If you give people a gun they can shoot themselves in the foot with, then a
certain percentage of the people will shoot themselves in the foot. (I know
someone who literally did that.) If the gun refuses to shoot you in the
foot, that is a better design for the gun. In the spirit of that analogy, I
am proposing that the entire transaction, conditional processing and all,
should be invoked by one method call. That would do a lot to help protect
the performance of the system.

Within the single-method constraint, there are two ways to go: (1) Specify
the entire transaction, conditional processing and all, within the stuff you
send over the network in the method invocation. (2) Invent stored WebDAV
procedures that specify WebDAV transactions (conditional processing and
all), and have the method pass parameters to the stored procedure. You could
think about taking either or both of these approaches.

It is an open question as to whether either of the above two approaches can
be done well without doing violence to HTTP and/or WebDAV.

Alan

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, September 09, 2002 7:34 AM
To: Pill, Juergen; [email-masked]
Subject: RE: Proposal: WebDAV and transactions

Jürgen,

I think the main issue isn't that people aren't interested -- it's more that
they (well, I) think that it's an extremely bad idea that doesn't adher to
the HTTP model.

In particular, it would make the meaning of an HTTP method invocation depend
on previous method calls. That makes it hard or impossible for
intermediaries to work with these messages.

Julian

--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Pill, Juergen
> Sent: Monday, September 09, 2002 4:17 PM
> To: [email-masked]'
> Subject: RE: Proposal: WebDAV and transactions
>
>
>
> Hello,
>
> I did not receive a lot of feed-back on the TA "proposal" (and the
> statements were both positive and negative).
>
> Would you agree on the statement, the WebDAV group would be currently not
> interested in standardizing TA methods?
>
> As we would need this extension for the API implementation, our
> option would
> be to implement a proprietary WebDAV TA extension available on our server
> only.
>
> If the interest on this topic would raise again - possibly at a
> later time -
> we would be happy to share our experience with you.
>
> Best regards,
>
> Juergen Pill
>
>
>
>  -----Original Message-----
> From: 	Pill, Juergen [mailto:[email-masked]]
> Sent:	Monday, September 02, 2002 12.24 PM
> To:	[email-masked]'
> Subject:	Proposal: WebDAV and transactions
>
>
> Hello,
>
> The current WebDAV specifications deal with a transactional support on a
> method base only. Either a single WebDAV method is completely and
> successfully executed, or the method is not executed at all (e.g. a
> PropPatch command will either modify all requested properties or none of
> them). A transactional support spanning multiple WebDAV methods
> is currently
> not specified.
> Multi user requirements are either handled in separate workspaces
> (Delta-V)
> or via the Lock method.
> During our discussion on the JSR 170 (Content Management API)
> implementation
> on top of WebDAV, we believe that we require longer transactions spanning
> multiple separate WebDAV methods. (This issue was already partially
> discussed in the Batch method thread). Do you also fell the
> need/requirement
> to get a standard on the way, concerning a transactional WebDAV protocol
> extension?
>
> A possible use case would be:
> An author wants to modify an html page. To successfully do so, he will PUT
> the updated html page back, PROPPATCH and UNLOCK it, DELETE all bitmap
> files, which are not references any more, and PUT (create) the newly
> referenced bitmaps files. If he wants to ensure, that either all
> his changes
> do apply (or none of them) he would surround those method call with an
> TA-BEGIN and a TA-COMMIT method call. If the author would be an API (e.g.
> JSR 170) exposing the TA functionality to an application, we believe this
> feature to be mandatory.
>
> A possible (starting) specification could be:
> WebDAV defines 3 additional methods: TA-BEGIN, TA-COMMIT and TA-ABORT. The
> TA-BEGIN method will return a TA token (e.g. via an XML response
> body). This
> TA token will be send to the following WebDAV methods, which are part of
> this transaction (e.g. via an additional header). The TA-COMMIT
> or TA-ABORT
> method will either commit or abort the transaction and invalidate the TA
> token.
> If and when an open TA, which is not used any more, is aborted, could be
> controlled similar to the lock method timeout header.
> All WebDAV methods, which either do not posses a TA token, or posses an
> invalid token are handled as today (the method is executed within its own
> TA), this would ensure upward compatibility.
> This extension would be optional, if a server supports the TA methods, it
> will state this in the OPTIONS request (similar to Delta-V).
>
>
> Does this make sense?
>
> Best regards
> Juergen Pill
>

---

From: "Eric Sedlar" <[email-masked]>
Date: Tue, 10 Sep 2002 00:51:57 -0700

I think it is clear that the best way to address this requirement is through
batching up WebDAV METHODS into a single request.  We also have requirements
for doing batch methods for other reasons, and I know Microsoft has also run
into this requirement and extended the standard this way.

--Eric

----- Original Message -----
From: "Babich, Alan" <[email-masked]>
To: "'Julian Reschke'" <[email-masked]>; "Pill, Juergen"
<[email-masked]>; <[email-masked]>
Sent: Monday, September 09, 2002 12:36 PM
Subject: RE: Proposal: WebDAV and transactions

---

From: "B. Shadgar" <[email-masked]>
Date: Wed, 11 Sep 2002 11:01:52 +0100

> into this requirement and extended the standard this way.

I think Batch method can help a lot in favor of transactions, But I don't know
why still it has not been accepted as a standard method in WebDAV protocol ?

Bita,

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 11 Sep 2002 12:08:13 +0200

> [mailto:[email-masked]]On Behalf Of B. Shadgar
> Sent: Wednesday, September 11, 2002 12:02 PM
> To: Eric Sedlar; [email-masked]
> Subject: Re: Proposal: WebDAV and transactions
> 
> ...
> 

Well, make a proposal that works and let the discussion start.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 11 Sep 2002 14:13:33 +0200

> into this requirement and extended the standard this way.

I don't think that is clear at all.  Batching methods does nothing to
help with transactions that cannot be accomplished just as easily
(and without breaking HTTP security) by including a transaction identifier
in each request.  Changing the packaging of a sequence of interrelated
requests does not change the fact that they are a sequence of interrelated
requests.  Eventually, something has to process those requests in sequence,
as a transaction.  The server simply needs a way to accumulate requests
and a method to commit them, and therefore it makes sense to have methods
for begin-, abort-, and commit-transaction.

>> If the gun refuses to shoot you in the
>> foot, that is a better design for the gun.

In that case, a gun that is incapable of shooting at all would be
a "better" design.  Considering requirements in isolation is a waste
of energy.

>> In the spirit of that analogy, I

It would do nothing to help protect the performance of the system.
There is no effective difference between a sequence of requests with
time-outs per request and a batch request with time-outs per read.
The server still has to accumulate requests in storage and is
subject to the same service problems with stateful behavior.

The Web already has stored procedures.  There is nothing preventing an
application developer from building complex transactions within a
single method handler.  The problem is when you allow multiple methods
with differing semantics to take place under the rubric of another
method, which removes HTTP's ability to govern access rights per method.

....Roy

---

From: "B. Shadgar" <[email-masked]>
Date: Wed, 11 Sep 2002 13:39:54 +0100

> Well, make a proposal that works and let the discussion start.

A proposal for Batch method is following:

Batch Method

The Batch method is introduce to processes a bunch of methods as one
single transaction and to improve the overload of internet connection.

The request message body of a BATCH method MUST contain one-transaction
and  request-set XML elements. It is included by one or more request XML
elements. processing of request elements MUST occur in the order request
elements are received (i.e., from top to bottom). Request elements can
either all be executed or some executed which will be specified by value
of one-transaction element. If the  value is true, all of request
elements MUST be executed, thus if any error occurs during processing
all executed  requests MUST be undone and resopnse MUST return the
result of failed request. Otherwise, when the one-transaction element's
value is false, some of requests can be executed and a proper response
with 207 statuse code  MUST be prepared to shows which requested has
been done and why the others refused.

The request XML element is included by header, body and
expected-status-code elements. Header and body element contains the
header and body of given request and expected-status-code is
representing the result is expected. It can be expressed by one value or
a range of values e.g  200 <= X < 300 by xml elements.

I consider Pill's example in his proposal and try to make my mean clear:

>> Request

BATCH  /bar/  HTTP/1.1
Host: www.foo.bar
Content-Type: text/xml; charset="utf-8"
Content-Length: xxxx

<?xml version="1.0" encoding="utf-8" ?>
<D:request-set xmlns:D="DAV:">
   <D:one-transaction>true</D:one-transaction>
   <D:request>
      <D:header> The header of PUTmethod  </D:header>
      <D:body> the Body of PUT method </D:body>
      <D:expected-status-code> <or>
                                                    <li>204</li>
                                                    <li>201</li>
                                                 </or>
      </D:expected-status-code>
   </D:request>
   <D:request>
      <D:header> The header of PROPPATCH method  </D:header>
      <D:body> the Body of PROPPATCH method </D:body>
      <D:expected-status-code> 200 </D:expected-status-code>
   </D:request>
   <D:request>
      <D:header> The header of UNLOCK method  </D:header>
      <D:body> the Body of UNLOCK method </D:body>
      <D:expected-status-code>204</D:expected-status-code>
   </D:request>
   <D:request>
      <D:header> The header of Delete method  </D:header>
      <D:body> the Body of Delete method </D:body>
      <D:expected-status-code> 204</D:expected-status-code>
   </D:request>
 </D:request-set>

>> Respons

HTTP/1.1  200 Ok

or

>> Response

HTTP/1.1 403 Forbidden
Content-Type: text/xml; charset="utf-8"
Content-Length: xxxx

    <?xml version="1.0" encoding="utf-8" ?>
    <D:response>
       <D:method>
               <D:status>HTTP/1.1 204 No Content </D:status>
       </D:method>
       <D:method>
               <D:status>HTTP/1.1 200 Ok</D:status>
       </D:method>
       <D:method>
                 <D:status>HTTP/1.1 204 No Content </D:status>
        </D:method>
       <D:method>
                  <D:href>http://www.foo.bar/bar/resource3>
                  <D:status>HTTP/1.1 403 Forbidden</D:status>
       </D:method>
     </D:response>

We can consider a bunch of delete methods when the one-transaction
element was false. In this case the format of the request and response
is the same, but the one-transaction element is set with false value,
and therefore failing some of Delete method dosn't cause to undone the
batch method. The response can inform the client which delete methods
are failed.

Does it make sence?

Regards,
Bita.

---

From: "B. Shadgar" <[email-masked]>
Date: Wed, 11 Sep 2002 14:20:40 +0100

> for begin-, abort-, and commit-transaction.

If you consider the proposal of batch method I have already sent to mailing
list, and if you consider that server would have the following schema of
handling a given method:

try {
     TA.begin();
     XXX Method();
    TA.commit();
} catch ( Exception e) {
    TA.rollback()
}

The whole of batch method would be considered as a one transaction without
having TA-methods like begin, abort and commit. For each request in the Batch
method we just simply call the corresponding method which is not a separate
transaction.

> a "better" design.  Considering requirements in isolation is a waste

I agree with this. Every body can make its choice to shoot or not.

> method, which removes HTTP's ability to govern access rights per method.

Although I think that Batch method can help regarding the transactions, but it
is not enough. Because, it can only handle a transaction which made by a
sequence of WebDAV methods. However it is not the whole problem of
transactions.
Consider an application which is using WebDAV protocol as a part of it work and
it needs to do some other functionality along with the WebDAV methods as  one
transaction. In this case my proposal of batch method doesn't help that
application. It would necessary need to TA-methods or change the server code to
provide its own function. The latter one is an anathema.

If I am wrong please make me correct.

Regards
Bita.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 11 Sep 2002 18:22:03 +0200

I'll try to explain.  It is fundamental to HTTP that intermediaries
are able to inspect the method of an HTTP request in order to determine
the semantics of a request, and thereby its associated access control
and/or authentication requirements.  What you are suggesting is that
we bypass that feature in HTTP via a new syntax that tunnels arbitrary
methods within XML via HTTP, effectively bypassing all of the benefits
of using HTTP in the first place.

Sound familiar? It should, since this is the same objection that I have
frequently given to the use of SOAP for tunneling arbitrary semantics
though an HTTP POST method.

Transactions can be accomplished by using a request to ask the server
for a transaction context, possibly multi-level, and then sending
the sequence of requests with a calculated transaction-request-id
in the header fields, to each the server responds with 202 Accepted
or an error, and finalized with a commit or abort method, to which
the server responds with the appropriate message for the entire
commit.  Such a transaction mechanism accomplishes the same thing
without violating the requirements of HTTP.

In any case, this is OUT OF SCOPE for WebDAV.  It is an HTTP issue
that is not specific to authoring and will not be solved by an
authoring-specific solution.

....Roy