Subject: Re: No partial success when it comes to deletion?

From: [email-masked]
Date: Thu, 30 Jan 2003 16:38:08 -0500

CAVEAT: This may appear very long but you can just skip over the example 
data if you want to get to the discussion bits at the bottom. /CAVEAT

Andrea wrote on 01/24/2003 10:36:56 AM CET:
> On Thu, Jan 23, 2003 at 03:43:40PM -0500, [email-masked] 
wrote:
> > expected.  (After all a VREPLY with a success response was already 
sent 
> > back).  I for one do not think that mandating a CS essentially be 
built 
> 
> Sorry? You mean, if the command succeeds for one target and fails
> for another one, right? 

Sure.  From Section 10.1.15 DELETE command I grok:

There MUST BE one "VREPLY" component returned for each object that is 
deleted or marked for delete. 

and the ABNF of:

delete-reply   = "BEGIN" ":" "VCALENDAR" CRLF
                 calprops   ; MUST include 'reply-cmd'
                 1*(delete-vreply)
                 "END" ":" "VCALENDAR" CRLF

so the DELETE command:

BEGIN:VCALENDAR
CMD;ID:"random but unique per CUA":DELETE
TARGET:work
TARGET:personal
BEGIN:VQUERY
QUERY:SELECT * FROM VEVENT WHERE ....
END:VQUERY
END:VCALENDAR

would result in responses like:

BEGIN:VCALENAR
TARGET:work
CMD;ID:"random but unique per CUA":REPLY
BEGIN:VREPLY
UID:abcd12345
REQUEST-STATUS:2.0
END:VREPLY
BEGIN:VREPLY
UID:abcd67890
REQUEST-STATUS:2.0
END:VREPLY
END:VCALENDAR
[Some pause here...]
BEGIN:VCALENAR
TARGET:personal
CMD;ID:"random but unique per CUA":REPLY
BEGIN:VREPLY
UID:1234567890
REQUEST-STATUS:2.0
BEGIN:VREPLY
UID:abcd12345
REQUEST-STATUS:2.0
END:VREPLY
END:VCALENDAR
[Another pause...]
BEGIN:VCALENAR
TARGET:work
CMD;ID:"random but unique per CUA":REPLY
BEGIN:VREPLY
UID:0987654321
REQUEST-STATUS:4.0
END:VREPLY
END:VCALENDAR
BEGIN:VCALENAR
TARGET:personal
CMD;ID:"random but unique per CUA":REPLY
BEGIN:VREPLY
UID:qwerty01
REQUEST-STATUS:2.0
END:VREPLY
END:VCALENDAR

All of the DELETEs worked except the one for UID:0987654321 in TARGET:work 
which failed with a REQUEST-STATUS:4.0. (made up value for demonstation 
purposes only).  Why should the previously successful DELETEs be undeleted 
because of an unrelated failure??  It makes no sense to me.

>                         But in any case, that's an implementation
> detail - you can't assume anything about when exactly the VREPLY
> gets sent. 

No.  A CS could craft and send VREPLYs as it goes or it can craft and 
queue them for sending when its done or it could craft VREPLYs and cache 
them into a buffer and only send when the cache got full or...  We never 
mandated either way.  Does it really matter??

I would expect that a CS would send results in a manner other than "hold 
all results until the command is done." since we have bounded latency on 
commands to factor in but thats just my philosophy...

>             That's what worries me more about the protocol 
specification:
> if my CUA sends a command with multiple TARGETs, there's not telling
> whether the response will be one RPY or multiple ANS, whether any
> of them will be a complete VCALENDAR component, which of them
> contains the reponse for which TARGET, and so on - not until it
> parses everything. There are so many ways of doing the same thing!

If matching the responses to each TARGET is important to you then you'd 
probably want to use separate DELETE commands with single TARGETs in them 
(on different channels even).  Is it critical for you to tell when the 
DELETE command is done per TARGET?  Are you displaying some kind of 
progress bar for each TARGET concurrently that you want to update the bar? 
 The VREPLYs are not that big nor complex (just the UID and REQUEST-STATUS 
really) so its not that difficult to quickly parse 'em and match them to 
the proper TARGET.

If ordering by TARGET is important to you then Id say that concurrent 
DELETEs on separate channels is the way you would probably want to go.  Or 
am I missing the root of the concern here?

> Anyway, back to the discussion. You still assume it IS meaningful
> to assume commands are not atomic; I contend that the CS simply
> has no way of knowing. Compare the following:

Not quite.  I think that atomicity of the command varys with the command. 
For example, a DELETE of a VEVENT in 1 VAGENDA succeeds and a positive 
VREPLY is sent back.  Then the attempted deletion of a different VEVENT 
fails.  Why should this make the first success now a failure??

A MOVE is slightly different in that it involves essentially a CREATE and 
then a DELETE and both must succeed for a single success VREPLY to be 
sent.  Again though the command scopped to the effected component, NOT to 
the TARGET or the entire command.

>  - delete every components more than a year old on both of my VAGENDAs
> 
> BEGIN:VCALENDAR
> CMD:DELETE
> TARGET:work
> TARGET:personal
> BEGIN:VQUERY
> QUERY:SELECT * FROM VEVENT WHERE ....
> QUERY:SELECT * FROM VTODO WHERE ....
> END:VQUERY
> END:VCALENDAR

See my example at top...

>  - propose two different but related meetings, book it on my VAGENDA
> 
> BEGIN:VCALENDAR
> CMD:CREATE
> TARGET:attendee1
> TARGET:attendee2
> TARGET:attendee3
> METHOD:REQUEST
> BEGIN:VEVENT
> ...
> END:VEVENT
> BEGIN:VEVENT
> ...
> END:VEVENT
> END:VCALENDAR
> BEGIN:VCALENDAR
> CMD:CREATE
> TARGET:work
> BEGIN:VEVENT
> ...
> END:VEVENT
> BEGIN:VEVENT
> ...
> END:VEVENT
> END:VCALENDAR

Umm, Ill assume each has a separate CMDID...

> I can agree #1 doesn't need to be atomic - if I get any failure
> I can simply retry.

Thats a start toward full agreement... ;^)

> In case #2, I would actually love to be able to perform both CREATE
> as a single atomic transaction. Unfortunately, there's no way I
> can do that.

Nope.  The act of creating the 2 entries on your calendar should happen 
first.  Otherwise you've sent out invitations to something that isn't on 
your calendar and thus any responses you get back from the invitees wont 
be able to matchable to where they should go.  Once you have the entries 
on your calendar you can then send out the invitations to other folks.  No 
safe way around this problem unless your design allows you to retract sent 
messages (like unsending that nasty gram to your boss after you decide NOT 
to quit...).  You'd also need to factor in that you may not have 
sufficient rights to delete the invitations, etc (nasty thing VCARs)...

For CREATE it could easily be argued that ALL contained components MUST be 
successfully created or all fail (but it would be on a per TARGET basis!). 
 I could also see sending back a success VREPLY and a failure VREPLY for 
your example too.  We just need to reach a concensus on behaviour and 
clearly spell it out on a per command basis.  (I will strongly resist 
deletions being "All or nothing" no matter the approach used for reasons 
already given.)

> Failing that, I surely want each CREATE to be an atomic transaction.
> What if there is an error, any kind of error, on one out of three
> VAGENDAs? 

Yeah, so?  The 1st and 2nd VAGENDAs allow you write permission so you can 
CREATE the invitation.  The 3rd does not (what a weenie for not wanting me 
to access their calendar!).  So does that mean that you were unable to 
invite the first 2 people?  Certainly not.  Does that mean the entire 
CREATE command should be failed?  Nope.

>           If the CUA wants to retry it would need to either rollback
> (via DELETE), 

What are you going to DELETE?  If you could not CREATE the invitation then 
you certainly cant DELETE it can you?  Then you also have to factor in 
VCARs that allow you to only write to the VAGENDA but not remove from it. 
How you gonna deal with that?

Does failing to invite #3 mean you have to uninvite the others?  I dont 
think so.  It just means you have to try again to invite #3...

>               or change the original command to eliminate for TARGETs
> it already succeeded. 

Why??  Just because you cant invite one person does not mean that you have 
to uninvite the others does it??  I just cant make that connection in my 
mind...

>                        And in case the error is non transient and
> CU intervention is required, it needs to either remember this.

Thats outside the scope of the CAP commands though.  A good thing to 
remember for later though.

> Couple this with the effective impossibility for a CUA to know
> whether it has permission to store into a TARGET (without retrieving
> the VCARs and trying to determine that by itself, that is).

The REQUEST-STATUS for the grouch TARGET _SHOULD_ provide some indication 
like this.  I see REQUEST-STATUS:6.4 in the CAP draft matches this just 
nicely...

> Look, I can see your point, but I see no reason why unreasonably
> simplistic implementations should effectively crimple the ability
> to have elegant and efficient implementations. It's not like we don't
> have free RDBMS after all...

Some actions CANNOT be just crammed into one massive command and treated 
like a Hail Mary operation.  See the example above about first sending 
invitations without first having the entry on your own VAGENDA as a good 
example why not.

Most vendors have some kind RDBMS handy (anyone for SQL and SQL.Slammer in 
the calendar? ;^]) but not everyone does and it makes building Open Source 
or lightweight servers impossible.  The entire thread was focused on 
controlling run amok commands based on bad inputs but it had some bad 
prose on it.  Ive proposed an alternative so that we do not _mandate_ 
RDBMS in every CS.  (You got an RDBMS that fits in a Palm so it can be a 
CS??)

Bruce
===========================================================================
Bruce Kahn                                INet: 
[email-masked]
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: Doug Royer <[email-masked]>
Date: Thu, 30 Jan 2003 15:55:37 -0700

> commands to factor in but thats just my philosophy...

I agree that both are valid. They could be sent as determined or
held until the last and sent. And I agree that it does not matter.

>  >             That's what worries me more about the protocol specification:
> probably want to use separate DELETE commands with single TARGETs in 
> them (on different channels even).  Is it critical for you to tell when 
> the DELETE command is done per TARGET?  Are you displaying some kind of 
> progress bar for each TARGET concurrently that you want to update the 
> bar?  The VREPLYs are not that big nor complex (just the UID and 
> REQUEST-STATUS really) so its not that difficult to quickly parse 'em 
> and match them to the proper TARGET.

I would agree with Bruce. Maybe there is something I am missing as I
do not see that it would be difficult to match up the TARGETs to
the replies.

> be able to matchable to where they should go.

Especially if they have a BOT doing work for them. Their BOT could
reply and fail and cause all kinds of confusion.

 > ...
> 
> For CREATE it could easily be argued that ALL contained components MUST 
> be successfully created or all fail (but it would be on a per TARGET 
> basis!).  I could also see sending back a success VREPLY and a failure 
> VREPLY for your example too.  We just need to reach a concensus on 
> behaviour and clearly spell it out on a per command basis.  (I will 
> strongly resist deletions being "All or nothing" no matter the approach 
> used for reasons already given.)

I agree that could be done on a per TARGET basis. However it is possible
for one to fail and another to succeed within a single TARGET. I could
allow you CREATE/REQUEST permission and disallow you CREATE/REPLY permission.
So I think we are going the correct direction to just make each VREPLY
independent of all others. Your point of if the CUA can not handle complex
replies, then it better not send complex commands - (send one TARGET at a
time).

> Yeah, so?  The 1st and 2nd VAGENDAs allow you write permission so you 
> can CREATE the invitation.  The 3rd does not (what a weenie for not 
> wanting me to access their calendar!).  So does that mean that you were 
> unable to invite the first 2 people?  Certainly not.  Does that mean the 
> entire CREATE command should be failed?  Nope.
> 
> What are you going to DELETE?  If you could not CREATE the invitation 
> then you certainly cant DELETE it can you?  Then you also have to factor 
> in VCARs that allow you to only write to the VAGENDA but not remove from 
> it.  How you gonna deal with that?

This debate is one of the reasons we decided to defer transactions.
Any transaction implementation will have to be a draft on its own
and consider and explain all of these situations.

>  > have free RDBMS after all...

The issue to consider is that it will take more time to consider
all of the interactions and the fact that we have moved forward
with this entire list saying we are not doing transactions at
this time in CAP. We can do transactions later. Allow the simplistic
implementations to be released now (or soon), release CAP, and
we can continue to expand CAP to allow transactions (and other
things) later.

-- 

  Doug Royer                     |   http://INET-Consulting.com
  -------------------------------|-----------------------------
  [email-masked]                 | Office: (208)612-INET
  http://Royer.com/People/Doug   |    Fax: (866)594-8574
                                 |   Cell: (208)520-4044

                 We Do Standards - You Need Standards

---

From: Andrea Campi <[email-masked]>
Date: Fri, 31 Jan 2003 09:39:03 +0100

> replies, then it better not send complex commands - (send one TARGET at a

I just want to make sure we all realize avoiding transactions
doesn't avoid the complexity per se, it just moves it to the CUA.
Or, it allows you to write naive CUAs.

> and consider and explain all of these situations.
[...]
> we can continue to expand CAP to allow transactions (and other

Ok, if this is the consensus I will agree.

Then I will focus my effort on making sure nothing prevents the CS from
implementing transactions when they are added down the road. Also, I will
start working on a proposal for adding transactions later on. Ok?

Bye,
	Andrea

-- 
Andrea Campi                              mailto:[email-masked]
I.NET S.p.A. - BT Ignite                  http://www.inet.it
Technical Dept. - R&D			  phone: +39 02 32863 ext 1
v. Darwin, 85 - I-20019			  fax: +39 02 32863 ext 7705
Settimo Milanese (MI), Italy