Subject: [Simple] MSRP message reliability(long)

From: Ben Campbell <[email-masked]>
Date: Thu, 08 Apr 2004 16:13:30 -0500

Hopefully, everyone has had a chance to look at Orit's draft giving an 
analysis and some recommendations concerning the way MSRP handles 
message acknowledgements. I responded separately with specific comments 
about that draft. However, I want to talk in general about the models 
that have been proposed, as I understand them.

First, I would like to clarify the reliability requirements, as I see 
them, a little. As I see it, when I send a message, there are three 
levels of reliability that I might care about. Consider the following 
scenarios:

1) I really don't care at all. (Strangely, this is the mode used by a 
lot of existing large scale systems. If we are dealing with highly 
interactive conversations between humans, humans provide their own 
reliability mechanism. I find myself typing "Hey, did you get my IM 
about X a few minutes ago" frequently.)

2) I want a good effort to deliver, and a good attempt to tell me if 
something went wrong. If I hear nothing, I assume it most likely worked.

3) Delivery is critical. I want to be told that it worked, and if I 
don't hear anything, I assume it most likely failed.

My personal opinion is that 1 is generally unacceptable (dispite current 
systems), 3 should be an option, but 2 will be the primary use case, and 
the one we should optimize for. The rest of this email takes that 
perspective; if consensus proves me wrong then my analysis will change.

We have a controversy about whether MSRP, in the general case where 
relays exist, should use per-hop transaction acknowledgements, or 
whether acknowledgments should be strictly end-to-end.

First, let us consider the success cases; that is, when everything 
works. The per-hop case looks something like the following:

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND------>
              <--------OK
                        SEND------>
                        <--------OK

This is not-optimal for the happy case, because each device has to keep 
transaction state until it gets a response. This state is _only_ useful 
if a failure occurs. (We will look at failure cases in a minute.) 
However, this state is not huge. As currently defined in MSRP, it would 
be the transaction identity and a timer. Adam proposed we do not need a 
mandatory timer, which would make it even smaller.

In a perfect world, the end-to-end approach would look like:

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>

This looks great on the surface. Not only can the relays be transaction 
stateless, only half as many messages happen. But, for reasons I will 
explain in a moment, this approach makes it impractical to tell A if 
something goes wrong downstream. Therefore, in a realistic 
implementation, it looks more like:

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>
                        <---DSN(success)
              <-------DSN
    <-------DSN

This has the same number of messages. It does allow the relays to be 
stateless.

So, now, lets assume a TCP failure happens between R1 and R2.

The hop-hop case looks like the following. A gets a failure notification 
shortly after R1 notices the failure.

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND----X
    <----DSN(fail)

One would hope to make the end-to-end case look similar. But, as I 
mentioned before, that is impractical. Assume R1 has had a TCP 
connection to R2 up for some time. R1 has sent a lot of messages in that 
time, including the one for A above. With hop-hop responses, R1 keeps 
state around for each message it relays, and destroys that state when it 
gets an OK response. When the TCP failure occurs, R1 sends failure 
reports for all messages for which it still has state.

But in the end-end case, R1 cannot report the failure. (We said R1 would 
keep no state for end-to-end, remember.) But even if it _did_ keep 
state, it would have no good way to know where in the message stream the 
error occured. Unless it does something _extremely_ stateful, like watch 
for success DSNs to go by, then when a TCP error occurs, it must assume 
that every message it has every relayed on that connection failed, or at 
least every message sent for the past 9 minutes or so failed. (9 minutes 
being the normal time requried for the _application_ to learn of some 
TCP failures.) Since 9 minutes is an eternity for a busy relay, I assume 
this is not acceptable. Rather, the relay would report nothing. The flow 
would look like the following:

    A         R1        R2        B
    SEND------>
              SEND---X

A would eventually figure out the message failure due to the lack of a 
success report. A would have to wait some time before making this 
decision to avoid a high likelyhood of false positives. Using the 9 
minute failure detection time above, this time would be something like 
(9*number of hops * 2) minutes.

So, back to our scenarios. For scenario 1, the end-end approach clearly 
wins, assuming we have the option to turn off DSN entirely. Orit's draft 
suggested that, with all responses turned off, a relay would have about 
a %20 scale advantage. However, her experiment was based on SIP 
messages, which tend to be larger, more complicated to parse, and have a 
more complex transaction model, even over TCP. I have no way of knowing 
how much the SIP vs MSRP aspect will impact things, but I think it is 
safe to say that the savings for MSRP will be real, but less than the 
%20 savings she observed for SIP.

For scenario 2, it is a harder call. The number of messages is the same 
for either approach. The work to do message parsing, etc is similar. 
Relays will have to keep state for the hop-hop approach, although if we 
accept Adam's proposal about removing transaction timers, this state is 
fairly small. Also, the time it requires for the sender to learn about 
the failure is considerably shorter for the hop-hop approach than for 
the end-end approach. Therefore, I personally think the hop-hop approach 
wins this one.

For scenario 3, an end-end receipt will be required anytime relays are 
present. This means the hop-hop approach will require more messages than 
the end-end, giving the round to the end-end.

So, considering that I believe 2 is the primary use case, I lean towards 
the hop-hop case.

I would also like to address a few arguments that I have heard, and 
believe to be red herrings:

1) A failure report cannot be interpreted with certainty to mean that a 
message was not delivered. There are a number of edge cases where you 
can successfully deliver a message, but report it failed. However, this 
is _also_ true with the end-end case; as it is possible for the success 
report itself to get lost. We can build a system that never claims 
success unless it was really successful, but we cannot build a system 
that never claims failure unless it really failed.

2) Since the TCP connections are used in both directions, responses may 
backup behind IMs sent in the reverse direction. This has been an 
acknowledged problem with the use of full-duplex connections from the 
beginning. But the problem exists for the end-end case as well, as 
success reports can _also_ get stuck in traffic. We are taking steps in 
MSRP (chunking, possible removal of transaction timers) that I think 
will mitigate this somewhat.

Conclusions:

I personally prefer the hop-hop approach, with DSN on failure and DSN on 
success as optional. However, if I am in the minority on that point, I 
can live with the end to end model. I do, however, want the transaction 
model to be consistent. That is, I don't want to negotiate whether we 
send hop-hop transaction responses to SEND requests. Either MSRP calls 
for them, or it does not.

Further, the VISIT request, and I assume BIND or whatever equivelant we 
come up with in the relay work, necessarily follow a hop-hop model. 
Making SEND end-end creates an inconsistency there.

If we do decide to change (back) to e2e SEND requests, the impact on the 
base spec is not huge; it would involve taking away the transaction 
response to SEND, and putting in a stronger dependency on DSN. There 
would be more impact on the relay spec.

Thanks!

Ben.

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: "Orit Levin" <[email-masked]>
Date: Fri, 9 Apr 2004 00:03:06 -0700

Ben,
I am replying to this and some of the points from your following mail
together.

[...]

OL:

My opinion on 1 is very different from yours: we will not educate the
existing and prosperous industry what it means "acceptable" vs.
"unacceptable" service by defining a rigid protocol. The industry will
just not use such a protocol at all. The only way to define an IM
protocol that will link the IM islands together is to make it appealing
to the industry and having the ability to upgrade the service (to better
quality) without replacing all the end applications. Therefore, in my
view, 1 is MUST.

Also, I am not sure at all that most of the applications will use 2.
Those applications that will prefer to use 1 for majority of the user
messages, in parallel will use 3 with a DSN receipt for application
level keep-alive mechanism potentially per MSRP session.

Why are you opposing to a flexible hop-by-hop mechanism (in addition to
DSN)? It doesn't necessarily require negotiation - only local policy for
setting the bit.
Let's assume for a moment that a bit per message exists saying
"response/don't response" to the previous hop.

This is how I can imagine the defined interface:

Primitive "Send and Forget": This is #1 - the bit is set to FALSE
Primitive "Send with no DSN": This is #2 - the bit can be either TRUE or
FALSE
Primitive "Send with DSN on Error only": This may be redundant - the bit
can be either TRUE or FALSE
Primitive "Send with DSN on success": This is #3 (application receipt) -
the bit can be either TRUE or FALSE

Orit.

---

From: "Orit Levin" <[email-masked]>
Date: Mon, 12 Apr 2004 06:12:00 -0700

Paul,
The scenario shows a "normal" server application that periodically performs other procedures than just processing the MESSAGEs. Even in this mixed scenario, the difference in performance is substantial - 20%.
 
In order to understand the benchmark, the easiest is to take a look at the "No response" column. You can see that out of all 1838.687 outgoing messages, only 1178.341 were MESSAGEs (i.e. the funning of the 49 MESSAGEs as a result of a single "trigger" MESSAGE being sent to the server) . The "trigger MESSAGEs" are hidden in the 660.389 of other SIP messages (e.g. REGISTER, INVITE, SUBSCRIBE, INFO, NOTIFY, ACK, etc.) being received and processed by the server that don't necessarily belong to the IM application itself. You can also see that the server sent  660.346 (1838.687 - 1178.341 ) - messages as a part of this "other" logic. 
 
It means that the server was busy with IM processing only ~ 47%  1178/(660 +1838) of its whole processing time. This statistic may represent some kind of an enterprise pattern...
 
According to the MSN data (typical public network), their deployment statistics shows that today ~10% CPU go to "session establishment" and ~90% CPU go to processing of IM data. If we apply this pattern to our MESSAGE benchmark, the results would be MUCH worse!
 
Hope, it helps,
Orit.

________________________________

From: [email-masked] on behalf of Paul Kyzivat
Sent: Fri 4/9/2004 12:29 PM
To: Ben Campbell
Cc: Simple WG
Subject: Re: [Simple] MSRP message reliability(long)

Sorry I have been absent - I have been preoccupied with other (work) things.

I took a quick look at Orit's draft. Before getting into the main
discussion, I don't think I understand the benchmark.

As I understand, the server is in the middle of all the sessions between
the 20000 endpoints. At any point in time there are a bunch of im
conferences, each with 50 participants. For each conference, one of the
participants is sending MESSAGE messages to the server, and the server
is sending 49 MESSAGE messages out to the other participants in that
conference.

If so, in the normal run, for each incoming MESSAGE into the conference,
the server will send one response, and will receive 49 responses. I
can't reconcile that with the data in the draft, which shows a ratio of
1:4 for incoming requests to incoming responses.

I don't want to use this data to draw conclusions about MSRP until I
understand if/how it applies.

Orit - am I just misinterpretting the data?

        Thanks,
        Paul

---

From: "Orit Levin" <[email-masked]>
Date: Mon, 12 Apr 2004 06:12:00 -0700

Paul,

 
 
Hope, it helps,
Orit.

        Thanks,
        Paul

---

From: [email-masked]
Date: Tue, 13 Apr 2004 11:21:14 +0300

First off, let me say that I do not like 1 at all. You need some sort of protocol level ACKs on messages. You cannot guarantee that if a TCP socket received a message that also an application on top has also received it. There could be a socket listing but the application on top has crashed.

Now, hop-by-hop vs. end-to-end:

In this scenario (hop-by-hop)

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND------>
              <--------OK
                        SEND------>
                        <--------OK

The OK does not tell the sender anything about the final delivery of the SEND. It only tells that the next hop has successfully received the SEND. Why is that important? Why is it important for A, R1 and R2 to know that the next hop has received the SEND? Why isn't TCP good enough here? That's the question that needs answering.

In this scenario (end-to-end)

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>
                        <------DSN
              <-------DSN
    <-------DSN

IMO, the relays need to keep state and a timer, and sending a DSN is mandatory. If the DSN does not arrive after x units of time, a failure DSN is generated by the last hop that forwarded the SEND. Something like

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND--X
              <-------DSN (failure)
    <-------DSN

Ben commented that "even if it (the relay) _did_ keep state, it would have no good way to know where in the message stream the error occured." My question is: is it important, say for R1, to know where the error occured (at R2, R3, R4 or B)?

Ben, can you also elaborate of this:

> If we do decide to change (back) to e2e SEND requests ... There 

Thanks,
Hisham

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> ext Ben Campbell
> Sent: 09.April.2004 00:14
> To: Simple WG
> Subject: [Simple] MSRP message reliability(long)
> 
>

---

From: [email-masked]
Date: Tue, 13 Apr 2004 16:16:33 +0300

Ok, I'm sold with the hop-by-hop solution. But for completeness sake, can you comment on my last question (I think you missed it). Here it is again:

> > Ben, can you also elaborate on this:
> > 
> > 

Thanks,
Hisham

> From: ext Ben Campbell [mailto:[email-masked]]
> Sent: 13.April.2004 15:53
> To: Khartabil Hisham (Nokia-TP-MSW/Helsinki)
> Cc: [email-masked]
> [email-masked] wrote:
> 
> > First off, let me say that I do not like 1 at all. You need 
> some sort of protocol level ACKs on messages. You cannot 
> guarantee that if a TCP socket received a message that also 
> an application on top has also received it. There could be a 
> socket listing but the application on top has crashed.
> > 
> > The OK does not tell the sender anything about the final 
> delivery of the SEND. It only tells that the next hop has 
> successfully received the SEND. Why is that important? Why is 
> it important for A, R1 and R2 to know that the next hop has 
> received the SEND? Why isn't TCP good enough here? That's the 
> question that needs answering.
> 
> The value comes when an error occurs. Lets say R1 has had the 
> connection 
> up to R2 for some time. Then the connection fails in mid session.. 
> Standard TCP interfaces give R1 know way of knowing how much data had 
> been sent before the error occured. Therefore, R1 must assume that 
> _nothing_ got delivered. This ties in with the DSN on error 
> case below.
> 
> > 
> > 
> > IMO, the relays need to keep state and a timer, and sending 
> a DSN is mandatory. If the DSN does not arrive after x units 
> of time, a failure DSN is generated by the last hop that 
> forwarded the SEND. Something like
> > 
> >     A         R1        R2        B
> >     SEND------>
> >               SEND------>
> >                         SEND--X
> >               <-------DSN (failure)
> >     <-------DSN
> > 
> > Ben commented that "even if it (the relay) _did_ keep 
> state, it would have no good way to know where in the message 
> stream the error occured." My question is: is it important, 
> say for R1, to know where the error occured (at R2, R3, R4 or B)?
> >
> 
> Sorry, I was unclear. I did not mean where the error occured 
> in space, 
> but rather in time. As in my example above, R2 has no way of 
> knowing how 
> much data got sent before the error occured. So how does he 
> know which 
> messages require DSN? He must assume the answer is "all of 
> them", or at 
> least "all of them for the last 9 or so minutes", since that is the 
> typical time TCP will keep retransmitting before it gives up.
> 
> 
> >>From: [email-masked] 
> [mailto:[email-masked]]On Behalf Of
> >>ext Ben Campbell

---

From: "Orit Levin" <[email-masked]>
Date: Tue, 20 Apr 2004 17:38:12 -0700

Hi Vicas,
I understand that you don't see a problem in unconditionally sending a
(hop-by-hop) response to every message (regardless the end-to-end mode
being used). If my understanding is correct, how are you planning to
address the wireless bandwidth problem in this case?

Thanks,
Orit.

-----Original Message-----
From: Vikas Tandon [mailto:[email-masked]] 
Sent: Friday, April 09, 2004 10:47 PM
To: 'Ben Campbell'
Cc: Orit Levin; 'Simple WG'
Subject: RE: [Simple] MSRP message reliability(long)

Hi Ben & Group,
By "this" i mean the earlier messages exchanged between me, Mr. Vijay
Gurbani, Mr. Hisham  and a couple of more participants. Yes i am
referring to the delivery reports.   
Regards,
-Vikas

-----Original Message-----
From: Ben Campbell [mailto:[email-masked]]
Sent: vendredi 9 avril 2004 20:10
To: Vikas Tandon
Cc: 'Orit Levin'; 'Simple WG'
Subject: Re: [Simple] MSRP message reliability(long)

Vikas Tandon wrote:

> Also as discussed earlier in this forum, this adds a lot of traffic to

> the client (specially the wireless mobile clients where the bandwidth 
> is an issue). If the workgroup plans to make it configurable to set 
> the delivery response for positive case off, then it has to be 
> terminated earlier than reaching the wireless medium. E.g. terminate 
> the positive response at the IM server in case the client has chosen 
> not to receive the positive acknowledgements of delivery of IM.

Hi, could you clarify what "this" refers to in the previous paragraph? 
Keep in mind that we are really discussion two things: the transaction 
responses between hops, and the sending of delivery reports. It sounds 
to me like you are talking delivery reports, but I am not certain.

> 
> If we fail to do this, imagine for every message I'm sending, there is

> some traffic I'm getting back to my mobile device (which is painful 
> because i may be paying by the amount of traffic i transact from my 
> device). So although it becomes transparent to me from device, by i 
> still pay for it. The workgroup needs to consider this matter. I 
> somehow still prefer - "No news is good news".
> 
> Would be good to know more thoughts on this.
> 
> Regards,
> Vikas Tandon
> 
> From: [email-masked] [mailto:[email-masked]] On Behalf 
> Of Ben Campbell
> Sent: vendredi 9 avril 2004 19:45
> To: Orit Levin
> Cc: Simple WG
> Orit Levin wrote:
>>Why are you opposing to a flexible hop-by-hop mechanism (in addition
>>to DSN)? It doesn't necessarily require negotiation - only local 
>>policy for setting the bit. Let's assume for a moment that a bit per 
>>message exists saying "response/don't response" to the previous hop.
> 
> 
> I oppose the ability to select whether you use transaction responses 
> or
> not because it adds a huge amount of complexity to the protocol and
any 
> implementations. Adding or removing transaction responses _completely_

> changes the protocol semantics. In effect, it would be saying we have 
> two different required protocols with the same syntax on top. If we
make
> 
> this selectable, we make every implementer build 2 protocols with
> different state machines, etc. I think that will be _much_ more of an 
> impediment to acceptance than performance differences in the range you

> predict.
> 
> My point is, if the working group were to decide we need to be able to
> run without transaction responses (a big if) , then we should _always_

> run without transaction responses. As it is, earlier attempts at a 
> message sessions did not have the transaction response, and the
working 
> group decided to put them it. (Does anyone remember the specific 
> argument used at that time?)
> 
> But, on all of these points, working group consensus trumps either of
> our opinions. Please, anyone who cares one way or another, state your 
> opinion asap!
>

---

From: "Orit Levin" <[email-masked]>
Date: Tue, 20 Apr 2004 17:38:12 -0700

Hi Vicas,

Thanks,
Orit.

-----Original Message-----
From: Vikas Tandon [mailto:[email-masked]] 
Sent: Friday, April 09, 2004 10:47 PM
To: 'Ben Campbell'
Cc: Orit Levin; 'Simple WG'
Subject: RE: [Simple] MSRP message reliability(long)

Hi Ben & Group,

-----Original Message-----
From: Ben Campbell [mailto:[email-masked]]
Sent: vendredi 9 avril 2004 20:10
To: Vikas Tandon
Cc: 'Orit Levin'; 'Simple WG'
Subject: Re: [Simple] MSRP message reliability(long)

any 

make

working

---

From: [email-masked]
Date: Wed, 21 Apr 2004 09:48:03 +0300

Hi Orit,

One message (the 200 ok for SEND) is hardly a problem to the air interface, which is between the terminal and the first relay only. Can you elaborate on why it is a problem?

Regards,
Hisham

> ext Orit Levin
> Sent: 21.April.2004 03:38
> To: Vikas Tandon
> Cc: Simple WG; Ben Campbell

---

From: [email-masked]
Date: Wed, 21 Apr 2004 09:48:03 +0300

Hi Orit,

Regards,
Hisham

---

From: "Orit Levin" <[email-masked]>
Date: Wed, 21 Apr 2004 12:18:40 -0700

Hisham,
My guess is that "200 OK" vs. minimum positive DSNs will have a similar
impact on the bandwidth between a phone and the first relay/switch. Of
course, using both simultaneously will have a double impact.

1. You can not use "200 OK" information for building (even somewhat)
reliable IM applications. These applications need to use DSN mechanism
instead (regardless the physical interface beneath).

2. The "200 OK" model is inapplicable for scalable IM conferencing
(please, see the draft for details).

3. Looking at a bigger picture (not the access interface only), core
network/service providers will have a MUCH BIGGER problem with "200 OKs"
compared to positive DSNs because of performance issues.

To summarize, by applying the "200 OK" model for EVERY message above (a
chain of) TCP connection(s), you basically loose the main motivation for
the "sessions mode" to start with. Although it is lighter than SIP and
MESSAGE page mode, it is still not as efficient as required compared to
existing commercial implementations. 

Orit. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, April 20, 2004 11:48 PM
To: Orit Levin; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: [Simple] MSRP message reliability(long)

Hi Orit,

One message (the 200 ok for SEND) is hardly a problem to the air
interface, which is between the terminal and the first relay only. Can
you elaborate on why it is a problem?

Regards,
Hisham

---

From: "Orit Levin" <[email-masked]>
Date: Wed, 21 Apr 2004 12:18:40 -0700

Hisham,

Orit. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, April 20, 2004 11:48 PM
To: Orit Levin; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: [Simple] MSRP message reliability(long)

Hi Orit,

Regards,
Hisham