Subject: RE: [Simple] MSRP message reliability(long)

From: [email-masked]
Date: Tue, 13 Apr 2004 11:21:14 +0300

First off, let me say that I do not like 1 at all. You need some sort of protocol level ACKs on messages. You cannot guarantee that if a TCP socket received a message that also an application on top has also received it. There could be a socket listing but the application on top has crashed.

Now, hop-by-hop vs. end-to-end:

In this scenario (hop-by-hop)

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND------>
              <--------OK
                        SEND------>
                        <--------OK

The OK does not tell the sender anything about the final delivery of the SEND. It only tells that the next hop has successfully received the SEND. Why is that important? Why is it important for A, R1 and R2 to know that the next hop has received the SEND? Why isn't TCP good enough here? That's the question that needs answering.

In this scenario (end-to-end)

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>
                        <------DSN
              <-------DSN
    <-------DSN

IMO, the relays need to keep state and a timer, and sending a DSN is mandatory. If the DSN does not arrive after x units of time, a failure DSN is generated by the last hop that forwarded the SEND. Something like

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND--X
              <-------DSN (failure)
    <-------DSN

Ben commented that "even if it (the relay) _did_ keep state, it would have no good way to know where in the message stream the error occured." My question is: is it important, say for R1, to know where the error occured (at R2, R3, R4 or B)?

Ben, can you also elaborate of this:

> If we do decide to change (back) to e2e SEND requests ... There 
> would be more impact on the relay spec.

Thanks,
Hisham

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> ext Ben Campbell
> Sent: 09.April.2004 00:14
> To: Simple WG
> Subject: [Simple] MSRP message reliability(long)
> 
> 
> Hopefully, everyone has had a chance to look at Orit's draft 
> giving an 
> analysis and some recommendations concerning the way MSRP handles 
> message acknowledgements. I responded separately with 
> specific comments 
> about that draft. However, I want to talk in general about the models 
> that have been proposed, as I understand them.
> 
> First, I would like to clarify the reliability requirements, as I see 
> them, a little. As I see it, when I send a message, there are three 
> levels of reliability that I might care about. Consider the following 
> scenarios:
> 
> 1) I really don't care at all. (Strangely, this is the mode used by a 
> lot of existing large scale systems. If we are dealing with highly 
> interactive conversations between humans, humans provide their own 
> reliability mechanism. I find myself typing "Hey, did you get my IM 
> about X a few minutes ago" frequently.)
> 
> 2) I want a good effort to deliver, and a good attempt to tell me if 
> something went wrong. If I hear nothing, I assume it most 
> likely worked.
> 
> 3) Delivery is critical. I want to be told that it worked, and if I 
> don't hear anything, I assume it most likely failed.
> 
> My personal opinion is that 1 is generally unacceptable 
> (dispite current 
> systems), 3 should be an option, but 2 will be the primary 
> use case, and 
> the one we should optimize for. The rest of this email takes that 
> perspective; if consensus proves me wrong then my analysis 
> will change.
> 
> We have a controversy about whether MSRP, in the general case where 
> relays exist, should use per-hop transaction acknowledgements, or 
> whether acknowledgments should be strictly end-to-end.
> 
> First, let us consider the success cases; that is, when everything 
> works. The per-hop case looks something like the following:
> 
>     A         R1        R2        B
>     SEND------>
>     <--------OK
>               SEND------>
>               <--------OK
>                         SEND------>
>                         <--------OK
> 
> 
> This is not-optimal for the happy case, because each device 
> has to keep 
> transaction state until it gets a response. This state is 
> _only_ useful 
> if a failure occurs. (We will look at failure cases in a minute.) 
> However, this state is not huge. As currently defined in 
> MSRP, it would 
> be the transaction identity and a timer. Adam proposed we do 
> not need a 
> mandatory timer, which would make it even smaller.
> 
> In a perfect world, the end-to-end approach would look like:
> 
>     A         R1        R2        B
>     SEND------>
>               SEND------>
>                         SEND------>
> 
> This looks great on the surface. Not only can the relays be 
> transaction 
> stateless, only half as many messages happen. But, for reasons I will 
> explain in a moment, this approach makes it impractical to tell A if 
> something goes wrong downstream. Therefore, in a realistic 
> implementation, it looks more like:
> 
>     A         R1        R2        B
>     SEND------>
>               SEND------>
>                         SEND------>
>                         <---DSN(success)
>               <-------DSN
>     <-------DSN
> 
> This has the same number of messages. It does allow the relays to be 
> stateless.
> 
> So, now, lets assume a TCP failure happens between R1 and R2.
> 
> The hop-hop case looks like the following. A gets a failure 
> notification 
> shortly after R1 notices the failure.
> 
>     A         R1        R2        B
>     SEND------>
>     <--------OK
>               SEND----X
>     <----DSN(fail)
> 
> One would hope to make the end-to-end case look similar. But, as I 
> mentioned before, that is impractical. Assume R1 has had a TCP 
> connection to R2 up for some time. R1 has sent a lot of 
> messages in that 
> time, including the one for A above. With hop-hop responses, R1 keeps 
> state around for each message it relays, and destroys that 
> state when it 
> gets an OK response. When the TCP failure occurs, R1 sends failure 
> reports for all messages for which it still has state.
> 
> But in the end-end case, R1 cannot report the failure. (We 
> said R1 would 
> keep no state for end-to-end, remember.) But even if it _did_ keep 
> state, it would have no good way to know where in the message 
> stream the 
> error occured. Unless it does something _extremely_ stateful, 
> like watch 
> for success DSNs to go by, then when a TCP error occurs, it 
> must assume 
> that every message it has every relayed on that connection 
> failed, or at 
> least every message sent for the past 9 minutes or so failed. 
> (9 minutes 
> being the normal time requried for the _application_ to learn of some 
> TCP failures.) Since 9 minutes is an eternity for a busy 
> relay, I assume 
> this is not acceptable. Rather, the relay would report 
> nothing. The flow 
> would look like the following:
> 
>     A         R1        R2        B
>     SEND------>
>               SEND---X
> 
> 
> A would eventually figure out the message failure due to the 
> lack of a 
> success report. A would have to wait some time before making this 
> decision to avoid a high likelyhood of false positives. Using the 9 
> minute failure detection time above, this time would be 
> something like 
> (9*number of hops * 2) minutes.
> 
> So, back to our scenarios. For scenario 1, the end-end 
> approach clearly 
> wins, assuming we have the option to turn off DSN entirely. 
> Orit's draft 
> suggested that, with all responses turned off, a relay would 
> have about 
> a %20 scale advantage. However, her experiment was based on SIP 
> messages, which tend to be larger, more complicated to parse, 
> and have a 
> more complex transaction model, even over TCP. I have no way 
> of knowing 
> how much the SIP vs MSRP aspect will impact things, but I think it is 
> safe to say that the savings for MSRP will be real, but less than the 
> %20 savings she observed for SIP.
> 
> For scenario 2, it is a harder call. The number of messages 
> is the same 
> for either approach. The work to do message parsing, etc is similar. 
> Relays will have to keep state for the hop-hop approach, 
> although if we 
> accept Adam's proposal about removing transaction timers, 
> this state is 
> fairly small. Also, the time it requires for the sender to 
> learn about 
> the failure is considerably shorter for the hop-hop approach than for 
> the end-end approach. Therefore, I personally think the 
> hop-hop approach 
> wins this one.
> 
> For scenario 3, an end-end receipt will be required anytime 
> relays are 
> present. This means the hop-hop approach will require more 
> messages than 
> the end-end, giving the round to the end-end.
> 
> So, considering that I believe 2 is the primary use case, I 
> lean towards 
> the hop-hop case.
> 
> I would also like to address a few arguments that I have heard, and 
> believe to be red herrings:
> 
> 1) A failure report cannot be interpreted with certainty to 
> mean that a 
> message was not delivered. There are a number of edge cases where you 
> can successfully deliver a message, but report it failed. 
> However, this 
> is _also_ true with the end-end case; as it is possible for 
> the success 
> report itself to get lost. We can build a system that never claims 
> success unless it was really successful, but we cannot build a system 
> that never claims failure unless it really failed.
> 
> 2) Since the TCP connections are used in both directions, 
> responses may 
> backup behind IMs sent in the reverse direction. This has been an 
> acknowledged problem with the use of full-duplex connections from the 
> beginning. But the problem exists for the end-end case as well, as 
> success reports can _also_ get stuck in traffic. We are 
> taking steps in 
> MSRP (chunking, possible removal of transaction timers) that I think 
> will mitigate this somewhat.
> 
> Conclusions:
> 
> I personally prefer the hop-hop approach, with DSN on failure 
> and DSN on 
> success as optional. However, if I am in the minority on that 
> point, I 
> can live with the end to end model. I do, however, want the 
> transaction 
> model to be consistent. That is, I don't want to negotiate whether we 
> send hop-hop transaction responses to SEND requests. Either 
> MSRP calls 
> for them, or it does not.
> 
> Further, the VISIT request, and I assume BIND or whatever 
> equivelant we 
> come up with in the relay work, necessarily follow a hop-hop model. 
> Making SEND end-end creates an inconsistency there.
> 
> If we do decide to change (back) to e2e SEND requests, the 
> impact on the 
> base spec is not huge; it would involve taking away the transaction 
> response to SEND, and putting in a stronger dependency on DSN. There 
> would be more impact on the relay spec.
> 
> Thanks!
> 
> Ben.
> 
> 
> _______________________________________________
> Simple mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/simple
> 

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Ben Campbell <[email-masked]>
Date: Tue, 13 Apr 2004 07:52:58 -0500

> The OK does not tell the sender anything about the final delivery of the SEND. It only tells that the next hop has successfully received the SEND. Why is that important? Why is it important for A,…

The value comes when an error occurs. Lets say R1 has had the connection 
up to R2 for some time. Then the connection fails in mid session.. 
Standard TCP interfaces give R1 know way of knowing how much data had 
been sent before the error occured. Therefore, R1 must assume that 
_nothing_ got delivered. This ties in with the DSN on error case below.

> Ben commented that "even if it (the relay) _did_ keep state, it would have no good way to know where in the message stream the error occured." My question is: is it important, say for R1, to know w…

Sorry, I was unclear. I did not mean where the error occured in space, 
but rather in time. As in my example above, R2 has no way of knowing how 
much data got sent before the error occured. So how does he know which 
messages require DSN? He must assume the answer is "all of them", or at 
least "all of them for the last 9 or so minutes", since that is the 
typical time TCP will keep retransmitting before it gives up.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 13 Apr 2004 07:52:58 -0500