Subject: [Simple] MSRP message reliability(long)

From: Ben Campbell <[email-masked]>
Date: Thu, 08 Apr 2004 16:13:30 -0500

Hopefully, everyone has had a chance to look at Orit's draft giving an 
analysis and some recommendations concerning the way MSRP handles 
message acknowledgements. I responded separately with specific comments 
about that draft. However, I want to talk in general about the models 
that have been proposed, as I understand them.

First, I would like to clarify the reliability requirements, as I see 
them, a little. As I see it, when I send a message, there are three 
levels of reliability that I might care about. Consider the following 
scenarios:

1) I really don't care at all. (Strangely, this is the mode used by a 
lot of existing large scale systems. If we are dealing with highly 
interactive conversations between humans, humans provide their own 
reliability mechanism. I find myself typing "Hey, did you get my IM 
about X a few minutes ago" frequently.)

2) I want a good effort to deliver, and a good attempt to tell me if 
something went wrong. If I hear nothing, I assume it most likely worked.

3) Delivery is critical. I want to be told that it worked, and if I 
don't hear anything, I assume it most likely failed.

My personal opinion is that 1 is generally unacceptable (dispite current 
systems), 3 should be an option, but 2 will be the primary use case, and 
the one we should optimize for. The rest of this email takes that 
perspective; if consensus proves me wrong then my analysis will change.

We have a controversy about whether MSRP, in the general case where 
relays exist, should use per-hop transaction acknowledgements, or 
whether acknowledgments should be strictly end-to-end.

First, let us consider the success cases; that is, when everything 
works. The per-hop case looks something like the following:

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND------>
              <--------OK
                        SEND------>
                        <--------OK

This is not-optimal for the happy case, because each device has to keep 
transaction state until it gets a response. This state is _only_ useful 
if a failure occurs. (We will look at failure cases in a minute.) 
However, this state is not huge. As currently defined in MSRP, it would 
be the transaction identity and a timer. Adam proposed we do not need a 
mandatory timer, which would make it even smaller.

In a perfect world, the end-to-end approach would look like:

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>

This looks great on the surface. Not only can the relays be transaction 
stateless, only half as many messages happen. But, for reasons I will 
explain in a moment, this approach makes it impractical to tell A if 
something goes wrong downstream. Therefore, in a realistic 
implementation, it looks more like:

    A         R1        R2        B
    SEND------>
              SEND------>
                        SEND------>
                        <---DSN(success)
              <-------DSN
    <-------DSN

This has the same number of messages. It does allow the relays to be 
stateless.

So, now, lets assume a TCP failure happens between R1 and R2.

The hop-hop case looks like the following. A gets a failure notification 
shortly after R1 notices the failure.

    A         R1        R2        B
    SEND------>
    <--------OK
              SEND----X
    <----DSN(fail)

One would hope to make the end-to-end case look similar. But, as I 
mentioned before, that is impractical. Assume R1 has had a TCP 
connection to R2 up for some time. R1 has sent a lot of messages in that 
time, including the one for A above. With hop-hop responses, R1 keeps 
state around for each message it relays, and destroys that state when it 
gets an OK response. When the TCP failure occurs, R1 sends failure 
reports for all messages for which it still has state.

But in the end-end case, R1 cannot report the failure. (We said R1 would 
keep no state for end-to-end, remember.) But even if it _did_ keep 
state, it would have no good way to know where in the message stream the 
error occured. Unless it does something _extremely_ stateful, like watch 
for success DSNs to go by, then when a TCP error occurs, it must assume 
that every message it has every relayed on that connection failed, or at 
least every message sent for the past 9 minutes or so failed. (9 minutes 
being the normal time requried for the _application_ to learn of some 
TCP failures.) Since 9 minutes is an eternity for a busy relay, I assume 
this is not acceptable. Rather, the relay would report nothing. The flow 
would look like the following:

    A         R1        R2        B
    SEND------>
              SEND---X

A would eventually figure out the message failure due to the lack of a 
success report. A would have to wait some time before making this 
decision to avoid a high likelyhood of false positives. Using the 9 
minute failure detection time above, this time would be something like 
(9*number of hops * 2) minutes.

So, back to our scenarios. For scenario 1, the end-end approach clearly 
wins, assuming we have the option to turn off DSN entirely. Orit's draft 
suggested that, with all responses turned off, a relay would have about 
a %20 scale advantage. However, her experiment was based on SIP 
messages, which tend to be larger, more complicated to parse, and have a 
more complex transaction model, even over TCP. I have no way of knowing 
how much the SIP vs MSRP aspect will impact things, but I think it is 
safe to say that the savings for MSRP will be real, but less than the 
%20 savings she observed for SIP.

For scenario 2, it is a harder call. The number of messages is the same 
for either approach. The work to do message parsing, etc is similar. 
Relays will have to keep state for the hop-hop approach, although if we 
accept Adam's proposal about removing transaction timers, this state is 
fairly small. Also, the time it requires for the sender to learn about 
the failure is considerably shorter for the hop-hop approach than for 
the end-end approach. Therefore, I personally think the hop-hop approach 
wins this one.

For scenario 3, an end-end receipt will be required anytime relays are 
present. This means the hop-hop approach will require more messages than 
the end-end, giving the round to the end-end.

So, considering that I believe 2 is the primary use case, I lean towards 
the hop-hop case.

I would also like to address a few arguments that I have heard, and 
believe to be red herrings:

1) A failure report cannot be interpreted with certainty to mean that a 
message was not delivered. There are a number of edge cases where you 
can successfully deliver a message, but report it failed. However, this 
is _also_ true with the end-end case; as it is possible for the success 
report itself to get lost. We can build a system that never claims 
success unless it was really successful, but we cannot build a system 
that never claims failure unless it really failed.

2) Since the TCP connections are used in both directions, responses may 
backup behind IMs sent in the reverse direction. This has been an 
acknowledged problem with the use of full-duplex connections from the 
beginning. But the problem exists for the end-end case as well, as 
success reports can _also_ get stuck in traffic. We are taking steps in 
MSRP (chunking, possible removal of transaction timers) that I think 
will mitigate this somewhat.

Conclusions:

I personally prefer the hop-hop approach, with DSN on failure and DSN on 
success as optional. However, if I am in the minority on that point, I 
can live with the end to end model. I do, however, want the transaction 
model to be consistent. That is, I don't want to negotiate whether we 
send hop-hop transaction responses to SEND requests. Either MSRP calls 
for them, or it does not.

Further, the VISIT request, and I assume BIND or whatever equivelant we 
come up with in the relay work, necessarily follow a hop-hop model. 
Making SEND end-end creates an inconsistency there.

If we do decide to change (back) to e2e SEND requests, the impact on the 
base spec is not huge; it would involve taking away the transaction 
response to SEND, and putting in a stronger dependency on DSN. There 
would be more impact on the relay spec.

Thanks!

Ben.

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 09 Apr 2004 15:29:00 -0400

Sorry I have been absent - I have been preoccupied with other (work) things.

I took a quick look at Orit's draft. Before getting into the main 
discussion, I don't think I understand the benchmark.

As I understand, the server is in the middle of all the sessions between 
the 20000 endpoints. At any point in time there are a bunch of im 
conferences, each with 50 participants. For each conference, one of the 
participants is sending MESSAGE messages to the server, and the server 
is sending 49 MESSAGE messages out to the other participants in that 
conference.

If so, in the normal run, for each incoming MESSAGE into the conference, 
the server will send one response, and will receive 49 responses. I 
can't reconcile that with the data in the draft, which shows a ratio of 
1:4 for incoming requests to incoming responses.

I don't want to use this data to draw conclusions about MSRP until I 
understand if/how it applies.

Orit - am I just misinterpretting the data?

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 09 Apr 2004 15:29:00 -0400

Thanks,
	Paul