Subject: Re: [Simple] FW: I-D ACTION:draft-levin-simple-msrp-review-00.txt

From: Ben Campbell <[email-masked]>
Date: Thu, 08 Apr 2004 14:49:13 -0500

Hi, thanks for sending this. I do have some comments and questions. This 
email is strictly in response to your draft. I will send general 
comments about MSRP with and without transaction responses in a separate 
email.

The big ones first:

  You make a number of assertions of normative behavior for relays. 
Considering that we do not have an MSRP relay draft yet, these 
assertions are a bit overstated. The recent discussions, both on and off 
the list, involved making some assumtions about how relays work, based 
on the SIMS draft. The reason for these assumptions was to determine 
what changes needed to be made to the MSRP base spec to minimally enable 
relay integration.

Second, you use a SIP transaction model to prove the impact of 
transaction responses. I need to think that through better, but my first 
inclination is to say that it is not a good comparison to MSRP. This is 
because the SIP transaction model is much more complex and timing 
sensitive than anything currently proposed for MSRP. Further, the MSRP 
syntax is (hopefully) lighter weight, and easier to parse. Therefore, I 
think that, if you had done this test using an MSRP device, and kept the 
other factors the same, the difference in performance between running 
with and without responses would be smaller than that for a SIP device.

Further, if I understand your data correctly, with the SIP device you 
see a difference in performance of around %20. Is this correct? If so, I 
do not find that to be highly convincing by itself. If we were talking 
about order-of-magnitude differences, I would be more convinced. Of 
course, if we end up accepting your other assertions that the 
transaction response brings no value, then a %20 change might be worthwhile.

And now, some specific comments:

>   The hop-by-hop mechanism (sometimes referred to as the "ACCEPT"
>    mechanism) is modeled after the SIP transaction model. 

We used the term "ACCEPT" in off line discussions as an abstraction to 
talk about transaction responses. I think in this draft, it would be 
less confusing to use the terminology from the MSRP draft. Perhaps 
"transaction response" would be more clear.

> (Negative responses are also defined but they are applicable only for
>    deployments with no relays in the network.)

There has not been any agreement to that effect that I am aware of. We 
had a side conversation that we might want to get rid of negative 
transaction responses for relays, but no wg consensus. I personally do 
not like the idea of having a different set of allowed transaction 
responses depending on whether relays are present.

> The negative "end-to-end" mechanism is mandatory for relays for error
>    reporting when connectivity problem is detected. In this context the
>    mechanism is interchangeably referred to as "DSR" or "DSN". Relays
>    MUST NOT use the end-to end mechanism for reporting positive
>    acknowledgements.

DSN would not necessarily be end-to-end, since, as you mention, relays 
might generate them. Also, I accidentally introduced the acronym DSR due 
to some email typos. I think we can drop it.

>  When an MSRP entity A (a user agent or a relay) sends/forwards a
>    message, it creates the message transaction state and starts a timer
>    (~30 sec). 

Adam suggested that we don't really need a mandatory timer here. He 
proposed that we say that a device MAY create a timer, and the value 
would be local policy. If a device(endpoint or relay) does _not_ create 
a timer, it would merely keep a record of the transaction until it got a 
transaction response, at which time it could completely forget about it. 
If a TCP error is encounted, it would then need to either send DSN(s) 
for any outstanding transactions (if a relay), or inform the user (if 
and endpoint.) I believe that removing the mandatory timer will make it 
possible for a relay to greatly reduce the transaction overhead. The 
only advantage in keeping such a timer is it could allow a device to 
detect a problem faster than TCP will by itself. This would come at the 
expense of more false failure notifications.

> If there is an
>    immediate problem in the local handling of this message by B, the
>    MSRP entity (B) MUST generate an error report and send it all the way
>    back to the originator (i.e. the original source) of the message
>    using the end-to-end mechanism described below.

I don't think we have consensus on that point. This goes back to the 
point about negative transaction responses. If the failure is immediate, 
B could have just sent a negative transaction response. The DSN is 
needed for failures that require too much time to detect to block the 
sending of the transaction response.

> In both cases, the
>    error reports must include a list the messages per originator that
>    haven’t been acknowledged by the next hop.

We still need to do work to determine whether a DSN can include a list 
of failed messages, or we have a separate DSN per message.

>    o  In addition, if as a result of the SDP negotiation, a user agent
>       agreed to provide an application layer positive and/or negative
>       report/receipt per message, the user agent MUST create the report
>       and send it back to the message originator in the same manner as
>       described above (i.e. inside the existing MSRP connection)

I am confused by this paragraph. The section so far talked about how you 
send DSN. This paragraph seems to say "And if you agreed to do so, you 
also have to send DSN. How is this not redundant?"

>    The following known issues still need to be addressed by the MSRP
>    specification:
> 
>    o  Relationship between the MSRP end-to-end procedures and the
>       "application above the user agent" end-to-end procedures

Our current expectation is the primary use of DSN would be to present 
the information to the user, who then does whatever he or she chooses. I 
expect it to be machine parseable, but do not expect to define any 
normative behavior from the application beyond telling the user.

>    o  The exact end-to-end operation modes and their negotiation using
>       the offer-answer SDP mechanism

That work is currently in progress. (In fact, Chris Boulon just proposed 
some text to me on DSN in general--Yeah Chris!!)

In the past, though, you have suggested per-message negotiation, rather 
than SDP based, i.e. per-session, negotiation. Are you saying you would 
now prefer per-session?

>    o  The exact format of the reports (both negative and positive)

In progress.

>    o  A mechanism needs to be defined so that all outstanding end-to-end
>       reports (i.e. belonging to sessions ended earlier by user agents)
>       are silently discarded by the system

The next MSRP draft will call out an open issue on whether DSN can be 
delivered after a session ends.

>    o  A mechanism needs to be defined so that NO end-to-end status
>       reports will be generated about delivery of earlier end-to-end
>       status reports

In progress.
>    o  It is unclear why the status report messages need to be
>       acknowledged hop-by-hop (as per current design) because they MUST
>       not be included in the status reports (as stated above)

It is a matter of balancing the complexity of having two different 
transaction models with the cost of having a transaction model that 
works well for the primary usage (sending IMs) but is less optimal for a 
secondary usage (sending DSN, or other status reports.)

Scalability analysis:

>    The most trivial consequence of the defined "hop-by-hop" mechanism is
>    the need to keep per message a state object (including a timer) and
>    to clear it upon receiving an acknowledgement for the message.
> 

As I mentioned above, we have a proposal to remove the requirement to 
keep a timer. This should have a significant affect on the overall impact.

> In order to fully realize the impact of this design on an IM system,
>    we need to translate it to numbers. For example, as of today, a
>    typical public IM service would have 150MM subscribed users with a
>    peak of 10MM simultaneous users.
> 
>    Consequently, in order to compare the cost of such IM solution to
>    "less statefull" IM architectures, every additional bit of memory and
>    CPU processing power required for keeping the state per "message
>    transaction" need to be multiplied at least (since message
>    transactions can be overlapped) by 10MM.

I'm confused. Are you saying that, if I have 10 million (I assume you 
mean MM to represent a million.) concurrently logged in users, I have 10 
million MSRP transactions in play at any moment? I would assume not all 
active users are in a session at any moment, and not all that _are_ in a 
session have an open transaction at any moment. Am I misunderastanding 
your point?

>    It is interesting to mention that even modern TCP implementations
>    have moved away from acknowledging every TCP segment and today
>    acknowledge every second segment only…
> 

While I don't find TCP's ack rate to be particularly relevent to the 
discussion, I believe that most implementations send _"at least that 
often", which is very different than  "no more often than".

> One of the concerns with the suggested hop-by-hop mechanism is that
>    although the mechanism is very simple, it suffers from some of the
>    known TCP problems but, unlike TCP, doesn’t address them.
>    Unfortunately, the current working assumption that the
>    "congestion-safe" TCP will automatically make the MSRP
>    congestion-safe is inaccurate.

I think the point of congestion-safety, is the presence of MSRP on the 
network should not interfere with the use of _other_ applications. It is 
not the same as saying an MSRP relay cannot itself get congested.

>   Because each message will be acknowledged (by Response) by B
>    automatically, A needs to control the rate of the messages it sends
>    to B in order to be able to process the incoming responses. In order
>    to do so, A buffers messages before it sends them to B.  In the same
>    time B sends its data messages (i.e. new Requests) asynchronously on
>    the same connection from B to A. As a result, the connection gets
>    throttled by new Requests while the old Requests (from A to B)
>    haven't been acknowledged yet. Provided A and B do not use common
>    throttling mechanism and/or have different processing power this
>    situation requires careful tuning (e.g. configuration or negotiation)
>    by each side.

It seems to me that using chunking to cut down on message size, and the 
removal of the transaction timer will help this issue quite a lot.

Functional Overview:

>    Detection of TCP connection failure earlier than reported by TCP
>    itself:
> 
>    o  This can be useful if TCP connection shows a property of
>       unreasonably long (e.g. up to 9 minutes) TCP retry procedures. The
>       reason for this pattern would be either improperly tuned TCP
>       configuration or/and transport layer intermediates (potentially
>       deployed for security or monitoring) that actively intervene the
>       TCP connection and, as a result, change its properties

The 9 minute figure you have heard Adam and I throw out is _not_ the TCP 
  retransmission timer value . Rather, it is roughly the total time that 
a TCP stack using default configurations is likely to continue 
retransmission/backoff cycles before it gives up and tells the 
application it has a problem. This is _not_ due to misconfiguration; 
rather, it is the behavior using the _default_ configuration. Not all 
implementations let you change this at all. I think Windows does, but at 
a whole interface scope.

>    o  Unfortunately, using the MSRP timer with shorter than TCP timeout
>       will increase the probability of message duplications as described
>       in the next section

See previous comments on the transaction timer.

> o  It’s possible that an MSRP timer of the relay has expired or a
>       local TCP connection failure has been detected by the relay when
>       there are still locally unprocessed positive acknowledgments from
>       the next hop in the TCP pipe. These acknowledgements can be to the
>       messages that had been already forwarded by the next hop and
>       eventually got delivered to the destination

Yes, I acknowledge this as true. (It is true of just about any protocol 
I have ever seen.)

>    o  Note that as shorter the local MSRP timer is, as more frequently
>       the scenario above can occur
>    o  Note also that the scenario above can theoretically happen to any
>       or all of the relays for the same message. This will result in a
>       flood of error reports towards the message originator even in the
>       case of successful message delivery to its final destination

Theoretically, yes. But in practice, rarely, as having multiple hops 
along the way _each_ decide that a transaction had failed when it in 
fact had succeeded is very unlikely.

MSRP Abstraction:

>    As shown above, the defined hop-by-hop mechanism is incapable of
>    addressing most of the basic IM applications requirements when more
>    than a single hop is involved.
> 

You showed _one_ potential requirement it does not address by itself. 
That is hardly "most".

>    As a result of the two statements above, this paper argues that the
>    end-to-end mechanism MUST be specified in the basic MSRP
>    specification:
> 

I believe we have already agreed that the base spec will describe 
endpoint processing of DSN, to the degree that a non-relay aware 
endpoint needs to understand to interwork with an endpoint that uses relays.

> 
> A New Internet-Draft is available from the on-line Internet-Drafts 
> directories.
> 
> 
> Title : Review of MSRP Delivery Mechanisms
> Author(s) : O. Levin
> Filename : draft-levin-simple-msrp-review-00.txt
> Pages : 16
> Date : 2004-4-6
>  
> This paper shows that the currently defined per-message hop-by-hop
>    mechanism doesn’t scale for large deployments and is not feasible for
>    scalable IM conferencing. Instead, the paper proposes to replace this
>    mechanism with an MSRP layer "per TCP connection keep-alive
>    mechanism". This paper also reinforces the need for the end-to-end
>    mechanism to be defined in the core MSRP specification and to become
>    the abstraction for all MSRP applications regardless the underlying
>    topology in use.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-levin-simple-msrp-review-00.txt
> 

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Ben Campbell <[email-masked]>
Date: Thu, 08 Apr 2004 14:49:13 -0500

The big ones first:

In progress.

In progress.