Subject: [Simple] FW: I-D ACTION:draft-levin-simple-msrp-review-00.txt

From: "Orit Levin" <[email-masked]>
Date: Wed, 7 Apr 2004 16:07:58 -0700

Hi!
This is the analysis that I promised to prepare during the last SIMPLE
meeting.
 
The proposed application TCP keep-alive mechanism may not be the only
way to resolve the problem, but my strong feeling is that the
scalability problem MUST be resolved in a reasonable way by not
requiring ACCEPT for each MSRP message and on each UA and each relay.
 
Thanks,
Orit.
 
------------------------------------------------------------------------
----------------------------------------------
A New Internet-Draft is available from the on-line Internet-Drafts
directories.

Title : Review of MSRP Delivery Mechanisms
Author(s) : O. Levin
Filename : draft-levin-simple-msrp-review-00.txt
Pages : 16
Date : 2004-4-6
 
This paper shows that the currently defined per-message hop-by-hop
   mechanism doesn't scale for large deployments and is not feasible for
   scalable IM conferencing. Instead, the paper proposes to replace this
   mechanism with an MSRP layer "per TCP connection keep-alive
   mechanism". This paper also reinforces the need for the end-to-end
   mechanism to be defined in the core MSRP specification and to become
   the abstraction for all MSRP applications regardless the underlying
   topology in use.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-levin-simple-msrp-review-00.tx
t

---

From: "Orit Levin" <[email-masked]>
Date: Wed, 7 Apr 2004 16:07:58 -0700

Hi!

Thanks,
Orit.

---

From: "Orit Levin" <[email-masked]>
Date: Thu, 8 Apr 2004 18:48:42 -0700

Ben,
Thank you very much for commenting on the draft in details.
I hope that the whole joined effort will help to keep the MSRP efforts
focused and a successful protocol being defined and widely used.

First, the two main points:

1. "I believe we have already agreed that the base spec will describe
endpoint processing of DSN, to the degree that a non-relay aware
endpoint needs to understand to interwork with an endpoint that uses
relays."
Excellent! Do we also agree that DSN contains at least the two
primitives: "send-and-forget" and "send-with-receipt: negative or
positive". Optionally also "send-negative-receipt-if possible", but I am
less sure about this one.

2. As we discussed in our previous conversations, just making the timer
optional will still require each entity to send the response per message
(50% gain only). Also, this way we blindly apply the loose local policy
to all the MSRP messages in the route. As we discussed earlier, a more
flexible solution would be marking the messages with "respond"/"don't
respond" bit being set by end-users and inspected by the relays/user
agents as they forward the message anyway (to the next hop or to the
applications). The other option would be the strict "single-hop"
mechanism per TCP connection being proposed in the draft. (BTW The
suggested value of the timer ~10 sec is a very poor choice (my fault)
and the value will depend on topology requirements.)

I am leaving for a 10 days vacation this evening, so I apologize for the
brief reply inline below...
Orit.

-----Original Message-----
From: Ben Campbell [mailto:[email-masked]] 
Sent: Thursday, April 08, 2004 12:49 PM
To: Orit Levin
Cc: Simple WG
Subject: Re: [Simple] FW: I-D
ACTION:draft-levin-simple-msrp-review-00.txt

Hi, thanks for sending this. I do have some comments and questions. This
email is strictly in response to your draft. I will send general
comments about MSRP with and without transaction responses in a separate
email.

The big ones first:

  You make a number of assertions of normative behavior for relays. 
Considering that we do not have an MSRP relay draft yet, these
assertions are a bit overstated. The recent discussions, both on and off
the list, involved making some assumtions about how relays work, based
on the SIMS draft. The reason for these assumptions was to determine
what changes needed to be made to the MSRP base spec to minimally enable
relay integration.

OL: True. I summarized the current assumptions because I haven't seen
them summarized anywhere else (beyond  separate e-mails) to explain
where my concerns and suggestions are coming from.

Second, you use a SIP transaction model to prove the impact of
transaction responses. I need to think that through better, but my first
inclination is to say that it is not a good comparison to MSRP. This is
because the SIP transaction model is much more complex and timing
sensitive than anything currently proposed for MSRP. Further, the MSRP
syntax is (hopefully) lighter weight, and easier to parse. Therefore, I
think that, if you had done this test using an MSRP device, and kept the
other factors the same, the difference in performance between running
with and without responses would be smaller than that for a SIP device.

OL: I hope that MSRP is lighter in general but not necessarily in %%.

Further, if I understand your data correctly, with the SIP device you
see a difference in performance of around %20. Is this correct? If so, I
do not find that to be highly convincing by itself. If we were talking
about order-of-magnitude differences, I would be more convinced. Of
course, if we end up accepting your other assertions that the
transaction response brings no value, then a %20 change might be
worthwhile.

OL: It is convincing to BIG deployments. It may be less crucial for
enterprises.

And now, some specific comments:

>   The hop-by-hop mechanism (sometimes referred to as the "ACCEPT"
>    mechanism) is modeled after the SIP transaction model. 

We used the term "ACCEPT" in off line discussions as an abstraction to
talk about transaction responses. I think in this draft, it would be
less confusing to use the terminology from the MSRP draft. Perhaps
"transaction response" would be more clear.

OL: Sure.

> (Negative responses are also defined but they are applicable only for
>    deployments with no relays in the network.)

There has not been any agreement to that effect that I am aware of. We
had a side conversation that we might want to get rid of negative
transaction responses for relays, but no wg consensus. I personally do
not like the idea of having a different set of allowed transaction
responses depending on whether relays are present.

OL: I don't like the different sets either. The point that I found
disturbing was that the negative responses are useless when received by
relays.

> The negative "end-to-end" mechanism is mandatory for relays for error
>    reporting when connectivity problem is detected. In this context
the
>    mechanism is interchangeably referred to as "DSR" or "DSN". Relays
>    MUST NOT use the end-to end mechanism for reporting positive
>    acknowledgements.

DSN would not necessarily be end-to-end, since, as you mention, relays
might generate them. Also, I accidentally introduced the acronym DSR due
to some email typos. I think we can drop it.

OL: No problem. That is why I used "". Still, currently these are the
same messages.

>  When an MSRP entity A (a user agent or a relay) sends/forwards a
>    message, it creates the message transaction state and starts a
timer
>    (~30 sec). 

Adam suggested that we don't really need a mandatory timer here. He
proposed that we say that a device MAY create a timer, and the value
would be local policy. If a device(endpoint or relay) does _not_ create
a timer, it would merely keep a record of the transaction until it got a
transaction response, at which time it could completely forget about it.

If a TCP error is encounted, it would then need to either send DSN(s)
for any outstanding transactions (if a relay), or inform the user (if
and endpoint.) I believe that removing the mandatory timer will make it
possible for a relay to greatly reduce the transaction overhead. The
only advantage in keeping such a timer is it could allow a device to
detect a problem faster than TCP will by itself. This would come at the
expense of more false failure notifications.

OL: I commented in the beginning of this mail.

> If there is an
>    immediate problem in the local handling of this message by B, the
>    MSRP entity (B) MUST generate an error report and send it all the
way
>    back to the originator (i.e. the original source) of the message
>    using the end-to-end mechanism described below.

I don't think we have consensus on that point. This goes back to the
point about negative transaction responses. If the failure is immediate,
B could have just sent a negative transaction response. The DSN is
needed for failures that require too much time to detect to block the
sending of the transaction response.

> In both cases, the
>    error reports must include a list the messages per originator that
>    haven't been acknowledged by the next hop.

We still need to do work to determine whether a DSN can include a list
of failed messages, or we have a separate DSN per message.

OL: Sure. I just took an assumption.

>    o  In addition, if as a result of the SDP negotiation, a user agent
>       agreed to provide an application layer positive and/or negative
>       report/receipt per message, the user agent MUST create the
report
>       and send it back to the message originator in the same manner as
>       described above (i.e. inside the existing MSRP connection)

I am confused by this paragraph. The section so far talked about how you
send DSN. This paragraph seems to say "And if you agreed to do so, you
also have to send DSN. How is this not redundant?"

OL: There are two mechanisms: one hop-by-hop (when the hop can be an end
user as well) and the other is end-to-end. Currently the hop-by-hop
negative responses are mandatory. If we keep the two mechanisms separate
(which we should) the first will report about a failure on the low MSRP
layer and stop the message processing. The second will try to report
about a different error in a higher mechanism . Therefore there is no
duplication here.

>    The following known issues still need to be addressed by the MSRP
>    specification:
> 
>    o  Relationship between the MSRP end-to-end procedures and the
>       "application above the user agent" end-to-end procedures

Our current expectation is the primary use of DSN would be to present
the information to the user, who then does whatever he or she chooses. I
expect it to be machine parseable, but do not expect to define any
normative behavior from the application beyond telling the user.

OL: I agree with this approach.

>    o  The exact end-to-end operation modes and their negotiation using
>       the offer-answer SDP mechanism

That work is currently in progress. (In fact, Chris Boulon just proposed
some text to me on DSN in general--Yeah Chris!!)

In the past, though, you have suggested per-message negotiation, rather
than SDP based, i.e. per-session, negotiation. Are you saying you would
now prefer per-session?

OL: My requirements are definitely per message. The point was that
currently I haven't seen a definition even per session.

>    o  The exact format of the reports (both negative and positive)

In progress.

>    o  A mechanism needs to be defined so that all outstanding
end-to-end
>       reports (i.e. belonging to sessions ended earlier by user
agents)
>       are silently discarded by the system

The next MSRP draft will call out an open issue on whether DSN can be
delivered after a session ends.

>    o  A mechanism needs to be defined so that NO end-to-end status
>       reports will be generated about delivery of earlier end-to-end
>       status reports

In progress.
>    o  It is unclear why the status report messages need to be
>       acknowledged hop-by-hop (as per current design) because they
MUST
>       not be included in the status reports (as stated above)

It is a matter of balancing the complexity of having two different
transaction models with the cost of having a transaction model that
works well for the primary usage (sending IMs) but is less optimal for a
secondary usage (sending DSN, or other status reports.)

OL: As we talked before, the bit "response/don't response" per message
will provide a solution to this problem as well.

Scalability analysis:

>    The most trivial consequence of the defined "hop-by-hop" mechanism
is
>    the need to keep per message a state object (including a timer) and
>    to clear it upon receiving an acknowledgement for the message.
> 

As I mentioned above, we have a proposal to remove the requirement to
keep a timer. This should have a significant affect on the overall
impact.

> In order to fully realize the impact of this design on an IM system,
>    we need to translate it to numbers. For example, as of today, a
>    typical public IM service would have 150MM subscribed users with a
>    peak of 10MM simultaneous users.
> 
>    Consequently, in order to compare the cost of such IM solution to
>    "less statefull" IM architectures, every additional bit of memory
and
>    CPU processing power required for keeping the state per "message
>    transaction" need to be multiplied at least (since message
>    transactions can be overlapped) by 10MM.

I'm confused. Are you saying that, if I have 10 million (I assume you
mean MM to represent a million.) concurrently logged in users, I have 10
million MSRP transactions in play at any moment? I would assume not all
active users are in a session at any moment, and not all that _are_ in a
session have an open transaction at any moment. Am I misunderastanding
your point?

OL: Yes. It is a simplistic example, but each user can simultaneously
have many open IM sessions with a number of outstanding messages on
each. So at the overage it can be one IM message for an active user at a
time.

>    It is interesting to mention that even modern TCP implementations
>    have moved away from acknowledging every TCP segment and today
>    acknowledge every second segment only...
> 

While I don't find TCP's ack rate to be particularly relevent to the
discussion, I believe that most implementations send _"at least that
often", which is very different than  "no more often than".

> One of the concerns with the suggested hop-by-hop mechanism is that
>    although the mechanism is very simple, it suffers from some of the
>    known TCP problems but, unlike TCP, doesn't address them.
>    Unfortunately, the current working assumption that the
>    "congestion-safe" TCP will automatically make the MSRP
>    congestion-safe is inaccurate.

I think the point of congestion-safety, is the presence of MSRP on the
network should not interfere with the use of _other_ applications. It is
not the same as saying an MSRP relay cannot itself get congested.

>   Because each message will be acknowledged (by Response) by B
>    automatically, A needs to control the rate of the messages it sends
>    to B in order to be able to process the incoming responses. In
order
>    to do so, A buffers messages before it sends them to B.  In the
same
>    time B sends its data messages (i.e. new Requests) asynchronously
on
>    the same connection from B to A. As a result, the connection gets
>    throttled by new Requests while the old Requests (from A to B)
>    haven't been acknowledged yet. Provided A and B do not use common
>    throttling mechanism and/or have different processing power this
>    situation requires careful tuning (e.g. configuration or
negotiation)
>    by each side.

It seems to me that using chunking to cut down on message size, and the
removal of the transaction timer will help this issue quite a lot.

Functional Overview:

>    Detection of TCP connection failure earlier than reported by TCP
>    itself:
> 
>    o  This can be useful if TCP connection shows a property of
>       unreasonably long (e.g. up to 9 minutes) TCP retry procedures.
The
>       reason for this pattern would be either improperly tuned TCP
>       configuration or/and transport layer intermediates (potentially
>       deployed for security or monitoring) that actively intervene the
>       TCP connection and, as a result, change its properties

The 9 minute figure you have heard Adam and I throw out is _not_ the TCP
  retransmission timer value . Rather, it is roughly the total time that
a TCP stack using default configurations is likely to continue
retransmission/backoff cycles before it gives up and tells the
application it has a problem. This is _not_ due to misconfiguration;
rather, it is the behavior using the _default_ configuration. Not all
implementations let you change this at all. I think Windows does, but at
a whole interface scope.

>    o  Unfortunately, using the MSRP timer with shorter than TCP
timeout
>       will increase the probability of message duplications as
described
>       in the next section

See previous comments on the transaction timer.

> o  It's possible that an MSRP timer of the relay has expired or a
>       local TCP connection failure has been detected by the relay when
>       there are still locally unprocessed positive acknowledgments
from
>       the next hop in the TCP pipe. These acknowledgements can be to
the
>       messages that had been already forwarded by the next hop and
>       eventually got delivered to the destination

Yes, I acknowledge this as true. (It is true of just about any protocol
I have ever seen.)

>    o  Note that as shorter the local MSRP timer is, as more frequently
>       the scenario above can occur
>    o  Note also that the scenario above can theoretically happen to
any
>       or all of the relays for the same message. This will result in a
>       flood of error reports towards the message originator even in
the
>       case of successful message delivery to its final destination

Theoretically, yes. But in practice, rarely, as having multiple hops
along the way _each_ decide that a transaction had failed when it in
fact had succeeded is very unlikely.

MSRP Abstraction:

>    As shown above, the defined hop-by-hop mechanism is incapable of
>    addressing most of the basic IM applications requirements when more
>    than a single hop is involved.
> 

You showed _one_ potential requirement it does not address by itself. 
That is hardly "most".

>    As a result of the two statements above, this paper argues that the
>    end-to-end mechanism MUST be specified in the basic MSRP
>    specification:
> 

I believe we have already agreed that the base spec will describe
endpoint processing of DSN, to the degree that a non-relay aware
endpoint needs to understand to interwork with an endpoint that uses
relays.

> http://www.ietf.org/internet-drafts/draft-levin-simple-msrp-review-00.
> txt
> 

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: "Chris Boulton" <[email-masked]>
Date: Tue, 13 Apr 2004 10:07:57 +0100

Comment in-line.

>than SDP based, i.e. per-session, negotiation. Are you saying you would

[Chris Boulton] Sorry for the late response ;-).  The current text
proposes per message negotiation with a default payload.  The only
negotiation that can occur at the session level is for an alternative
payload type to be negotiated for DSN.  

This message has been scanned for viruses by MailControl - www.mailcontrol.com

---

From: "Chris Boulton" <[email-masked]>
Date: Tue, 13 Apr 2004 10:07:57 +0100

Comment in-line.