Subject: RE: MSRP: Are REPORTs per-chunk or per-message? (was Re: [Simple]Review of draft-ietf-simple-message-sessions-08 - Byte Ranges inREPORTs)

From: [email-masked]
Date: Mon, 11 Oct 2004 10:50:50 +0300

Here are my thoughts:

Success Reports: I see some value in byte-range success reports, but if we allow it in the protocol, then we are really complicating the protocol. And as David Oran said, turning MSRP into a file transfer protocol.

Lets look at a scenario: Sender is sending a large message and requests success report. If the receiver is to report byte-range success, then the benefit is that the sender's implementation can determine if there has been failure in delivering a byte range and re-send those. 

Yes, this makes the protocol more robust, BUT we need to examine the requirements here and ask the question of what does it mean for the sender to request success reports? Do we want success reports to aid the user in determining what happened to the message s/he sent, or the implementation in recovering from a lost byte range? I think we want the former. There has never been requirements for the latter. Therefore I vote to success reports per message. It is left as a user decision what to do if that success report is never received.

Failure Reports: I see also some value in byte-range failure reports which also complicates the protocol.

Lets look at a scenario: Sender is sending a large message and requests failure report. If the receiver is to report byte-range failure, then the benefit is that the sender's implementation can determine if there has been failure in delivering a byte range and re-send those.

Again, I ask the question of requirements. Are the failure reports meant for user consumption or for implementation to recover? I'm inclined to say we want the former and therefore suggest that failure reports to be per message. It is left as a user decision what to do if that failure report is received.

Remember, this is not a file transfer protocol. It is an instant messaging protocol. If the message is large, it is chunked, but it is still a message and success and failure reports should be per message. A message is message, large or small.

Lets now look at Cullen's Elevator problem: Cullen is sending Ben a large file using his IM client running on his mobile device. Cullen walks into an elevator and looses radio service. When Cullen walks out of the elevator, he expects his IM client to continue transmitting from where it left off.

When Cullen walks into the elevator, any TCP connections are lost and therefore Ben's IM client can't report the failure anyway. Ben's client can only wait for x number of minutes hoping that Cullen's client comes back online and re-establishes the session. Cullen's client does not know the byte range that Ben's client has received and can therefore only commence from where it has sent the last byte. Is that ok?

If we continue to think that a message is a message, even if spread across IM sessions, then Ben's client can report success or failure of the message that was sent across both those sessions.

Regards,
Hisham

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]]On Behalf
> Of ext Paul Kyzivat
> Sent: 09.October.2004 01:51
> To: Cullen Jennings
> Cc: Adam Roach; [email-masked]; Niemi Aki (Nokia-NRC/Helsinki); Rohan
> Mahy
> Subject: Re: MSRP: Are REPORTs per-chunk or per-message? (was Re:
> [Simple]Review of draft-ietf-simple-message-sessions-08 - Byte Ranges
> inREPORTs)
> 
> 
> 
> 
> Cullen Jennings wrote:
> > I think Paul clarified this for me. The semantics of the 
> protocol are that
> > reports are for the byterange specified in the report. No 
> more or less is
> > implied on how this range lines up with any chunk or message.
> > 
> > If someone has asked for reports that a message succeeded, 
> you can't do that
> > till you have all the message so you don't send a SUCCESS 
> report until you
> > have all the bytes in the message.
> 
> You *can* do it on a per-byte-range basis. There is a 
> question of *why* 
> you would do that - it would only be because it was of some value in 
> error recovery.
> 
> > If you know some bytes have failed, and the sender has 
> asked for failure
> > reports, you send that as soon as you know which bytes have 
> failed. You
> > might later send another one that other bytes have failed. 
> The byte ranges
> > in a FAILURE report will correspond to some chunk somewhere 
> but that may not
> > correspond to any chunk that the first sender sent since an 
> intermediate
> > relay may have chunked differently.
> 
> Right.
> 
> > The issue of tracking which bytes you have received and not 
> received already
> > existed as soon as we did chunks even if there was no 
> reporting. This is not
> > hard to implement with linked lists of chunks.
> 
> Yes. But if you want to send partial success reports, there is some 
> complexity in deciding when to do so, and what range to include in it.
> 
> We haven't mentioned the issue, but if we can lose chunks of 
> message, we 
> can probably lose REPORTs too. If there is known to be only 
> one success 
> report, and it fails to come, the sender can time and declare 
> a failure.
> 
> If there are multiple success reports for disjoint byte 
> ranges, and you 
> don't get them all, then what? I guess you wait until you have sent 
> everything once, and then timed out waiting for all the 
> success reports, 
> and then you could try retransmitting the ranges that haven't been 
> confirmed. But then what? Do you go thru another timeout cycle?
> 
> > When you receive a negative report on a byterange. You 
> renegotiate the
> > connection and resend that chunk.
> 
> That works for negative ones, but not the positive kind. But negative 
> ones are even less reliable. If things aren't working well so that 
> chunks are being lost, what is chance that the failure will 
> be lost too?
> 
> Another question that comes to mind: Suppose I have sent bytes 
> 1-1,000,000. I get a failure report for bytes 500,000-501,000. If I 
> renegotiate a new connection, right away, and close the old 
> one, what do 
> I retransmit? Just 500,000-501,000, or everything starting 
> from 500,000? 
> There could be failure reports for other byte ranges that I would 
> receive if I kept listening to the old connection - even for earlier 
> byte ranges than the one I received. If I kill the connection 
> and start 
> another then I won't get those. I can just retransmit starting from 
> 500,000 and hope that nothing earlier failed too.
> 
> > As some level this is complicated - as soon as we said that 
> we wanted a
> > system that multiplexed message onto one TCP transport, we 
> choose this level
> > of complexity. None of this is that complicated - we have 
> been around this
> > over and over for the last 1.5 years, if we remove any part 
> of this system
> > we break requirements that people said were absolutely 
> critical to them.
> 
> I agree that some scheme like this is ultimately not that complicated.
> But it isn't written down anywhere. The question is whether 
> we want to 
> take the time to write it all down.
> 
> The various combinations of success and failure reports with 
> and without 
> byte ranges at least makes a lot of cases to explain and for 
> people to 
> understand and implement correctly.
> 
> Forbidding byte ranges in success reports would at least prune away a 
> few of the options.
> 
> 	Paul
> 
> > On 10/8/04 7:39 AM, "Paul Kyzivat" <[email-masked]> wrote:
> > 
> > 
> >>
> >>Cullen Jennings wrote:
> >>
> >>>>I had a conversation with Cullen, where he expressed the 
> same opinion
> >>>>for failure reports. Imagine, in your LoTR example, you 
> were using a
> >>>>relay, and it has a transport failure between itself and 
> the next hop.
> >>>>It sends you one or more failure reports for the chunks 
> for which it
> >>>>could not confirm delivery. You establish a new session, 
> and continue;
> >>>>resending the failed chunks.
> >>>>
> >>>>I'd like to close on this quickly, so I offer the 
> following questions
> >>>>to anyone who cares. I would appreciate opinions asap, so 
> I can get
> >>>>this into the next revision.
> >>>>
> >>>>1) Should failure reports be sent per chunk or per 
> complete message? (I
> >>>>think it is per chunk.)
> >>>
> >>>sort of both - yes they have to be per chunk so the chunk 
> had a fialure in
> >>>transmission but the message is fine. If the whole message 
> is going to be
> >>>bad, because of something like the body type is not 
> understood, then they
> >>>are sent on the whole message. You indicate they are on 
> the whole message by
> >>>using a * in the byterange
> >>
> >>This is confusing. The failure reports are really for byte 
> ranges, not
> >>chunks, since there is no consistent understanding by all parties of
> >>what the chunking is. And the failure reports aren't necessarily all
> >>sent by the same node.
> >>
> >>It seems that if failure reports for byte ranges are 
> supported at all,
> >>then the sender will need to in effect keep a separate 
> status for every
> >>byte sent. The status for each byte is one of:
> >>- unknown
> >>- failure reported
> >>- success reported
> >>
> >>(Obviously there are optimizations on how to store this status for a
> >>message of many bytes.) It appears that there could be at 
> least as many
> >>reasons for overlapping byte ranges in reports as in sends, so that
> >>needs to be handled as well.
> >>
> >>Then the sender will need to decide what to do if failure 
> is reported
> >>for one or more bytes. It could decide to just give up on 
> the message
> >>and stop sending. Or it could just retransmit a chunk 
> containing the bad
> >>byte(s). (For that to be useful, it would have to assume 
> that some relay
> >>was at fault and will recover.) Or it could negotiate a replacement
> >>session and then send a chunk containing the bad byte(s).
> >>
> >>This is starting to look very ugly.
> >>
> >>
> >>>>2) Should success reports be sent per chunk or per 
> complete message?
> >>>>Note that, if we send them per-chunk, then the sender 
> must accumulate 1
> >>>>or more reports covering all the bytes in the message 
> before it can
> >>>>consider the message successful. These reports may or may not
> >>>>correspond one-to-one with the chunks it sent, as a relay 
> may re-chunk.
> >>>>
> >>>>(Again, I vote per-chunk.)
> >>>
> >>>I think we have to have a success for the whole message or 
> there is no way
> >>>to know if everything has arrived - questions to me is do 
> we also need to
> >>>have chunk by chunk acknowledgments along the way. (This 
> is all assuming
> >>>that positive reports where requested). I'm not sure I see 
> the reason that
> >>>these would be needed but I feel like I am forgetting something.
> >>
> >>Success reports only make sense end to end. Ultimately you need
> >>confirmation of success of the *whole* message. So if byte 
> ranges are
> >>used then the sum total of them must cover the entire message.
> >>
> >>This is complicated by the fact that REPORTs aren't 
> confirmed, and so
> >>might be lost. (E.g. by a relay.)
> >>
> >>If byte ranges in success reports are to be used, then I 
> think the best
> >>strategy might be for the receiving node to send cumulative 
> ones. For
> >>instance, send one for the largest range of bytes received 
> that includes
> >>previously unconfirmed bytes. (With in-order delivery this 
> would mean
> >>that each report would cover everything received so far.) 
> These reports
> >>might not be sent for every chunk received - just every so often.
> >>
> >>This is of course just as complicated for the sender as the failure
> >>reports above, and more complicated for the receiver.
> >>
> >>
> >>>>3) Do we need to say anything about how long the 
> _receiver_ holds onto
> >>>>the chunks of an incomplete message in hopes any remaining chunks
> >>>>arrive? In particular, do we need to talk about this for 
> holding chunks
> >>>>after a session fails, in case the sender establishes a 
> new session and
> >>>>sends the remaining chunks?
> >>>
> >>>No, I think this is like how long the end to end timer 
> should be - it is
> >>>totally situation dependent. Probably needs to say 
> something along lines off
> >>>"for awhile"
> >>
> >>I am inclined to agree.
> >>
> >>
> >>>>(I vote that we say no more than we already did in 08 concerning
> >>>>re-sending chunks on new sessions.)
> >>>>
> >>>>4) Do we need to enable the receiver to send a failure 
> report about
> >>>>_missing_ chunks? This does not make sense with success reports
> >>>>requested, but might make sense when only failure reports are
> >>>>requested.
> >>>
> >>>I don't think so - if the sender cared about the data, 
> someone will sent a
> >>>negative at the appropriate time. The chunks can arrive 
> out of order and
> >>>this makes it hard for the receiver to know when to 
> request a missing block.
> >>
> >>I agree this doesn't make sense.
> >>
> >>Because of the complexity that is now apparent, I only see 
> a couple of
> >>useful ways forward:
> >>
> >>- Specify how to handle recovery, driven by byte ranges in 
> responses.
> >>
> >>- Put off recovery to a future draft. Remove support for byte ranges
> >>   in reports. But figure out how we could gracefully 
> incorporate that
> >>   future draft and migrate to support of it.
> >>
> >>- Entirely abandon any thought of recovery in MSRP. Remove 
> support for
> >>   byte ranges from reports. Leave it for a replacement protocol.
> >>
> >>I think leaving byte ranges in reports without specifying how to use
> >>them for recovery will result in major interoperability problems.
> >>
> >>I am of mixed thought about which of those alternative I 
> prefer. Given
> >>the time pressures and priorities, I could easily be convinced to
> >>abandon any thought of recovery, though I am not happy about it.
> >>
> >>Paul
> >>
> >>
> > 
> > 
> > 
> > 
> 
> 
> _______________________________________________
> Simple mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/simple
> 

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 11 Oct 2004 17:22:32 +0200

I agree with Dave's characterization of MSRP. That was pretty much the
requirements for MSRP for transferring IMs.

If you believe IM are short, then you don't need MSRP, a SIP Message message
will work fine.  If you believe they are not short, then transferring them
is very close to the problem of file transfer.  I note that some IM's that
say nothing more than "R U There :-)" with complex formatting are looking
pretty big. I guess this is hard to believe but then again I work in a
standards group that can publish its very largest document (RFC 3261) in
about 640K. Though Bill once thought this was enough, he seems to have moved
on to technology that uses more than this.

If the working group believes we do not need to to transfer large IM
messages, I propose we kill all the MSRP work immediately and call get
session mode to work in some much simpler way. I do not believe this. I
believe that transferring large messages is required. I am talking about
transferring large IM messages here - I'm not taking about file transfer. I
also think that a commercially viable IM system needs file transfer but this
could be done with some other undefined protocol and the working group has
told me before that file transfer is not a requirement for MSRP. However, we
previously came to the conclusion that transferring large IM's was required.

Right now we have a fairly concrete proposal on the table to make this work.
It does use negative reports on byte ranges and uses positive and negative
reports on complete messages. It does not need anything else thought the
positive reports on byte ranges can allow the sender to better understand
the status of a message and I suggest they be optionally allowed but not
required.

Unless someone has a really good reason why the current proposal does not
meet their needs, I would really rather not start over. The current proposal
has many things in it to meet the needs of different people. That is how we
got the complex system on saying what type of status reports needs to be
generated. The current proposal meets the needs off everyone who got
involved. If people feel that is is too complex and meets too many needs,
then we need to redo our requirements and take out several of our current
requirements because I don't think they can be meant with anything much
simpler than this.

I would appreciate guidance on this from the WG and the WG chairs before the
00 draft cutoff. Personally I think that if we restart the requirements
discussions, we will come to almost exactly the requirements we currently
have so I am strongly not in favor of doing this.

> Here are my thoughts:
All reports include a byte range but some are reporting on the complete
message some only a chunk of it.

> Success Reports: I see some value in byte-range success reports, but if we
> allow it in the protocol, then we are really complicating the protocol. And as
> David Oran said, turning MSRP into a file transfer protocol.

I don't think the receiver of a positive report on a chunk of the message
needs to do anything with it. The discussion of this seems to have spiraled
into discussing why all the reports are there.

> 
> Lets look at a scenario: Sender is sending a large message and requests
> success report. If the receiver is to report byte-range success, then the
> benefit is that the sender's implementation can determine if there has been
> failure in delivering a byte range and re-send those.
>
A sender figures out what needs to be resent using the negative not positive
reports on a chunk.
 
> Yes, this makes the protocol more robust, BUT we need to examine the
> requirements here and ask the question of what does it mean for the sender to
> request success reports? Do we want success reports to aid the user in
> determining what happened to the message s/he sent, or the implementation in
> recovering from a lost byte range? I think we want the former. There has never
> been requirements for the latter. Therefore I vote to success reports per
> message. It is left as a user decision what to do if that success report is
> never received.
> 

We took as a requirements that error recovery for a lost chunk was faster
than the time we were willing to set as a timeout for the complete message
assumed failed because a positive report for the complete message could had
not been received. This forced the  design to have negative reports. We had
to accept this because some messages will start on fast links and move to
slow (read wireless) links and the error recover time expect on a fast link
was less than the total transit time of the message on a slow link.

> Lets look at a scenario: Sender is sending a large message and requests
> failure report. If the receiver is to report byte-range failure, then the
> benefit is that the sender's implementation can determine if there has been
> failure in delivering a byte range and re-send those.
> 
> Again, I ask the question of requirements. Are the failure reports meant for
> user consumption or for implementation to recover? I'm inclined to say we want
> the former and therefore suggest that failure reports to be per message. It is
> left as a user decision what to do if that failure report is received.
>

The current document has reports for failure of a chunk as meant for
retransmission purposes.
 
> Remember, this is not a file transfer protocol. It is an instant messaging
> protocol. If the message is large, it is chunked, but it is still a message
> and success and failure reports should be per message. A message is message,
> large or small.
>

Exactly the opposite requirement was stated before. The argument made before
was it I had just transferred 20k over my wireless link and lost the last 1
k block, we did not want to resend the whole 20k. Adam and I both pointed
out at that time that this was going to complicate stuff.
 
> Lets now look at Cullen's Elevator problem: Cullen is sending Ben a large file
> using his IM client running on his mobile device. Cullen walks into an
> elevator and looses radio service. When Cullen walks out of the elevator, he
> expects his IM client to continue transmitting from where it left off.
> 
> When Cullen walks into the elevator, any TCP connections are lost and
> therefore Ben's IM client can't report the failure anyway. Ben's client can
> only wait for x number of minutes hoping that Cullen's client comes back
> online and re-establishes the session. Cullen's client does not know the byte
> range that Ben's client has received and can therefore only commence from
> where it has sent the last byte. Is that ok?
>

no this would not be ok. The relay that was sending a chunk to me when the
connection failed will report an error on that chunk (an any others that it
is holding) and that data will get retransmitted.
 
> If we continue to think that a message is a message, even if spread across IM
> sessions, then Ben's client can report success or failure of the message that
> was sent across both those sessions.
>

In this case, Ben and I only had one IM session. It was just that it used a
couple of MSRP sessions to transport the data in it.

---

From: David R Oran <[email-masked]>
Date: Mon, 11 Oct 2004 15:08:59 -0400

There's a difference between "bigger than a couple of packets", and 
LoTR. I'm guess I'm saying that there is a middle ground between 
something that benefits from chunking to reduce latency when you 
multiplex through a relay, and something which needs to do incremental 
recovery because the MTTR is too long compared to the MTBF for a single 
message.

Dave.

On Oct 11, 2004, at 11:22 AM, Cullen Jennings wrote:

>> Failure Reports: I see also some value in byte-range failure reports 
>> which
>> also complicates the protocol.
>>
>
David R. Oran
Cisco Fellow
Cisco Systems
7 Ladyslipper Lane
Acton, MA 01720 USA
Tel: +1 978 264 2048
Email: [email-masked]

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 12 Oct 2004 11:27:14 +0200

> recovery because the MTTR is too long compared to the MTBF for a single

I agree. 

We went down an incremental path that started with stage 1 the sender runs a
timer waiting for ACK of whole message. Problem here was wanted faster
recovery than acceptable timer level. So got to stage 2 where relay could
NACK a message and ask for immediate resent. But people pointed out wasted
of resend particularly in wireless and got to stage 3 where the NACK
includes the byte range that got lost.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 12 Oct 2004 20:54:30 -0500

On Oct 11, 2004, at 2:50 AM, [email-masked] wrote:

> Here are my thoughts:
>
> Success Reports: I see some value in byte-range success reports, but 
> if we allow it in the protocol, then we are really complicating the 
> protocol. And as David Oran said, turning MSRP into a file transfer 
> protocol.
>
> Lets look at a scenario: Sender is sending a large message and 
> requests success report. If the receiver is to report byte-range 
> success, then the benefit is that the sender's implementation can 
> determine if there has been failure in delivering a byte range and 
> re-send those.
>
> requirements here and ask the question of what does it mean for the 
> sender to request success reports? Do we want success reports to aid 
> the user in determining what happened to the message s/he sent, or the 
> implementation in recovering from a lost byte range? I think we want 
> the former. There has never been requirements for the latter. 
> Therefore I vote to success reports per message. It is left as a user 
> decision what to do if that success report is never received.
>

I am willing to accept the idea of sending success reports only for 
complete messages. If the sender desires chunk-level restart, it needs 
to request failure reports. (More on that below.)

> which also complicates the protocol.
>
> Lets look at a scenario: Sender is sending a large message and 
> requests failure report. If the receiver is to report byte-range 
> failure, then the benefit is that the sender's implementation can 
> determine if there has been failure in delivering a byte range and 
> re-send those.
>
> Again, I ask the question of requirements. Are the failure reports 
> meant for user consumption or for implementation to recover? I'm 
> inclined to say we want the former and therefore suggest that failure 
> reports to be per message. It is left as a user decision what to do if 
> that failure report is received.
>
> Remember, this is not a file transfer protocol. It is an instant 
> messaging protocol. If the message is large, it is chunked, but it is 
> still a message and success and failure reports should be per message. 
> A message is message, large or small.

The issue I have, is it seems like more work to send _failure_ 
responses for whole messages than it does per chunk. If a relay is 
going to send a failure report, are we going to require it to check 
that it has not already sent a report for a previous chunk in the 
message? That would require it to keep state around for _failed_ 
messages, which seems like a bad idea.

So, another alternative is to let it send a failure report for each 
failed chunk, but not to include a byte-range. If we are going to go 
that far, it seems to be no more difficult to go ahead and throw in the 
byte range.

>
> Lets now look at Cullen's Elevator problem: Cullen is sending Ben a 
> large file using his IM client running on his mobile device. Cullen 
> walks into an elevator and looses radio service. When Cullen walks out 
> of the elevator, he expects his IM client to continue transmitting 
> from where it left off.
>
> therefore Ben's IM client can't report the failure anyway. Ben's 
> client can only wait for x number of minutes hoping that Cullen's 
> client comes back online and re-establishes the session. Cullen's 
> client does not know the byte range that Ben's client has received and 
> can therefore only commence from where it has sent the last byte. Is 
> that ok?

I don't see how it can do that without at least having some sort of 
byte-range reporting. A TCP failure will generally not be detected for 
some minutes after the actual failure occurs, unless a complete reset 
of the interface occurs. The last byte it sent almost certainly did not 
arrive at the receiver, so this approach will not likely work.

I think, if we do not have any sort of byte-range reporting, then the 
only thing it can do is start the message over from byte 0.

>
> If we continue to think that a message is a message, even if spread 
> across IM sessions, then Ben's client can report success or failure of 
> the message that was sent across both those sessions.

See my comment above on failure reports.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 12 Oct 2004 21:16:03 -0500

> includes the byte range that got lost.

I have to point out that we do not require the sender to do anything 
about a failure message. It is allowed to attempt to resend on a new 
session, but it is also allowed to just give up in disgust.