Subject: [Simple] Re: ABNF Issues in draft-ietf-simple-message-sessions-08.txt

From: "Peter Ridler" <[email-masked]>
Date: Thu, 14 Oct 2004 18:54:42 -0400

Thanks Ben -- I removed things we agree on from this reply and commented
inline on the rest....

>>
>> 2) Grouping not required on "headers" rule (nit)
>>
>
> Sorry, I don't understand the comment.
>

Sorry -- my typo should be "header" rule (still a nit)

Shown as::
 header = ( To-Path
    / From-Path
    / Message-ID
    / Report-Success
    / Report-Failure
    / Byte-Range
    / Status
    / ext-header )

Should be::
 header = To-Path
    / From-Path
    / Message-ID
    / Report-Success
    / Report-Failure
    / Byte-Range
    / Status
    / ext-header

>> 3) Section 5 defines the MSRP URL with ABNF but not present in Section
>> 8
>> (shown here with corrections)
>>
>> ; "userinfo", "hostport", and "unreserved" are detailed in RFC2396
>>
>> MSRP-url = msrp-scheme "://" [userinfo "@"] hostport ["/" resource] ";"
>> transport
>>
>> msrp-scheme = "msrp" / "msrps"
>>
>> resource = 1*unreserved
>>
>> transport = "tcp" / ALPHANUM
>
>The only change I see is the removal of the "s" char in MSRP-urls (which
was a typo.) Are there other
>changes I have missed? Also, I _think_ you are recommending moving this to
section 8, correct?

Yes, just added the comment to reference RFC2396 in ABNF (nit) and yes
adding/moving the ABNF syntax to Section 8 is requested.

>>
>>
>> 12) "Content-ID" and "Content-Description" NOT defined in RFC2045
>>     "Content-Disposition" NOT defined in RFC2183
>>     "mime-extension-field" should be defined as "MIME-extension-field" 
>> for RFC2045 (nit)
>>
>>
>> Other-Mime-Header = id                    ; Content-ID defined in RFC2045
>>                   / description           ; Content-Description defined
in RFC2045
>>                   / disposition           ; Content-Disposition defined
in RFC2183
>>                   / MIME-extension-field  ; defined in RFC2045
>>
>
>Sorry, I am not sure I understand. Are you saying I have the wrong RFC
numbers? Or just that I
> should present this differently?
>

I would say to present it as shown.  Example - in RFC2045 the
"Content-Description" rule is really defined as "description".  This is
required to make ABNF compilers find the correct rules.  There is no rule
defined in RFC2045 called "Content-Description".  The comments are to aid
human readability...

>
>
>> 15) The definition of "data" states ZERO to INFINITY, and assuming 
>> that we
>> have defined a MSRP message limit, then data will eat up all the bytes 
>> of
>> information AND the CRLF of "content-stuff" as well as the "end-line"
>> message as well.  "data" must have some bounds defined for it!
>>
>> Here is a suggestion to make "data" more clear (and conform to ABNF 
>> RFC2234
>> logic rules)
>>
>> a) You could use "text" as defined in RFC2822 (because of
>> MSRP->RFC2183->RFC2045->RFC2822)
>>
>> data = *text    ; defined in RFC2822
>>
>> ---- OR ------
>>
>> b) You could use "ptext" as defined in RFC2045.  This allows for 
>> escaped
>> bytes using "=" (hex-octet)
>>
>> data = *ptext    ; defined in RFC2045
>>
>> ---- OR ------
>>
>> c) Just define our own type (similar to text in RFC2822)
>>
>> data = *(%x01-09 / %x0B-0C / %x0E-7F)	; Characters excluding CR
and LF
>>
>> ---------------
>>
>> I like point #b, but could live with any of them.....
>
>If I understand right, your point is we need to exclude CR and LF from 
>data, so that we can find the end. Problem is, the body may well 
>contain CRs and LFs. That would seem to preclude A and C. And I don't 
>really like having to escape things in the body if we can possibly 
>avoid it.
>
>We had assumed that an MSRP parser would scan for occurrence of the 
>"-------" and transaction-ID in the end-line. These values are chosen 
>to make it highly unlikely they will collide with the message content. 
>But I don't know a way to express this concept in ABNF, at least not 
>with the formality needed to make a parser created by an ABNF compiler 
>to just do the right thing.
>
>Does anyone else have thoughts on this matter? Is this lever of ABNF 
>formality required?
>

Yes you read me correctly.  I also don't like to escape the CR/LF as in "b"
either, but when you have "data" in the middle of the message you must be
able to delimit it somehow.  I think the route of the problem is the
"end-line" at the end of the message.

   end-line = "-------" transact-id continuation-flag CRLF
   continuation-flag = "+" / "$" / "#"

We can make this work by parsing the message out in the transport layer and
reformat the message so that the simple ABNF logic will work (and I think if
it is hard to make the simple ABNF logic work - then you may not be logical
in your approach).  We would then re-parse the (reformatted) message in the
transaction layers.  A lot of resources in use here....

My suggestion is why not define the message structure similar to what is
used in RFC3261 (SIP) with a headers of "Continuation-Flag" and a
"Content-Length".  We are already using SIP and have this message processing
in the Transport Layer anyway.  This will leave "data = *OCTET" at the end
of the message where it is easy to parse out and abide by the simple logic
rules of ABNF...  This would also improve the MIME / muilt part MIME parsing
performed on the message payload section ("data") as well. 

---Peter

_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Ben Campbell <[email-masked]>
Date: Thu, 14 Oct 2004 18:07:05 -0500

> Sorry -- my typo should be "header" rule (still a nit)

Ah, that makes more sense.

> adding/moving the ABNF syntax to Section 8 is requested.

OK. I will add the reference. I will also move the URL definition to 
section 8 unless someone objects. (If anyone objects, please do it 
asap.)

> defined in RFC2045 called "Content-Description".  The comments are to 

OK, I understand now.

> performed on the message payload section ("data") as well.

We originally defined it exactly that way. We had a long running 
controversy over the use of content-length, vs. the use of a delimiter. 
The work group chose to go with the delimiter. The reasoning for this 
is, it allows you to abort a request without destroying framing for the 
connection. If you use a content-length, then you can't abort before 
you are finished.

If we were strictly p2p, this would not be a big deal. But we also want 
to allow relays, and have the relays multiplex multiple sessions over a 
connection with an adjacent relay. If framing is lost for that 
connection, it affects all of the sessions.

(Cullen or Rohan can state this much more eloquently that I 
can--perhaps they will jump in and do so :-)  )

---

From: "Peter Ridler" <[email-masked]>
Date: Fri, 15 Oct 2004 14:10:38 -0400

Inline (at bottom).... 

> -----Original Message-----
> From: Ben Campbell [mailto:[email-masked]] 
> Sent: Thursday, October 14, 2004 7:07 PM
> To: Peter Ridler
> Cc: [email-masked]
> Subject: Re: ABNF Issues in draft-ietf-simple-message-sessions-08.txt
> 
> 
> > Thanks Ben -- I removed things we agree on from this reply and 
> > commented inline on the rest....
> >
> >>>
> >>> MSRP-url = msrp-scheme "://" [userinfo "@"] hostport ["/" 
> resource] 
> >>> ";"
> >>> transport
> >>>
> OK. I will add the reference. I will also move the URL 
> definition to section 8 unless someone objects. (If anyone 
> objects, please do it
> asap.)
> 
> >
> >>>
> >>>
> > required to make ABNF compilers find the correct rules.  
> There is no 
> > rule
> > defined in RFC2045 called "Content-Description".  The 
> comments are to 
> > aid
> > human readability...
> >
> 
> OK, I understand now.
> 
> >>
> >>
> > Yes you read me correctly.  I also don't like to escape the 
> CR/LF as 
> > in "b"
> > either, but when you have "data" in the middle of the 
> message you must 
> > be
> > in your approach).  We would then re-parse the 
> (reformatted) message 
> > in the
> > My suggestion is why not define the message structure 
> similar to what 
> > is
> > in the Transport Layer anyway.  This will leave "data = 
> *OCTET" at the 
> > end
> > of the message where it is easy to parse out and abide by 
> the simple 
> > logic
> > rules of ABNF...  This would also improve the MIME / muilt 
> part MIME 
> > parsing

Well I don't want to rake up old issues.  I understand (and agree with) the
reasons on the affects of framing messages.

I don't see the difference through -- read X (Content-Length) bytes from a
stream -- or -- read more or less bytes from a stream and scan the buffered
data for "------- transact-id continuation-flag CRLF", and then re-sync the
framing if I read too much data from the stream !!!! I would like it if
Cullen or Rohan could enlighten me why they like the later (maybe on the
side)...

If we keep to this message format then I really fell strong that we MUST
escape the CR LF out of the data payload section of the message (I don't
think the extra processing a big deal for this).

Yes, I can fudge the ABNF Compiler with a special "data" processing rule
that would handle this, but, I feel that the simple logic constructs of ABNF
keep the basic rules of logic in check and if the rules don't work, then,
maybe there is a issue with the way we are doing things.

---Peter

---

From: Ben Campbell <[email-masked]>
Date: Fri, 15 Oct 2004 13:34:22 -0500

On Oct 15, 2004, at 1:10 PM, Peter Ridler wrote:

[...]

> Cullen or Rohan could enlighten me why they like the later (maybe on 

Imagin I am sending a fairly long request. If I have a content-length 
header, then once I have transmitted that header, I cannot change its 
value. So I have to transmit that number of bytes, period. If I don't, 
the receiver will read into the next message, and things go downhill 
from there. This gets really bad with relays. Imagine a relay that does 
not attempt to buffer the entire message, but rather forwards the bytes 
as they come in. If the sender does not complete the message (maybe it 
crashed), the relay would have to pad out the difference, or risk 
breaking framing on the downstream connection.

With the closing delimiter approach, I can abort at any time by simply 
writing out the closing.

> maybe there is a issue with the way we are doing things.

It seems like we have to balance the complexity of escaping CRLF vs the 
advantage of using unmodified ABNF compiler generated parsers. I 
personally prefer sacrificing the parsers to requiring the escaping, 
but I can accept consensus in either direction.

  I am curious if anyone else has an opinion on this.

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 15 Oct 2004 15:46:34 -0400

Ben Campbell wrote:
> personally prefer sacrificing the parsers to requiring the escaping, but 
> I can accept consensus in either direction.
> 

I originally preferred use of content-length, but I have seen the 
advantages of this approach and am now convinced they outweigh the 
problems it causes.

This does result in some hacking to deal with the framing. But in my 
experience, some hacking is required around the framing no matter which 
we we go. (With Content-Length you can't pull the messages off the 
stream for later parsing without doing some redundant parsing just to 
find and understand the content-length header.)

	Paul

---

From: Ben Campbell <[email-masked]>
Date: Sat, 16 Oct 2004 14:29:20 -0500

Responds to self in-line (at end):

On Oct 15, 2004, at 1:34 PM, Ben Campbell wrote:

> from there. This gets really bad with relays. Imagine a relay that 
> does not attempt to buffer the entire message, but rather forwards the 
> bytes as they come in. If the sender does not complete the message 
> (maybe it crashed), the relay would have to pad out the difference, or 
> risk breaking framing on the downstream connection.
>
> It seems like we have to balance the complexity of escaping CRLF vs 
> the advantage of using unmodified ABNF compiler generated parsers. I 

Okay, after thinking this through a bit, I have further thoughts:

Even if we escape CRLF, I think we do not solve the ABNF problem. We 
just make it easier to modify an ABNF compiled parser to look for the 
CRLF delimiter.

But, we went to a great deal of trouble to design the closing string in 
a way so that it was both highly unlikely to collide with the body, and 
also very efficient for parsing. We trade a little bit of development 
complexity in parsing for the closing for the simplicity of not 
escaping _anything_ in the body.

So, my opinion is thus: The work group has already spent a lot of time 
thinking about this, and reached a consensus as reflected in the 
current draft revision. We should not change it now, unless we find 
evidence that it is unworkable. I don't _think_ we have found any such 
evidence.

Thanks!

Ben.

---

From: "Peter Ridler" <[email-masked]>
Date: Tue, 19 Oct 2004 07:08:59 -0400

Ben,

OK on this point.  As I said I don't want to rake up old issues (but I wish
I were there!). You added in all my other ABNF changes, so, I will give on
this one.

Thanks
--Peter 

> Sent: Saturday, October 16, 2004 3:29 PM
> To: Ben Campbell
> Cc: Peter Ridler; Simple WG
> Subject: Re: [Simple] MSRP: Escaping in body (was Re: ABNF 
> Issues in draft-ietf-simple-message-sessions-08.txt)
> 
> >>>> "Content-Length".  We are already using SIP and have 
> this message 
> >>>> processing in the Transport Layer anyway.  This will 
> leave "data =
> >>> *OCTET" at the
> >>>> end
> >>> The work group chose to go with the delimiter. The reasoning for 
> >>> this is, it allows you to abort a request without 
> destroying framing 
> >>> for the connection. If you use a content-length, then you can't 
> >>> abort before you are finished.
> >>>
> >>> If we were strictly p2p, this would not be a big deal. 
> But we also 
> >>> want to allow relays, and have the relays multiplex multiple 
> >>> sessions over a connection with an adjacent relay. If framing is 
> >>> lost for that connection, it affects all of the sessions.
> >>>
> >> I don't see the difference through -- read X 
> (Content-Length) bytes 
> >> from a stream -- or -- read more or less bytes from a 
> stream and scan 
> >> the buffered data for "------- transact-id 
> continuation-flag CRLF", 
> >> and then re-sync the framing if I read too much data from 
> the stream 
> >> !!!! I would like it if Cullen or Rohan could enlighten me 
> why they 
> >> like the later (maybe on the side)...
> >
> >> If we keep to this message format then I really fell 
> strong that we 
> >> MUST escape the CR LF out of the data payload section of 
> the message 
> >> (I don't think the extra processing a big deal for this).
> >>
> >> Yes, I can fudge the ABNF Compiler with a special "data" 
> processing 
> >> rule that would handle this, but, I feel that the simple logic 
> >> constructs of ABNF keep the basic rules of logic in check 
> and if the 
> >> rules don't work, then, maybe there is a issue with the way we are 
> >> doing things.
> >>
> >
> Even if we escape CRLF, I think we do not solve the ABNF 
> problem. We just make it easier to modify an ABNF compiled 
> parser to look for the CRLF delimiter.
> 
> But, we went to a great deal of trouble to design the closing 
> string in a way so that it was both highly unlikely to 
> collide with the body, and also very efficient for parsing. 
> We trade a little bit of development complexity in parsing 
> for the closing for the simplicity of not escaping _anything_ 
> in the body.
> 
> So, my opinion is thus: The work group has already spent a 
> lot of time thinking about this, and reached a consensus as 
> reflected in the current draft revision. We should not change 
> it now, unless we find evidence that it is unworkable. I 
> don't _think_ we have found any such evidence.
> 
> Thanks!
> 
> Ben.
> 
>