Subject: Re: [SIP] whitespace in SIP & SDP

From: [email-masked]
Date: Thu, 17 Aug 2000 17:24:23 +0530

That is correct :-)

(On a related note: A few days ago, someone pointed out an interesting case
of one line having a string without an ending quote, but slashed at the
end, the next line having an LWS and then some text and then the
terminating quote. - in which case I guess we cannot ignore the LWS
like we usually do for continuation lines". Just thought I'd re-mention
this interesting "torture" test wrt all our continuation line parsing
routines
we have in our implementations:-) )

Regds
Arjun

--
Arjun Roychowdhury @ Hughes Software Systems

[email-masked] (Richard Verhoeven) on 08/17/2000 03:19:02 PM

To:   [email-masked]
cc:

Subject:  Re: [SIP] whitespace in SIP & SDP

> SIP is lenient with LWSs.
> So typically one or more LWSes between/before/after tokens should be
> accepted.
> So the white spaces acceptance of SIP is implicit .
>
>
> From = {LWS}("From" | "f") ":" {LWS}(name-addr | addr-spec)
> but that is typically assumed .

I assume you mean:

From = ("From" | "f") {LWS} ":" {LWS}(name-addr | addr-spec)

The possible LWS at the beginning of a line will be interpreted
as a continuation newline for the previous line in the message.

Richard Verhoeven.

_______________________________________________
SIP mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/sip

_______________________________________________
SIP mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/sip

---

From: Gethin Liddell <[email-masked]>
Date: Thu, 17 Aug 2000 13:35:33 +0100

> we have in our implementations:-) )

do you mean, for example something like:

Header: headerStuff ;param="This is a
   wrapped quoted string"

i dont think we can really expect implementations to maintain the
quantity of white space within the quoted parameter.

Section C.1 (page 118 of bis-01)

"SIP header field values can be folded onto multiple lines if the
continuation line begins with a space or horizontal tab. All linear
white space, including folding, has the same semantics as SP. A
recipient MAY replace any linear white space with a single SP before
interpreting the field value or forwarding the message downstream."

whether it is in a quoted string or not i think is irrelevant.  it
still MAY be interpreted as a single white space.

-- 
Gethin Liddell
Ubiquity Software Corporation

http://www.ubiquity.net
mailto:[email-masked]

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 17 Aug 2000 23:24:22 -0400

> still MAY be interpreted as a single white space.

I'm not sure here. The rules in SIP regarding implicit LWS apply only to
between words (where words include tokens and quoted strings - the
entire quoted string, that is) and separators, and between tokens. The
space inside of a quoted string does not meet the definitions of
implicit LWS. 

However, I think you also would need to escape the CR and/or LF that
might appear, so that you can detect headers without parsing the headers
themselves.

-Jonathan R.

-- 
Jonathan D. Rosenberg                       72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.cs.columbia.edu/~jdrosen         PHONE: (732) 741-7244
http://www.dynamicsoft.com

---

From: [email-masked] (Richard Verhoeven)
Date: Fri, 18 Aug 2000 10:43:50 +0200 (MET DST)

> might appear, so that you can detect headers without parsing the headers

I'm not sure what you mean by this.  It seems that allowing an escaped
[CR] or [LF] in a string will lead to problems.  For example, how do you
interpret the next lines:

X-Organization:  "A conspiracy organisation \[LF]
X-From:  "Some alien \[LF]
From: outer space" <[email-masked]>[LF]

Within a string, the backslash works as an escape, so the [LF] is
not interpreted as a end of line and the next line is
part of a continued line, eventhough it doesn't start with
whitespace (if I interpret the ABNF grammar for quoted strings
correctly).  However, you can't determine whether the backslash
is part of a string or not, especially when extension fields
are used.  In the above example, the X-Organization field is
followed by an UTF8-Text, so double-quotes and backslashes are
not interpreted.  The X-From field contains a string, so the
backslash should be interpreted and the next line (starting
with "From: ") is part of string.  However, a parser will be
unable to determine whether a quoted-string or UTF8-text
is used, so it has to handle both cases similarly.

Therefore, the statement in Section C.1 (page 118 of bis-01),
as quoted above, should be stronger and state that the ONLY
way to continue a line is indenting the next line.  The backslash
at the end of a line should be regarded as an error if it appears
within a quoted string, which can be achieved by changing the
definition of quoted-pair to something link:

qouted-pair  = "\" <any CHAR except CR and LF>

Best regards,

Richard Verhoeven
EESI, Eindhoven

---

From: Gethin Liddell <[email-masked]>
Date: Fri, 18 Aug 2000 09:29:47 +0100

For SIP to be efficient and quick, as i'm sure you'll agree it must be,
it MUST be possible to parse in a message with out having to parse each
individual header.

Ignoring the \CR and \LF, it is possible to get the structure of the
messages and the headers split up by looking for the combination of
CR, LF and white space indentation. great, this can be done quickly by
simple byte-by-byte comparison.

This all goes out the window however when we allow, or expect, that in
`special cases' a `\' at the end of the line actually marks another
form of continuation.

for example, parsing the fast, efficient, lazy way, the header:

Waring: 307 foo "some info \
some more info"

Will be parsed as 2 headers, with the second header being very wrong
and breaking everything.

If we want this to be parsed properly, then from the very begining we
have to parse the header and understand it.

So what if we don't understand it?

eg.

foo: foodata "\
To: sip:a@b

how do we parse this?  we know nothing about the header.  is it a quoted
string, or does it just happen that the " is present?

this is impossible to parse.

I think the BNF in Section C.1 (page 119 bis-01) is wrong where it says:

quoted-pair = "\"CHAR

perhaps it should be:

quoted-pair = "\" (0x00-09 | 0x0B-0C | 0x0E-0x7F)

i.e. remove CR and LF from the quoted pair.

or should we go further and remove all control characters:

quoted-pair = "\" (0x20 - 0x7E | 0x09) (including SP and TAB)

thoughts?

http://www.ubiquity.net