Subject: Re: [Fwd: [SIP] Sip URL and escaped characters. RFC 2543 Section 2]

From: "Rick Workman" <[email-masked]>
Date: Mon, 01 May 2000 12:01:38 -0400

I hesitate to wade into this because things tend to get somewhat
philosophical, but I think there are some important underlying design
issues. In any protocol (or grammar) design, there's syntax and semantics.
Unfortunately, they're not always cleanly separated in the specification. As
noted, ABNF is pretty good notation for syntax but really wasn't designed to
express semantics. Unfortunately, designers often try to capture semantics
in ABNF because it's a concise and formal way of doing so *when it works*.
But it's a slippery slope, and you often wind up with an ambiguous or
unusable syntactic specification as a result.

The URL parameter is a good example of this. It appears that the
"other-param" rule completely captures the syntax. The various other
parameter rules (transport-param, user-param, etc.) really capture semantics
rather than syntax. Now if all parameter names and values were enumerated,
these "enumerating" rules would capture both syntax and semantics and
"other-param" wouldn't be necessary. So presumably it was included to
account for yet unforeseen future parameters where the syntax needs to be
generically specified, but the semantics (including names and values) aren't
yet known. However in doing so, the semantic checking is eliminated;
"transport=fuddle" is perfectly valid syntax, so the apparent semantic
constraint implied by the transport-param rule has been lost. Is this a
transport parameter with an illegal value or a yet to be defined "other"
parameter. A syntactic or semantic violation, or no violation at all?

My own preference would be to define a core SIP syntax which is captured in
the ABNF. (ABNF does have a few problems, but that's another discussion.)
The current grammar and any future extensions should be parsable using this
syntax specification, because there's no attempt to define semantics.
Barring a formal semantic notation, all semantics is captured in the text as
is currently done, e.g., the text describing the  From header states:

The SIP-URL MUST NOT contain the "transport-param", "maddr-param",
"ttl-param", or "headers" elements. A server that receives a SIP-URL with
these elements removes them before further processing.

Note that the semantics includes not only legal parameter names and values
in that context, but what should occur when the "semantics" is violated.

So I think a more rigorous syntactic definition of SIP which is separate
from semantics would lead to a more concise core syntax, a more
understandable spec, and possibly better implementation designs, since there
would be less tendency to enforce semantic constraints in parsers which
would get re-written whenever an extension is added. I note that SIP is no
better or worse than many of the other IETF protocols in this respect.
(Computer language designers seem to have done a better job on this probably
because there is a much clearer separation of syntax and semantics in their
domain.)

Rick Workman
Nortel Networks
Ottawa

Henning Schulzrinne wrote:

> Maybe time to summarize and see if there's an issue left:
>
> - The host part of the URL cannot contain %, either as itself or as an
> escape mechanism:
>
> host          = hostname | IPv4address
> hostname      = *( domainlabel "." ) toplabel [ "." ]
> domainlabel   = alphanum | alphanum *( alphanum | "-" ) alphanum
> toplabel      = alpha | alpha *( alphanum | "-" ) alphanum
> alphanum      = alpha | digit
>
> - For "token" elements in URLs, this is a bit tricky, since we have two
> escape mechanisms that are intersecting:
>
>   (1) the normal token definition, i.e., any letter, digit, and a large
> number of symbols (-, #, $, &, %, ^, *, _, +, |, ~, ')
>
>   (2) the rules for URLs, where some of these characters (&, +) are
> reserved and thus need to be escaped if there's the possibility of
> confusion with a separator. As far as I can tell, neither of them are
> syntactically relevant for SIP URLs in this context (the & is only used
> to separate 'header' items), so neither of them would need to be
> escaped.
>
> Thus, it might be better to define
>
> other-param = pname [ "=" pvalue ]
>
> pname     = 1*tokenchar
> pvalue    = 1*tokenchar | quoted-string
> tokenchar = alphanum | escaped | "!" | "'" | "*" | "."
>             | "^" | "_" | "~" | "|" | "%" | "-" | "#" | "/" | ":" | "@"
> | "&" | "+" | "$"
>
> (since "," is not valid in token, I didn't include it. Any that I'm
> missing?)
>
> Here, escaped can only refer to the legal characters (alphanum and the
> others) to maintain "tokenness", but I can't express this in the BNF.
> However, in practice this is not a problem except for spec-writers since
> the pname and pvalue tokens have to be drawn from an enumeration, so
> that most character combinations are invalid simply because they are not
> valid parameter names.
>
> - To make matters slightly more confusing, we do have a different
> potential problem, namely the " in the quoted string. The <"> is a
> "delims" character and thus must be escaped (RFC 2396, pg. 9/10). One
> could argue that it only needs to be escaped if there's a possibility
> for confusion, e.g., on a web page or if the URL appears in a quoted
> string. However, either should be accepted by the parser.
>
> - In URLs, %hex hex (where allowed) is always interpreted as an escape
> mechanism. It is mechanically translated to the equivalent character,
> while maintaining its distinct identity as a non-separator (if it could
> be treated as such). Thus,
>
> [email-masked]
>
> can't just be translated into foo@[email-masked] and then be parsed, since
> the result would not be as expected. This is not fundamentally different
> from other escape mechanisms for delimiters, like \" or "".
>
> Thus, one should *first* split the URL into components and then
> translate those parts where escaping is possible (which is pretty much
> anywhere except in scheme and host).
>
> - As was pointed out, BNF specifies syntax. It cannot express certain
> things, including
>   . restrictions on ordering  (an | does not imply that the elements
> have to appear in order)
>   . any restriction on duplication
>   . the role of identifiers or restrictions on their names (e.g., "int
> int" may be legal in a BNF specification but the compiler may still
> complain)
>
>   For details, see for example, pg. 172 of Aho/Sethi/Ullman:
>
>   "Certain constraints on the input, such as the requirement that
> identifiers be declared before they are used, cannot be described by a
> context-free grammar."
>
> If there are places where the grammar needs to be tightened up, let me
> know.
>
> --
> Henning Schulzrinne   http://www.cs.columbia.edu/~hgs
>
> _______________________________________________
> SIP mailing list
> [email-masked]
> http://lists.bell-labs.com/mailman/listinfo/sip

---

From: Rohan Mahy <[email-masked]>
Date: Mon, 01 May 2000 10:11:45 -0700

Hi,

I just want to make sure that we can handle/describe the escaping of a 
nested monster such as this:  (sorrry for the wrapping at hyphens)

<a 
href="<sip:[email-masked]:5060;method=TRANSFER?Transfer-To=<sip: 
[email-masked]:5060;user=phone?Request-Disposition=no-fork&Accep 
t-Contact=*;mobility="mobile",*;media="audio">&Request-Disposition=parallel>">

this would be a SIP URI inside an HTML page, that asks userA's "Butler" 
(wherever he/she might be) to call (presumably userA's) cell phone.

this particular example may be a bit contrived, but i can come up with 
legitimate uses of this kind of functionality.

thanks,
-rohan

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 02 May 2000 01:01:08 -0400

You definitely want to handle things like this. There are many
innovative services that are enabled by placing URLs like this in web
pages, Contact headers in redirects, or inside of instant messages.

-Jonathan R.

-- 
Jonathan D. Rosenberg                       72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (732) 741-4778
http://www.cs.columbia.edu/~jdrosen         PHONE: (732) 741-7244
http://www.dynamicsoft.com