Subject: Re: [Fwd: [SIP] Sip URL and escaped characters. RFC 2543 Section 2]

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 29 Apr 2000 14:37:21 -0400

Maybe time to summarize and see if there's an issue left:

- The host part of the URL cannot contain %, either as itself or as an
escape mechanism:

host          = hostname | IPv4address
hostname      = *( domainlabel "." ) toplabel [ "." ]
domainlabel   = alphanum | alphanum *( alphanum | "-" ) alphanum
toplabel      = alpha | alpha *( alphanum | "-" ) alphanum
alphanum      = alpha | digit

- For "token" elements in URLs, this is a bit tricky, since we have two
escape mechanisms that are intersecting:

  (1) the normal token definition, i.e., any letter, digit, and a large
number of symbols (-, #, $, &, %, ^, *, _, +, |, ~, ')

  (2) the rules for URLs, where some of these characters (&, +) are
reserved and thus need to be escaped if there's the possibility of
confusion with a separator. As far as I can tell, neither of them are
syntactically relevant for SIP URLs in this context (the & is only used
to separate 'header' items), so neither of them would need to be
escaped.

Thus, it might be better to define

other-param = pname [ "=" pvalue ]

pname     = 1*tokenchar
pvalue    = 1*tokenchar | quoted-string
tokenchar = alphanum | escaped | "!" | "'" | "*" | "." 
            | "^" | "_" | "~" | "|" | "%" | "-" | "#" | "/" | ":" | "@"
| "&" | "+" | "$"

(since "," is not valid in token, I didn't include it. Any that I'm
missing?)

Here, escaped can only refer to the legal characters (alphanum and the
others) to maintain "tokenness", but I can't express this in the BNF.
However, in practice this is not a problem except for spec-writers since
the pname and pvalue tokens have to be drawn from an enumeration, so
that most character combinations are invalid simply because they are not
valid parameter names.

- To make matters slightly more confusing, we do have a different
potential problem, namely the " in the quoted string. The <"> is a
"delims" character and thus must be escaped (RFC 2396, pg. 9/10). One
could argue that it only needs to be escaped if there's a possibility
for confusion, e.g., on a web page or if the URL appears in a quoted
string. However, either should be accepted by the parser.

- In URLs, %hex hex (where allowed) is always interpreted as an escape
mechanism. It is mechanically translated to the equivalent character,
while maintaining its distinct identity as a non-separator (if it could
be treated as such). Thus, 

[email-masked]

can't just be translated into foo@[email-masked] and then be parsed, since
the result would not be as expected. This is not fundamentally different
from other escape mechanisms for delimiters, like \" or "".

Thus, one should *first* split the URL into components and then
translate those parts where escaping is possible (which is pretty much
anywhere except in scheme and host).

- As was pointed out, BNF specifies syntax. It cannot express certain
things, including
  . restrictions on ordering  (an | does not imply that the elements
have to appear in order)
  . any restriction on duplication
  . the role of identifiers or restrictions on their names (e.g., "int
int" may be legal in a BNF specification but the compiler may still
complain)

  For details, see for example, pg. 172 of Aho/Sethi/Ullman:

  "Certain constraints on the input, such as the requirement that
identifiers be declared before they are used, cannot be described by a
context-free grammar."

If there are places where the grammar needs to be tightened up, let me
know.

-- 
Henning Schulzrinne   http://www.cs.columbia.edu/~hgs

_______________________________________________
SIP mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/sip