Subject: [SIP] Sip URL and escaped characters. RFC 2543 Section 2

From: Gethin Liddell <[email-masked]>
Date: Wed, 26 Apr 2000 11:39:34 +0100

Hi,

I'm querying the idea of escaped characters in Sip Url's.
This is discussed in both the rfc and bis in Section 2.

The RFC says:

"Note that reserved characters have to be
   escaped and that the "set of characters reserved within any given URI
   component is defined by that component. In general, a character is
   reserved if the semantics of the URI changes if the character is
   replaced with its escaped US-ASCII encoding" [12]."

What I am querying is the use of this escaped notation within the hostname
of the SIP-URL.

The bnf in Section 2 Figure 3 defines the hostname to consist of only alphas,
numbers and `-' characters.  It does NOT allow for escaped characters to 
exist as no `%' char is allowed.

However, the bis draft provides an example at the end of Section 2.1 as follows:

"Thus, the following URLs are equivalent:

sip:juser@%65xample.com:5060
sip:[email-masked];Transport=UDp"

Thus we have a contradication to the bnf with the address "%65xample.com".  
It seems to me that there would be no cause to have escaped characters in the 
host name as it could never be resolved to an actual IP address.  And to have to 
revert the escaped char back to its original form is very inefficent as extra checks
will have to be made for each character to see if it is in the escaped form.  Plus the
actual escaped char may be reverted to an invalid char for a host.

The bnf does however allow escaped characters in the userinfo and header elements, 
but not in parameter values.

To summarize, as I understand it:

Escaped characters can only be used in the following elements:

1) user
2) password
3) hname
4) hvalue

and in these cases, when doing a comparison check, the escaped values MUST be
compared in their reverted (non-escaped form).

Escaped values MUST not be used in the following elements:
1) host
2) url-parameter

Please can you confirm and clarify this matter.

thanx

gethin

_______________________________________________
SIP mailing list
[email-masked]
http://lists.bell-labs.com/mailman/listinfo/sip

---

From: Igor Slepchin <[email-masked]>
Date: Wed, 26 Apr 2000 13:01:12 -0400

I agree that escaping characters in host name makes little sense as that
host name wouldn't be resolvable anyway; it is explicitly disallowed by
the BNF, and the example is inconsistent with the BNF as you correctly
noted. However, I don't think the spec prohibits escaped chars in url
parameters:

other-param     = ( token | ( token "=" ( token | quoted-string )))

and BNF of token allows '%' sign (explicitly in 2543bis).

---
Igor Slepchin

---

From: Gethin Liddell <[email-masked]>
Date: Thu, 27 Apr 2000 09:00:30 +0100

> and BNF of token allows '%' sign (explicitly in 2543bis).

I almost agree.

Yes, the `%' is allowed in tokens, but only as a character in its own right,
not as the start of an escaped value.  The format ("%" HEX HEX) of escaped 
values are valid but escaped values themselves are not explicity 
allowed or defined in a token.

Thus, if a url parameter has the value:

%61

is this considered to be the char `a' or the three characters `%' `6' and `1'

As the bnf stands, it would be the latter.

---

From: Igor Slepchin <[email-masked]>
Date: Thu, 27 Apr 2000 13:48:17 -0400

I think BNF is only supposed to define grammar (i.e., syntax), not
semantics. Thus, it's sufficient for the BNF to allow the % sign in
tokens; it's up to the application to interpret % HEX HEX as an escaped
char. I agree that the grammar would be cleaner and easier to parse if
the BNF for token listed "escaped" instead of '%' (otherwise you should
pay attention to things like %XYZ - not an escaped char).

---
Igor Slepchin

---

From: Gethin Liddell <[email-masked]>
Date: Fri, 28 Apr 2000 09:37:55 +0100

> pay attention to things like %XYZ - not an escaped char).

That is fundamentally incorrect.

The "escaped" BNF element is a lexeme in its own right.  The "token" BNF element
is a separate lexeme.  One does not exist in / is not part of the other.  During lexical
analysis, the format "% HEX HEX" will NOT be put into the "escaped" lexeme IF it
is analyzing a "token". 

i.e. The bnf DOES define the semantics.

or put another way, consider the tree representation of BNF. When parsing a token, 
the only lower nodes that exist are those of alphanum and  

"-"  "."  "!" "%" "*" "_" "+" "`" "'"

the "escaped" element does not exist below token so it knows nothing about it.  The two
are used in isolation to each other.

This is the only way the grammar can remain context-free.

If it was not context-free then it would be impossible to parse correctly as it would not be
known which context is being used.

---

From: Igor Slepchin <[email-masked]>
Date: Fri, 28 Apr 2000 16:00:48 -0400

I see what you are saying. I guess you might have a point here; however,
I'm still not sure that the BNFs in the SIP specs are (or even should
be) designed in a way that allows feeding them directly into a parser
generator and not worrying about anything else. Note that there are
still quite a few places in the spec where the behavior of the grammar
needs to be explained in words. In general, BNF has its limitations and
cannot describe everything, e.g., if you go by the book, you'll notice
that many header BNFs allow multiple instances of the same parameter,
even though this is often meaningless; a special wording is needed to
describe the use of brackets in To headers, etc. The combination of the
BNF and the description gives you the complete grammar and semantics, at
least currently.

With that said, it surely is nice to make BNF as expressive as possible.
Having a separate ecsape-token to describe tokens with % HEX HEX might
make the life of parser implementors somewhat easier. Does this mean the
current BNF is broken (or doesn't allow escaped chars in url-params,
which started this thread)? I don't believe so.

Regards,
Igor Slepchin

Gethin Liddell wrote:
> <...>