Subject: [core] #4: URI Encoding

From: "core issue tracker" <[email-masked]>
Date: Fri, 11 Jun 2010 09:31:46 -0000

#4: URI Encoding
--------------------------------+-------------------------------------------
 Reporter:  zach@…              |       Owner:  zach@…            
     Type:  enhancement         |      Status:  new               
 Priority:  major               |   Milestone:                    
Component:  coap                |     Version:                    
 Severity:  -                   |    Keywords:                    
--------------------------------+-------------------------------------------
 The working group needs to consider the encoding of URIs in order to save
 overhead. The WG needs to converge on an approach to integrate in coap-01.

 The following approaches have been suggested so far:

 - Encode the URI with 7-bit ASCII encoding.
 - Encode the URI with Base64 encoding (requires mapping table).
 - Encode the URI with Ascii85 encoding (works on blocks of 5 characters)
 - Define a custom ASCII encoding only for valid URI characters.

 There is an obvious tradeoff of space savings vs. complexity in these
 approaches. At the minimum 7-bit ASCII encoding is doable.

 Furthermore, the Uri-code option was dropped in favor of integrating
 "short" URIs into the normal URI string option. Here is one proposed
 approach from Carsten:

 - We could do an ASCII form of at least 4000 or so using /.xy or /!xy
 style shortcuts (essentially encoding the "codedness" in the first ASCII
 character).

-- 
Ticket URL: <https://wiki.tools.ietf.org/wg/core/trac/ticket/4>
core <http://tools.ietf.org/core/>

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Fri, 11 Jun 2010 16:52:25 +0200

I've two different proposals on this topic:

1) Compressed URI using a dictionary:

Exploiting the fact the URI is a 7-bit ASCII, we can represent most
frequent URI characters (regex format: [0-9a-zA-Z/]) using a 6 bit
encoding, a NULL (\x00) octet signals that an uncompressed character
is following. Every 3 compressed chars we have 6 unused bits that can
be used to represent another character (0.75 ratio if characters are
all from the dictionary).

Maybe some predefined dictionaries can be agreed, and the used
dictionary can be chosen in a preceding field (another example:
[a-zA-Z%+._/\|=&?-], etc.)

Reference: http://www.ietf.org/mail-archive/web/core/current/msg00166.html

2) Binary-URI encoding:

Binary-URI is a compressed mapping of the URI:
* It is N bytes long, where N is the depth of the resource in the URI. example:

(a) /document/letter/order.html DEPTH: 3
(b) / DEPTH: 0
(c) /index.html DEPTH: 1

* byte K is equal to the sum modulo 256 of ASCII values forming the
K-level resource name in the tree:

(a) document = 95, letter = 144, order.html = 255. So Binary-URI = {
95, 144, 255 } (c-style representation)
(b) no byte so Binary-URI = {}
(c) index.html = 251. Binary-URI = { 251 }

Here is the code of the program used to compute the values:

sum.c
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char* argv[]) {
       if (argc==2) {
               unsigned char* p = argv[1];
               int sum=0;
               do {
                       sum+=*p;
               } while (*(++p));
               printf("word: \"%s\" sum: %d\n", argv[1], sum&0xff);
       }
}

Resources created on a CoAP server should not collide with each-other,
to achieve this when we try to register a new resource the CoAP server
should check if it is colliding with other available resources and
accept the request only if this condition is satisfied.

Reference: http://www.ietf.org/mail-archive/web/core/current/msg00133.html

In my opinion both are useful, because (1) is useful when a CoAP node
wants to access a resource which cannot be represented in Binary-URI
format (CoAP node -> HTTP server on the internet); (2) can be used
when a CoAP server (a node) provides only non-colliding resources and
thanks to this its URI can be represented using the very short and
hierarchical Binary-URI format.

Best,
Angelo

On Fri, Jun 11, 2010 at 11:31, core issue tracker <[email-masked]> wrote:
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: "Stuber, Michael" <[email-masked]>
Date: Fri, 11 Jun 2010 12:00:19 -0700

I like #1.  I recognize that #2 may be useful, but it seems it would force you into walking a URI structure on a given server.  Put another way, let's say I have a resource like /machine/pump/flowstate.  (I recognize you may want to use shorter names, but I just want something readable for this example.)  Now I have several nodes that each offer this resource.  If I read #2 correctly, the beautiful thing is that I could compress this URI to a mere three bytes (woot!), but I have no reason to believe it could, would, or should, be consistent between these nodes.  Thus, it would seem (unless I have misunderstood), that I would have to walk the resource tree of each of these nodes to find where /machine/pump/flowstate is.  With option #1, I could still get some compression, and, should I choose to use fixed URIs, avoid navigating the resource tree.

Or have I misunderstood the potential implications of #2?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Angelo P. Castellani
Sent: Friday, June 11, 2010 7:52 AM
To: core issue tracker
Cc: core
Subject: Re: [core] #4: URI Encoding

Exploiting the fact the URI is a 7-bit ASCII, we can represent most frequent URI characters (regex format: [0-9a-zA-Z/]) using a 6 bit encoding, a NULL (\x00) octet signals that an uncompressed character is following. Every 3 compressed chars we have 6 unused bits that can be used to represent another character (0.75 ratio if characters are all from the dictionary).

Maybe some predefined dictionaries can be agreed, and the used dictionary can be chosen in a preceding field (another example:
[a-zA-Z%+._/\|=&?-], etc.)

2) Binary-URI encoding:

* byte K is equal to the sum modulo 256 of ASCII values forming the K-level resource name in the tree:

(a) document = 95, letter = 144, order.html = 255. So Binary-URI = { 95, 144, 255 } (c-style representation)
(b) no byte so Binary-URI = {}
(c) index.html = 251. Binary-URI = { 251 }

sum.c
#include <stdio.h>
#include <stdlib.h>

Resources created on a CoAP server should not collide with each-other, to achieve this when we try to register a new resource the CoAP server should check if it is colliding with other available resources and accept the request only if this condition is satisfied.

In my opinion both are useful, because (1) is useful when a CoAP node wants to access a resource which cannot be represented in Binary-URI format (CoAP node -> HTTP server on the internet); (2) can be used when a CoAP server (a node) provides only non-colliding resources and thanks to this its URI can be represented using the very short and hierarchical Binary-URI format.

Best,
Angelo

On Fri, Jun 11, 2010 at 11:31, core issue tracker <[email-masked]> wrote:
> #4: URI Encoding
> --------------------------------+-------------------------------------
> --------------------------------+------
>  Reporter:  zach@...              |       Owner:  zach@...
> --------------------------------+-------------------------------------
> --------------------------------+------

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 11 Jun 2010 15:31:00 -0400

Two remarks:

- see http://www.ietf.org/id/draft-ietf-iri-3987bis-00.txt - it would seem advisable to at least consider this ahead of time
- see the Punycode translation for DNS for the translation (this is not a compression)

Henning

---

From: Paul Duffy <[email-masked]>
Date: Fri, 11 Jun 2010 18:08:33 -0400

DEFLATE appears to be popping up quite frequently re: various 
constrained node capabilities.

Viable for URIs?

On 6/11/2010 10:52 AM, Angelo P. Castellani wrote:
> #include<stdio.h>
> #include<stdlib.h>
> On Fri, Jun 11, 2010 at 11:31, core issue tracker<[email-masked]>  wrote:
>    
>> #4: URI Encoding
>> Ticket URL:<https://wiki.tools.ietf.org/wg/core/trac/ticket/4>
>> core<http://tools.ietf.org/core/>
>>

---

From: "core issue tracker" <[email-masked]>
Date: Tue, 29 Jun 2010 07:35:52 -0000

#4: URI Encoding
--------------------------------+-------------------------------------------
 Reporter:  zach@…              |        Owner:  zach@…            
     Type:  enhancement         |       Status:  closed            
 Priority:  major               |    Milestone:                    
Component:  coap                |      Version:                    
 Severity:  -                   |   Resolution:  fixed             
 Keywords:                      |  
--------------------------------+-------------------------------------------
Changes (by zach@…):

  * status:  new => closed
  * resolution:  => fixed

Comment:

 After discussion and contributions on the mailing list, this ticket can be
 closed for now like this for coap-01:

 - No special URI encoding as recommended in draft-bormann-coap-misc-01

 - Servers can already assign static short URIs for their resources. This
 doesn't require anything new to the coap-00 mechanisms. However the
 resource description should include a field (sh=) to indicate such an
 alternative static URI exists.

 - The concept of TeRIs proposed in draft-bormann-coap-misc-01 should be
 experimented with first before considering integration with the base
 document.

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/core/trac/ticket/4#comment:1>
core <http://tools.ietf.org/core/>