Subject: Re: [lp-wan] Field Length in schc

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 3 Oct 2017 11:27:39 +0000

Hello Ana

From: lp-wan [mailto:[email-masked]] On Behalf Of Ana Minaburo
Sent: vendredi 29 septembre 2017 12:17
To: Arun <[email-masked]>
Cc: [email-masked]
Subject: Re: [lp-wan] Field Length in schc

Hello,

I think that the best solution is the one that sends less bits
[Pascal] Hello Ana: sending bits over wire is probably OK. Sending additional bits over the air is not. But here, I think we are talking about the former so we do not want to create too much complexity and bug / error prone protocols. The whole discussion started because the current spec was not specific enough and lead to different understandings on how the length was obtained. Now we want an unambiguous clarity, knowing that  modules (e.g. Network GWs)  may need this spec to learn in run time how to compress for new types of devices. I think unambiguous comes first, before raw amount of bits over the (real) wire or in memory.

Proposal 1: use more memory in the devices but sends the actual bits: length bits plus LSB value

Proposal 2: is similar to the first one, you are not change the actual number of bits

Proposal 3: instead of LSB bits you are sending the complete value, for instance you are not sending the length bits (4, or 8, or 16)

Proposal 4: you are eliminating the length bits you send and you are sending the LSB bits.

So for me the best for the transmission view if the proposal 4

[Pascal] Pont taken but are we certain that the resulting bit encoding is better? Until we have numbers, my vote is for the actual length when we have it, and a function that explains how it is obtained otherwise.

Take care,

Pascal

Ana

On 26 Sep 2017, at 21:00, Arun <[email-masked]<mailto:[email-masked]>> wrote:

Hello all,
following are some proposals which you can vote to choose one, or can propose a better one :)
"Proposal 1:
add a length column:
    - actual length to represent static fields.
    -  -1 to represent variable length fields."
"Proposal 2:
add a length column:
      - ""variable"" to represent variable length fields
      - ""static"" to represent static fields, length here is obtained with the help from parser."
"Proposal 3:
 no extra column:
      - use check_len(x) as MO, where x is the length of the variable field
      - static fields, length here is obtained with the help from parser.
      - there is a limitation to use LSB with MSB as MO for variable length field since we don't know the msb_val."
"Proposal 4:
 no extra column:
     - static fields, length here is obtained with the help from parser.
     - check_len(x),for variable length fields.
     - msb_check(x,y) for variable length fields to use LSB. x can be length, and y be msb_val."

thanks,

--

Arunprabhu Kandasamy

arun at ackl.io<http://ackl.io/>

2 bis rue de la Châtaigneraie

35510 Cesson-Sévigné

France

Phone: +33 (0)2.99.12.24.14

On 25/09/2017 09:55, Arun wrote:
Hi Ana,
 It becomes trickier when we use LSB without argument. In that case, check_length overloads the MSB MO with additional argument specifying the msb bits?

thanks,
arun

On 15/09/2017 10:22, Ana Minaburo wrote:
Hello LPWANers,

I understand why you need the length column, but I’m not sure that adding a column is the best way to solve this problem. Correct me if I’m wrong you need the length in order to get the size of the variable fields of the CoAP header as path, but this length is used only for this variable fields the rest of the header fields are known, with the idea of creating a column with the length values you will repeat this known values as much times as rules you have and it will use memory.

I propose the following: if we need to have the length of this variable fields, we create a Matching Operator that gives this information, for example MO called check-length(x) so when you need to read your field you know that you need to read x bits.

rule id 1
+-------------------+------+-------+----------------+-----------------------+----------------+
|         Field        | dir   | TV    |MO               |CDF                     |  Sent           |
+===========+===+====+=========+=============+==========+
|CoAP version  | bi     |    01  |    equal         | not-sent              |                   |
|CoAP Type      | bi    |          |   ignore         | value-sent           |TT              |
|CoAP TKL       | bi    |  0       |   equal          | not-sent              |                    |
|CoAP Code     | bi    | ML1  | match-map   | matching-sent      | CC CCC    |
|CoAP MID       | bi    |  0000  |  MSB(7 )      | LSB(9)                |        M-ID   |
|CoAP Uri-Path | dn  |           | check_length(24)  | value-sent |      XXX        |
+-------------------+-----+--------+-----------------+-----------------------+----------------+

Ana

Ana Minaburo
Research Division
Phone: +33 (0)2.99.12.24.14
ana at ackl.io<http://ackl.io/>

2 bis rue de la Châtaigneraie
35510 Cesson-Sévigné
France

www.acklio.com<http://www.acklio.com/>

On 13 Sep 2017, at 10:38, Laurent Toutain <[email-masked]<mailto:[email-masked]>> wrote:

Hi,
thanks Arun,
That's a good idea to  add an entry in the rule for length, especially for CoAP compression.

In my views the description of the entry can be:
The Field Length entry contains either the size in bit of the field or a keyword indicating the way the length is defined:
- variable: indicates that the field length is variable and is coded as described in section 2.5
- static : indicates that the compressor and the decompressor learn the length by another means.
The static is useful for the Token field. The size is variable, but there is no need to send it since the Token Length field indicates this length.
Laurent

On Tue, Sep 12, 2017 at 6:09 PM, Arun <[email-masked]<mailto:[email-masked]>> wrote:
Hello All,
 Sending again to restart the discussion in the ML after today's interim meeting.
somethings which arose during the call:
 -  We need some sensible way to represent the variable length. 0 doesn't seem good as Pascal pointed out that there can be fields of 0 length.
 -  Do we need  to define the type for the length , so that we can represent it in json or some other format.
 -  As type int and -1 value could be also used, as suggested by Alex.
any thoughts?

thanks,

On 03/09/2017 22:00, Arun wrote:
Hello All,
We need a column/field in the context to represent the length of the field.
For the fixed length field the actual length in bits shall be used and '0' shall be used to indicate variable length fields with the length being sent on wire.
Ref: https://tools.ietf.org/html/draft-ietf-lpwan-ipv6-static-context-hc-05, Section:6

"The receiver must be able to find the size of each compressed field which can be given by the

   rule or may be sent with the compressed header

thanks,

--

Arunprabhu Kandasamy

arun at ackl.io<http://ackl.io/>

2 bis rue de la Châtaigneraie

35510 Cesson-Sévigné

France

Phone: +33 (0)2.99.12.24.14<tel:02%2099%2012%2024%2014>

_______________________________________________

lp-wan mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/lp-wan

--

Arunprabhu Kandasamy

arun at ackl.io<http://ackl.io/>

2 bis rue de la Châtaigneraie

35510 Cesson-Sévigné

France

Phone: +33 (0)2.99.12.24.14<tel:02%2099%2012%2024%2014>

_______________________________________________
lp-wan mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lp-wan

_______________________________________________
lp-wan mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lp-wan

_______________________________________________

lp-wan mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/lp-wan

--

Arunprabhu Kandasamy

arun at ackl.io<http://ackl.io>

2 bis rue de la Châtaigneraie

35510 Cesson-Sévigné

France

Phone: +33 (0)2.99.12.24.14

_______________________________________________

lp-wan mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/lp-wan

--

Arunprabhu Kandasamy

arun at ackl.io<http://ackl.io/>

2 bis rue de la Châtaigneraie

35510 Cesson-Sévigné

France

Phone: +33 (0)2.99.12.24.14
_______________________________________________
lp-wan mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lp-wan

---

From: Laurent Toutain <[email-masked]>
Date: Sat, 7 Oct 2017 18:49:39 +0200

Hi

Sorry for my late reply but this is not an easy question :-).

For the length we want to solve the following problem. if a variable
length field has a fix size in a specific rule (for example an URI-path is
always 3 byte long), we can save 4 bits in the air if the
length not have to be sent before the value.

If we add a column in the rule with the length, we can either specify that
a Field is variable or have a fixed length
If we don't add a column, we have to add this property in the MO arguments.
We can add a check_length MO which
replace "ignore" and test the field size.

The problem comes with MSB, if we have for example a query k=XXXX, and XXXX
must be 4 byte longs, we need to
create a 2-argument MO "check_length_MSB (L, V)" with L specifying the
length and V the first bits to check.

In my point of view the question can be summarized on how we can represent
in the rule
arguments to MO. Until now we have only 1 arg for MSB, so the rule looks
like:

Currently we have something like this

*  |Fid|Pos|dir|TV|MO|arg|CDF|arg*

*  |   |   |   |  |  | X |   |  *

If we want to add another arguments, we have two solutions.

Either add a column

*  |Fid|Pos|dir|TV|MO|arg1|arg2|CDF|arg*
*  |   |   |   |  |  | X  | Y  |   |*

or in the same column:
*  |Fid|Pos|dir|TV|MO|arg|CDF|arg*
*  |   |   |   |  |  |X,Y|   |   |*

The first approach is less flexible but easier to manage since it is quite
easy to access separately to each arguments.
The second is more flexible since the number of arguments is not limited,
but it is a little bit more complex to access
to a specific argument.

To try do to math, if N is the number of columns without arguments columns
and M the number of argument columns, the number of
bits to access a argument will be log2(N+M) in the first solution and in
the other log2(N)+log2(M).

So the second solution takes more bits for managements.

Therefore the best solution depends of what we want to optimize:
- evolution of the protocol. If we want to implement new MO with several
arguments, the second solution is better
- on the air they are equivalent, except for rule management where the
second one may add 1 byte or 2 over the air.
For memory usage on device, they seems to be quite equivalent.

So my answer will be to create one column for length because it is
something that could be useful to
optimize uri-path and especially uri-query and stop here.
Length and values are independent so it seems logical to have it.

If we find in the future some values that need to be added to the rule,
will can add them in the argument list using the
second approach (list of arg in the arg column).

we will have

*  |Fid  |Pos|dir|Length|TV|MO |arg|CDF|arg*
*  |path | 1 |   |24    |  |ign|   |v-s|   |*
*  |path | 2 |   |var   |  |ign|   |v-s|   |*
*  |query|   |   |32    |K=|MSB|16 |LSB|   |*

Laurent

> length bits plus LSB value

agree, we add a column with length and keep using MSB with one argument

> Proposal 3: instead of LSB bits you are sending the complete value, for
> instance you are not sending the length bits (4, or 8, or 16)
>

no, for the same reason

> Proposal 4: you are eliminating the length bits you send and you are

yes, but we have to represent it in the rule

> On 26 Sep 2017, at 21:00, Arun <[email-masked]> wrote:
>
> Hello all,
>  -1 is a convention, we can
also use names as variable and number for direct length. We need also to
sole the token field case which
do not enter in those 2 categories. We don't need to send its length since
another field contains it and token is
not a static field.

It is the same for MO of CDF we used a name but it can be converted into
numbers in implementations.

>  This is not really an optimization.

>       - there is a limitation to use LSB with MSB as MO for variable
> length field since we don't know the msb_val."
>
>
no, if we add a length columns, check_length is implicitly made on each
field, MSB is not changed.

> no, but we need to be aware that some MO may need several arguments in the
future and the current standard
must allow it.

> arun at ackl.io
> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
>
>
>  It becomes trickier when we use LSB without argument. In that case,
> check_length overloads the MSB MO with additional argument specifying the
> msb bits?
>
>
> thanks,
> arun
>
> I understand why you need the length column, but I’m not sure that adding
> a column is the best way to solve this problem. Correct me if I’m wrong you
> need the length in order to get the size of the variable fields of the CoAP
> header as path, but this length is used only for this variable fields the
> rest of the header fields are known, with the idea of creating a column
> with the length values you will repeat this known values as much times as
> rules you have and it will use memory.
>
> I propose the following: if we need to have the length of this variable
> fields, we create a Matching Operator that gives this information, for
> example MO called check-length(x) so when you need to read your field you
> know that you need to read x bits.
>
> rule id 1
> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
> ana at ackl.io
>
> www.acklio.com
> On 13 Sep 2017, at 10:38, Laurent Toutain <laurent.toutain@imt-
> atlantique.fr> wrote:
>
> Hi,
>
> thanks Arun,
>
> *The Field Length entry contains either the size in bit of the field or a
> keyword indicating the way the length is defined:*
>
> *- variable: indicates that the field length is variable and is coded as
> described in section 2.5*
>
> *- static : indicates that the compressor and the decompressor learn the
> length by another means.*
> The static is useful for the Token field. The size is variable, but there
> is no need to send it since the Token Length field indicates this length.
>
> Laurent
>
> On Tue, Sep 12, 2017 at 6:09 PM, Arun <[email-masked]> wrote:
>
>> Hello All,
>>  -  We need some sensible way to represent the variable length. 0 doesn't
>> seem good as Pascal pointed out that there can be fields of 0 length.
>>  -  Do we need  to define the type for the length , so that we can
>> represent it in json or some other format.
>> For the fixed length field the actual length in bits shall be used and
>> '0' shall be used to indicate variable length fields with the length being
>> sent on wire.
>>
>> arun at ackl.io
>> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
>>
>>
>>
>> lp-wan mailing [email-masked]://www.ietf.org/mailman/listinfo/lp-wan
>>
>>
>> --
>> Arunprabhu Kandasamy
>> arun at ackl.io
>>
>> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
>>
>>
>> [email-masked]
> [email-masked]
> lp-wan mailing [email-masked]://www.ietf.org/mailman/listinfo/lp-wan
>
>
> --
> Arunprabhu Kandasamy
> arun at ackl.io
>
> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
>
>
>
> lp-wan mailing [email-masked]://www.ietf.org/mailman/listinfo/lp-wan
>
>
> --
> Arunprabhu Kandasamy
> arun at ackl.io
>
> Phone: +33 (0)2.99.12.24.14 <02%2099%2012%2024%2014>
>
> [email-masked]
> [email-masked]