Subject: [core] draft-ietf-core-sid-00, SID allocation

From: peter van der Stok <[email-masked]>
Date: Mon, 14 Nov 2016 01:12:55 +0100

Dear YANG-SID authors,

I want to propose a change to the allocation of SIDs.
Currently, the SIDs are divided in ranges and prospective users can 
register a range with IANA.
When the range is already assigned, they need to select a new not 
allocated range.
I think that this will discourage many future SDOs who may want to use 
YANG + COMI. Many of these SDOs like to figure out the best number 
structure for their uses, and will be very disappointed when they cannot 
acquire the range. Actually, I believe, they will abandon the use of 
YANG + COMI.

My proposal is to assign numbers to modules, and let IANA handle the 
module number registration as proposed for the SIDs. The assignment of 
SIDs to YANG identifiers, as proposed in the draft remains, the same. 
The difference is the complete freedom to choose the SIDs in any given 
module. The advantage is that all modules can pick their values from the 
small number range.

The change is in the discovery and the structure of the resource path. 
In COMI I want to define another resource type called YANG module with 
name core.c.module. The discovery will return the path: 
/prefix/module-number-in-binary64. For example, with an empty prefix and 
for module 32, discovery will return /g. To retrieve a specific YANG 
instance with numeric identifier sid in module 32, the statement GET 
coap/example.com/g/sid will do. With two characters, modules with 
numbers < 4096 are covered; probably 3 characters will cover all 
modules.  Given that the SIDs are small, the total URI size will not 
increase due to this modification.

When the full datastore is accessed, the path /c is currently used. We 
can reserve c, meaning module number 28 is already assigned. Another 
method is returning a long path name, such as /whole_store. The size of 
the related URI is not important in this case. However, the proposed 
module allocation necessitates a small modification to the CBOR 
representation of the contents of the full datastore. This is attained 
by representing the full datastore as a CBOR map containing (key, value) 
pairs, where key is the module number and value is the content of the 
module as specified in yang-cbor document.

For the PATCH and the FETCH methods, this representation will also work, 
given the content-formats that are currently looked at.

I hope you like my proposal. The advantage is a simpler IANA 
administration, SID allocation freedom within a module, and shorter 
SIDs. The disadvantage is a slight complexity increase in the CBOR 
representation of the full datastore.

Hope this helps,

Peter

-- 
Peter van der Stok
vanderstok consultancy
mailto: [email-masked]
www: www.vanderstok.org
tel NL: +31(0)492474673     F: +33(0)966015248

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 14 Nov 2016 09:17:17 +0900

> On 14 Nov 2016, at 09:12, peter van der Stok <[email-masked]> wrote:
> 
> Dear YANG-SID authors,
> 
> Currently, the SIDs are divided in ranges and prospective users can register a range with IANA.
> When the range is already assigned, they need to select a new not allocated range.
> I think that this will discourage many future SDOs who may want to use YANG + COMI. Many of these SDOs like to figure out the best number structure for their uses, and will be very disappointed when they cannot acquire the range. Actually, I believe, they will abandon the use of YANG + COMI.

Well, the idea was that an SDO that is in the business of writing YANG modules gets a large amount of SID space they then can administer in any way they like.  So they are never in a position to have to go to IANA again to allocate specific parts of that SID space.

> My proposal is to assign numbers to modules, and let IANA handle the module number registration as proposed for the SIDs. The assignment of SIDs to YANG identifiers, as proposed in the draft remains, the same. The difference is the complete freedom to choose the SIDs in any given module. The advantage is that all modules can pick their values from the small number range.

Again, there is this complete freedom in the existing SID proposal, too, as long as the organization remembered to get a SID space in the first place.  (We’ll have to coax^w educate them to do this.)

Still, the idea of separating module numbers and intra-module IDs is worth exploring.

Grüße, Carsten

> The change is in the discovery and the structure of the resource path. In COMI I want to define another resource type called YANG module with name core.c.module. The discovery will return the path: /prefix/module-number-in-binary64. For example, with an empty prefix and for module 32, discovery will return /g. To retrieve a specific YANG instance with numeric identifier sid in module 32, the statement GET coap/example.com/g/sid will do. With two characters, modules with numbers < 4096 are covered; probably 3 characters will cover all modules.  Given that the SIDs are small, the total URI size will not increase due to this modification.
> 
> When the full datastore is accessed, the path /c is currently used. We can reserve c, meaning module number 28 is already assigned. Another method is returning a long path name, such as /whole_store. The size of the related URI is not important in this case. However, the proposed module allocation necessitates a small modification to the CBOR representation of the contents of the full datastore. This is attained by representing the full datastore as a CBOR map containing (key, value) pairs, where key is the module number and value is the content of the module as specified in yang-cbor document.
> 
> For the PATCH and the FETCH methods, this representation will also work, given the content-formats that are currently looked at.
> 
> I hope you like my proposal. The advantage is a simpler IANA administration, SID allocation freedom within a module, and shorter SIDs. The disadvantage is a slight complexity increase in the CBOR representation of the full datastore.
> 
> Hope this helps,
> 
> Peter
> 
> 
> -- 
> Peter van der Stok
> vanderstok consultancy
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: Andy Bierman <[email-masked]>
Date: Sun, 13 Nov 2016 17:17:22 -0800

Hi,

One question to ask:  How many modules do we expect to have?
How many objects do we expect to have?  In 30 years of writing MIB modules
we have never even come close to a million objects. (Maybe 100,000?)

A million modules, each with a million objects, would still be less than 40
bits.
So why to we need complex range assignments and non-contiguous numbering
within a module?  Presumably to preserve numbers and not waste them.
But since the SID encoding is based on deltas, and the full SID is only used
once per transaction, this does not seem to be a real concern.

Andy

On Sun, Nov 13, 2016 at 4:12 PM, peter van der Stok <[email-masked]>
wrote:

> acquire the range. Actually, I believe, they will abandon the use of YANG +
> COMI.
>
> module number registration as proposed for the SIDs. The assignment of SIDs
> to YANG identifiers, as proposed in the draft remains, the same. The
> difference is the complete freedom to choose the SIDs in any given module.
> The advantage is that all modules can pick their values from the small
> number range.
>
> The change is in the discovery and the structure of the resource path. In
> COMI I want to define another resource type called YANG module with name
> core.c.module. The discovery will return the path:
> instance with numeric identifier sid in module 32, the statement GET coap/
> example.com/g/sid will do. With two characters, modules with numbers <
> 4096 are covered; probably 3 characters will cover all modules.  Given that
> the SIDs are small, the total URI size will not increase due to this
> modification.
>
> When the full datastore is accessed, the path /c is currently used. We can
> reserve c, meaning module number 28 is already assigned. Another method is
> returning a long path name, such as /whole_store. The size of the related
> URI is not important in this case. However, the proposed module allocation
> necessitates a small modification to the CBOR representation of the
> contents of the full datastore. This is attained by representing the full
> datastore as a CBOR map containing (key, value) pairs, where key is the
> module number and value is the content of the module as specified in
> yang-cbor document.
>
> administration, SID allocation freedom within a module, and shorter SIDs.
> The disadvantage is a slight complexity increase in the CBOR representation
> of the full datastore.
>
> Hope this helps,
>
> Peter
>
>
> --
> Peter van der Stok
> vanderstok consultancy

---

From: Alexander Pelov <[email-masked]>
Date: Mon, 14 Nov 2016 10:45:02 +0900

Dear Peter, Andy,

Excellent points, thanks for bringing this up!

A major difference between SIDs and MIBs is that we cannot allocate prefixes in SIDs (as is the case with MIB-OID). The system works perfectly well - it is just NOT feasible for IANA to allocate ranges to individual modules on a global scale. It’s also not feasible for developers/companies to ask IANA for individual allocations per module (and anything else boils down to what we’re currently doing).

The SID draft functions under a Two-tier registration system. IANA -> SDOs/Registrars -> Modules. IANA allocates chunks of IDs to other SDOs/registrars. A module is then registered by these third parties and the particular SIDs get assigned to the individual modules.

Note, that IANA can also register SIDs for individual modules, e.g. for all YANG modules published by the IETF. I would suppose the same policy could apply to other SDOs.

Hope that this clarifies the question.

Best,
Alexander

> Le 14 nov. 2016 à 10:17, Andy Bierman <[email-masked]> a écrit :
> 
> Hi,
> 
> 
> A million modules, each with a million objects, would still be less than 40 bits.
> On Sun, Nov 13, 2016 at 4:12 PM, peter van der Stok <[email-masked] <mailto:[email-masked]>> wrote:
> Dear YANG-SID authors,
> 
> The change is in the discovery and the structure of the resource path. In COMI I want to define another resource type called YANG module with name core.c.module. The discovery will return the path: /prefix/module-number-in-binary64. For example, with an empty prefix and for module 32, discovery will return /g. To retrieve a specific YANG instance with numeric identifier sid in module 32, the statement GET coap/example.com/g/sid <http://example.com/g/sid> will do. With two characters, modules with numbers < 4096 are covered; probably 3 characters will cover all modules.  Given that the SIDs are small, the total URI size will not increase due to this modification.
> 
> mailto: [email-masked] <mailto:[email-masked]>
> www: www.vanderstok.org <http://www.vanderstok.org/>
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>
>

---

From: Andy Bierman <[email-masked]>
Date: Sun, 13 Nov 2016 17:58:37 -0800

On Sun, Nov 13, 2016 at 5:45 PM, Alexander Pelov <[email-masked]> wrote:

> A major difference between SIDs and MIBs is that we cannot allocate
> prefixes in SIDs (as is the case with MIB-OID). The system works perfectly
> well - it is just NOT feasible for IANA to allocate ranges to individual
> modules on a global scale. It’s also not feasible for developers/companies
> to ask IANA for individual allocations per module (and anything else boils
> down to what we’re currently doing).
>

You want to manually manage ranges.  Why?

Why not allocate module-ids with a fixed range of objects within a module?

e.g.,32 bits for modules,  20 bits for objects --> module ID 1 has objects
0x100000 to 0x1fffff.

What evidence do you have that we need more than 4 billion modules,
each with up to 1 million objects?

There is no operational experience with SIDs but there is plenty with
managed objects
in general.

Andy

>
> The SID draft functions under a Two-tier registration system. IANA ->
> SDOs/Registrars -> Modules. IANA allocates chunks of IDs to other
> SDOs/registrars. A module is then registered by these third parties and the
> particular SIDs get assigned to the individual modules.
>
> Note, that IANA can also register SIDs for individual modules, e.g. for
> all YANG modules published by the IETF. I would suppose the same policy
> could apply to other SDOs.
>
>> can reserve c, meaning module number 28 is already assigned. Another method
>> is returning a long path name, such as /whole_store. The size of the
>> related URI is not important in this case. However, the proposed module
>> allocation necessitates a small modification to the CBOR representation of
>> the contents of the full datastore. This is attained by representing the
>> full datastore as a CBOR map containing (key, value) pairs, where key is
>> the module number and value is the content of the module as specified in
>> yang-cbor document.
>>

---

From: Michel Veillette <[email-masked]>
Date: Mon, 14 Nov 2016 16:27:20 +0000

Hi Peter.

The allocation by range have been introduced to simplify the life of SDOs, not to make their life more difficult.
With your proposal, the SDO must interact with IANA for each module created.
With the allocation by ranges, the allocated range can be large enough to allow the SDO to be autonomous for multiple years of development, for dozen of modules.

Updating the different drafts to identify YANG items using (module ID, module item ID) is possible, but not with some impacts on message size and complexity.

Regards,
Michel

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of peter van der Stok
Sent: Sunday, November 13, 2016 7:13 PM
To: Core <[email-masked]>
Subject: [core] draft-ietf-core-sid-00, SID allocation

Dear YANG-SID authors,

My proposal is to assign numbers to modules, and let IANA handle the module number registration as proposed for the SIDs. The assignment of SIDs to YANG identifiers, as proposed in the draft remains, the same. 
The difference is the complete freedom to choose the SIDs in any given module. The advantage is that all modules can pick their values from the small number range.

The change is in the discovery and the structure of the resource path. 
In COMI I want to define another resource type called YANG module with name core.c.module. The discovery will return the path: 
/prefix/module-number-in-binary64. For example, with an empty prefix and for module 32, discovery will return /g. To retrieve a specific YANG instance with numeric identifier sid in module 32, the statement GET coap/example.com/g/sid will do. With two characters, modules with numbers < 4096 are covered; probably 3 characters will cover all modules.  Given that the SIDs are small, the total URI size will not increase due to this modification.

Hope this helps,

Peter

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Nov 2016 09:12:02 -0800

On Mon, Nov 14, 2016 at 8:27 AM, Michel Veillette <
[email-masked]> wrote:

> With the allocation by ranges, the allocated range can be large enough to
> allow the SDO to be autonomous for multiple years of development, for dozen
> of modules.
>
>
The interpretation of the number is completely independent of the
allocation of the number.  One could allocate a block of module IDs
just as easily as a block of object IDs.

The disconnect here is the number of available IDs vs. the number of
IDs needed.  You seem to think this is a precious resource.  I do not.

Andy

> Updating the different drafts to identify YANG items using (module ID,
> module item ID) is possible, but not with some impacts on message size and
> complexity.
>
> Regards,
> Michel
>
> to YANG identifiers, as proposed in the draft remains, the same.

---

From: Michel Veillette <[email-masked]>
Date: Mon, 14 Nov 2016 20:47:09 +0000

Hi Andy

=== About the size of the SID range allocated to each YANG module

The 20 bits range proposed is not incompatible with 'draft-ietf-core-sid'.
Each entity allocating SIDs (e.g. SDOs, End users) is free to define the size of the SID range allocated to each YANG module.

Do you propose to mandate a fixed range size (e.g. 20 bits / 1048576 IDs) ?

'draft-bierman-core-yid' (https://tools.ietf.org/html/draft-bierman-core-yid-00#section-2.1)
supports this concept of flexible range size (field 'local-bits'),
what will be the advantage to revert to a fixed rage size?

== About the capability to add an extra SID range to a YANG module

The solution can't impose a hard limit on the number of YANG items within a YANG module.
The ability of adding an extra SID range removes this hard limit and remove any range anxiety.

Regards,
Michel

From: core [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Sunday, November 13, 2016 8:59 PM
To: Alexander Pelov <[email-masked]>
Cc: Core <[email-masked]>
Subject: Re: [core] draft-ietf-core-sid-00, SID allocation

On Sun, Nov 13, 2016 at 5:45 PM, Alexander Pelov <[email-masked]<mailto:[email-masked]>> wrote:
Dear Peter, Andy,

e.g.,32 bits for modules,  20 bits for objects --> module ID 1 has objects 0x100000 to 0x1fffff.

There is no operational experience with SIDs but there is plenty with managed objects
in general.

Andy

Best,
Alexander

Le 14 nov. 2016 à 10:17, Andy Bierman <[email-masked]<mailto:[email-masked]>> a écrit :

Hi,

Andy

On Sun, Nov 13, 2016 at 4:12 PM, peter van der Stok <[email-masked]<mailto:[email-masked]>> wrote:
Dear YANG-SID authors,

The change is in the discovery and the structure of the resource path. In COMI I want to define another resource type called YANG module with name core.c.module. The discovery will return the path: /prefix/module-number-in-binary64. For example, with an empty prefix and for module 32, discovery will return /g. To retrieve a specific YANG instance with numeric identifier sid in module 32, the statement GET coap/example.com/g/sid<http://example.com/g/sid> will do. With two characters, modules with numbers < 4096 are covered; probably 3 characters will cover all modules.  Given that the SIDs are small, the total URI size will not increase due to this modification.

Hope this helps,

Peter

--
Peter van der Stok
vanderstok consultancy
mailto: [email-masked]<mailto:[email-masked]>
www: www.vanderstok.org<http://www.vanderstok.org/>
tel NL: +31(0)492474673     F: +33(0)966015248

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Nov 2016 13:31:01 -0800

On Mon, Nov 14, 2016 at 12:47 PM, Michel Veillette <
[email-masked]> wrote:

> Each entity allocating SIDs (e.g. SDOs, End users) is free to define the
> size of the SID range allocated to each YANG module.
>
>
>

I prefer 1 range per module, not N

> remove any range anxiety.

Your previous solution allocated 10 bits for schema nodes within a module.
We already have modules with more than 1024 objects. Most modules
have less than 300 nodes, so a limit of 1M nodes will never  be a real
restriction.

The real issue is multiple ranges per module.
If this is removed the rest of the details cancel each other out.

Andy

>
>
> *From:* core [mailto:[email-masked]] *On Behalf Of *Andy Bierman
> *Sent:* Sunday, November 13, 2016 8:59 PM
> *To:* Alexander Pelov <[email-masked]>
> *Cc:* Core <[email-masked]>
> *Subject:* Re: [core] draft-ietf-core-sid-00, SID allocation
>
>
>
>
>
>
>
> core.c.module. The discovery will return the path: /prefix/module-number-in-binary64.
> For example, with an empty prefix and for module 32, discovery will return
> /g. To retrieve a specific YANG instance with numeric identifier sid in
> module 32, the statement GET coap/example.com/g/sid will do. With two
> characters, modules with numbers < 4096 are covered; probably 3 characters
> will cover all modules.  Given that the SIDs are small, the total URI size
> will not increase due to this modification.
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 15 Nov 2016 06:44:47 +0900

I would think we can go for a standard size such as 2000 (2048?), with a larger size possible if the need for that can possibly occur.
That would leave multiple ranges only as an emergency for initial allocations that turned out way too small.
Is that a reasonable solution?

Grüße, Carsten

> On 15 Nov 2016, at 06:31, Andy Bierman <[email-masked]> wrote:
>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Nov 2016 14:02:44 -0800

On Mon, Nov 14, 2016 at 1:44 PM, Carsten Bormann <[email-masked]> wrote:

> I would think we can go for a standard size such as 2000 (2048?), with a
> larger size possible if the need for that can possibly occur.
> That would leave multiple ranges only as an emergency for initial
> allocations that turned out way too small.
I don't think that reduces any complexity if all tools still need to deal
with
multiple ranges per module.

If it took 30 years to get to 500K MIB objects, how long will it take to
get to 4 billion?  64 billion? Maybe never.

The cost of the larger id (e.g., 18 - 20 bits per module) seems to only
impact
the 1 anchor ID needed per exchange. I think the deltas work out the same.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Mon, 14 Nov 2016 22:49:25 +0000

Hi Andy

What are these

Hi Andy

What are those complexities that "all tools still need to deal with"?

Supporting multiple ranges of SIDs increase the complexity of a single tool, the tool used to assign them.
Supporting multiple ranges of SIDs have no impacts on the CoMI protocol.
The only requirement put on SIDs by the protocol is to be unique.

In the case of the pyang sid.py plugin, this complexity is implemented by the get_next_sid() function.
https://github.com/core-wg/yang-cbor/blob/master/sid.py
This function have 9 lines of code.
Is it real or just a perceive issue?

def get_next_sid(self, sid):
        sid += 1
        for i in range(len(self.content['assignment-ranges'])):
            if sid < self.content['assignment-ranges'][i]['entry-point'] + self.content['assignment-ranges'][i]['size']:
                return sid
            else:
                if i + 1 < len(self.content['assignment-ranges']):
                    if sid < self.content['assignment-ranges'][i+1]['entry-point']:
                        return self.content['assignment-ranges'][i+1]['entry-point']

        raise SidParcingError("SID range(s) exhausted, extend the allocation range or add a new one.")

Regards,
Michel

From: Andy Bierman [mailto:[email-masked]]
Sent: Monday, November 14, 2016 5:03 PM
To: Carsten Bormann <[email-masked]>
Cc: Michel Veillette <[email-masked]>; Core <[email-masked]>
Subject: Re: [core] draft-ietf-core-sid-00, SID allocation

On Mon, Nov 14, 2016 at 1:44 PM, Carsten Bormann <[email-masked]<mailto:[email-masked]>> wrote:
I would think we can go for a standard size such as 2000 (2048?), with a larger size possible if the need for that can possibly occur.
That would leave multiple ranges only as an emergency for initial allocations that turned out way too small.
Is that a reasonable solution?

I don't think that reduces any complexity if all tools still need to deal with
multiple ranges per module.

The cost of the larger id (e.g., 18 - 20 bits per module) seems to only impact
the 1 anchor ID needed per exchange. I think the deltas work out the same.

Grüße, Carsten

Andy

> On 15 Nov 2016, at 06:31, Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:
>

---

From: peter van der Stok <[email-masked]>
Date: Mon, 14 Nov 2016 23:51:05 +0100

Hi Michel,

One module per SDO, may be 5?
They can structure the module with submodules to their liking.

My point is about freedom to choose SID values.
For example, SDOs have a history to assign numbers to attribute types, 
device types, etc..
These numbers have a meaning in the group.
Using the same numbers for the SID values when appropriate may be 
welcome indeed.

Peter

Michel Veillette schreef op 2016-11-14 17:27:
> Hi Peter.
> 
> The allocation by range have been introduced to simplify the life of
> SDOs, not to make their life more difficult.
> With the allocation by ranges, the allocated range can be large enough
> to allow the SDO to be autonomous for multiple years of development,
> for dozen of modules.
> 
> module item ID) is possible, but not with some impacts on message size
> and complexity.
> 
> Regards,
> Michel
> 
> structure for their uses, and will be very disappointed when they
> cannot acquire the range. Actually, I believe, they will abandon the
> use of YANG + COMI.
> 
> module. The advantage is that all modules can pick their values from
> the small number range.
> 
> /prefix/module-number-in-binary64. For example, with an empty prefix
> and for module 32, discovery will return /g. To retrieve a specific
> YANG instance with numeric identifier sid in module 32, the statement
> GET coap/example.com/g/sid will do. With two characters, modules with
> method is returning a long path name, such as /whole_store. The size
> of the related URI is not important in this case. However, the
> proposed module allocation necessitates a small modification to the
> CBOR representation of the contents of the full datastore. This is
> attained by representing the full datastore as a CBOR map containing
> (key, value) pairs, where key is the module number and value is the
> content of the module as specified in yang-cbor document.
> 
> For the PATCH and the FETCH methods, this representation will also
> work, given the content-formats that are currently looked at.
>