Subject: [IPFIX] basicList clarification

From: Gerald Naveen A <[email-masked]>
Date: Sat, 12 Dec 2015 13:59:57 +0530

Hi all,

It is my understanding from the RFCs that basicList type is an encoding for
list of IPFIX Information Elements.

To create the template much more strongly typed, we have defined a custom
(enterprise) Information Element with a new ID (say X), which is defined as
a basicList of specific items (eg., InterfaceName).

My question to you all is: Is it mandatory that we have a separate
information element called InterfaceName (say ID Y) and define X as a
basicList of Y ? For our need, it is enough for us, that we define X as a
basicList of string (as X is already strongly defined). But string is a
type, not info element.

We would like to retain X (instead of defining a template directly with a
standard basicList). And we don't have any other need to define a new
Information Element Y.

I realize that the basicList encoding has a information-element ID of the
item in payload -- is this mandatory? Or for a strongly typed item X, we
could expect the IPFIX parser to be aware of contents (interface name) by
contract?

Pls let me know if my question is unclear.

Thanks in advance.
- Gerald Naveen

---

From: Paul Aitken <[email-masked]>
Date: Tue, 15 Dec 2015 13:14:20 +0000

Gerald,

> Hi all,
>
> It is my understanding from the RFCs that basicList type is an 
> encoding for list of IPFIX Information Elements.

Yes, where the list contains zero or more instances of a single IE.

> To create the template much more strongly typed, we have defined a 
> custom (enterprise) Information Element with a new ID (say X), which 
> is defined as a basicList of specific items (eg., InterfaceName).
>
> basicList of Y ?

Yes, that's how the protocol is defined.

> For our need, it is enough for us, that we define X as a basicList of 
> string (as X is already strongly defined). But string is a type, not 
> info element.
>
> We would like to retain X (instead of defining a template directly 
> with a standard basicList). And we don't have any other need to define 
> a new Information Element Y.

Y defines the list elements which tells the Collector that the list 
contains strings, and it defines the semantics of those strings - eg 
whether they contain interface names, user names, a list of 
selectorNames, or a list of Application names. Then a basicList of Y 
tells the Collector that you're exporting zero or more of them.

Perhaps you should define Y as a single instance of X (so Y is well 
defined), then redefine X as a basicList of Y. Or if you don't want/need 
to define both X and Y, then use the basicList IE #291 to export a 
basicList of (strongly defined) Y.

> I realize that the basicList encoding has a information-element ID of 
> the item in payload -- is this mandatory?

Yes. The encoding would not be interoperable without it.

> Or for a strongly typed item X, we could expect the IPFIX parser to be 
> aware of contents (interface name) by contract?

No. Every collector which implements RFC 6313 (IPFIX Structured Data) 
should be able to decode a basicList of Y. However, the mechanism which 
you propose requires the collector to have special knowledge of the 
internals of X, which is not scalable. In short, such an Exporter would 
not be interoperable with all Collectors.

P.

> Pls let me know if my question is unclear.

The custom IE which you've defined must contain meta-data about the list 
such as an element count, element lengths, delimiters or separators. It 
creates a new way of encoding list information which the collector must 
also understand in addtion to the basicList encoding, which is not 
useful. In short, you should not create list type elements; instead, use 
the basicList mechanism.

>
>
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix

---

From: Gerald Naveen A <[email-masked]>
Date: Tue, 15 Dec 2015 20:34:17 +0530

Hi Paul,

Thanks for the responses. It clarifies for most part; while I would also
like to clarify few things on my question. (tagged [Gerald] inline)

So it appears we would need to define Y as IE and mark X as basicList of Y.

Thanks again

On Tue, Dec 15, 2015 at 6:44 PM, Paul Aitken <[email-masked]> wrote:

> Gerald,
>
> Hi all,
>
> It is my understanding from the RFCs that basicList type is an encoding
> for list of IPFIX Information Elements.
>
>
> string (as X is already strongly defined). But string is a type, not info
> element.
>
> whether they contain interface names, user names, a list of selectorNames,
> or a list of Application names. Then a basicList of Y tells the Collector
> that you're exporting zero or more of them.
>
[Gerald] This is definitely a choice. However, as I mentioned, this makes
the template weakly typed (ie., the element is just #291 basicList). We
would like to enforce stronger types at the template itself (this was the
motive behind defining X)

> defined), then redefine X as a basicList of Y. Or if you don't want/need to
> define both X and Y, then use the basicList IE #291 to export a basicList
> of (strongly defined) Y.
>
[Gerald] Yes, this is also a choice; but we don't need Y for any other
reason except to fit into the basicList detail. So wanted to know if that
can be avoided. I got my answer: No (if strongly typed templates).

> item in payload -- is this mandatory?
>
>
> should be able to decode a basicList of Y. However, the mechanism which you
> propose requires the collector to have special knowledge of the internals
> of X, which is not scalable. In short, such an Exporter would not be
> interoperable with all Collectors.
>
[Gerald] X is not a different encoding. X is a basicList (sort of more
strongly typed. ie., X is a basicList of Y). So the encoding is just the
same as #291 standard basicList. But when the template declares a element
as X, it is clearly typed even without having to look into the data (or it
also doesn't let the exporter send X of Z on the same template).

> also understand in addtion to the basicList encoding, which is not useful.
> In short, you should not create list type elements; instead, use the
> basicList mechanism.
>
[Gerald] Agree. We aren't trying to redefine basicList encoding for sure.

> IPFIX mailing [email-masked]://www.ietf.org/mailman/listinfo/ipfix
>
>
>

---

From: Paul Aitken <[email-masked]>
Date: Tue, 15 Dec 2015 23:24:52 +0000

Gerald,

> Hi Paul,
>
> Thanks for the responses. It clarifies for most part; while I would 
> also like to clarify few things on my question. (tagged [Gerald] inline)

See [PJ] inline...

> So it appears we would need to define Y as IE and mark X as basicList 

What you're proposing is possible if there's a good reason. However we 
want to avoid unnecessary duplication of existing IEs into new "list of" 
IEs as much as possible, so you should only create X if there's a strong 
justification (eg it models an existing object).

> Thanks again
>
> On Tue, Dec 15, 2015 at 6:44 PM, Paul Aitken <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Gerald,
>
>>     Hi all,
>>
>>     To create the template much more strongly typed, we have defined
>>     a custom (enterprise) Information Element with a new ID (say X),
>>     which is defined as a basicList of specific items (eg.,
>>     InterfaceName).
>>
>>     My question to you all is: Is it mandatory that we have a
>>     separate information element called InterfaceName (say ID Y) and
>>     define X as a basicList of Y ?
>
>>     For our need, it is enough for us, that we define X as a
>>     basicList of string (as X is already strongly defined). But
>>     string is a type, not info element.
>>
>>     We would like to retain X (instead of defining a template
>>     directly with a standard basicList). And we don't have any other
>>     need to define a new Information Element Y.
>
>     Y defines the list elements which tells the Collector that the
>     list contains strings, and it defines the semantics of those
>     strings - eg whether they contain interface names, user names, a
>     list of selectorNames, or a list of Application names. Then a
>     basicList of Y tells the Collector that you're exporting zero or
>     more of them.
>
> [Gerald] This is definitely a choice. However, as I mentioned, this 
> makes the template weakly typed (ie., the element is just #291 
> basicList). We would like to enforce stronger types at the template 
> itself (this was the motive behind defining X)

[PJ] This was an objection from the Collector side when we were writing 
the IPFIX Structured Data draft, because the collector can no longer 
tell what information it's going to receive ahead of time (just "a 
list"); it has to check each Data Record to know what information the 
list contains. On the other hand, since the basicList is essentially a 
wrapper around any other IE, it's not necessary to create new 
strongly-defined list IEs for each of the existing (and potentially all 
future) IEs.

>     Perhaps you should define Y as a single instance of X (so Y is
>     well defined), then redefine X as a basicList of Y. Or if you
>     don't want/need to define both X and Y, then use the basicList IE
>     #291 to export a basicList of (strongly defined) Y.
>
> reason except to fit into the basicList detail. So wanted to know if 
> that can be avoided. I got my answer: No (if strongly typed templates).

[PJ] Strongly typed templates would require a new "list of" IE for every 
existing (and future) IE that was to become a list, which is incredibly 
wasteful.

>>     I realize that the basicList encoding has a information-element
>>     ID of the item in payload -- is this mandatory?
>

[PJ] Here I understood the question to be whether the IE ID field was 
required in the basicList encoding - ie, it seemed that you were 
proposing a new encoding where, since X is already strong defined as a 
basicList of Y, there's no need to include Y in the export.

>>     Or for a strongly typed item X, we could expect the IPFIX parser
>>     to be aware of contents (interface name) by contract?
>
>     No. Every collector which implements RFC 6313 (IPFIX Structured
>     Data) should be able to decode a basicList of Y. However, the
>     mechanism which you propose requires the collector to have special
>     knowledge of the internals of X, which is not scalable. In short,
>     such an Exporter would not be interoperable with all Collectors.
>
> strongly typed. ie., X is a basicList of Y). So the encoding is just 
> the same as #291 standard basicList. But when the template declares a 
> element as X, it is clearly typed even without having to look into the 
> data (or it also doesn't let the exporter send X of Z on the same 
> template).

[PJ] What you're proposing is possible and may be useful in some 
circumstances. For example, for the MIB export draft, we've defined new 
IEs (#443, #444) which are subTemplateLists, in order to give strong 
typing information because the IE contains a row from a table.

(See https://tools.ietf.org/html/draft-ietf-ipfix-mib-variable-export-10 
and http://www.iana.org/assignments/ipfix/ipfix.xhtml)

P.

>     The custom IE which you've defined must contain meta-data about
>     the list such as an element count, element lengths, delimiters or
>     separators. It creates a new way of encoding list information
>     which the collector must also understand in addtion to the
>     basicList encoding, which is not useful. In short, you should not
>     create list type elements; instead, use the basicList mechanism.
>
>>     [email-masked] <mailto:[email-masked]>

---

From: Andrew Feren <[email-masked]>
Date: Wed, 16 Dec 2015 22:41:05 +0000

Hi Paul and Gerald,

I have slightly mixed feelings on this, but my current bias is towards not requiring a second IE.  I plan to look at this a bit more closely, but here are a few thoughts.

see [ACF] inline...

________________________________
From: IPFIX [[email-masked]] on behalf of Paul Aitken [[email-masked]]
Sent: Tuesday, December 15, 2015 6:24 PM
To: Gerald Naveen A
Cc: [email-masked]
Subject: Re: [IPFIX] basicList clarification

Gerald,

Hi Paul,

Thanks for the responses. It clarifies for most part; while I would also like to clarify few things on my question. (tagged [Gerald] inline)

See [PJ] inline...

What you're proposing is possible if there's a good reason. However we want to avoid unnecessary duplication of existing IEs into new "list of" IEs as much as possible, so you should only create X if there's a strong justification (eg it models an existing object).

Thanks again

On Tue, Dec 15, 2015 at 6:44 PM, Paul Aitken <[email-masked]<mailto:[email-masked]>> wrote:
Gerald,

Hi all,

It is my understanding from the RFCs that basicList type is an encoding for list of IPFIX Information Elements.

To create the template much more strongly typed, we have defined a custom (enterprise) Information Element with a new ID (say X), which is defined as a basicList of specific items (eg., InterfaceName).

My question to you all is: Is it mandatory that we have a separate information element called InterfaceName (say ID Y) and define X as a basicList of Y ?

For our need, it is enough for us, that we define X as a basicList of string (as X is already strongly defined). But string is a type, not info element.

We would like to retain X (instead of defining a template directly with a standard basicList). And we don't have any other need to define a new Information Element Y.

Y defines the list elements which tells the Collector that the list contains strings, and it defines the semantics of those strings - eg whether they contain interface names, user names, a list of selectorNames, or a list of Application names. Then a basicList of Y tells the Collector that you're exporting zero or more of them.
[Gerald] This is definitely a choice. However, as I mentioned, this makes the template weakly typed (ie., the element is just #291 basicList). We would like to enforce stronger types at the template itself (this was the motive behind defining X)

[PJ] This was an objection from the Collector side when we were writing the IPFIX Structured Data draft, because the collector can no longer tell what information it's going to receive ahead of time (just "a list"); it has to check each Data Record to know what information the list contains. On the other hand, since the basicList is essentially a wrapper around any other IE, it's not necessary to create new strongly-defined list IEs for each of the existing (and potentially all future) IEs.

[ACF] From the collector side I still prefer strongly typed templates.  There may be some middle ground possible like an option template that reports the range of expected values for a template with a basicList (or other generic structure).  I started a draft on this a while back, but it got back burnered.   Might be time to dust it off again.

Perhaps you should define Y as a single instance of X (so Y is well defined), then redefine X as a basicList of Y. Or if you don't want/need to define both X and Y, then use the basicList IE #291 to export a basicList of (strongly defined) Y.
[Gerald] Yes, this is also a choice; but we don't need Y for any other reason except to fit into the basicList detail. So wanted to know if that can be avoided. I got my answer: No (if strongly typed templates).

[PJ] Strongly typed templates would require a new "list of" IE for every existing (and future) IE that was to become a list, which is incredibly wasteful.

[ACF]  I think there can be different considerations for lists in standard space and list IEs with a PEN.

I realize that the basicList encoding has a information-element ID of the item in payload -- is this mandatory?

[PJ] Here I understood the question to be whether the IE ID field was required in the basicList encoding - ie, it seemed that you were proposing a new encoding where, since X is already strong defined as a basicList of Y, there's no need to include Y in the export.

[ACF] I think the quibble here is whether Y needs an IE if it is only used as a list.  If an IE already exists you can use it.  Requiring Y becomes wasteful (to borrow your earlier word) in a different way.  Assume there are many lists of A, B, C, D and X.  There isn't a need to ever export A, B, C, D or X as any thing other than a list.  The second IE isn't really needed.

Or for a strongly typed item X, we could expect the IPFIX parser to be aware of contents (interface name) by contract?

No. Every collector which implements RFC 6313 (IPFIX Structured Data) should be able to decode a basicList of Y. However, the mechanism which you propose requires the collector to have special knowledge of the internals of X, which is not scalable. In short, such an Exporter would not be interoperable with all Collectors.
[Gerald] X is not a different encoding. X is a basicList (sort of more strongly typed. ie., X is a basicList of Y). So the encoding is just the same as #291 standard basicList. But when the template declares a element as X, it is clearly typed even without having to look into the data (or it also doesn't let the exporter send X of Z on the same template).

[PJ] What you're proposing is possible and may be useful in some circumstances. For example, for the MIB export draft, we've defined new IEs (#443, #444) which are subTemplateLists, in order to give strong typing information because the IE contains a row from a table.

(See https://tools.ietf.org/html/draft-ietf-ipfix-mib-variable-export-10 and http://www.iana.org/assignments/ipfix/ipfix.xhtml)

[ACF] A different compromise might be some generic IEs, for exampe, a string IE.  Then you could have a list of aNames or bNames.  Each of which put the (currently non existent) string IE ID as the ID in the strongly typed list.  Of course that opens the door to generic basicList of generic strings.  Having a single list encoding is nice, but leaving out the ID probably isn't any worse than fixed length vs var length strings.

I reserve the right to change my opinions as I reread 6313,
-Andrew

P.

Thanks in advance.
- Gerald Naveen

The custom IE which you've defined must contain meta-data about the list such as an element count, element lengths, delimiters or separators. It creates a new way of encoding list information which the collector must also understand in addtion to the basicList encoding, which is not useful. In short, you should not create list type elements; instead, use the basicList mechanism.
[Gerald] Agree. We aren't trying to redefine basicList encoding for sure.

_______________________________________________
IPFIX mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ipfix