Subject: RE: [ipfix] Variant Field Types.

From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" <[email-masked]>
Date: Tue, 30 Sep 2003 12:37:52 -0400

Hi,
 
  Although the use of variant types may seem convenient from the perspective
of an
exporter, I think the various consumers of the data would benefit from
sticking with
an explicit type system.
 
  Consider operations which involve summing fields, storing them to a DB,
etc.
 
  Explicit typing ensures that these consuming applications can handle these
values
appropriately and also makes the information modeller think about how these 
types are to be used.
 
  The examples you cite:
 
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports and AS's), counts
and intervals.
 
  For the identifiers, I believe the size is fixed by the relevant RFC's
(i.e. why would I 
have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a valid
question, however
these can be handled by making them explicit (i.e. the information model
says if
I get this value it is a 32-bit quantity or a 64-bit quantity and the code
will address
it accordingly).  For intervals, the resolution would determine which size
is most
appropriate.
 
  The option always exist to define variations on these data items
explicitly.  E.g.
byteCount32 and byteCount64.
 
  If there is a need to vary the sizes for various fields I'd much rather
see the rationale
for each discrete sizing made explicit vs. forcing all the downstream
consumers of this \
information have to deal with special cases and ambiguity.

Regards,
 
  Jeff Meyer

-----Original Message-----
From: Ganesh Sadasivan [mailto:[email-masked]]
Sent: Monday, September 29, 2003 6:00 PM
To: [email-masked]
Subject: [ipfix] Variant Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only

has types for fixed length objects and variable length 

string like objects.

There are a number of information element types (listed below)

that would be better defined in terms of fixed-size run-time

specified objects. The type variation comes due to difference

in the size of the objects used because of an implementation,

as demanded by the network or otherwise.

The way to make this work is to use the template

length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt

    if (element.length == 1)

         type = unsignedByte

    else if (element.length <= 2)

         type = unsignedShort

    else if (element.length <= 4)

         type = unsignedInt

    else if (element.length <= 8)

         type = unsignedLong

   else type = undefined

type varInt

    if (element.length == 1)

         type = byte

    else if (element.length <= 2)

         type = short

    else if (element.length <= 4)

         type = int

    else if (element.length <= 8)

         type = long

   else type = undefined

The candidates that fit into the description above are (I may

have missed some also):

- ingressPort 

- egressPort 

- packetCount 

- byteCount 

- sourceAS 

- destinationAS

- nextHopAS 

- droppedPacketCount

- samplingInterval

- droppedByteCount

Please comment.

Thanks

Ganesh

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 30 Sep 2003 18:06:16 +0100

MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
> Hi,
>  
>   Although the use of variant types may seem convenient from the 
> perspective of an

Variant types allow better use of resources and the ability to upgrade
fields without the need to firstly get IETF approval, and secondly
synchronise deployment of the exporter and collector upgrade.

Anything that counts is a good example. To cope with the high end
systems we really need to make them all 64bits. That means that
tiny system that are memory and export limited and which may never
wrap a 16 or 32 bit counter need to devote resources to 64 bits.

If we were designing this say 5 years ago we would probably not have
considered 32 bit AS numbers, now we need to. There are bound to be
other IE that need to be upgraded, and for each and every one we need
to synchronise both exporter and collector.

By introducing this flexibility we get greater scope for how we handle
these upgrades.

>   Consider operations which involve summing fields, storing them to a 
> DB, etc.
>  

The collector storage is less likely to be the long pole in the tent
than the exporter or the transmission bandwidth to the collector. The
collector can therefore more readily be implemented to accept the
largest reasonable value than can the exporter.

>   Explicit typing ensures that these consuming applications can handle 
> these values

But they have to think worst case during deployment, and my contention
is that this places undue stress on the exporter and on the network
bandwidth.

>  
>   The examples you cite:
>  
> - ingressPort
> - egressPort
> - packetCount
> - byteCount
> - sourceAS
> - destinationAS
> - nextHopAS
> - droppedPacketCount
> - samplingInterval
> - droppedByteCount
>  
>   fall into a few categories, identifiers (such as ports and AS's), 
> counts and intervals.
>  
> have a 1 byte Port?).  

Clearly but not long ago you would have said "why would I have a 4 byte
AS number"

Also you are thinking in terms of the "examples" we have in the current
RFCs. We need to design this both for future RFCs and for vendor specific
IEs.

> For counts, the use of 32 vs. 64 bits is a valid 
> question, however
> I get this value it is a 32-bit quantity or a 64-bit quantity and the 
> code will address
> it accordingly).  For intervals, the resolution would determine which 
> size is most
> appropriate.

But this then makes it harder for tiny systems or cramps large systems.

> byteCount32 and byteCount64.

Thay leads to two (or more) varients that the collector has to support.
The alternative is for the collector to support the larger, and the exporter
to support what ever size it wants. An alternative way to think of this
is as a data compression technology.

>  
>   If there is a need to vary the sizes for various fields I'd much 
> rather see the rationale

That requires too much (unnecessary) synchronisation between exporter and collector.

Regards

Stewart

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Tal Givoly" <[email-masked]>
Date: Tue, 30 Sep 2003 10:21:12 -0700

Ganesh,

Jeff makes very good points IMHO. Having the complexity of arbitrary length
should have strong justification. In particular, for all numeric fields that
have fixed length (e.g. identifiers), those should be preferred rather than
having variable length.

Tal
-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf Of
MEYER,JEFFREY D (HP-Cupertino,ex1)
Sent: Tuesday, September 30, 2003 9:38 AM
To: [email-masked]
Subject: RE: [ipfix] Variant Field Types.

  Hi,

    Explicit typing ensures that these consuming applications can handle
these values
  appropriately and also makes the information modeller think about how
these
  types are to be used.

    The examples you cite:

  - ingressPort
  - egressPort
  - packetCount
  - byteCount
  - sourceAS
  - destinationAS
  - nextHopAS
  - droppedPacketCount
  - samplingInterval
  - droppedByteCount

  Regards,

    Jeff Meyer

    -----Original Message-----
    From: Ganesh Sadasivan [mailto:[email-masked]]
    Sent: Monday, September 29, 2003 6:00 PM
    To: [email-masked]
    Subject: [ipfix] Variant Field Types.

Hi,

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte

type varInt
    if (element.length == 1)
         type = byte

Please comment.

Thanks
Ganesh

---

From: Ganesh Sadasivan <[email-masked]>
Date: Tue, 30 Sep 2003 10:44:30 -0700

> exporter, I think the various consumers of the data would benefit from 

It would benefit the exporter, the n/w, flexibility of interpretation of 
fields in the
information model  and the collector.

>   Consider operations which involve summing fields, storing them to a 

One way to solve this problem is if the collector assumes the largest 
size for
computations?

>   For the identifiers, I believe the size is fixed by the relevant 
> RFC's (i.e. why would I
> have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a 
> valid question, however
> these can be handled by making them explicit (i.e. the information 
> model says if

For AS the size could be 2 or 4 bytes.
You may prefer to use a 1 byte port if you are a small system as against 
a 4 byte for
a gigantic system. Both have their own implementation reasons.

> byteCount32 and byteCount64.

Yes. But is'nt this not more of an overhead to introduce new field type 
as the requirements
come in and then make sure that it gets deployed in exporter and collector?

> information have to deal with special cases and ambiguity.

This is not a special case handling as you can see that the list above 
covers about
40% of the fields defined today in the information model. It is way to 
handle
ambiguity arising out of implementation differences.

Thanks
Ganesh

>     *From:* Ganesh Sadasivan [mailto:[email-masked]]
>     *Sent:* Monday, September 29, 2003 6:00 PM
>     *To:* [email-masked]
>     *Subject:* [ipfix] Variant Field Types.
>
>Hi,
>