Subject: Re: [IPFIX] new work item candidate #6: order of information elements

From: Hitoshi Irino <[email-masked]>
Date: Sun, 05 Aug 2007 17:03:34 +0900

Dear Gerhard, all,

I am sorry about my late reply.

(2007/07/27 23:45), Gerhard Muenz wrote:
> Dear Hitoshi, all,
> 
> I had a look at your slides. Here are some thoughts about it.
> 
> With respect to the Template structure, I see three aspects which affect
> the performance of the collector:
> 
> - Order of IEs.
> - Reduced size encoding.
> - Position of variable length IEs in the record.
> 
> Let's assume the collector has to copy the record data into an internal
> data structure positioned at a certain address in memory.
> (Only) if the following two conditions are fulfilled, a performance gain
> can be achieved by copying blocks of consecutive IEs:
> 
> 1. Order of IEs (within the block) is the same in the Template and in
>    the internal data structure
> 2. No reduced size encoding is applied.
> 
> As you wrote in your email, your collector makes use of this block
> copying feature. And this feature is the main reason why your collector
> is so fast when using the "suggested order" at both exporter and collector.
> 
> Jürgen: Common collectors not supporting this block copying feature
> won't show the same performance gain. So, just repeating the experiments
> with other collector implementations does not seem to be very promising.

I agree that current collectors do not support processing multiple IEs. 
(Maybe anyone except me have not thought to treat multiple IEs at once.)
Although it does not mean always that other implementations can not be 
implemented this features, I think.
(By the way, the logic of this feature is implemented by about 30 lines 
on my implementation. I extended a function which decodes template. Of 
course I recognize this issue depends on each implementation.)

> Here are some further thoughts:
> - Having the same order of IEs while using reduced size encoding at
>   the same time does not enable to copy multiple IEs at once.
>   => no performance gain

Yes. Therefore I separate position for constant fixed length IEs from 
other IEs to enable copy multiple IEs in my suggested order.

> - Not using reduced size encoding at all while allowing random IE
>   orders, we can omit type casts in the collector, but casts usually
>   do not actually require computational resources.
>   => no performance gain

I did not contain type casts into the reason of difference of the 
performance. I have thought the performance is differed by difference 
between aligning data boundary and not aligning.
I recently tested same evaluation without coping multiple IEs. Using 
random order (which do not align data boundary) template is about 1% 
slower than others constantly. Though it is only a difference of 1% 
after all, it is better to consider data boundary. It is my opinion.

> - The treatment of variable length IEs always results in performance
>   drawbacks because the offsets of the different IEs cannot be
>   calculated in advance. So, you would probably not use variable length
>   IEs at all if performance is an issue. Also, I do not see how the
>   performance can be increased significantly if variable length IEs are
>   positioned at the end of the record. You still need to calculate all
>   offsets to determine record and set boundaries.

I have not tested templates containing variable length IEs. My 
implementation treats data per data records and the implementation 
calculate offsets of each IEs when read Template records. Therefore I 
suppose that if variable length IEs are collected into end or top of the 
record, the performance do not worsen too much because collector can 
process these IEs independently of other IEs. On the other hand, I 
suppose that if variable length IEs are positioned at middle of the 
record, the performance worsen  because recalculations offset of each 
following IEs are needed.

> A final comment: You often need to convert values from network-byte
> order to host-byte order. If you do this, you have to process the
> corresponding IEs separately and cannot use the block copy.

If collector uses little endian CPU and all IEs have to be converted 
byte order, I agree with you.
However I evaluated the performance without byte order conversion because:
  1) the necessity of byte order conversion depends on CPU architecture. 
(If collector uses big endian CPU, the cost of byte order conversion is 
zero.)
  2) I do not think that all IEs are needed to convert host byte order, 
because some standard C language functions relating DNS presuppose that 
data is network byte order.

Probably counter and timestamps need to be converted, however these IEs 
did not effects the performance in my evaluation introduced in my 
presentation.

I think applications of mediation often copy from fields of incoming 
data records to fields of outgoing data records without byte order 
conversion.
Is it correct? > Kobayashi-san

> Conclusion: Only under the specific conditions described above, using a
> specialized collector, the performance can be improved.

As you pointed, my suggestion and the gained performance are made up of 
some conditions. However I think my suggestion does not affect except 
the cost of implementation, on the other hand it has possibility to make 
good performances.

Best regards,
Hitoshi IRINO

> Hitoshi Irino wrote:
>> Dear all,
>>
>> (2007/07/26 15:38), Juergen Quittek wrote:
>>> Dear all,
>>>
>>> Candidates #6 to become a new IPFIX work item is described by
>>> draft draft-irino-ipfix-ie-order-02.txt.
>>>
>>>> 6. Order of Information Elements (Irino)
>>>>    Interesting, need more people to measure possible efficiency gain.
>>>>    Would this be better as an individual draft?
>>> This draft suggests a special ordering of information elements in a
>>> flow record.  At the very brief presentation at our session, the author
>>> explained that he can improve the performance of IPFIX exporter and
>>> collector significantly. To prove this, he compared a random order
>>> of information elements with an order according to his proposal
>>> and achieved significant performance improvements.
>>>
>>> Speaking as technical contributor, I think that if these results would
>>> be the same also for other implementations of IPFIX, then this looks like
>>> a very good IPFIX work item.
>>>
>>> Speaking as co-chair, I would like to ask everybody who has an IPFIX
>>> implementation to conduct a short experiment in order to find out if
>>> the improvement is the same for his/her implementations.
>>>
>>> Would any implementor be willing to do so?
>>> If yes, until when would you have results?
>>>
>>> Thanks,
>>>
>>>     Juergen
>> A day before yesterday, I had a presentation about a performance result
>> when exporters and collectors use same order, however I didn't have any
>> time to talk about the reason of difference of performance.
>>
>> In the slide,
>> ( http://www3.ietf.org/proceedings/07jul/slides/ipfix-10.pdf )
>> I use very primitive collector which I implemented. This collector
>> receive data records and write the data to file.
>> This collector has internal proprietary format to store to data to file.
>> The format consists of some Information Elements whose size are full
>> size. (For example, bgpSourceAsNumber is 4 octets.)
>>
>> This collector process following steps:
>>  1) read the data record.
>>  2) convert received data records to the collector's format on memory.
>>  3) write to file from memory.
>>
>> I tested each combination of 3 kind of orders for template of exporter
>> and format of collector.
>>  1) An order same as NetFlow version 5
>>  2) Suggested Order in my draft (separating constant fixed length IEs
>> and reduced size encoding applicable IEs)
>>  3) Random order
>> In this test, exporter's template includes set of Information Elements
>> same as NetFlow version 5 fields.
>> These templates and formats are shown in page 9 in the slide.
>>
>> I measured 10 times this test.
>> The matrix table in page 4 is the result of this test. Upper one shows
>> total time, lower one shows ratio.
>>
>> My collector program can process bulk of sequential Information
>> Elements. For example in the case reading from date records ordered by
>> suggested order to writing format formed by suggested order, my
>> collector program copies first 20 octets (from sourceIPv4Address to
>> ipClassOfService) at once.
>>
>> I described counts for processing in from page 10 to 12 in the slide.
>> For example, in the best case that a exporter uses suggested order and a
>> collector uses suggested order, the counts for processing (processing
>> means coping data in this case) is 8 (on middle table in page 11). In
>> the worst case a exporter uses random order and collector uses random
>> order, the counts is 17 (on right table in page 11). Difference of the
>> counts for processing effects their performance.
>> my presentation shows one of the best case.
>>
>> Therefore, there are effective case and ineffective case by ordering.
>>
>> Example of effective case
>>  writing to files
>>  real time flow filtering (comparing bulk data)
>>
>> Example of ineffective case
>>  when process each Information Element
>>
>> By the way, if all exporter supports draft-muenz-ipfix-configuration and
>> operators can set configuration about template completely freely, my
>> draft is not important, because operator can adjust configuration for
>> collector.
>> However currently NetFlow v9 implementations of some router makers are
>> not configurable at all. Operators can choice preset template on these
>> implementations. If this state continues in future, reference order is
>> valuable for these exporters' implementations and collectors'
>> implementations. Therefore I've thought a reference order should be
>> defined as Informational. I suggested an order adjusted for collector's
>> performance, however perhaps it is not suitable for exporter. So I hope
>> to discuss on WG.
>>
>> thanks,
>> Hitoshi Irino
>>
> 

-- 
Hitoshi Irino
NTT Network Service Systems Laboratories
9-11 Midori-cho 3-Chome, Musashino-shi, Tokyo 180-8585 Japan
Tel: +81-422-59-4403  Fax: +81-422-59-4549

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

---

From: Hitoshi Irino <[email-masked]>
Date: Sun, 05 Aug 2007 17:03:34 +0900

Dear Gerhard, all,

Best regards,
Hitoshi IRINO