Subject: [ipfix-info] Re: draft-boschi-ipfix-biflow and software implementation

From: Brian Trammell <[email-masked]>
Date: Tue, 7 Feb 2006 10:47:13 -0500

Alexandre,

On Feb 7, 2006, at 10:11 AM, Alexandre Dulaunoy wrote:

> I'll  try  to  "over"  summarize  back  what  I  understand  with  the
> information       you      gave       me      and       the      draft
> (draft-boschi-export-perpktinfo-01.txt) :
>
> The FlowID  is a 32bit id  unique across one observation  point and is
> (always?) generated by the exporter.
>
> We   (at  SES   ASTRA  and   CSRRT-LU)  are   trying  to   generate  a
> "unique/calculable" FlowID  among a specify dataset of  records. Why ?
> We  have  a  bunch  of  "exporters" (routers,  devices)  with  various
> versions  of Netflow  and  we have  to  live with  that  for the  time
> being. What we are doing currently  ? we use a simple hash function to
> generate a  "unique" FlowID per  data set based  on source/destination
> IP,  source/destination  PORT  and  (protocol).   We  use  the  lowest
> source/destination (IP/PORT) to generate a unified hash id per biflow.
>
> So it  seems that I was mixing  our and your definition  of FlowID but
> what do  you think ?  Can  we call that  a FlowID or something  else ?
> Have you think of a 'possible'  function to generate a FlowID based on
> records ? Like  that, we can rebuild biflow(s)  from uniflow(s) ? Does
> it make sense ?

flowId, as defined in -info-11, does not presently appear flexible  
enough to cover this use case; its "unique within Observation Domain"  
constraint appears to make multiple Flow Records sharing the same  
flowId impossible. Thinking about it for a moment, I think this is a  
fault in the information model. To increase the flexibility of this  
IE, I'd propose changing its definition to the following:

5.1.7.  flowId

    Description:
       An identifier of a Flow that is unique within an Observation
       Domain.  This Information Element can be used to distinguish
       between different Flows if Flow Keys such as IP addresses and  
port
       numbers are not reported or reported in separate records.  
Multiple
       Flow Records observed within the same Observation Domain may  
share
       the same flowId if they describe the same Flow as determined  
by the
       Metering Processes within that Domain.
    Abstract Data Type: unsigned32
    Data Type Semantics: identifier
    ElementId: 148
    Status: current

(Elisa - this also expands flowId as required for -biflow-02 section  
3.2)

> Brian : A site note, looking into naf source code (the pcap exporter),
> I can't  find the part of generating  the FlowID. Can you  point me to
> the correct part ?

We don't actually use the flowId anywhere (hence the cheating) - we  
do single-record biflow export per draft-boschi-ipfix-biflow section  
3.4 (section 4 in the forthcoming -02 draft), with CERT-private IEs  
standing in for IANA-assigned IEs pending completion of that draft.  
The relevant code is in nafcore.c. NAF unifies uniflows (from pcap or  
SiLK) into biflows the hard way; SiLK asymmetric unidirectional data  
is collected using the SiLK tools' proprietary protocol into a single  
database, relevant records are selected out of that database and  
sorted by time, mingling incoming and outgoing records and re- 
matching them before aggregation.

YAF also cheats; it's designed solely for deployment at symmetric  
routing points, or for use on packet capture data that can be  
prearranged to appear to have come from a symmetric routing point.

Regards,

Brian

---

From: Alexandre Dulaunoy <[email-masked]>
Date: Thu, 9 Feb 2006 18:46:56 +0100

On 07/Feb/06 10:47 -0500, Brian Trammell wrote:

Hi Brian,

> flowId, as defined in -info-11, does not presently appear flexible enough to cover this use case; its "unique within Observation Domain" 
> constraint appears to make multiple Flow Records sharing the same flowId impossible. Thinking about it for a moment, I think this is a fault in 
> the information model. To increase the flexibility of this IE, I'd propose changing its definition to the following:
> 
> 5.1.7.  flowId
> 
>    Description:
>       between different Flows if Flow Keys such as IP addresses and port
>       numbers are not reported or reported in separate records. Multiple
>       Flow Records observed within the same Observation Domain may share
>       the same flowId if they describe the same Flow as determined by the
> (Elisa - this also expands flowId as required for -biflow-02 section 3.2)

Thanks for the feedback. On our side (outside the scope of the draft),
we  will keep  the  approach to  uniquely  identify the  flow among  a
dataset. In  that case, the  flowId is often  bigger than 32  bits but
it's not sequential(?) like the flowId expressed in the draft.

Another question (more in scope with  the draft), how do you deal with
the uniqueness of the flowId ?   Is it up to the implementation on the
exporter to handle the uniqueness ?

> We don't actually use the flowId anywhere (hence the cheating) - we do single-record biflow export per draft-boschi-ipfix-biflow section 3.4 
> (section 4 in the forthcoming -02 draft), with CERT-private IEs standing in for IANA-assigned IEs pending completion of that draft. The 
> relevant code is in nafcore.c. NAF unifies uniflows (from pcap or SiLK) into biflows the hard way; SiLK asymmetric unidirectional data is 
> collected using the SiLK tools' proprietary protocol into a single database, relevant records are selected out of that database and sorted by 
> time, mingling incoming and outgoing records and re-matching them before aggregation.

OK. understood.

Thanks a lot for the feedback,

Have a nice day,

adulau

-- 
--                   Alexandre Dulaunoy (adulau) -- http://www.foo.be/
--                          http://www.gnu.org/philosophy/free-sw.html
--         "Knowledge can create problems, it is not through ignorance
--                                that we can solve them" Isaac Asimov

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/