Subject: [ipfix] multiple Exporting Processes in IPFIX-ARCH

From: Paul Aitken <[email-masked]>
Date: Mon, 29 May 2006 12:15:29 +0100

Dear all,

This text in IPFIX-ARCH:

    * IPFIX Device

       An IPFIX Device hosts at least one Observation Point, a Metering
       Process and an Exporting Process.

seems to say that there is one one MP and one EP, which is clearly
wrong. eg, Figure 3 shows one IPFIX device with multiple OPs, multiple
MPs and multiple EPs:

    +------------------------+
    |IPFIX Device(i)         |
    |[Observation Point(s)]  |
    |[Metering Process(es)]  |
    |[Exporting Process(es)] |
    +------------------------+

The text should say

       An IPFIX Device hosts at least one Observation Point, at least
       one Metering Process and at least one Exporting Process.

Figure 4 doesn't make clear whether there may be multiple exporters
associated with each MP. I believe that should be possible to provide
redundancy, for export over different transports, to convey the MP
output to multiple collectors, or even to impliment different export
protocols.

Without being spelt out in the text, the way it's drawn does seem to
constrain each MP to just one EP. I don't see any architectural reason
for that.

Cheers.
-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Benoit Claise <[email-masked]>
Date: Mon, 29 May 2006 14:06:59 +0200

Paul,
> wrong. eg, Figure 3 shows one IPFIX device with multiple OPs, multiple
I agree. I modified [IPFIX-PROTO].
Some PSAMP drafts should be modified as well, if I recall correctly

Regards, Benoit.

---

From: kobayashi atsushi <[email-masked]>
Date: Mon, 29 May 2006 21:58:14 +0900

Dear Paul and all,

According to IPFIX-ARCH, I recognized that one exporting process can
send same data stream to multiple collectors. And also, one exporting
process seems to be able to send same data stream to multiple collector
using different transport protocol.
This one exporting process can treat same data stream against multiple
collector. But, it can't treat different data stream against multiple
collector. In that case, exporting process should be separated.
Please comment, if it is wrong.

I believed that one exporting process is created per the export policy
of administrator, if this IPFIX device can create multiple exporting
processes.

I think that capability by one exporting process should be made more clear. 

Regards,
Atsushi KOBAYASHI

On Mon, 29 May 2006 12:15:29 +0100

> wrong. eg, Figure 3 shows one IPFIX device with multiple OPs, multiple

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5652

---

From: Paul Aitken <[email-masked]>
Date: Mon, 29 May 2006 15:42:56 +0100

Kobayashi-san

> using different transport protocol.

It's allowed by the architecture.

However, a simple exporter implimentation might only be able to connect 
to one collector or speak one transport protocol - with the intention 
that there would be multiple Export Processes, one for each protocol or 
collector.

And that might be a good design, because then the failure of one Export 
Process has no impact on other exports.

> collector.

Why do you say that?

> In that case, exporting process should be separated.

I think it's possible. eg, in IPFIX-ARCH figure 4, I see the Exporting 
Process receiving data from multiple Metering Processes. And I can 
imagine the same Exporting Process sending packets to multiple collectors.

> of administrator, if this IPFIX device can create multiple exporting

Perhaps. Or perhaps by a policy built into the system by the designer. 
Perhaps the adminsitrator does not even know or need to know how many 
seperate processes there actually are?!

> I think that capability by one exporting process should be made more clear. 

I agree.

---

From: Gerhard Muenz <[email-masked]>
Date: Mon, 29 May 2006 18:26:24 +0200

Paul, Kobayashi,

> imagine the same Exporting Process sending packets to multiple collectors.

I assume Kobayashi thinks that in figure 4, all collectors receive the
same records, i.e. the same data stream. Whereas two exporting processes
would be necessary to send different sets of records to different
collectors.
This is also according to my understanding.

-- 
Dipl.-Ing. Gerhard Münz
Computer Networks and Internet
Wilhelm Schickard Institute for Computer Science
University of Tuebingen
Auf der Morgenstelle 10C 9P16, D-72076 Tuebingen, Germany
Phone: +49 7071 29-70534 / Fax: +49 7071 29-5220
EMail: [email-masked]
WWW:   http://net.informatik.uni-tuebingen.de/~muenz

---

From: Paul Aitken <[email-masked]>
Date: Mon, 29 May 2006 18:39:03 +0100

Gerhard,

> would be necessary to send different sets of records to different

That might be a common implimentation, but IPFIX neither requires nor 
mandates that.

eg, section 5.5 of IPFIX-ARCH discusses:

    There may be additional rules defined within an Observation Domain so
    that only certain Flow Records are exported.  This may be done by
    either one or a combination of Si, Fi, as described in the section on
    "Selection Criteria for Packets".

It'd be straightforward to impliment a single Exporting Process capable 
of exporting to multiple collectors, which implimented independant sets 
of filters on each output stream as described.

eg,  export to 1.1.1.1 port 2222 match dst-addr 192.0.33.5
      export to 3.3.3.3 port 4444 match dst-addr 192.0.11.30 sample 1 in 2

It would also be straighforward to impliment those rules as seperate 
Export Processes.

IPFIX doesn't tell us what to do here; it's an implimentation choice.

Cheers.

---

From: kobayashi atsushi <[email-masked]>
Date: Tue, 30 May 2006 16:39:16 +0900

Dear Paul and Gerhard,

I considered what Gerhard has said.
The Exporting Process seems not to have filter function on record. Therefore,
single Exporting Process receives records from multiple Metering
Processes, and it sends same records to multiple Collection Processes.
Exporting Process seems not to be able to handle different datastream
against multiple Collection Processes.

Certainly, IPFIX neither requires nor mandates that, but, it can be considered
like above.

In section 5.5, a combination of Si, Fi is one function of the Metering process.
I wonder that this description is depicted in section 5.5.
According to this example, single or more than one exporting processes receives
flow records multiple metering processes that have different rules.

In that case, whichever single or two is sufficient to IPFIX.
Which a implementation choice or a configuration choice of policy is?
Is it clearly implementation choice? I think that it should made more
clear.

Regards,
Atsushi KOBAYASHI

On Mon, 29 May 2006 18:39:03 +0100

---

---

From: kobayashi atsushi <[email-masked]>
Date: Tue, 30 May 2006 17:00:55 +0900

Dear Paul,

I'm sorry for not enough.

I mean that the relationship between Metering process and Exporting
process is configured by policy of administrator.
Certainly, administrator does not need to know about how many processes.
But, when administrator wants to apply new export policy, Exporting
process is likely to be created and be linked with Metering process , after
configuration.

Regards,
Atsushi KOBAYASHI

On Mon, 29 May 2006 15:42:56 +0100

---

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 30 May 2006 10:11:17 +0200

Kobayashi, Paul,

The current text in 5.5 does not explicitly allow different "additional
rules" for different collectors (nor prohibits such an implementation).

I think the motiviation for this discussion is to determine where to
locate filtering/sampling parameters within a MIB module. The MIB module
will always be an abstraction of the actual implementation, i.e. the
borders between different processes can be drawn differently or
disappear in the implementation.

The MIB module is to provide a coherent view on the IPFIX device and it
should be possible to describe any possible state and configuration.
On the other hand, it should not be possible to describe the same
state/configuration in many different ways. Hence, in a MIB module, it
could be better to assign sampling/filtering parameters to the metering
processes only and use different exporting processes for different data
streams.

Regards,
Gerhard

---

From: kobayashi atsushi <[email-masked]>
Date: Wed, 31 May 2006 10:46:25 +0900

Hi Gerhard,

> disappear in the implementation.

My motivation for this discussion is to consider what IPFIX-MIB meets IPFIX-ARCH.
I think that it might be inconsistent to IPFIX-ARCH, if we refer objects
of concerning Exporting Process of PSAMP-MIB as IPFIX-MIB.
I would like to arrange what parameter is configurable in IPFIX-MIB.
The topics concerning exporting process is one of these issues.

> processes only and use different exporting processes for different data

I also think so.

> Dipl.-Ing. Gerhard M|nz

---