Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to draft-ietf-ipfix-reqs-00.txt

From: Juergen Quittek <[email-masked]>
Date: Thu, 14 Feb 2002 23:29:53 +0100

Hi Peter,

So what is your point? Do you want to have in-band configuration
for the exported attributes/IEs in the requirements document?

What would be the application requiring this? Or from which
general requirement would you derive this one?

Cheers,

    Juergen
-- 
Juergen Quittek     [email-masked]     Tel: +49 6221 90511-15
NEC Europe Ltd.,    Network Laboratories     Fax: +49 6221 90511-55
Adenauerplatz 6, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

--On 14 February 2002 11:32 -0800 Peter Ludemann <[email-masked]> wrote:

>
> Whether or not we have in-band configuration, we do need a mechanism for advertising the data that a device can deliver (i.e., self-defining data). In terms of design and implementation, supporting a simple negotiation ("Don't send me x." "OK, here's a
> revised list of what I'm sending") isn't much more work.
> If you configure a device with a MIB or CLI, how do you tell the receivers about what data is being sent? How do you coordinate amongst thousands of sending devices and dozens of receivers ... doing this by hand would be a nightmare. (I'm not making up
> this scenario; but I can't give more details because of confidentiality issues.)
>
> There is a very pragmatic reason for having the receiver specify the fields it doesn't want. Some fields are expensive (for example, we have a probe which can look inside data streams and extract information such as URLs and response codes; it can also
> compute performance metrics such as response times ... when all of these are turned on, the performance can drop significantly).
> Having in-band configuration does not mean that a device needs to support anything beyond advertising the data that it will send. For example, with Kevin's CRANE proposal, the in-band configuration consists of:
> - the device ("meter") advertises the records it can send
> - the receiver requests restrictions on what is sent (which kinds of records, which fields in those records)
> - the device can ignore the restriction request
>
> So, this configuration simply offers the possibility of optimization but does not force the optimization.
>
> This kind of configuration does not specify frequency of data delivery, aggregation, filtering, etc. Currently, we do this out of band (with a CLI, MIB, or registry file) because it does not raise the coordination issues that data formats cause. We're
> thinking of a generic mechanism for doing configuration inside the transfer protocol, but that's a future issue that can easily be added to the existing protocol.
> - peter
>
>
> -----Original Message-----
> From: majordomo listserver [mailto:[email-masked]]On Behalf Of Reinaldo Penno
> Sent: Thursday, February 14, 2002 7:24 AM
> To: [email-masked]; Benoit Claise
> Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
> Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to draft-ietf-ipfix-reqs-00.txt
>
>
>
> Hello,
>
> Last I counted we had most people saying that it's useful but we should tackle this later, except for Me, Kevin and Ram that would like to see it now. K.C said it was nice, Paul and Mike that it was for after "IPfixv1" and Benoit, Ganesh, Juergen and
> Will saying no.
>
> So in terms of number of vendors (I agree with Benoit that the important thing is the number of vendors adopting it) it seems there is an agreement to investigate this problem at some point. We need then to choose the words that reflect this position
> on the arch draft/req. Suggestions?
>
> Having said that I would like to clarify some points:
>
> One is that the idea was never to have a "common provisioning protocol". Provisioning seems a pretty heavy word. The in-band config would facilite things like template negotiation associate with flow metering capabilities and the like. So, in fact it
> would help configuring vendor specific features like extended flow information, etc
>
> Second, the fact that "the meter will be built partially in hardware which is costly. Here we can
> save a lot by reducing some functions. " is a implementation issue...Somebody else cold say that their meter would be build on software (a PC-appliance for instance) so they do not care. So trying to be fair with all possible IPfix users, I think we
> should drop this specific point altogether.
>
> Now, as I said, I think Benoit had a good point on the adoption of such a mechanism, ie, How many vendors would implement this in-band configuration. If we can base the vendors on the people that work for them, I would say up to this point 4-5 (not
> sure what percentage of companies that will really build an IPfix collector or exporter this represents).
>
> regards,
>
> Reinaldo
>
>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]]
>> Sent: Thursday, February 14, 2002 6:24 AM
>> To: Benoit Claise
>> Cc: [email-masked]; [email-masked]; [email-masked];
>> [email-masked]
>> Subject: Re: in band configuration? RE: [ipfix-req] Proposed changes to
>> draft-ietf-ipfix-reqs-00.txt
>>
>>
>>
>> I vote no in-band configuration in IPFIX V1.
>>
>> Benoit Claise wrote:
>>>
>>> Hi Ram and All,
>>>
>>> We've discussing the in-band configuration for a few times
>> without conclusions.
>>> I propose to use this thread to arrive to a rough consensus
>> and close the discussion one for all. Otherwise we're turning
>> in circles and loosing everybody's time!
>>> And if we can't get a consensus, the chairmen and area
>> directors will have to take a decision.
>>>
>>> Note: please let's try to concentrate on just one issue:
>> in-band configuration and not capability negotiation and
>> transport negotiation. One step at the time please.
>>>
>>> >
>>> >
>>> > > Our charter tells us to "select a protocol by which
>>> > > IP flow information can be transferred" and I would
>>> > > prefer not to go beyond this.
>>> >
>>> > Transfer  of flow information may not refer only to
>> monitoring. I interpreted as
>>> > one  can monitor and then transfer ( exchange) flows
>> between endpoints . To do this one has
>>> > to  configure first what we are interested in - that means
>> configure, monitor and transfer
>>> > flows.
>>> >
>>> > I am not sure and would like to get clarify my doubt,
>>> >
>>> > If we have in-band management what is the problem? It's is
>> a good choice whether
>>> > there is no flaw and it is good to control from one place
>> to do both configure and manage the network
>>> > elements. If we provide proper security mechanism and
>> ensure that the devices gets
>>> > the proper commands  similar to CLI then  what's the problem ?
>>>
>>> What is the problem, you're asking?
>>> There are actually no problems. I agree it would be the
>> perfect world to have in-band configuration. But here are some
>> arguments against the idea. Actually I even think a little bit
>> further: the exporter configuration is out of the scope of
>> this charter.
>>>
>>> 1. Do you want the working group to spend the time (the
>> money) to developp a comon provisioning protocol, IF this is
>> even possible to have an agreement on that accross the
>> different vendor (probes and routers), which I think is impossible.
>>>
>>> 2. What would happen with proprietary features, which most
>> likely will not be part of the IPFIX provisioning protocol. So
>> anyway we will have to use a different way of provisioning the
>> device, like CLI for example, next to IPFIX. This will bypass
>> the use of IPFIX provisioning protocol.
>>>
>>> 3. How many vendors would implement this in-band configuration?
>>> Because at the end this is the final question. What's the
>> point to have a IETF protocol that no/not many vendors are
>> going to implement.
>>> Do you think that vendors will spend time and money to
>> developp again another configuration interface, next to CLI,
>> SNMP, XML, etc...?  And this just for an IPFIX protocol!
>>>
>>> 4. SNMP is widely used. Why? because the S stands for simple.
>>> And now, along the time, it has been evolving because there
>> are new needs.
>>> I would like to start with something Simple, that everybody
>> agrees on.
>>> And if there is a need for in-band configuration, then we
>> can think of IPFIX2.
>>> Why trying to have an ISO standard from version 1? ;)
>>> You can give me a long list of nice idea about in-band
>> configuration that I'm sure I would love to have in a perfect
>> world but let's start with something Simple.
>>> Don't forget that simplicity is the key for interoperability.
>>> If not yet convinced: have you already compared Snmp and CMIP?
>>>
>>> 5. From the charter, "select a protocol by which IP flow
>> information can be transferred" as Juergen said already. I
>> would not beyond this point.
>>> OK, you (Ramg) have a different view.
>>>
>>> 6. IPFIX stands for Flow Information eXport. So we have to
>> standardize the export, not the configuration. So the
>> direction:exporter -> collector and not the direction
>> collector -> exporter. At least for version 1.
>>>
>>> 7. Email from Nevil Brownlee, Subject: [ipfix] Comments on
>> 'goals of IPFX WG'
>>> http://ipfx.doit.wisc.edu/list/ipfix/archive/0388.html
>>>       2. IPFIX is *not* trying to standardise a flow meter,
>> i.e. a network
>>>       entity which measures flows - we already have RFCs
>> which do that,
>>>       e.g. those for RTFM (2720 .. 2724).  Existing widely-used
>>>       implementations can remain different, but might
>> perhaps be extended
>>>       to report flow information in the IPFIX lingua franca.
>>>
>>>       3. Instead, we recognise that network equipment
>> vendors already have
>>>       flow measurement systems implemented as part of their
>> equipment.
>>>       Our goal is to make the IPFIX flow information export system
>>>       so good technically that all vendors will be happy to implement
>>>       it, for the benefit of all users of the data.  A
>> useful side-effect
>>>       will be that IPFIX will make it easier to build and maintain
>>>       interoperable flow data collecting systems.
>>>
>>> As we're not supposed to standardise a flow meter, why would
>> we standardize its configuration? This email just expresses
>> that the exporter configuration is out of the scope of the charter.
>>>
>>> 8. Juergen's argument: "Please consider that the meter will
>> be built partially in hardware which is costly. Here we can
>> save a lot by reducing some functions.
>>> The data collector implemented by software can be much more flexible
>>> with less cost."
>>>
>>> Regards, Benoit.
>>>
>>> >
>>> > I can understand  currently most of network elements
>> configuration happens thro'   CLI or other mechanisms....
>>> >
>>> > Take for example SNMP is used only as monitoring tool and
>> but people are working on to making configuration
>>> > also thro' SNMP and seperate activity is going on in IETF
>> community itself.
>>> >
>>> >
>>> > Regards
>>> > Ramg
>>> >
>>> >
>>> >
>>> >
>>>
>>> --
>>> Help        mailto:[email-masked] and say
>> "help" in message body
>>> Unsubscribe mailto:[email-masked] and say
>>> "unsubscribe ipfix" in message body
>>> Archive     http://ipfix.doit.wisc.edu/archive/
>>
>> --
>> Help        mailto:[email-masked] and say "help"
>> in message body
>> Unsubscribe mailto:[email-masked] and say
>> "unsubscribe ipfix" in message body
>> Archive     http://ipfix.doit.wisc.edu/archive/
>>
>

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Peter Ludemann" <[email-masked]>
Date: Fri, 15 Feb 2002 08:53:17 -0800

> for the exported attributes/IEs in the requirements document?

Yes. The CRANE protocol solves real problems that we've encountered; and we
have a number of partners who are using it (sorry, I can't announce these
right now).

> general requirement would you derive this one?

My answer is a little long. Our requirements go beyond just delivering the
kind of data that IPFIX is discussing; but I think that they are very
applicable to IPFIX.

If all you care about is delivering one set of values which you'll never
change, then there's no need for any in-band configuration. (This would be
true only if you think that the data architecture folks will agree on a
specification with no SHOULDs in the document ...)

If you anticipate change, then at minimum you need to specify the version of
the data that you're delivering. A more general solution is to use
self-describing data.  XML would be fine but it's much too verbose for
delivering thousands of records per second from a small network device.

So, the minimal requirement is to have some way for the device to advertise
what it has.  I don't see a need for all of XML's nesting capabilities; a
simple 2-level scheme (type of record + fields) should suffice.

The receiver could quietly take this advertisement and process it. But our
experience in "IP data mediation" is that people usually only want specific
fields; and they typically want to post-process them. Examples of such
post-processing: associating values with IP addresses (e.g., reversing NAT,
mapping IP to customer), aggregation, filtering, conditional splitting, etc.
plus a host of database-oriented processing, such as looking for error
patterns, for people who hog bandwidth, etc.

If the sending device can be configured by a MIB or CLI, the receiver can
adapt to what it advertises. But there are good reasons for such
configuration to be done by response from the receiver:
  - the needed fields can be derived from the post-processing configuration,
    so they naturally reside on the receiver, not the sender
  - there can be tens or even thousands of sending devices, so
    centralized configuration is desirable (especially if they can't all
    have their versions changed simultaneously)
  - there needs to be a way of coordinating amongst multiple receivers,
    for fail-over deployments
(these aren't hypothetical requirements: we have customers who are looking
at deploying thousands of sending devices; and who require fail-over)

Once you decide that the sender should advertises its data, it doesn't take
much more work to allow the receiver to respond with a list of fields that
it does or doesn't want.  The sender can just mark the fields it won't send.
The advantages of this are:
  - bandwidth saving (probably not very important)
  - computation saving on the sender (this can be significant
    in our experience)

Our proposal makes it OPTIONAL for the sender to respond to the
configuration request.  The sender can ignore the configuration request and
still send all the fields -- the receiver will just have to accept them (and
throw away what it doesn't want).  But if the sender can take advantage of
the configuration request (e.g., turning off expensive computations), then
it has a simple mechanism for doing so.

----

Peter Ludemann
Chief Architect, XACCT
+1.408.330.5732

---

From: Juergen Quittek <[email-masked]>
Date: Sat, 16 Feb 2002 16:21:43 +0100

Hi Peter,

> simple 2-level scheme (type of record + fields) should suffice.

As far, as I followed the ongoing difscussion, (almost) everyone agrees
to this.

>   - computation saving on the sender (this can be significant

I think we are discussing two different issues. Your suggestion
covers - as far as I have understood so far - the negotiation
between sender an receiver about individual attributes/fields/IEs
to be sent. But as 'in-band configuration' i understand the more
basic configuration of meter and exporter including which ports
to observe, which filtering rules to apply, which flow keys to use,
which sampling rate to choose, etc.

But coming back to your point: If you say the exporter may ignore it,
why then making it a requirement? Requirements should be essentially
needed and not just nice to have, but opyionally ignored at any time.

I think your point can be a valuable contribution to the architecture,
but following your argument it does not belong into the requirement
docvument.

    Juergen

---

From: "Peter Ludemann" <[email-masked]>
Date: Sat, 16 Feb 2002 10:01:46 -0800

Juergen:

Thank-you for pointing out the apparent self-contradiction in my argument.
I neglected to complete me argument.

I'm glad that everyone agrees on the need for self-describing data. It
wasn't clear from all the emails that were flying back and forth. :-)

I'd like to address the two other issues that are not yet agreed to:
 - content negotiation (which extends content advertisement)
 - configuration of processing (sampling rates, ports, etc.)

Some devices are very simple and have no need for any kind of content
negotiation. Others (such as our PacketSight probe, which can have over 200
fields) are quite complex and can benefit greatly from content negotiation.

The capability to respond to a content request is NOT optional. Every
request MUST be answered; but the answer can be to reject the request, to
honour it, or to partially honour it. For example:

    exporter advertises fields A,B,C,D
    receiver requests only fields A,D
    exporter responds with:
      fields A,B,C,D -- if it ignores the request
      fields A,D     -- if it completely honours the request
      fields A,C,D   -- if it partially honours the request

One of our concerns with content negotiation was in minimising the
development work for putting the transport protocol onto a network element
(our target is 2 weeks or less). If the device already has the data
available in a particular form, we want to be able to export it as-is. For
example, the exporting device determines whether big-endian or little-endian
numbers are used; the receiver translates as necessary. This has the
advantages:
   - simpler implementation on the exporting device
   - minimal performance impact on the exporting device
   - minimal space requirements on the exporting device
Of course, if the exporting device wants to switch the data from
little-endian to big-endian (network byte order), we aren't going to prevent
it. But we cater to the exporter's needs, not the receiver's.

If the exporting device can take advantage of sending fewer fields, it can;
but if simplicity of implementation or performance concerns would make it
better to ignore content requests, that's also OK.

Moving on to the second point:
   But as 'in-band configuration' i understand the more
   basic configuration of meter and exporter including which ports
   to observe, which filtering rules to apply, which flow keys to use,
   which sampling rate to choose, etc.

I would like this kind of configuration also; but I think it is less
necessary than content negotiation. Content information must be communicated
amongst all the receivers (in a fail-over configuration), so simply using
MIBs or a CLI won't work very well. However, the issues of determining
sampling rate, observed ports, etc. do not need to be distributed in the
same way (if needed, they can be exported as just another kind of record
from the network device).

I think that this kind of configuration will always remain "machine
specific". But it can fit into a very simple structure, something like a
list of attribute/value pairs being sent from the receiving device and a
accepted/rejected message being sent back [the configuration must do
multiple items together because there can be interactions amongst them].
This is similar to the way configuration is done using a MIB, except that it
would get around some of the problems with MIB configuration (interactions
amongst attributes, confirmation of result, multiple tools needed to do
configuration, etc.).

To sum up: we need a framework for all the configuration, without requiring
agreement on the content (which we might never be able to, especially with
backwards support for existing equipment). This is similar to the
self-describing data delivery ... the requirement is that data be
self-describing and that it contain type information, not that we all agree
to a specific data model (there eventually will be standard data models, of
course). Similarly, we should agree to a method of in-band content
negotiation and processing configuration.

- peter

----
Peter Ludemann           [email-masked]
Chief Architect          +1.408.330.5732
XACCT Technologies, Inc.
2900 Lakeside Drive      Santa Clara, CA 95054

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf
Of Juergen Quittek
Sent: Saturday, February 16, 2002 7:22 AM
To: Peter Ludemann; Reinaldo Penno; [email-masked]; Benoit
Claise
Cc: [email-masked]; [email-masked];
[email-masked]; Kevin Zhang
Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to
draft-ietf-ipfix-reqs-00.txt

Hi Peter,

    Juergen

--
Help        mailto:[email-masked] and say "help" in message
body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/