Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to d raft-ietf-ipfix-reqs-00.txt

From: "Christian Gilby"<[email-masked]>
Date: Tue, 19 Feb 2002 00:28:37 -0800

Content Negotiation
  I agree that the negotiation of the set of attributes that a collector is
interested in has a lot of value.  Depending on the application, a collector
may be interested in a small or large set of attributes.  If the list of
requested attributes can be advertised to the mesuring device (which can
indicate which will be sent) then, IF the device does not ignore the request
it can minimize the processing and export data volume required.  I.e. no
need to send extraneous attributes that the collector will ignore.

In-band Configuration
  The ability to perform the configuration in-band (done in a lightweight
manner) will have payoffs in large network deployments with mixed devices
from various vendors.  It will be a challenge to deal with configuration of
existing implementations which can be difficult to add in-band config to
(they may need to be configured manually on the box) but believe it is
worthwile to have a framework for performing this config in-band for devices
in a common way since we are trying to collect a common set of information
across the devices.

Regards,
Chris
-----Original Message-----
From: Peter Ludemann [mailto:[email-masked]]
Sent: Saturday, February 16, 2002 10:02 AM
Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to
draft-ietf-ipfix-reqs-00.txt

Juergen:

Thank-you for pointing out the apparent self-contradiction in my argument.
I neglected to complete me argument.

I'm glad that everyone agrees on the need for self-describing data. It
wasn't clear from all the emails that were flying back and forth. :-)

I'd like to address the two other issues that are not yet agreed to:
 - content negotiation (which extends content advertisement)
 - configuration of processing (sampling rates, ports, etc.)

Some devices are very simple and have no need for any kind of content
negotiation. Others (such as our PacketSight probe, which can have over 200
fields) are quite complex and can benefit greatly from content negotiation.

The capability to respond to a content request is NOT optional. Every
request MUST be answered; but the answer can be to reject the request, to
honour it, or to partially honour it. For example:

    exporter advertises fields A,B,C,D
    receiver requests only fields A,D
    exporter responds with:
      fields A,B,C,D -- if it ignores the request
      fields A,D     -- if it completely honours the request
      fields A,C,D   -- if it partially honours the request

One of our concerns with content negotiation was in minimising the
development work for putting the transport protocol onto a network element
(our target is 2 weeks or less). If the device already has the data
available in a particular form, we want to be able to export it as-is. For
example, the exporting device determines whether big-endian or little-endian
numbers are used; the receiver translates as necessary. This has the
advantages:
   - simpler implementation on the exporting device
   - minimal performance impact on the exporting device
   - minimal space requirements on the exporting device
Of course, if the exporting device wants to switch the data from
little-endian to big-endian (network byte order), we aren't going to prevent
it. But we cater to the exporter's needs, not the receiver's.

If the exporting device can take advantage of sending fewer fields, it can;
but if simplicity of implementation or performance concerns would make it
better to ignore content requests, that's also OK.

Moving on to the second point:
   But as 'in-band configuration' i understand the more
   basic configuration of meter and exporter including which ports
   to observe, which filtering rules to apply, which flow keys to use,
   which sampling rate to choose, etc.

I would like this kind of configuration also; but I think it is less
necessary than content negotiation. Content information must be communicated
amongst all the receivers (in a fail-over configuration), so simply using
MIBs or a CLI won't work very well. However, the issues of determining
sampling rate, observed ports, etc. do not need to be distributed in the
same way (if needed, they can be exported as just another kind of record
from the network device).

I think that this kind of configuration will always remain "machine
specific". But it can fit into a very simple structure, something like a
list of attribute/value pairs being sent from the receiving device and a
accepted/rejected message being sent back [the configuration must do
multiple items together because there can be interactions amongst them].
This is similar to the way configuration is done using a MIB, except that it
would get around some of the problems with MIB configuration (interactions
amongst attributes, confirmation of result, multiple tools needed to do
configuration, etc.).

To sum up: we need a framework for all the configuration, without requiring
agreement on the content (which we might never be able to, especially with
backwards support for existing equipment). This is similar to the
self-describing data delivery ... the requirement is that data be
self-describing and that it contain type information, not that we all agree
to a specific data model (there eventually will be standard data models, of
course). Similarly, we should agree to a method of in-band content
negotiation and processing configuration.

- peter

----
Peter Ludemann           [email-masked]
Chief Architect          +1.408.330.5732
XACCT Technologies, Inc.
2900 Lakeside Drive      Santa Clara, CA 95054

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf
Of Juergen Quittek
Sent: Saturday, February 16, 2002 7:22 AM
To: Peter Ludemann; Reinaldo Penno; [email-masked]; Benoit
Claise
Cc: [email-masked]; [email-masked];
[email-masked]; Kevin Zhang
Subject: RE: in band configuration? RE: [ipfix-req] Proposed changes to
draft-ietf-ipfix-reqs-00.txt

Hi Peter,

--On 15 February 2002 08:53 -0800 Peter Ludemann <[email-masked]>
wrote:

> Juergen Quittek wrote Thursday, February 14, 2002 2:30 PM:
>
>> So what is your point? Do you want to have in-band configuration
>> for the exported attributes/IEs in the requirements document?
>
> Yes. The CRANE protocol solves real problems that we've encountered; and
we
> have a number of partners who are using it (sorry, I can't announce these
> right now).
>
>> What would be the application requiring this? Or from which
>> general requirement would you derive this one?
>
> My answer is a little long. Our requirements go beyond just delivering the
> kind of data that IPFIX is discussing; but I think that they are very
> applicable to IPFIX.
>
> If all you care about is delivering one set of values which you'll never
> change, then there's no need for any in-band configuration. (This would be
> true only if you think that the data architecture folks will agree on a
> specification with no SHOULDs in the document ...)
>
> If you anticipate change, then at minimum you need to specify the version
of
> the data that you're delivering. A more general solution is to use
> self-describing data.  XML would be fine but it's much too verbose for
> delivering thousands of records per second from a small network device.
>
> So, the minimal requirement is to have some way for the device to
advertise
> what it has.  I don't see a need for all of XML's nesting capabilities; a
> simple 2-level scheme (type of record + fields) should suffice.

As far, as I followed the ongoing difscussion, (almost) everyone agrees
to this.

> The receiver could quietly take this advertisement and process it. But our
> experience in "IP data mediation" is that people usually only want
specific
> fields; and they typically want to post-process them. Examples of such
> post-processing: associating values with IP addresses (e.g., reversing
NAT,
> mapping IP to customer), aggregation, filtering, conditional splitting,
etc.
> plus a host of database-oriented processing, such as looking for error
> patterns, for people who hog bandwidth, etc.
>
> If the sending device can be configured by a MIB or CLI, the receiver can
> adapt to what it advertises. But there are good reasons for such
> configuration to be done by response from the receiver:
>   - the needed fields can be derived from the post-processing
configuration,
>     so they naturally reside on the receiver, not the sender
>   - there can be tens or even thousands of sending devices, so
>     centralized configuration is desirable (especially if they can't all
>     have their versions changed simultaneously)
>   - there needs to be a way of coordinating amongst multiple receivers,
>     for fail-over deployments
> (these aren't hypothetical requirements: we have customers who are looking
> at deploying thousands of sending devices; and who require fail-over)
>
> Once you decide that the sender should advertises its data, it doesn't
take
> much more work to allow the receiver to respond with a list of fields that
> it does or doesn't want.  The sender can just mark the fields it won't
send.
> The advantages of this are:
>   - bandwidth saving (probably not very important)
>   - computation saving on the sender (this can be significant
>     in our experience)

I think we are discussing two different issues. Your suggestion
covers - as far as I have understood so far - the negotiation
between sender an receiver about individual attributes/fields/IEs
to be sent. But as 'in-band configuration' i understand the more
basic configuration of meter and exporter including which ports
to observe, which filtering rules to apply, which flow keys to use,
which sampling rate to choose, etc.

But coming back to your point: If you say the exporter may ignore it,
why then making it a requirement? Requirements should be essentially
needed and not just nice to have, but opyionally ignored at any time.

I think your point can be a valuable contribution to the architecture,
but following your argument it does not belong into the requirement
docvument.

    Juergen
--
Juergen Quittek     [email-masked]     Tel: +49 6221 90511-15
NEC Europe Ltd.,    Network Laboratories     Fax: +49 6221 90511-55
Adenauerplatz 6, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

>
> Our proposal makes it OPTIONAL for the sender to respond to the
> configuration request.  The sender can ignore the configuration request
and
> still send all the fields -- the receiver will just have to accept them
(and
> throw away what it doesn't want).  But if the sender can take advantage of
> the configuration request (e.g., turning off expensive computations), then
> it has a simple mechanism for doing so.
>
>
> ----
>
> Peter Ludemann
> Chief Architect, XACCT
> +1.408.330.5732
>

---

From: [email-masked]
Date: Tue, 19 Feb 2002 10:32:03 -0500

Christian Gilby wrote:
> 
> Content Negotiation
>   I agree that the negotiation of the set of attributes that a
> collector is interested in has a lot of value.  Depending on the
> application, a collector may be interested in a small or large set of
> attributes.  If the list of requested attributes can be advertised to
> the mesuring device (which can indicate which will be sent) then, IF
> the device does not ignore the request it can minimize the processing
> and export data volume required.  I.e. no need to send extraneous
> attributes that the collector will ignore.
> 
> In-band Configuration
>   The ability to perform the configuration in-band (done in a
> lightweight manner) will have payoffs in large network deployments
> with mixed devices from various vendors.  It will be a challenge to
> deal with configuration of existing implementations which can be
> difficult to add in-band config to (they may need to be configured
> manually on the box) but believe it is worthwile to have a framework
> for performing this config in-band for devices in a common way since
> we are trying to collect a common set of information across the
> devices.

	I don't think anyone indicated that it is not worthwhile.
	But rather the difficulty might make it better solved
	in a subsequent version.

	Paul

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/