Subject: RE: [ipfix-eval] Discussion of Candidate Protocols

From: "Harrington, David" <[email-masked]>
Date: Fri, 4 Oct 2002 17:38:56 -0400

Hi,

Let me be clear that I am advocating no particular solution. I am trying to
cut through any hype and understand the technical pros and cons of each
approach.

Am I biased? You decide. I don't believe so, and want to dispel any bias
assumptions.
I am a network management technology analyst for the CTO of Enterasys (which
used to be Cabletron).

1) I analyzed LFAP in 1999(?) for its potential as an industry standard and
found it lacking. Paul has since addressed most of the points I raised, and
we've had many discussions about its pros and cons. It has both.
2) I was a member of the IPDR and reviewed their early specs. I did not find
it compelling. Its development in a fee-based members-only forum limited the
industry's ability to review it as it developed. There were obviously many
talented individuals involved, but some companies had a larger influence in
its design, which makes me believe it may not be very vendor-neutral. It
included a number of people from telco companies, where usage accounting and
billing has a long history. I pushed for a while to have them submit it to
the IETF, but they resisted. As a long-time IETFer, I didn't see much
justification for not working within an open standards organization like the
IETF. 
3) I have some serious reservations about Netflow's technical design,
especially after talking candidly with many Cisco developers at IETF
meetings. Nonetheless, I promoted the replacement of LFAP with Netflow in
Enterasys devices because of Netflow's wide-spread deployment and
third-party application support. We did not want to be in the LFAP
accounting application business. Cisco is our primary competitor, and I
would like to see them not control the flow accounting standard. I believe
Netflow has both pros and cons technically.
4) Xacct sought out Enterasys as a potential partner in 2000(?). I did the
technology analysis on CRANE for the due diligence. My largest concern was
the proprietary nature of the protocol. Just like IPDR, it was developed in
a members-only agreement and was denied the opportunity for wide-spread
industry review. This was done to capture the intellectaul property rights
before making it public. Another major concern was the push for
standardizing "buckets for transporting proprietary data in"; I don't
believe that transporting a lot of proprietary data models really helps
standardization and interoperability. CRANE has pros and cons.
5) I co-authored RFC2975 "Introduction to Internet Accounting" which
discusses many of the requirements identified here.
6) As the co-chair of SNMPv3 WG, I am glad nobody proposed SNMP as an IPFIX
candidate because it would likely not be very satisfactory for flow
accounting purposes. SNMP has pros and cons, but I don't think SNMP (as
currently defined) is well suited to this particular task.

I am not advocating any particular solution, and I believe all the proposals
offer some benefits for the stated purpose.

That being said, I have chosen to reply to Jeff's mail because it uses SNMP
for comparison and I fail to fully understand the point being made. 

Comments inline.

dbh

> -----Original Message-----
> From: Jeff Meyer [mailto:[email-masked]]
> Sent: Friday, October 04, 2002 12:58 PM
> To: Sebastian Zander
> Cc: Peter Ludemann; [email-masked]
> Subject: Re: [ipfix-eval] Discussion of Candidate Protocols
> 
> 
> Hi,
> 
>    There seems to be some confusion over the separation
> between a formal, machine readable description of
> the data model (as IPDR defines), and the requirements put
> on an implementation (especially in an IPFIX Middlebox).
> 
>    Presumably folks are familiar with SNMP and the
> separation between what a MIB defines and how the
> protocol operates.
> 
>    Basically IPDR's use of XML schema is producing a
> MIB for sets of accounting information.
> 

So IPDR provides a formal machine readable description of a data model just
as SNMP does. The difference is IPDR uses an XML schema and MIBs use SMI
schemas.

OK. 

> 
>    Why not use MIB's?  Well, SNMP is really geared towards
> providing views/access into a current system state. Historical
> information tends to be global or other coarse grain
> counters.  Fine grain counters in SNMP are hindered by
> SNMP's OID indexing model (i.e. you end up with something
> like RMON, a clever use of SNMP, but not much fun to
> code to).

You make these assertions about SNMP, without any supporting analysis. You
seem to do this as the setup for a comparison with IPDR's XML schema, but
you don't follow through. 

When you say SNMP is geared towards providing views/access into a current
system state, are you talking about the SNMP protocol being geared to this,
or is it just that currently existing SNMP mibs seem geared to this task?

You talk about historical information being global or coarse-grain counters.
Again, are you talking about the SNMP protocol constraining this, or is this
an artifact of the mibs that have been defined? How does XML do this
differently? If we can model this in XML, couldn't we also model this in a
mib? (Does this mean you're advocating SNMP as a viable solution? ;-)

I am missing the point about how OIDs hinder fine-grained counters. An OID
is used to name an object such as a counter. I don't think the naming has a
significant effect on how fine-grained the counter is. Are you asserting
that somehow XML-naming better supports fine-grained counters?

Can you explain further?
> 
>    IP Flow information is just a specifically defined
> set of metrics which are being collected based on
> certain observed network traffic.   

SNMP mibs typically are also a specifically defined set of metrics which are
being collected based on certain observed network traffic, such as bytes
in/out or the RMON flow statistics, or other events. So IP Flow information
is just a specifically defined set of metrics, like a MIB. Are you
advocating we solve the problem by writing a mib module for IP Flows?

> It just so happens
> that this can produce a huge amount of info, which
> often cannot be stored on the device itself for any
> period of time.  For this reason, using SNMP for gathering
> flow info is probably a bad idea, and I'm glad to
> see no one signed up as the SNMPv3 advocate ;-)

Off-loading large amounts of information quickly could be done using SNMP
Informs. I certainly agree that using SNMP to gather flow info is probably a
bad idea if you mean gathering it up and waiting for applications to poll
for it. I would have concerns about the efficiency of SNMP Informs compared
to a protocol designed to handle off-loading large quantities of flow
information quickly. See RFC2975.

As you pointed out, SNMP makes a clear separation between the data modeling
and the transport mechanism. I wouldn't recommend SNMPv3 for IPFIX primarily
because of the transport constraints associated with 484 byte messages over
UDP, lack of support for streaming data, and repetitive OID attribute
identifiers, not because of the data modeling capabilities.

I don't see that an XML data model addresses those problems.
> 
> 
>    But the essential concept of MIBs, which SMIng is also
> looking at revising is important. 

I disagree that SMIng is looking at revising the essential concept of MIBs.
They are looking at revising the SMI to improve ease-of-use and add
object-orientation. The essential concept of MIBs as "data models that are
extensible without protocol changes" remains the same.

> The extensibility
> model which MIBs provided for SNMP is what has led
> to its large scale adoption for many problem domains.

Agreed. An extensible data model has been important to SNMP's large scale
adoption, and I believe an extensible data model should be supported by
IPFIX. Keep in mind however, that there  are trade-offs. An extensible data
model adds overhead and complexity, which may reduce the efficiency of an
extensible IPFIX. 

A vendor-extensible data modeling language also leads to many vendor
extensions which greatly reduces standardization and the ability to
aggregate/compare/contrast information from multiple vendors. This has been
a major factor in SNMP applications all offering similar limited sets of
functionality, and the large number of applications that are little more
than mib browsers - device vendors do not provide solid support for
fully-compliant industry standard mib implementations, but rather offer lots
of proprietary mib data using a standardized modeling language and a
standardized transport mechanism.

The more data elements that can be named, the longer the names need to be to
differentiate them. OIDs work well in SNMP to provide a data naming space
that can be extended as needed without naming conflicts. Integers work well
in LFAP to name a limited set of data elements. 

XML is extensible, and will suffer the tradeoffs of long names and lots of
proprietary extensions just like SNMP.

> 
>    It just so happens that the profile of flow exporting
> is not appropriate for the SNMP domain.

Here you lose me. What is the "profile" you refer to that makes flow
exporting inappropriate for the SNMP domain? How does IPDR deal with those
same profile characteristics differently than SNMP and the other IPFIX
proposals?
> 
> 
>    However, the statement that IP flow is somehow unique
> in its data requirements and as such a more generalized
> "data mover" is somehow problematic, is just plain wrong.

If your argument is that IP flow is just another data model and that a
generalized "data mover" is acceptable, then are you advocating we model IP
flow in a MIB and move the data using SNMP? 

If your argument is that IP flow is just another data model and that a
generalized "data mover" is acceptable as long as it meets performance
requirements, then would a MIB transported over another protocol be
acceptable? or would a mib transported over an SNMP that supported TCP,
larger messages, and OID suppression/compression be acceptable? If so, you
might want to check on some of the work being done by the EOS WG.

> 
>    As a mediation product vendor I see lots of devices
> with lots of vendor specific protocols.  And there
> are several which have the same protocol requirements
> as IPFIX.  Just different data models.
> 
>    Consider a middlebox which looks not only at flows,
> but also at protocols above layer 4.  Would it not
> make sense for the additional information produced
> by this to use the same basic protocol as IPFIX?

So you're arguing that if IPFIX already solves the protocol problems
associated with fast-offloading of bulk data, it should be reused for
transporting any type of data?

Let me make sure I follow your logic:
You're arguing that we should keep the data model and protocol separate.
You're arguing that we should reuse rather than develop more task-specific
protocols.

Then shouldn't you also be advocating using existing mechanisms for data
modeling of formal, machine readable descriptions of the data model, such as
the SMI rather than XML?

> 
>    IPFIX does identify extensibility as a requirement,
> is there some bounded set of extensions which are
> envisioned?
> 

One of the real strengths of SNMP's MIB approach is the allocation of a
partitioned namespace, because industry standard data models were developed,
and because the enterprise-specific namespace encouraged vendors to use
SNMP. 

One of the real strengths of SNMP's MIB approach is the specification of a
common data modeling language, with very limited syntax options, that allows
machine-parsing of any mib and display of any mib information.

One of the real weaknesses of SNMP's MIB approach is the allocation of an
enterprise-specific namespace, because vendors used it to develop
proprietary data models which hindered interoperability across vendors. 

I think it is very important to understand the tradeoffs involved in
deciding how extensible something should be.

>    If not, then a trivial numeric id scheme is a pretty
> lousy namespace to work from.  Diameter's vendorId/
> avaCode model is not much better, but at least has
> some partitioning.

SNMP's OID hierarchy has been found to be a good namespace to work from for
its extensibility aspects and partitioning, and it has lots of room for
extension. Are you advocating SNMP again? tsk, tsk. ;-)

> 
> 
>    The only possible unique aspect I could picture is
> based on observing NFv2,5,7.  In all of these versions,
> all data items were fixed width (typically 32-bit
> integers).  So...  If IPFIX wants to say that only
> fixed with data values may be sent via the IPFIX
> protocol, then I'd say, maybe you could optimize
> beyond IPDR for that.
> 
> 
>    A minimal implementation of IPDR on a middlebox
> would NOT need to have any XML knowledge whatsoever.
> As a producer, the system could merely hardcode the
> information model produced (or make it configurable).
> 
>    The encoding itself follows XDR format, which has
> worked alright for NFS and RPC, and is the precursor
> to the encoding for DCE and CORBA encoding.  It is
> a very simple model.  Read RFC 1832.  An implementation
> for what IPDR requires is pretty trivial.  IPDR
> members have access to opensource for this in
> both Java and C.  And as Stas pointed out in an
> earlier memo there are > 10 implementations out
> there.
> 
>    The other benefits from IPDR which I cited, such
> as XML mapping, DB, binary file format, etc. are
> just that, additional benefits.  They ARE NOT
> REQUIRED to be implemented.
> 
> 
> Regards,
> 
>    Jeff Meyer
> 
> 
> 
> 
> Sebastian Zander wrote:
> 
> > Jeff Meyer wrote:
> > 
> > [...]
> > 
> >>
> >>   So you have one description language, based on
> >> XML Schema, a wire protocol, a binary format,
> >> an XML format and a DB mapper, in a single document.
> >>
> >>
> >>   Show me how that is done w/ the  other proposed
> >> drafts?
> > 
> > 
> > Jeff,
> > 
> > 
> > its very nice to have these things but I don't think its
> > a good idea to have them in one document. IMHO the main
> > focus here is on the protocol. Working on all the things
> > at once will only slow down the standardization process.
> > I am also wondering if the IETF would ever standardize
> > things like binary formats. Probably not.
> > 
> > Furthermore I am not convinced that it is a good idea to
> > standardize things like DB mapper or binary formats. E.g.
> > there exist a number of different formats and some people
> > may want to continue using them. But informational RFCs
> > could cover all these things.
> > 
> > To summarize my point: the evaluation should focus on the
> > protocol and its capabilities and not on things storage
> > formats etc.
> > 
> > BTW Diameter has a proposed XML definition as well
> > (draft-frascone-aaa-xml-dictionary-00.html). Another proposal
> > was to use SMIng (draft-schoenw-sming-dia-00.txt).
> > 
> > Cheers,
> > 
> > Sebastian
> > 
> > 
> 
> 
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

---

From: "Peter Ludemann" <[email-masked]>
Date: Sat, 5 Oct 2002 11:35:01 -0700

RE: [ipfix-eval] Discussion of Candidate ProtocolsDavid Harrington wrote
Friday, October 04, 2002 2:39 PM :

The CRANE spec will not be "proprietary" if adopted, so do you have any
other significant objections to it (there have been some improvements since
2000)? Even if it isn't adopted, I still appreciate criticism, so that I can
improve it as a product.

- peter

---

From: Jeff Meyer <[email-masked]>
Date: Tue, 08 Oct 2002 12:57:10 -0700

David,

  Quite a lot of info and questions in your e-mail.

  Regarding your evaluation of IPDR.  If you evaluated this a year ago,
then you wouldn't have seen the additional work for the compact IPDR
format in addition to XML. Since the IPDR advocacy draft is focused
just on Compact IPDR, you might want to read the latest spec (NDM-U
3.1)

  Regarding SNMP and MIBs.  It sounds like we are in violent agreement
that a MIB could be written to address the delivery of IPFIX data via 
SNMP, using
either informs or by defining some indexing scheme to allow individual
flow records to be pulled via get, getnext or getbulk; it's just that it 
wouldn't
produce very satisfactory results.

  My advocacy for IPDR's XML-Schema based approach to data modelling
is based on the following two shortcomings I see with MIBs as they stand
today for accounting information like IPFIX:

   1.  SMI's definition language is based on a subset of ASN.1, although 
this
     has stood in good stead these years, I would submit that the 
toolset available
     for processing XML documents is larger and will continue to outpace
     ASN.1.     In the SMIng draft, the issues on p. 52 state:

   Learn from ODL, XML, ODBMS -  Look at the ODL proposal from TINAC.
      Look at the XML schema work from W3C.  Look at the ODBMS work.

 2. SMIv1 and v2 all require each use of an identifier to
   have an OID bound to it.  In section 3.3 of the SMIng 
   draft they explicitly state that OIDs should not be used
   for any bindings other than SNMP or COPS.

 So I believe that the IPDR XML-Schema subset defined today
will address the needs of IPFIX.  I believe the proposed 
protocol using this data model will efficiently address the
transfer and other requirements.

  I also believe that this approach (with some extensions
using the XML-Schema "annotation" mechanism), could better
address the problem space defined by SMIng, than the
current draft.  But that's another topic...

  Some additional comments inline.

Regards,

  Jeff Meyer

Harrington, David wrote:

> Hi,
>
> Let me be clear that I am advocating no particular solution. I am 
> trying to cut through any hype and understand the technical pros and 
> cons of each approach.
>
> Am I biased? You decide. I don't believe so, and want to dispel any 
> bias assumptions.
> I am a network management technology analyst for the CTO of Enterasys 
> (which used to be Cabletron).
>
> 1) I analyzed LFAP in 1999(?) for its potential as an industry 
> standard and found it lacking. Paul has since addressed most of the 
> points I raised, and we've had many discussions about its pros and 
> cons. It has both.
>
> 2) I was a member of the IPDR and reviewed their early specs. I did 
> not find it compelling. Its development in a fee-based members-only 
> forum limited the industry's ability to review it as it developed. 
> There were obviously many talented individuals involved, but some 
> companies had a larger influence in its design, which makes me believe 
> it may not be very vendor-neutral. It included a number of people from 
> telco companies, where usage accounting and billing has a long 
> history. I pushed for a while to have them submit it to the IETF, but 
> they resisted. As a long-time IETFer, I didn't see much justification 
> for not working within an open standards organization like the IETF.
>
> meetings. Nonetheless, I promoted the replacement of LFAP with Netflow 
> in Enterasys devices because of Netflow's wide-spread deployment and 
> accounting application business. Cisco is our primary competitor, and 
> I would like to see them not control the flow accounting standard. I 
> believe Netflow has both pros and cons technically.
>
> 4) Xacct sought out Enterasys as a potential partner in 2000(?). I did 
> the technology analysis on CRANE for the due diligence. My largest 
> concern was the proprietary nature of the protocol. Just like IPDR, it 
> was developed in a members-only agreement and was denied the 
> opportunity for wide-spread industry review. This was done to capture 
> the intellectaul property rights before making it public. Another 
> major concern was the push for standardizing "buckets for transporting 
> proprietary data in"; I don't believe that transporting a lot of 
> proprietary data models really helps standardization and 
> interoperability. CRANE has pros and cons.
>
> 6) As the co-chair of SNMPv3 WG, I am glad nobody proposed SNMP as an 
> IPFIX candidate because it would likely not be very satisfactory for 
> flow accounting purposes. SNMP has pros and cons, but I don't think 
> SNMP (as currently defined) is well suited to this particular task.
>
> I am not advocating any particular solution, and I believe all the 
> proposals offer some benefits for the stated purpose.
>
> That being said, I have chosen to reply to Jeff's mail because it uses 
> SNMP for comparison and I fail to fully understand the point being made.
>
> Comments inline.
>
> dbh
>
> So IPDR provides a formal machine readable description of a data model 
> just as SNMP does. The difference is IPDR uses an XML schema and MIBs 
> use SMI schemas.
>
> OK.
>
> >
> You make these assertions about SNMP, without any supporting analysis. 
> You seem to do this as the setup for a comparison with IPDR's XML 
> schema, but you don't follow through.
>
> When you say SNMP is geared towards providing views/access into a 
> current system state, are you talking about the SNMP protocol being 
> geared to this, or is it just that currently existing SNMP mibs seem 
> geared to this task?
>
> You talk about historical information being global or coarse-grain 
> counters. Again, are you talking about the SNMP protocol constraining 
> this, or is this an artifact of the mibs that have been defined? How 
> does XML do this differently? If we can model this in XML, couldn't we 
> also model this in a mib? (Does this mean you're advocating SNMP as a 
> viable solution? ;-)
>

We could define a MIB yes, see the two shortcomings I've outlined in the 
introduction.

> I am missing the point about how OIDs hinder fine-grained counters. An 
> OID is used to name an object such as a counter. I don't think the 
> naming has a significant effect on how fine-grained the counter is. 
> Are you asserting that somehow XML-naming better supports fine-grained 
> counters?
>

If you want to define the information produced by IPFIX in a MIB, then I 
would
expect it to be in the form of a table definition, since a given 
category of IPFIX
"records" will all have the same field information.

In defining a table in SNMP, you need to address SNMPs requirement about
being able to lexicographically order each information item.  So the 
indexing
scheme would need srcip,dstip,srcport,dstport and time at least to make a
unique index.

In the XML model you define the record information (using the complexType
declaration) and there is no requirement around defining an indexing scheme.

So I not saying it couldn't be done, just that it introduces information 
which
is not (in my opinion) useful for the IPFIX requirements.

> SNMP mibs typically are also a specifically defined set of metrics 
> which are being collected based on certain observed network traffic, 
> such as bytes in/out or the RMON flow statistics, or other events. So 
> IP Flow information is just a specifically defined set of metrics, 
> like a MIB. Are you advocating we solve the problem by writing a mib 
> module for IP Flows?
>

No, see above.

> Off-loading large amounts of information quickly could be done using 
> SNMP Informs. I certainly agree that using SNMP to gather flow info is 
> probably a bad idea if you mean gathering it up and waiting for 
> applications to poll for it. I would have concerns about the 
> efficiency of SNMP Informs compared to a protocol designed to handle 
> off-loading large quantities of flow information quickly. See RFC2975.
>
> As you pointed out, SNMP makes a clear separation between the data 
> modeling and the transport mechanism. I wouldn't recommend SNMPv3 for 
> IPFIX primarily because of the transport constraints associated with 
> 484 byte messages over UDP, lack of support for streaming data, and 
> repetitive OID attribute identifiers, not because of the data modeling 
> capabilities.
>

See above.

> I disagree that SMIng is looking at revising the essential concept of 
> MIBs. They are looking at revising the SMI to improve ease-of-use and 
> add object-orientation. The essential concept of MIBs as "data models 
> that are extensible without protocol changes" remains the same.
>

One key revision is the restriction of OID usage to SNMP and COPS. 
 IPFIX does
not need OIDs.

> Agreed. An extensible data model has been important to SNMP's large 
> scale adoption, and I believe an extensible data model should be 
> supported by IPFIX. Keep in mind however, that there  are trade-offs. 
> An extensible data model adds overhead and complexity, which may 
> reduce the efficiency of an extensible IPFIX.
>
> aggregate/compare/contrast information from multiple vendors. This has 
> been a major factor in SNMP applications all offering similar limited 
> sets of functionality, and the large number of applications that are 
> little more than mib browsers - device vendors do not provide solid 
> support for fully-compliant industry standard mib implementations, but 
> rather offer lots of proprietary mib data using a standardized 
> modeling language and a standardized transport mechanism.
>
> The more data elements that can be named, the longer the names need to 
> be to differentiate them. OIDs work well in SNMP to provide a data 
> naming space that can be extended as needed without naming conflicts. 
> Integers work well in LFAP to name a limited set of data elements.
>
> XML is extensible, and will suffer the tradeoffs of long names and 
> lots of proprietary extensions just like SNMP.
>

Indeed, good points, but there are also plenty of very useful "RFC 
standard MIBs", these
do not suffer this problem.

Likewise the IPDR schema definition for IPFIX requirements should be 
standardized.

> exporting inappropriate for the SNMP domain? How does IPDR deal with 
> those same profile characteristics differently than SNMP and the other 
> IPFIX proposals?
>

By "profile" I mean the volume of information.

> generalized "data mover" is acceptable, then are you advocating we 
> model IP flow in a MIB and move the data using SNMP?
>

No, but something like SNMP which addresses the requirements of IPFIX.

> acceptable? or would a mib transported over an SNMP that supported 
> TCP, larger messages, and OID suppression/compression be acceptable? 
> If so, you might want to check on some of the work being done by the 
> EOS WG.
>
> >
> You're arguing that we should reuse rather than develop more 
> task-specific protocols.
>
> Then shouldn't you also be advocating using existing mechanisms for 
> data modeling of formal, machine readable descriptions of the data 
> model, such as the SMI rather than XML?
>

You've got all the arguments above, but the conclusion is different
based on my statement above.

> One of the real strengths of SNMP's MIB approach is the allocation of 
> a partitioned namespace, because industry standard data models were 
> developed, and because the enterprise-specific namespace encouraged 
> vendors to use SNMP.
>
> One of the real strengths of SNMP's MIB approach is the specification 
> of a common data modeling language, with very limited syntax options, 
> that allows machine-parsing of any mib and display of any mib information.
>
> One of the real weaknesses of SNMP's MIB approach is the allocation of 
> an enterprise-specific namespace, because vendors used it to develop 
> SNMP's OID hierarchy has been found to be a good namespace to work 
> from for its extensibility aspects and partitioning, and it has lots 
> of room for extension. Are you advocating SNMP again? tsk, tsk. ;-)
>
> >
> >

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Benoit Claise <[email-masked]>
Date: Wed, 09 Oct 2002 13:57:59 +0200

David,

> believe Netflow has both pros and cons technically.
Any serious reservations about NetFlow's technical design that you would 
like to share?

Regards, Benoit.