Subject: [ippm] comments on draft-ietf-ippm-reporting-mib-04.txt

From: Andy Bierman <[email-masked]>
Date: Thu, 13 Nov 2003 08:02:36 -0800

1. General comments

Although a good job has been done documenting this MIB,
IMO it is too complex to be deployable.  The resource
requirements in the agent are very large.  Some of the 
tables can get extremely large, which would lead to 
poor SNMP performance.  The resource limitation mechanisms 
do not align well with actual SNMP agent implementation 
constraints.

This MIB is way too complicated and contains too many 
features that should be provided by an EMS or NMS.  There
are likely to be many implementation difficulties due to
undocumented (and unforeseen) interactions between features.
The MIB should be simplified to collect metric values
and provide aggregation reports which can be retrieved
via SNMP.  Analysis of the reports, threshold based
exception handling, sending email and pages to administrators, 
etc. should be left to the application.

2. sec. 3) SNMP Framework

This boilerplate is out of date.
Use the new template at:
http://www.ops.ietf.org/mib-boilerplate.html

3. sec 4, para 4) RMON model

This paragraph is incorrect. RMON can collect data from 
several interfaces which are not required to be in the
same chassis.

4. sec. 4.1.3.1) Internet addresses 

Why are special address types needed for Src and Dest?

5. sec. 4.1.4)   GMTTimeStamp 

Why is it important to start the timestamp from 2000
instead of 1900?  This is extra work since implementations
are likely to support the standard "seconds since 1900"
format, not this new format.

6. sec. 4.1.6)   Report definition 

This TC is very complex and overloads many functions
into a single bit string.  It should be broken out
into several MIB objects for simplicity and clarity.

7. MIB, TypeP TC)

This design is flawed because the authoritative name
of the protocol is based on the descriptive string found
in an RMON protocol identifier.  However this text string
is not normative and not required to be globally unique
or even consistent across implementations.

8. MIB, IppmReportDefinition TC

This TC overloads event management, report filtering,
delivery options, and report cleanup options in a
long bit string.  This should be split into at least 
4 TCs (or MIB objects).

9. MIB, ippmSystemTime object

There should not need to be a special object for the
system time.  Also, how does the NMS know if the probe
is in proxy mode?

10. MIB, Clock sync objects

   ippmSystemSynchronizationType
   ippmSystemSynchronizationDesc
   ippmSystemClockResolution

These objects should be in a separate MIB module
since this is a generic problem not specific to
IPPM reporting.

11. MIB, ippmSynchronizationTable

Why is a history table of up to 64K clock sync events
needed?  This will take up a lot of memory for very
questionable value.  

12. MIB, point of measure table

  IppmPointOfMeasureEntry ::= SEQUENCE { 
   ippmPointOfMeasureIndex                Unsigned32, 
   ippmPointOfMeasureMgmtAddrType         InetAddressType, 
   ippmPointOfMeasureMgmtAddress          InetAddress, 
   ippmPointOfMeasureTestAddrTypeP        TypeP, 
   ippmPointOfMeasureTestAddr             TypePaddress, 
   ippmPointOfMeasureMetrics              IppmStandardMetrics 
  } 

Why is there a management address for each test point?
How is this used?  Why does the NMS need to know this?
Doesn't the NMS just contact the IPPM SNMP agent to
retrieve data?

Why is the test address of the measurement point needed?
Why isn't it an InetAddress?  How is this used?

13. MIB, ippmMetricEntry

This table is broken because there is only one entry per
metric.  This assumes that the collection capabilities
are the same for every point of measure, which is not very
likely.  The ippmPointOfMeasureIndex should be added
(first) to the INDEX clause in this table.

14. MIB, ippmMetricCapabilities

  ippmMetricCapabilities OBJECT-TYPE 
   SYNTAX INTEGER { 
   notImplemented(0), 
   implemented(1) 

Why is this object needed?  Why not just populate the
table for metrics that are implemented.  The other objects
in this table are irrelevant or unknown if the metric is
not implemented by the agent.

15. MIB, ippmOwnersTable

  IppmOwnersEntry ::= SEQUENCE { 
   ippmOwnersIndex              Unsigned32, 
   ippmOwnersOwner              IppmOwnerString, 

The ippmOwnerOwner object is the object that is used
to create unique index values in other tables (e.g.,
that represent history entries), yet this object is
required to be unique in this table.  The ippmOwnersIndex
object has no value at all and should be removed.  This 
table should be indexed by the ippmOwnersOwner object.

16. MIB, owner address

   ippmOwnersIpAddressType      InetAddressType, 
   ippmOwnersIpAddress          InetAddress, 

Why are these objects in the MIB?  Why does it
matter what the owner's address is in this MIB module?
If this is for sending notifications, then use the
MIBs in the SNMP Applications document, not this table.

   ippmOwnersEmail              SnmpAdminString, 
   ippmOwnersSMS                SnmpAdminString, 

These objects should be removed and the functions of
sending IPPM metric reports in email or phone messages
should be removed.  These functions are too complex for an
SNMP agent.  In addition, the format of the email
or SMS is not defined in any way.  This will not be
interoperable in any way.

17. MIB, ippmHistoryTable

This table is very complex and can get very huge, or entries
will be so short-lived that the table will be unstable and
very difficult to retrieve.  Keeping a timestamp and value
for every metric value collected by the probe is overkill.

18. MIB, ippmHistorySequence

       Network metrics:  
       It's the sequence number of a measurement packet. Typically, it 
       identifies the order of the packet in the stream of packets sent 
       by the source. 
        
       Aggregated metrics: 
       It is the sequence number of the computed aggregated metric 
       result." 

Why is this object useful if the table already has an
index for the measure instance?  Why is an additional
index for the packet sequence needed?  What if the
entry is for a metric that utilizes more than one packet
in the measurement?

19. MIB, ippmHistoryValue
   SYNTAX Integer32 

Why is this an Integer32 instead of Unsigned32?
Could it ever be a negative number?

20. MIB, ippmNetMeasureTable

       " The IppmNetMeasureTable is mandatory, and its content is read 
       only. It means that the measurement software handles the table 
       internally. The setup of the network measure is not permitted 
       through the IPPM REPORTING MIB. As an example, OWAP provides a 
       setup protocol to setup and tear down networks measures. 

This goes entirely against the spirit of SNMP to contain
a configuration table that is populated in an unspecified
manner, and not allowing SNMP to be used for configuration.
These objects should be read-create with a MIN-ACCESS of
read-only.

Also, this MIB should not include this table because this 
table indicates the configuration of test traffic generators.  
The comments in the previous paragraph apply to the MIB module 
that is eventually used for traffic generation.  The 
Synthetic Sources for Performance Measurements MIB (SSPM-MIB) 
should be used for this purpose.

The status and statistics objects in this table should
be split out and placed in another table.  That table 
may be appropriate for this MIB module.

21. MIB, IppmAggrMeasureEntry

   ippmAggrMeasureOwner                  IppmOwnerString, 
   ippmAggrMeasureIndex                  Unsigned32, 
   ippmAggrMeasureName                   SnmpAdminString, 
   ippmAggrMeasureMetrics                IppmStandardMetrics, 
   ippmAggrMeasureBeginTime              GMTTimeStamp, 
   ippmAggrMeasureAggrPeriodUnit         TimeUnit, 
   ippmAggrMeasureAggrPeriod             Unsigned32, 
   ippmAggrMeasureDurationUnit           TimeUnit, 
   ippmAggrMeasureDuration               Unsigned32, 
   ippmAggrMeasureHistorySize            Unsigned32, 
   ippmAggrMeasureStorageType            StorageType, 
   ippmAggrMeasureHistoryOwner           IppmOwnerString, 
   ippmAggrMeasureHistoryOwnerIndex      Unsigned32, 
   ippmAggrMeasureHistoryMetric          Unsigned32, 
   ippmAggrMeasureAdminState             INTEGER, 
   ippmAggrMeasureFastReport             OBJECT IDENTIFIER, 
   ippmAggrMeasureMap                    SnmpAdminString, 
   ippmAggrMeasureResultsMgmt            INTEGER, 
   ippmAggrMeasureLastUpdate             GMTTimeStamp, 
   ippmAggrMeasureOperState              INTEGER, 
   ippmAggrMeasureNbPktsTreated          Counter64, 
   ippmAggrMeasureStatus                 RowStatus 

This configuration table is rather complex.
The ippmAggrMeasureName object is redundant, since
this info is already available in another table.
Recording this long string in every entry is wasteful.

The ippmAggrMeasureHistoryOwner object makes this too complex.
The same value for the owner in this table and the history
table should be used.

The ippmAggrMeasureHistoryMetric object is confusing.  How does
it relate to the ippmAggrMeasureMetrics object in this table?

It is not clear why the ippmAggrMeasureAdminState object
is needed and what it means to start or stop this entry
at any time.

The ippmAggrMeasureFastReport object is complex and
it is not clear what value it adds.  Why is this needed
and ippmReport needed?

22. MIB, ippmAggrMeasureMap
   SYNTAX SnmpAdminString 
   MAX-ACCESS read-create 
   STATUS     current 
   DESCRIPTION 
       "This object allows for classification of the measure. It is 
       typically the name of an administrative map. 

I have no idea what this object is for.  There are no
guidelines for populating this object and therefore
no interoperability possible.

23. MIB, ippmReportPathToResults 

This URL object is a global scalar, which is odd since
multiple reports for multiple owners are created.
Why are results pushed off the box to a file?  What is
the format of this file?  Just SNMP access should be
provided.

24. MIB, IppmReportSetupEntry

This table sets up a report but it looks like each entry
represents one metric.  This seems broken since the
parameters would be the same for each metric collected
in the set (IppmStandardMetrics OCTET STRING).  A different
complex report configuration for each metric is too
complicated.  Since some metrics are related to each
other, it doesn't really make sense to have a report
for each metric.  It would be much better to have
one report for each set of metrics collected.

25. MIB, ippmReportSetupMeasureOwner

Why is this object needed?  This is too complex. Any type
of access control should be done by VACM, not by matching
values of IppmOwnerStrings. Just the integer index pointer
is needed.

26. MIB, thresholding objects in this table

This thresholding mechanism is too complex and too generic 
to be included in this table. 

27. MIB, ippmReportSetupNMS
   DESCRIPTION 
       "The recipient of the report may be provided in the setup. By 

This object is not needed.  SNMP should be used to retrieve data
and other MIBs exist to determine where to send notifications.

28. MIB, ippmReportSetupNotification
   SYNTAX     OBJECT IDENTIFIER 
   MAX-ACCESS read-create 
   STATUS     current 
   DESCRIPTION 
       "Even though the notification to use is defined in the report 
       definition, the object ippmReportSetupNotification provides 
       flexibility to select another notification. " 
   DEFVAL { zeroDotZero } 

This object is too vague and there is no chance for 
interoperability.  This is also too complicated on the agent
to be told by the NMS which notification to generate
for each report.

29. MIB, ippmReportSetupMap
   DESCRIPTION 
       "An administrative name of a map to which the report belongs." 

This object is too vague and has no chance for
interoperability. It should be removed.

30. MIB, ippmReportEntry

This table doesn't seem to add much value beyond the
history table.  It is also too complicated to have
separate results for every metric collected in a
report.  How are all metrics collected for a given
report correlated?  The network and aggregated features
seem too complicated and are not well defined.
A single report for all metrics collected together for
a given test should be provided instead.

31. MIB, ippmReportValue
   SYNTAX Integer32 
   MAX-ACCESS read-only 
   STATUS     current 
   DESCRIPTION 
       "The value." 

Why is this object an Integer32 instead of Unsigned32?
The description is not very useful.

32. MIB, Notifications

It is not very clear the exact conditions and configuration
used by the agent to decide when to send each notification.
It is not very clear why so many different notification types
are needed.  All notifications include a lot of static info that
is not really needed.

33. Security considerations

Very good security section!

34. References

Check the MIB template for update to SNMP references.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: Matthew J Zekauskas <[email-masked]>
Date: Thu, 13 Nov 2003 11:23:09 -0500

As chair, I'd like to say that Andy's comment below is what
I've heard informally, but no one has to-date posted to the
list.  I'd like to ask others to read the draft, and comment
on the complexity (or any other area) --  do you agree, or
do you think that the current draft is on the right track.

With my chair hat off, I concur with Andy's assessment below.
What I always intended to implement with Surveyor was a simple
scheme to export the basic data types; if there was any complex
manipulation, it would be done by the software playing the role
as a network management system.

--Matt

---

From: Matthew J Zekauskas <[email-masked]>
Date: Thu, 13 Nov 2003 11:23:09 -0500

--Matt

---

From: "Henk Uijterwaal (RIPE-NCC)" <[email-masked]>
Date: Wed, 19 Nov 2003 14:01:16 +0100 (CET)

> as a network management system.

Also with my chair hat off.

The way I currently see this (and the way that we've implemented this) is
something like:

                       +------+
             /-------- | User |----------------\
          Control      +------+             Control
             |                                 |
             V                                 V
         +--------+                      +----------+
         | Sender |                      | Receiver |
         +--------+                      +----------+
          |  |                             ^    |
          |  |                             |    |
          |  \--------Measurement ---------/    |
          |                                     |
          V                                     V
        +--------------------------------------------+
        |     Results                                |
        +--------------------------------------------+
              |
              V
        +------------+
        | Processing |
        +------------+
              |
              V
             User

The user requests up a measurement session by sending a control message to
both sender and receiver.  Sender and receiver set up the measurement
using a home-grown mechanism now, perhaps OWAMP-control in the future.
The measurement is done, currently with our own protocol, perhaps
OWAMP-test in the future.  Both sender and receiver record data regarding
the measurement in a file.  After the measurement is over, these files are
pulled to a central point at regular intervals. Here data is processed:
combine what sender and receiver know about a measurement, aggregate,
filter, plot and print on web pages or so, depending on what the user
wants.

The last steps are often done more than once, people tend to aggregate
over short intervals as well as long intervals, make 2 or more plots of
the same data etc.

When talking to users, the biggest issue is that the data is not available
until the file has been written.

A buffer with the data from the last X minutes would solve that.  An
application can then query the buffer and do whatever it wants to do with
the data.  This is where I see a standard format (MIB) and protocol
(SNMP) come in.

What people are not interested in, is a standard way to aggregate and
filter data.  Requirements vary a lot here: time interval, loops over the
data, format (numbers vs a plot), etc, and the one size-fits all approach
doesn't seem right.

Henk

> > etc. should be left to the application.

------------------------------------------------------------------------------
Henk Uijterwaal                             Email: [email-masked]
RIPE Network Coordination Centre            WWW: http://www.ripe.net/home/henk
P.O.Box 10096          Singel 258           Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam    Fax: +31.20.5354445
The Netherlands        The Netherlands      Mobile: +31.6.55861746
------------------------------------------------------------------------------

That problem that we weren't having yesterday, is it better? (Big ISP NOC)

---

From: "Henk Uijterwaal (RIPE-NCC)" <[email-masked]>
Date: Wed, 19 Nov 2003 14:01:16 +0100 (CET)

+------+

Henk