Subject: Re: [i2rs] Bulk data collection use cases draft submitted

From: Robert Raszuk <[email-masked]>
Date: Wed, 6 Nov 2013 19:34:57 +0100

Hi Scott,

As mentioned yesterday it would be helpful to provide reasons why MRT
(Multi-Threaded Routing Toolkit (MRT) Routing Information Export
Format) http://tools.ietf.org/html/rfc6396 or perhaps it's minor
extension would not work for you.

Sorry for misstating the "M", but for me everything is multiprotocol
:-) And MRT already supports OSPFv2/v3, BGP, ISIS & RIB exports.

Thx,
R.

On Mon, Oct 21, 2013 at 11:02 PM, Scott Whyte <[email-masked]> wrote:
> We've submitted a draft of what we believe to be interesting bulk data
> collection use cases for I2RS, and the concomitant functionality
> missing from existing solutions.  Please read and comment.
>
> Normative and informative references are still to be added, and the
> draft is still rough around the edges.  Data model discussions are
> deliberately left out, hopefully the scope is sufficient.  And of
> course other interesting use cases would be welcome.
>
> Thanks for having a look.
>
> -Scott
>
> ---------- Forwarded message ----------
> From:  <[email-masked]>
> Date: Mon, Oct 21, 2013 at 11:11 AM
> Subject: New Version Notification for
> draft-swhyte-i2rs-data-collection-system-00.txt
> To: Scott Whyte <[email-masked]>, Marcus Hines <[email-masked]>,
> Warren Kumari <[email-masked]>
>
>
>
> A new version of I-D, draft-swhyte-i2rs-data-collection-system-00.txt
> has been successfully submitted by Scott Whyte and posted to the
> IETF repository.
>
> Filename:        draft-swhyte-i2rs-data-collection-system
> Revision:        00
> Title:           Bulk Network Data Collection System
> Creation date:   2013-10-21
> Group:           Individual Submission
> Number of pages: 10
> URL:
> http://www.ietf.org/internet-drafts/draft-swhyte-i2rs-data-collection-system-00.txt
> Status:
> http://datatracker.ietf.org/doc/draft-swhyte-i2rs-data-collection-system
> Htmlized:
> http://tools.ietf.org/html/draft-swhyte-i2rs-data-collection-system-00
>
>
> Abstract:
>    Collecting large amounts of data from network infrastructure devices
>    has never been very easy.  Existing methods generate CPU and memory
>    loads that may be unacceptable, the output varies across
>    implementations and can be difficult to parse, and these methods are
>    often difficult to scale.  I2RS programmatic interfacing with the
>    routing system may exacerbate this problem: state needs to be
>    collected from nodes and fed to consumers participating in the
>    control plane that may not be physically close to the nodes.  This
>    state includes not only control plane information, but elements of
>    the data plane that have a direct impact on control plane behavior,
>    like traffic engineering.
>
>    This document outlines a set of use cases requiring a flexible
>    framework to collect routing system data, and the features and
>    functionality needed to make such a framework useful for these use
>    cases.
>
>
>
>
>
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
>
> The IETF Secretariat
> _______________________________________________
> i2rs mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i2rs

---

From: Joe Marcus Clarke <[email-masked]>
Date: Wed, 06 Nov 2013 14:35:38 -0500

> Thanks for having a look.

Thanks for raising this, Scott.  In terms of the streaming (i.e., live 
data feeds), I see this as a very valuable thing for our traceability 
draft as well.  I feel there is a good compliment there.  Early on I had 
thought I2RS itself (via notifications) could make logging data 
available as a "stream."

With your idea of bulk information collection and querying/filtering, we 
can install a filter to collect specific log messages or specific parts 
of log messages.  This would be very beneficial for troubleshooting and 
consistency validation.

Joe

-- 
Joe Marcus Clarke, CCIE #5384,         |          |
SCJP, SCSA, SCNA, SCSECA, VCP        |||||      |||||
Distinguished Services Engineer ..:|||||||||::|||||||||:..
Phone: +1 (919) 392-2867         c i s c o  S y s t e m s
Email: [email-masked]

----------------------------------------------------------------------------

---

From: Scott Whyte <[email-masked]>
Date: Thu, 7 Nov 2013 14:45:11 -0800

On Wed, Nov 6, 2013 at 11:35 AM, Joe Marcus Clarke <[email-masked]> wrote:
> Thanks for raising this, Scott.  In terms of the streaming (i.e., live data
> feeds), I see this as a very valuable thing for our traceability draft as
> well.  I feel there is a good compliment there.  Early on I had thought I2RS
> itself (via notifications) could make logging data available as a "stream."
>
> With your idea of bulk information collection and querying/filtering, we can
> install a filter to collect specific log messages or specific parts of log
> messages.  This would be very beneficial for troubleshooting and consistency
> validation.

Joe,
Thanks for the feedback.  I think there is a good fit for this
information source in traceability as well;  having consumers
subscribe to different streams, with different parameters, for
different purposes should be very powerful.  Ideally the same system
could support streaming all the logs off to a pub-sub message bus
system for filtering, especially if the number of consumers and
overlapping data is high, which should have some nice scaling
properties.

-Scott

-- 
There was never any more inception than there is now,
Nor any more youth or age than there is now,
And will never be any more perfection than there is now,
Nor any more heaven or hell than there is now.
  -Walt Whitman

---

From: Scott Whyte <[email-masked]>
Date: Thu, 7 Nov 2013 15:28:37 -0800

> :-) And MRT already supports OSPFv2/v3, BGP, ISIS & RIB exports.

Funny enough, I know what MRT is, but didn't have any idea it moved
into the IETF, so the context gave me a bit of cognitive dissonance.
I hear that's common at IETF :)

I read over rfc6369.  I think MRT is a similar use-case to IPFIX or
BMP: if a network node supports exporting MRT data, I would like to
see this framework enable that via a subscription.

If you feel i2rs should leverage MRT as its _standard_ format for
exporting RIB snapshots, I suppose we need to wait a bit to see if
there is enough interest in the more abstract ideas here first.

Thanks for the feeback!
-Scott