Subject: Re: [i2rs] Call for WG Adoption: draft-atlas-i2rs-problem-statement-01

From: Scott Whyte <[email-masked]>
Date: Wed, 4 Sep 2013 10:02:15 -0700

On Thu, Aug 15, 2013 at 12:06:35PM -0400, Alia Atlas wrote:
> Exactly - we need to give some real thought to the notifications and how
> the filtering is set up.  There's also a distinction between event
> notifications and information streams, even though subscription may be
> similar.
> 
> For notifications, we have ideas such as:
>     register for a notification when state written by client X is changed
>     register for a notification when a particular state is changed
>     notify when route is installed in FIB
>     notify if interface utilization is over high-water threshold or under
> low-water threshold
> 
> For info streams, one could have:
>     subscribe to OSPF peer up/downs
>     subscribe to route changes (perhaps by protocol or within a prefix
> range)
> 
> There's the question of describing the desirable notifications and info
> streams - and then figuring out how to support them in protocol mechanisms
> (and the practicality for a real implementation).

Alia,

I would like to see a subscription for either information streams or
notifications, perhaps a field in the subscription offer could determine
whether its push or pull.  Similarly, a single system should be able
to provide data subscriptions with sub-second resolution and
multi-hour resolution, by negotiating the frequency at a subscription
start.

In the architecture draft you speak of structured, filterable
information and events.  It would ideal if we can focus on the
structure and filter mechanisms, and leave it up to an implementation
to decide what to provide.  

In order to get as much data off of a node with as little impact as
possible, we clearly need to require as little as possible in the way of
complexity on the node.  If the node can provide a list of things it
publishes, with structure discoverable via introspection, filtering
on the node should be as simple as only subscribing to what a consumer
is interested in.  Anything beyond that should be out of scope IMO and
more complex filtering, authentication, reliability, etc lies in the
pub/sub system deployed to collect the data.

Apache Thrift has some interesting serialization solutions, in
particular I think we should strongly consider decoupling transport
from serialization, and providing a way to map in-memory
data-structures to a negotiable wire format.

-Scott
> 
> Alia
> 
> 
> On Thu, Aug 15, 2013 at 12:00 PM, Andy Bierman <[email-masked]> wrote:
> 
> > Hi,
> >
> > Maybe it won't be that hard to automatically setup filters
> > for common use-cases. E.g., if the 'inject' operation
> > had parameters like 'notify-when-ready' and 'notify-if-bounced',
> > then the server can send those notifications (just to that client)
> > if the "inject" fails or gets deleted later, for the specific data entry.
> >
> >
> > Andy
> >
> >
> >
> >
> > On Thu, Aug 15, 2013 at 7:39 AM, Alia Atlas <[email-masked]> wrote:
> > > Hi Andy,
> > >
> > > I meant that the traceability and syslog aspects are a 2nd order problem.
> > >
> > > I agree very strongly that notifications are a primary concern.  That's
> > how
> > > clients learn
> > > about network changes, changes affecting their state, etc.  I believe
> > that
> > > we need a
> > > sophisticated mechanism to filter and send just the relevant events to
> > the
> > > specific client.
> > > Sending all notifications to all clients would be bad in terms of scaling
> > > and security.
> > >
> > > I don't have a fixed number for the notifications per second.  I'm
> > curious
> > > if others have thought
> > > about it?
> > >
> > > Alia
> > >
> > >
> > >
> > > On Thu, Aug 15, 2013 at 10:31 AM, Andy Bierman <[email-masked]>
> > wrote:
> > >>
> > >> Hi,
> > >>
> > >> We are going to run into several long-unsolved NM problems
> > >> like "standardized log files".  (Knowing that adding such a huge
> > >> task to the charter would be a mistake is the difference between
> > >> boiling an ocean or just a lake. ;-)
> > >>
> > >> Notifications may not be a 2nd order problem because the
> > >> WG punted on the "I2RS broker" role.  Now it is up to
> > >> each I2RS Client to know that the RIB state it injected
> > >> got replaced by a higher priority client, or know a higher
> > >> priority entry was deleted and now maybe it can try again.
> > >>
> > >> IMO, the notification mechanism will need to be quite
> > >> sophisticated to filter just the events relevant to a specific client.
> > >> More likely all clients will each get all notifications, creating
> > >> N fire-hoses of 'add' and 'delete' events.
> > >>
> > >> At the BoF, the metric "5000 updates a second" was mentioned
> > >> as the answer to the question "what is fast-path?"  How many
> > >> notifications per second are reasonable for I2RS?
> > >>
> > >>
> > >> Andy
> > >>
> > >> On Thu, Aug 15, 2013 at 7:10 AM, Alia Atlas <[email-masked]> wrote:
> > >> > Juergen,
> > >> >
> > >> > Thanks for the useful information.  I agree that a focus on
> > traceability
> > >> > is
> > >> > not an initial primary goal, but Joe's point that syslog isn't
> > >> > vendor-agnostic is a good one too.  I'm not sure that I see a network
> > >> > application doing automated troubleshooting and recovery -
> > >> > but perhaps I'm insufficiently optimistic.
> > >> >
> > >> > Alia
> > >> >
> > >> >
> > >> > On Thu, Aug 15, 2013 at 3:06 AM, Juergen Schoenwaelder
> > >> > <[email-masked]> wrote:
> > >> >>
> > >> >> On Wed, Aug 14, 2013 at 05:46:30PM -0400, Joe Marcus Clarke wrote:
> > >> >> > On 8/14/13 5:35 PM, Alia Atlas wrote:
> > >> >> > > I can see having it in a file for displaying or sending - but
> > >> >> > > actually
> > >> >> > > having the I2RS agent crawl through it to report the data back???
> > >> >> >
> > >> >> > A file/buffer is fine, but will lead to vendor-specific ways of
> > >> >> > getting
> > >> >> > at the data.  As I read the architecture concerning information
> > >> >> > collection, I thought this might be very useful information for a
> > >> >> > Client
> > >> >> > to collect from the Agent.
> > >> >> >
> > >> >> > I know Joel disagrees in favor of syslog.  I think syslog is okay,
> > >> >> > but
> > >> >> > only if you're receiving messages since the beginning of time
> > (i.e.,
> > >> >> > before a problem is identified).  While I very well may be off base
> > >> >> > here, having the a record of what Clients did what on a particular
> > >> >> > Agent
> > >> >> > in a vendor-agnostic way as information to be collected via I2RS
> > >> >> > seemed
> > >> >> > reasonable.
> > >> >>
> > >> >> Just for your information: RFC 5277 defines a NETCONF extension for
> > >> >> notification delivery which supports the playback of event streams.
> > >> >> This essentially allows implementations to buffer notifications
> > (which
> > >> >> might be syslog messages) for a certain time (or a certain volume)
> > and
> > >> >> one can request a replay of the notifications from a given start time
> > >> >> until a certain stop time, optionally applying a filter to it.
> > >> >>
> > >> >> For debugging purposes, such a mechanism (buffering and selected and
> > >> >> filtered playback) seems a reasonable solution that is also
> > relatively
> > >> >> easy to implement. If, however, complete accountability tracking is
> > >> >> the goal, then you need to stream all notifications to an external
> > >> >> server and you probably also need something like SYSLOG signatures
> > >> >> (RFC 5848) to be able to prove that the notification log is complete
> > >> >> etc.
> > >> >>
> > >> >> Personally, I think i2rs is likely better off with postponing work on
> > >> >> event notification logging until the primary goals of i2rs have been
> > >> >> met. For early implementors, protocols like SYSLOG likely are good
> > >> >> enough.
> > >> >>
> > >> >> /js
> > >> >>
> > >> >> --
> > >> >> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> > >> >> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> > >> >> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
> > >> >
> > >> >
> > >> >
> > >> > _______________________________________________
> > >> > i2rs mailing list
> > >> > [email-masked]
> > >> > https://www.ietf.org/mailman/listinfo/i2rs
> > >> >
> > >
> > >
> >

> _______________________________________________
> i2rs mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i2rs