Subject: Re: [Netconf] review of "event-notification" documents

From: Andy Bierman <[email-masked]>
Date: Thu, 1 Dec 2016 10:40:57 -0800

Hi,

I worked on the filtering problem a bit more.
Here is filter.yang, a filtering framework, and topic.yang, and example of
a filter type
that can be used in filter.yang. Topic-based filtering was briefly
discussed in Seoul.

IMO filters will continue to evolve over time so the solution needs to allow
them to be added and combined with other filters.  The current YANG
choice-stmt
is not extensible enough or powerful enough.

Andy

On Tue, Nov 29, 2016 at 7:11 PM, Eric Voit (evoit) <[email-masked]> wrote:

> *From:* Andy Bierman, November 29, 2016 8:30 PM
>
> On Tue, Nov 29, 2016 at 8:05 AM, Eric Voit (evoit) <[email-masked]>
> wrote:
>
> Thanks Martin,
>
> More in line.   Also I extracted areas where we already agree to make this
> easier to follow.
>
> > From: Martin Bjorklund, November 29, 2016 6:40 AM
> > > >If I understand the intention correctly, this document is supposed to
> > > >*define* how notifications are sent over NETCONF.  But there is no
> > > >such definition in this document.  Instead it simply repeats
> > > >information already defined in draft-ietf-netconf-rfc5277bis-01.txt,
> > > >and provides lots of examples of how the YANG operations defined in
> > > >rfc5277bis are encoded in XML and sent over NETCONF.
> > > >
> > > >I suggest that this document is rewritten.  Since the idea is to
> > > >replace RFC 5277, it needs to focus on how notifications are sent
> > > >over NETCONF, and not how RPCs are encoded in XML.
> > > I agree -- maybe get rid of it and just have rfc5277bis contain this
> > > text
> > >
> > > <ev> 5277bis is supposed to allow transports other than NETCONF.  If
> we put
> > the NETCONF specific stuff in here we lose that separation.
> >
> > We need *some* document that defines how notifications are sent over
> > NETCONF.  This document needs to have the specification for the
> <notification>
> > element.
> >
> > Then we need a protocol-independent document that defines the concept of
> > streams and subscriptions, stream discovery, etc.
> >
> > I *think* that your intention is that new clients really should be using
> > <establish-subscription> instead of <create-subscription>, since it is
> protocol-
> > independent and support modification and deletion.
> >
> > If we also want to be fully backwards compatible with 5277, I think we
> should
> > create a document that is much closer to the current 5277 - essentially
> just
> > creating a YANG model for the config false data and for the "old"
> <create-
> > subscription>.
>
> We absolutely want to have a full backwards compatible capability.  The
> question is how to best frame this in documents.  It is possible to rebuild
> RFC-5277 with a YANG model.  But then you can't just layer on new
> capabilities driving this work.  (And this is why we need separate
> namespaces.)
>
>
>
>
>
> We need to parse and understand "full backwards compatible".
>
>
>
> Do we want existing implementations to be leveraged into the new
> solution?  Yes
>
> A server should be capable of supporting <create-subscription> for
>
> old, deprecated subscriptions, and <establish-subscription> for the new
> current subscriptions.
>
>
>
> Do we need to update RFC 5277 or replace it? IMO, replace it.
>
> Since the <create-subscription> RPC was never in a YANG module,
>
> it can be left out of the new module.
>
>
>
> <ev> I believe at minimum that <create subscription> should be pulled out
> of the new module.  It needs its separate namespace.   Perhaps nobody is
> ready to advocate for a parallel 5277-legacy YANG model since new
> development should go to the new paradigm anyway.  In this case, we could
> just have a standalone legacy 5277 section in the document for anything
> needed.  This would make things simpler and easier to tease apart.
>
>
>
> Even more radical, I think streams should be removed, even the NETCONF
> stream.
>
> They really serve no purpose now that subscriptions are formalized and can
>
> even be configured.  It is also bad design to couple the output message
> encoding
>
> into the input stream. (e.g., NETCONF stream MUST be XML encoded).
>
>
>
> <ev> Even as we move away from streams, I still can see reasons for it.
> (Adding Balazs & Alex as they have been proponents.)  The biggest reason
> for streams is that a robust customer designed filters are hard.  If a
> vendor/customer/etc. is able to pre-filter notifications or datastore in an
> interesting and useful way not supportable by our filtering semantics, this
> is a good way to allow the pre-filtering. Some examples which could be
> interesting:
>
> ·       Event notifications when new devices connect to a network
>
> ·       Alarms potentially set across multiple YANG models
>
>
>
> Currently, filters are defined as a choice-stmt.
>
> This implied OR-expr is too simplistic. An explicit combination of OR,
> AND, and NOT is required
>
> for different types of filters.  (similar to YANG 1.1 if-feature-stmt
> syntax).
>
>
>
> <ev> Totally agree that we need robust filters.  Figuring this problem
> space out is a full time job.  Figuring out how to encoding filtering
> capabilities supported on different types of constrained platforms is
> non-trivial.  I would love to see someone take this on for the industry.
> Any takers?
>
>
>
> Eric
>
>
>
> Andy
>
>
>
>
>
> As layering upon RFC-5277 cannot give the new capabilities being requested
> of us in places like RFC-7923 (e.g., multiple subscriptions/session), we
> are moving now to put all elements just needed for backwards compatibility
> in the netconf transport draft.  We could also separate all this out into
> another independent backwards compatibility extension.  But we felt we had
> enough drafts in progress where we didn't want/need a fifth one.
>
> > > [AG] FWIW, the scope of each doc is summarized on
> > > https://www.ietf.org/proceedings/96/slides/slides-96-netconf-5.pdf
> > > (slide
> > > #5)
> > > [AG] The key is that the spec for NC comes from the union of 5277-bis
> > > and the NC transport doc
> > > (draft-ietf-netconf-netconf-event-notifications-01.txt) The NC
> > > transport doc is not meant to stand alone.
> > > The doc contains how 5277-bis concepts are realized when using NC and
> > > NC-specific aspects. E.g.:
> > > - the use of NC call-home for configured subscriptions
> > > - backwards compatibility
> > >    - the existence of a NETCONF stream
> > >    - support of /netconf/streams
> > > <ev> Yes, any 5277bis topic specific to only NETCONF transport should
> > > be in netconf-event-notifications
> > >
> > > I agree with Martin that duplicating normative text is bad.
> > > Not having any normative text is even worse.
> > >
> > > <ev> +1.  To help address that, I just built a whole list of pending
> changes
> > across the four drafts.  And in quite a few places I pulled out
> duplicative text.
> > >
> > > - the definition of create-subscription may be moved to this doc so
> > > that other transports would ignore create-subscription and use only
> > > establish-subscription, simplifying the solution
> > >
> > >
> > > That seems wrong since 5277 had create-subscription so it should stay
> > > in 5277bis
> > >
> > > <ev> It is really a style thing so it doesn’t matter that much either
> way.
> > Current thinking is that as we need both the new and old namespaces.
> > Therefore it seems simpler to have anything in the old namespace
> (“create-
> > subscription”) in netconf-event-notification draft.
> >
> > I agree with Andy that anything that comes from 5277 that you need to
> keep
> > for backwards compatibility reasons should go into 5277bis.
>
> Right now it is in 5277bis.  But we already can't have everything needed
> for backwards compatibility since the 5277bis is transport (NETCONF)
> independent.  So it seemed logical to put it NETCONF transport draft.
> (Again we were trying to limit the proliferation of drafts.)
>
> In the end, I am ok as long as it lands somewhere.  So if people prefer,
> we could also have this as a completely separate backwards compatibility
> section + YANG model in 5277bis.
>
> > > - how to issue notifications in JSON are sent using NC (this is also
> > > in 5277-bis). Arguably, it belongs in the NC transport doc
> > >
> > >
> > >
> > > This is poorly defined.
> > > NETCONF does not support JSON encoding and IMO should not define JSON
> > > encoding unless the entire protocol supports it cleanly.
> > > The proposal seems to be to use XML for <rpc> and <rpc-reply>, but
> > > allow some special mode where <notification> is sent in JSON.
> > >
> > > >
> > > >o  Section 2.1
> > > >
> > > >  The text says:
> > > >
> > > >   The NETCONF event stream contains all
> > > >   NETCONF XML event notifications supported by the publisher,
> > > >
> > > >  First of all, since this document is protocol-agnostic, should it
> > > > really define the stream "NETCONF"?
> > >
> > > <ev> Agree, which is why this is going to netconf-event-notification.
> > >
> > > >  Secondly, this would be a new requirement.  There is nothing in RFC
> > > >  5277 that says that a notification is sent on "NETCONF" be default.
> > >
> > > <ev> 5277 section 3.2.3 talks about the default event stream which has
> > > all NETCONF event notifications
> >
> > You're right.  The question is then what is an "NETCONF XML event
> > notification"?  I think the intention was that these would be
> "notifications
> > related to NETCONF", rather than "all YANG-defined notifications".  This
> needs
> > some disucssion.
>
> Agree
>
> > > >  I think this text should be removed.  How notifications are mapped
> > > > to streams is should be out of scope for this document.
> > >
> > > <ev> Yes, streams as a whole were something we deferred for a while.
> Latest
> > thinking is we minimize streams to the degree possible.  Look for legacy
> stuff to
> > be in netconf-event-notification.
> >
> > Do you mean that you plan to update the text around streams?  If so,
> that's
> > fine.
>
> Yes
>
> > > >  In list "filter", change "filter-id" to "id".
> > > >
> > > >  In list "subscription", change "subscription-id" to "id".
> > >
> > > <ev> Model purity-wise you are correct.  With both subscription id and
> filter
> > id, several people expressed they wanted the objects to be immediately
> and
> > obviously differentiable.   Hopefully others will chime in here.
> >
> > I think we should try to keep the same style across IETF documents.
> > Most models do not use redundant qualifiers, especially not for generic
> names
> > like 'id' or 'name' when used as a key.
>
> I am happy with whatever convention the WG chooses.
>
> > > >  In list "subscription", change "startTime" to "start-time" and
> > > > "stopTime" to "stop-time", for consistency.
> > >
> > > <ev> we kept the old names for backwards equivalency to 5277.
> >
> > But there is nothing to be backwards compatible with in this case.
> > The input paramters to the existing <create-subscription> cannot be
> changed,
> > but new nodes should be kept consistent.
>
> Ok.  So you want start-time in the YANG model, and startTime in the RPC.
> That can be done.
>
> > > >  In list "subscription", change choice "push-source" to a better
> > > > name, maybe "egress-interface" (this is how it is described).
> > >
> > > <ev> push-source can also be an IP Address.  Another name possibility
> for this
> > might be “Originates-from”, that is the basic idea.
> >
> > The current draft has:
> >
> >        choice push-source {
> >          description
> >            "Identifies the egress interface on the Publisher from
> >             which notifications will or are being sent.";
> >
> > You probably need to adjust this, and make it clear what the ip-address
> case
> > really means.
>
> agree
>
> > > >  In list "receiver", what is a "multipoint address"?
> > >
> > > <ev> we are trying not to limit receivers to hosts. Perhaps multicast
> address is
> > ok.  Really we would be good with type: inet:host.
> >
> > The type is inet:host already.
> >
> > You should probably clarify the descriptions.
>
> agree
>
> > > >  Remove the leaf "source-vrf"; this should eventually be aligned
> > > > with  draft-ietf-rtgwg-ni-model.
> > >
> > > Perhaps a place for schema-mount?
> >
> > Not really, rather an augment.
>
> Happy to go with whatever convention people want to use.
>
> > > We should leave source-vrf in
> > > place until we have the proper definition.
> >
> > No I say remove it until you have a proper definition.  If you keep it
> you need to
> > have a proper definition of what it is, and it needs to be interoperable
> across
> > implementations.
>
> We will address with the proper convention in the next draft
>
> > > But we could update the text showing there is a pending decision.
> > >
> > > >  You have made the stream name an identity.  In RFC 5277 it was a
> > > > string.  By using an identity, you severly limit how it can be used;
> > > > with a string new streams can be dynamically created at run-time,
> > > > but with an identity stream names must be known at design-time.
> > > >  I think the stream name should be changed back to a string.
> > >
> > > <ev> as the majority of the people in the informal design team were
> against
> > the expansion of streams, this is likely a moot point.
> >
> > I don't know what "expansion of streams" means, and I don't understand
> what
> > "this" refers to in "this is likely a moot point".
> >
> > But if we keep the stream name as an identity we're no longer backwards
> > compatible with RFC 5277, and we severly limit the functionality.  I
> strongly
> > object to such a change.
>
> I am fine with string, especially as:
> (a) we are moving away from strings in favor of filters
> (b) custom streams are likely to be the dominant use.
> Anyone have an objection  to this change?
>
> > > >o  Section 4.1
> > > >
> > > >  The text says:
> > > >
> > > >   If the
> > > >   request is rejected because the publisher is not able to serve it,
> > > >   the publisher SHOULD include in the returned error what
> subscription
> > > >   parameters would have been accepted for the request when it was
> > > >   processed.
> > > >
> > > >  I think this is a pretty weird idea.  It seems extremely difficult
> > > > to implement, and the use case is not clear at all.  In an
> > > > automation deployment, do we expect that the client application code
> > > > contains logic to rewrite itself to send proper requests the next
> > > >  time?   If it is for debugging purposes I think this should be up to
> > > >  implementations to figure out.  We shouldn't add such things to
> > > > standard RPCs.
> > >
> > > <ev> there has been lots of discussion on this one.  The biggest issue
> has been
> > that there are enough variations of parameters where the guidance on what
> > might be acceptable is the only way to make some scenarios work.  (Was
> it the
> > period which was a problem?  Was it the complexity of the filter?)
> Obviously
> > we do need to bound what could be provided back to the subscriber.
> >
> > So then the text should encourage implementations to provide good error
> > messages.
>
> yes
>
> > > The good news is that if a publisher cannot support negotiation, it
> can just
> > send back a failure.  Which is why the requirement is only a SHOULD.
> >
> > SHOULD is too strong.  And even so, this just adds complexity to the
> > specification.  I think this should be removed.
>
> RFC7923 has it as a MUST (see section 4.2.2.)  Going to SHOULD is already
> easing off the requirement.
>
> > > A worse outcome would be if a Subscriber kept guessing at acceptable
> > parameters and pounding the Publisher with load on this.  This would take
> > more resources than providing hints.
> >
> > That would also be quite weird.  But I can't imagine a use case where a
> client
> > needs a certain combination of parameters, the server rejcets them but
> suggest
> > some other parameters that will give the same result, and then the client
> > would use them?  Or worse, the server suggest something that gives
> another
> > result and the client somehow adjust to them?
>
> When a subscription is rejected, we can provide a hint at why.  This is a
> new dampening period, a suggestion to use on-change, etc.  Without this
> hint, a subscriber could just keep guessing at parameters without
> guidance.  That is all negotiation is.
>
> > > >o  Section 6
> > > >
> > > >  The text says:
> > > >
> > > >   The event notifications must also include the subscription-id if
> the
> > > >   establish-subscription was used in its establishment, or if it was
> > > >   configured via an operational interface.
> > > >
> > > >  How is this "sucbscription-id" supposed to be included?  Where?
> > > >  There is no such field defined in a <notification>.
> > >
> > > <ev> Unlike yang-push, the Notification events are not specified via
> the
> > document.   The examples following the requirement do not include a
> > subscription-id when they absolutely should.  (And this proves the point
> that
> > these are needed :-).   We will update the examples.
> >
> > Well, examples are good, but you also need a normative definition.
>
> Will do.
>
> Eric
>
> > /martin
> >
> >
> > >
> > > Thanks again,
> > > Eric
>
>
>