Subject: Re: [Netconf] subscribed-notifications-12

From: Martin Bjorklund <[email-masked]>
Date: Thu, 07 Jun 2018 15:43:42 +0200 (CEST)

"Eric Voit (evoit)" <[email-masked]> wrote:
> Hi Tom,
> 
> Thanks for the review.  You can see the updated draft at:
> https://github.com/netconf-wg/rfc5277bis/blob/master/draft-ietf-netconf-subscribed-notifications-13.txt 
> 
> Some details inline...
> 
> > -----Original Message-----
> > From: t.petch, April 19, 2018 7:42 AM
> > To: Eric Voit (evoit) <[email-masked]>; Kent Watsen <[email-masked]>;
> > Alexander Clemm <[email-masked]>; [email-masked]
> > Subject: subscribed-notifications-12

[...]

> > There are a number of Informative References which I think should be
> > Normative namely

[...]

> > 7950
> 
> I believe this one is an informative reference.  I.e., the reference
> in the YANG model is to an example of analogous behavior from HTTP2.
> My reading of rfc7950 section 7.21.4 doesn't seem to require a
> reference to be normative.  If it does, I can move the information
> into the description.

7950 needs to be normative b/c you define a YANG 1.1 module in this
document.

/martin

> 
> Thanks again,
> Eric
> 
>  
> > Tom Petch
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 7 Jun 2018 13:55:36 +0000

> > > To: Eric Voit (evoit) <[email-masked]>; Kent Watsen
> > > <[email-masked]>; Alexander Clemm <[email-masked]>;
> > > [email-masked]

Updated.   It makes sense that every YANG 1.1 draft should have this as a normative reference.

Eric

---

From: Kent Watsen <[email-masked]>
Date: Tue, 12 Jun 2018 22:38:17 +0000

Please look for <Kent9> below.

Next, I view this as a Security Consideration, since nefarious things can happen when a device reboots and note that a DoS could extend the gap to hours or days.  I think that this draft (the text above) is watering down the issue.  In my view, this is a huge consideration, along the lines of a receiver really MUST always try to use a dynamic subscription to fill in any gaps.

<Eric6> I agree this is a best practice, but a MUST would require telemetry receivers to have to support dynamic subscriptions.  I can see IoT receiver implementations where this wouldn’t be likely.

<Kent6> Good point, but I don't see in the YANG module establish-subscription being optional to implement.  Is that an oversight?

<Eric7> This is ok.  It is totally fine for a publisher to support dynamic subscriptions.  But a receiver need not.  Perhaps a lightweight IoT client just will just be a configured subscription receiver.

<Kent7> I think you misunderstood me.  Using your IoT example, even though a device (or the entire IoT space) only uses configured subscriptions, the current module doesn't enable a server to not support dynamic subscriptions.   For constrained devices, having to implement something never used could be a problem…

<Eric8>  The requirement is that a publisher must support a dynamic subscription.  There is no requirement for that on a configured receiver.   To clarify this, I have tweaked early “Configured Subscriptions” section text to say:

On the publisher, supporting configured subscriptions is optional and advertised using the "configured" feature. On a configured receiver, support for dynamic subscriptions is optional except where replaying missed event records is required.

<Kent8> I understand that supporting dynamic subscriptions is currently a requirement.  I am challenging that requirement.  Why is it a requirement?  Does it have to be a requirement?

What if an IoT device only wants to support configured subscriptions and having code to support dynamic is wasting space?    FWIW, I realize that not supporting dynamic subscriptions also means that it would be impossible to filling in gaps introduced by a reboot, but maybe that's a decision that the vendor can/should make for themselves?

<Eric9> In RFC-5277, all you have is dynamic subscriptions.  So support for that older spec by definition makes dynamic subscriptions mandatory.  Beyond that, newer specifications like RFC-7923 as well as sections of other documents like RFC-7921, section 7.6 identify dynamic subscriptions as mandatory for a subscription service.  So at least some use cases exist where such dynamic support is mandatory.

<Kent9> Does it?   I mean, this draft doesn't obsolete 5277, so it seems that server can optionally support one or the other or both, and when it supports this draft, can't it use a feature statement to limit dynamic subscriptions?

With your IoT publisher use case above you are asserting that dynamic subscriptions are not needed for configured subscription only publishers – i.e., there are a class of publishers which have been driven by use cases not considered by the documents referenced above.  So who has documented the need configured subscription only publishers?   I can’t point to such documentation (beyond IoT case above).  Is such a possibility worth slowing down this spec?     In the end making the fix for this specification which you seem to want is itself really quite trivial: we can make both dynamic and configured subscriptions optional.  The reason I have been resisting it is that this solution (a) leads to more complexity for implementers as yet another feature would have to be advertised as optional, (b) this waters down the mandatory capabilities support of the YANG module, and (c) we would need to include some a constraint that at least one of the two optional features needs to be supported.  Also for (c) AFAIK, features don’t support the application of such constraints, so it would have to be done in the feature descriptions themselves.

I guess the text above is a long way of saying that if you assert the optional dynamic subscription is mandatory to progress the document, I will make the change.  But the change will impose complexity costs which to me are hard to justify.

<Kent9> that's a reasonable answer, but mind you that it was your IoT use-case originally.   I'd like to get other opinions.  Yes, trivial to add now, hard to add later, more flexibility for servers, almost no additional effort for clients.  FWIW, I'm planning to add a feature statement for "periodic connections" in the ietf-[net|rest]conf-client-server drafts for similar reasons, that the server just might not want to support them, and I don't want the minimal bar to be higher than needed.

<Kent8> Separately, "configured receiver" is not a defined term, though I see it appearing eight times in the -13 draft.  FWIW, there are no instances of "configured publisher".   Firstly, I'm not entirely sure what a "configured receiver" is, I think it's suppose to mean "a receiver of a configured subscription", yes?  Next, I'm unsure if the fact that the subscription was configured vs dynamic is important/relevant for these eight instances; if not, then s/configured receiver/receiver/, else if it is relevant/important, then maybe s/configured receiver/receiver of a configured subscription/?

<eric9> Made the change to “receiver of a configured subscription”

 <Kent9> ok

<Kent6> that's helpful.  Perhaps tack onto the end of the last sentence something like ", assuming the publisher supports the ['dynamic' and] 'replay' feature[s]"?

<Eric7>  Added
“, assuming the publisher supports the "replay" feature.”
As dynamic isn’t an optional feature.

<Kent7> okay, that's expected, but see my previous Kent7 comment, I think that we might want dynamic to be optional.

<Eric8> Per above, receiver support for dynamic is what is optional.

<Kent8> I understand that this is what's written, but I'm challenging that premise (see my previous comment on this)

<Eric9> This comment will then fate-share the resolution from the previous comment.

<Kent9> ack.

It almost begs the question for why configured subscriptions are supported at all.

<Eric6> Because there have been lots of industry requests.  And lots of implementations with non-IETF solutions.  Personally I prefer dynamic subscriptions.

<Kent6> I can see the appeal of configured subscriptions, but if each receiver of a configured subscription SHOULD always do a dynamic subscription on every publisher reboot, it is already being coded to support dynamic subscriptions, at which point it seems that maybe it should've just been implemented to support dynamic subscriptions from the start (on top of the publisher-initiated call-home transport, of course).   Can you say some more about these industry requests and, in particular, if the requests are still strong even when made aware of the need to also do dynamic subscriptions?

<Eric7>  Here is an industry request:  stream event record entries being placed into security logs (e.g., Integrity Measurement Architecture audit events).   These events are only relevant starting at boot time.  Losing events means a system cannot be validated.  And days after a boot, streaming all the events since boot would result in too many events.   So, in this environment, doing a dynamic subscription to fill the gap is acceptable.

<Kent7> meh

<snip/>

<Kent4> this I agree with, but I really don't like the fact that receiver MUST do a short-lived dynamic subscription to scoop-up any possibly-missed logs, for which there may be none.  Perhaps we could add more values into the "subscription-started" notification message that would enable to receiver to make a local determination if such a dynamic subscription would be  helpful?

<Eric5> I recommend against providing extra objects/reasons in the “subscription-started” at this time.  Publishers might not want to advertise a reboot, and they might not want to advertise why there was loss in event continuity.   All that should matter to a receiver is that such a discontinuity existed, and they have a way to try to fill event the gap should they care.  If the need for more data and the cause of the discontinuity turns out to be required, we can always augment here with future objects.

<KENT5> first, I'm still not 100% sure if this is just a reboot problem, or any time the subscription is restarted/resumed.

<Eric6> Per above: retrieving missing event records is not a reboot specific problem.  But unintentionally replicating event records is reboot specific.  (Otherwise the configured replay-start-time would drive a repeat of everything on each and every reboot.)

<Kent6> okay, I think I got it this time.  Having a *configurable* replay-start-time is so confusing.  Is it really worth having?

<Eric7>   Yes it is worth having.

(a) In many environments, reboot is very infrequent.  Without configurable start time, an operator setting up a configured subscription would not have the ability to designate what to send.  It could only send the full log (at whatever size).

(b) on-publisher security or troubleshooting diagnostics might identify a breach or some event where streaming recent historical event records is a MUST.  As a result, it might want to stream a subset of event records off a box going back in time to potential events which might have been evidence or contributing factors.

<Kent7> Let me come at this another way.  Assume we drop all support for *configurable* replay-start-time.  As such, configured subscriptions always start with the next-generated event (no replay at all).   This covers most use-cases, right?   For those receivers that really wanted the older logs, can't they just do a dynamic subscription to collect them, same as we've been discussing above?

<Eric8> Some reasons this might not always be practical:

(a) IoT devices just might want to passively listen to event streams of Telemetry.  (I.e., this would force configured receivers to support dynamic subscriptions.)

(b) This forces complexity onto applications which only ever need to track what has happened since boot.  (E.g., per above, continuous Integrity Measurement Architecture (IMA) boot log streaming and evaluation.)

(c) Publisher access permissions for who can use the establish-subscription RPC might have to be expanded to include lots of configured receivers.  This might open up a vector to control plane DDoS.  Right now the access permissions would just have to allow the receiver read access to the event records.

(d) A publisher may choose to firewall classes of receivers (or locations of receivers) into a listen-only mode without the ability to establish subscriptions.

<Kent8> This response seems to address the "can't they just do a dynamic subscription" aspect of my comment, but doesn't really address the "why is it important" (I paraphrase) part.  My contention is that the concept of a *configurable* replay-start-time seems confusing and of low value.   I acknowledge that there is some value, but it seems like the value is limited to a one-time start-up optimization that can be alternatively addressed by a dynamic subscription to fetch earlier events (assuming it's allowed, per your points b-d).   Additionally, FWIW, I've never seen such a feature implemented before, and logging mechanisms have been around for decades, so this makes me think that this is something that probably isn't worth having.

<Eric9> As you point out, the why "can't they just do a dynamic subscription" is covered, and we shouldn’t always assume away (b)-(d) as they can matter in some scenarios.  So if we want to support the use case of streaming log entries made after boot, but before the transport session is available, the only alternative I see is to have a configured replay-flag rather than a configuring a start-time.  Are you ok with a flag instead?  Or do you have an alternative suggestion?

<Kent9> see below.

In terms of using this configured replay capability, Cisco’s Integrity Verification application

https://www.cisco.com/c/dam/en/us/td/docs/cloud-systems-management/application-policy-infrastructure-controller-enterprise-module/1-5-x/integrity_verification/user-guide/Cisco_Integrity_Verification_Application_APIC-EM_User_Guide_1_5_0_x.pdf<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.cisco.com_c_dam_en_us_td_docs_cloud-2Dsystems-2Dmanagement_application-2Dpolicy-2Dinfrastructure-2Dcontroller-2Denterprise-2Dmodule_1-2D5-2Dx_integrity-5Fverification_user-2Dguide_Cisco-5FIntegrity-5FVerification-5FApplication-5FAPIC-2DEM-5FUser-5FGuide-5F1-5F5-5F0-5Fx.pdf&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=YLzifR1978kb_hHj64ZtYbrlHE2fJaofeSKu9OAFQXg&s=Vc8m5WAJJE8YkQIpZuxlnVTgAtVKQZ-n0dyoRKX3Eao&e=>

does do a shell access event log fetch of the full log after boot, and then just does incremental fetch the deltas of the log (based on log line numbers).  This application is interested in configured subscriptions subsequent to boot for this purpose.  So such incremental streaming of portions of syslog after boot seems like a typical/common need to me.

<Kent9> it might be typical/common desire, but it's still once in the lifetime of the configured subscription.  It seems like, if the device supports dynamic subscriptions, after receiving subscription-started, the client could a) pause the configured subscription, b) use a dynamic subscript to fetch the missing logs, and then c) resume the flow of logs from the configured subscriptions.

/Kent9

<Eric4>  Tweaked a Section 2.4.2.1 sentence to say:

This document puts no restrictions on the size or form of the log, where it resides within the publisher, or when event record entries in the log are purged.

I suggest adding text that clarifies this, and details the need for a short-lived dynamic-subscription.

<Eric4> The tweak above, with the suggested text in the Implementation Considerations section above hopefully covers this.

<KENT5> the "purged" part helps, but whyis this information buried inside a section titled "Requesting a replay of event records"?

<Eric6>   This is the second paragraph of the first section which discussed replay.  It is the sentence after the optional feature of replay is introduced.

<Kent6> I understand how it fits into this section, but it seems like it should be in a section called something like "replay log", since it's equally applicable for configured subscriptions (and s2.4 is about dynamic subscriptions).  Of course, if we remove replay from configured subscriptions, then leaving it here makes sense…

<Eric7> Per above, there is lots of value in configured replay.

<Kent7> Yes, but see again my challenge to that assertion.  I'm leaving this here in case the decision is flipped…

<snip/>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Wed, 13 Jun 2018 22:33:58 +0000

Now <Eric10>

From: Kent Watsen, June 12, 2018 6:38 PM

<Eric10> Per below, I am ok to make dynamic subscription support optional (even if I don’t believe this is the right decision).  Part of the fix in the YANG Model description text would be to note that either dynamic or configured must be supported.

<Eric10> Lets go with whatever opinions people have.  I will adapt accordingly.   Do you want me to start an independent thread?

<snip>

<Kent9> see below.

<Eric10> Your proposal still precludes (b)-(d) above.   In addition for your step a), there is no RPC or action which allows the event records from a configured (or dynamic) subscription to be paused.  The solution also adds complexity into the client to recognize that early events might be missing, to issue an establish-subscription, and then to tie the results of the independent subscriptions together.

Supporting a “replay” flag on a configured publisher is simpler.

Eric

/Kent9

<snip/>

---

From: Kent Watsen <[email-masked]>
Date: Fri, 22 Jun 2018 21:01:43 +0000

Please look for <Kent10> below.

<Kent10> why don't you ask the WG?  "Should we support servers having only configured subscriptions (i.e. no dynamic subscriptions)?"  FWIW, the ietf-*conf-server modules have features around both the "listen" and "call-home" subtrees.  Heck, you might think "listen" would be mandatory (per RFC 6241), but still we support the possibility of a server only supporting call-home…

<Kent10> yes, please ask the WG

<Kent9> see below.

<Kent10> pausing can be implemented by the receiver not reading any more from the TCP socket, or something else.  How is it any more complex for the client/receiver than the following in the SN draft already?

   When a receiver of a configured subscription gets a new

   "subscription-started" message for a known subscription where it is

   already consuming events, the receiver SHOULD retrieve any event

   records generated since the last event record was received.  This can

   be accomplish by establishing a separate dynamic replay subscription

   with the same filtering criteria with the publisher", assuming the

   publisher supports the "replay" feature.

Eric

/Kent10

<snip/>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Sat, 23 Jun 2018 05:50:33 +0000

<Eric11>

From: Kent Watsen, June 22, 2018 5:02 PM

 <Eric11>  Started thread.

<Kent9> see below.

<Kent10> pausing can be implemented by the receiver not reading any more from the TCP socket, or something else.

<Eric11> There is no mechanism for a receiver to pause a single subscription without pausing other subscriptions on the TCP session (as subscriptions typically would share a common TCP.)

How is it any more complex for the client/receiver than the following in the SN draft already?

<Eric11> It is the same general process.  But it turns the SHOULD into a MUST for applications which need to know the events since boot.  It also doesn’t deliver the events in order to the application, delaying application event analysis.

Eric

Eric

/Kent10

---

From: Kent Watsen <[email-masked]>
Date: Mon, 25 Jun 2018 20:28:30 +0000

Please look for <Kent11> below.

 <Kent11> thanks!

<Kent9> see below.

<Kent11> Different "receivers" of different configured subscriptions pointing to the same underlying netconf or restconf call-home connection?

<Kent11> here's another question that might be good to raise to the WG level.   Please be sure to capture my general concern and also the availability of this workaround.  Thanks.

Eric

/Kent11

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Mon, 25 Jun 2018 21:39:15 +0000

And <Eric12>

From: Kent Watsen, June 25, 2018 4:29 PM

<snip>

<Kent9> see below.

<Eric12> Yes

<Eric12>  You are welcome to take the question to the WG level.  I have no desire to waste people’s time with such an obvious question:

- The current solution does not add the extra complexity described above for configured subscription replay.

- The current solution supports deployment scenarios (b)-(d) above.

- The current solution has far less implementation complexity and error reconciliation states for the client.

/Eric12

Eric

/Kent11

---

From: Kent Watsen <[email-masked]>
Date: Thu, 28 Jun 2018 15:19:01 +0000

Please look for <Kent12> below.

<Kent9> see below.

<Eric12> Yes

<Kent12> Ack.  So, *if* we were to do this, the client would either have to pause all the subscriptions, or do a dynamic fetch in parallel.  Hmmm, given that we're talking about the *configured* replay-start-time, which kicks in after a reboot, all the subscriptions would be restarted simultaneously (right?), so maybe this isn't a big issue?

<Kent12> No Eric, you are the Editor.  We can take this to Montreal if you prefer.   We need more opinions to break the standoff, and I don't think folks are watching this thread.  Please try to present the tradeoffs in a fair manner.   BTW, a==b and c==d, AFAICT.  a/b seems true and c/d also seems true, but 1) it is already a SHOULD for the receiver to do a dynamic fetch for already started subscriptions (see quoted paragraph above), so having another SHOULD for newly started configured subscription doesn't threaten of a-d any more than already, 2) it seems really weird to have persistent configuration that only gets used once in the lifetime of a configured subscription, 3) it is good to remove frivolous features, and 4) the current text in draft is confusing about replay-start-time.   #4 is what started this fork in the thread, but rather than fix the text, I'm thinking it might be better to remove the feature.

Kent  // contributor