Subject: Re: [netmod] system configuration sync mechanism

From: "maqiufang (A)" <[email-masked]>
Date: Wed, 4 Aug 2021 03:33:38 +0000

Hi, Andy,

Apologize for my chiming in. Please see my reply inline.

From: netmod [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Wednesday, August 4, 2021 1:11 AM
To: Fengchong (frank) <[email-masked]>
Cc: Balázs Lengyel <balazs.lengyel=[email-masked]>; [email-masked]
Subject: Re: [netmod] 答复: system configuration sync mechanism

On Tue, Aug 3, 2021 at 12:34 AM Fengchong (frank) <[email-masked]<mailto:[email-masked]>> wrote:
Hi juergen,
  Please see my comments inline.

  /frank

-----邮件原件-----
发件人: Jürgen Schönwälder [mailto:[email-masked]<mailto:[email-masked]>]
发送时间: 2021年8月3日 12:15
收件人: Fengchong (frank) <[email-masked]<mailto:[email-masked]>>
抄送: Andy Bierman <[email-masked]<mailto:[email-masked]>>; Kent Watsen <[email-masked]<mailto:[email-masked]>>; Balázs Lengyel <balazs.lengyel=[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
主题: Re: [netmod] 答复: system configuration sync mechanism

On Tue, Aug 03, 2021 at 01:45:40AM +0000, Fengchong (frank) wrote:
> Hi andy and all.
>
> I don’t think get-data with origin can solve the issues below:
>
>
> 1.       Some leafs like interface type CAN NOT be modified by user, but can be referenced by other config nodes(e.g. using leafref or occur in when/must). The validation will be fail if these leafs are not be configured by user now explicitly (we assume these leafs are optional and no default value).

The interface type in the RFC 8343 module is config true. YANG does not allow you to refer to config false nodes in constraints that apply to config true nodes. A core principle is that the content of a configuration datastore can be validated without knowing the actual state of the system.

Frank: yes, interface type is a config node, but it isn't allow to be modified. If a config node reference if-type, in order to validate data successfully ,user must have to config if-type with determined value again. For example, user must config if-type with 'ethernet' for ethernet0/0/1. I don't think it makes sense. The if-type has been created by system, when running config is merged to <operational>, it can work and the validation is successful. If you think running datastore is self-contained, I will think the system configuration should be imported to running datastore automatically to ensure successful validation

I doubt another rewrite of ietf-interfaces would gain much support.

There are many objects that have user access restrictions that are not standardized
with machine-readable statements.  I am interested in the 2 use cases that Balazs described.
The system does its own access control (not NACM) wrt/ these objects. The server just
rejects the edit requests, even though the standard YANG indicates it should be writable.
Some standard extensions to tag the data would help applications better predict server behavior.

There is no way in YANG 1.1 to have a config=true node reference config=false in the XPath.
(Well, we implement it but the developer has to enable it).

I do not see the point of the system datastore if it contains config=true nodes,
just so that XPath can reference it.  I do not agree that it is useful or easy to implement,
especially since the contents of <system> are visible in <intended>.  Representing the possible
instances is non-trivial (e.g. pattern allowed, not a fixed name, resulting in 6 million possible values).
[Qiufang Ma] I don’ t see any system configuration handling behavior standards in non-NMDA; NMDA defines that system configuration should only be included in <operational>. I think it necessary to distinguish between system configuration and operational state.
Since there may be some requirements to reference the system configuration, it doesn’t seem sensible to treat system configuration as operational state. I don’t think retrieval from <operational> should be the only way to get system configurations, given the definition of operational datastore.
That is to say, <operational> contains all applied configurations which go through configuration combination and template expansion. Only the system configurations which take effect are included and will be tagged as “origin=intended” once being referenced in <running>. I believe there would be a need to get the pure system configurations regardless of whether being applied or not.

If the system datastore contains config=false nodes, then you are talking about a rewrite
of the YANG validation architecture and a new version of YANG.  IMO there is no need for that.
[Qiufang Ma] I don’t think <system> would contain config=false nodes. A rewrite of the YANG validation architecture and a new version of YANG are not our intention.

Best Regards,
Qiufang Ma

Andy

> 2.       Some instances are generated by system, but these instances can be referenced by other config nodes. The validation must be fail if these instances are not be recreated by user explicitly now.

Yes, a configuration datastore is self-contained. If a client wants to configure the interface x, it has to define the interface x in the configuration. Note that this should not be confused with a system generating an interface x after probing the hardware. Note the difference between operational state, applied config, and running config.

Frank: for example , predefined policies are provided by system, user configuration can reference these predefined policy directly. but because predefined policies are not configured by user explicitly, the validation will be fail. If you want to get a successful validation , user MUST have to redefined the system predefined policies.
So, system predefined data are useless. I think it is unreasonable.

Let me add that the underlying model is that the client(s) have control over the configuration. A system making ad-hoc changes to the config (even with the best intentions) will be surprising. In this model, the only way to inject config into running on system boot is to have a client making changes to running following the normal procedures - at least conceptually. This means that conceptually the other clients need to be aware that there is a system client injecting configuration.

If you follow this logic, it seems wrong to define a system datastore that is somehow magically merged into running - and it is not needed.

Frank: system configuration not only includes some instances generated on reboot time, but also includes the configuration when hardware is plugged in, or a function is enabled (for example, in huawei's implementation, when QOS function is enabled, many qos predefined policies are created by system) or a user-created list instance is created, some leafs' value will be created by system. So , we need a system datastore to hold these configuration.

> 3.       User may need know what the original system configuration is, if we get data from <operational>, you may get the modified system configuration.(for example, user modify or template is expanded, or only active instances)

If you have multiple clients managing shared configuration, then yes it is good if they are aware of what is going on. I am not sure yet that exposing other clients intentions via additional datastores and defining merge mechanisms and semantics is the way to go.

> I don’t care about whether system datastore is imported to running or intended datastores. But I think if a config node reference a system node, the validation (running or intended datastores) will be successful even if the system node is not configured by user explicitly.

I am concerned about having to define what "is imported" means precisely and whether moving to a model multiple datastores have to be merged before validation is the way to go. We already acknowledged that there are template expansions in some implementations without working out how they work.

> Especially on the client side,  if a client need validate all data retrieved from server, the validation SHOULD be successful. If system configuration are not imported to running, at a minimum, the client needs to know what the original system configuration is. Another way is adding with-system-used parameter to get-config operation to retrieved all user configuration and system configuration referenced by user configuration.

Let me repeat, in the original model, the running configuration datastore is self-contained and can be validated without knowing additional datastores.

/js

--
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 3 Aug 2021 20:48:39 -0700

Hi,

I do not agree that a new datastore is needed to supposedly retrieve the
system created nodes that exist (already supported by get-data) or
the nodes that could be created by the server but do not currently exist.
The  <config> element is way too simplistic to represent all the real-world
possibilities.

Current XPath procedures define several contexts for evaluating expressions.
None of these procedures support an additional <system> datastore that is
somehow
merged with the other datastores for XPath purposes.  YANG 1.1 does not
support
this proposal at all.

Andy

On Tue, Aug 3, 2021 at 8:34 PM maqiufang (A) <[email-masked]> wrote:

> *From:* netmod [mailto:[email-masked]] *On Behalf Of *Andy
> Bierman
> *Sent:* Wednesday, August 4, 2021 1:11 AM
> *To:* Fengchong (frank) <[email-masked]>
> *Cc:* Balázs Lengyel <balazs.lengyel=[email-masked]>;
> [email-masked]
> *Subject:* Re: [netmod] 答复: system configuration sync mechanism
>
>
>
>
>
>
>
> On Tue, Aug 3, 2021 at 12:34 AM Fengchong (frank) <
> [email-masked]> wrote:
>
> Hi juergen,
> 发件人: Jürgen Schönwälder [mailto:[email-masked]]
> 发送时间: 2021年8月3日 12:15
> 收件人: Fengchong (frank) <[email-masked]>
> 抄送: Andy Bierman <[email-masked]>; Kent Watsen <[email-masked]>;
> Balázs Lengyel <balazs.lengyel=[email-masked]>;
> [email-masked]
> > 1.       Some leafs like interface type CAN NOT be modified by user, but
> can be referenced by other config nodes(e.g. using leafref or occur in
> when/must). The validation will be fail if these leafs are not be
> configured by user now explicitly (we assume these leafs are optional and
> no default value).
>
> The interface type in the RFC 8343 module is config true. YANG does not
> allow you to refer to config false nodes in constraints that apply to
> config true nodes. A core principle is that the content of a configuration
> datastore can be validated without knowing the actual state of the system.
>
> Frank: yes, interface type is a config node, but it isn't allow to be
> modified. If a config node reference if-type, in order to validate data
> successfully ,user must have to config if-type with determined value again.
> For example, user must config if-type with 'ethernet' for ethernet0/0/1. I
> don't think it makes sense. The if-type has been created by system, when
> running config is merged to <operational>, it can work and the validation
> is successful. If you think running datastore is self-contained, I will
> think the system configuration should be imported to running datastore
> automatically to ensure successful validation
>
>
>
>
>
> *[Qiufang Ma] I don’ t see any system configuration handling behavior
> standards in non-NMDA; NMDA defines that system configuration should only
> be included in <operational>. I think it necessary to distinguish between
> system configuration and operational state.*
>
> *Since there may be some requirements to reference the system
> configuration, it doesn’t seem sensible to treat system configuration as
> operational state. I don’t think retrieval from <operational> should be the
> only way to get system configurations, given the definition of operational
> datastore.*
>
> *That is to say, <operational> contains all applied configurations which
> go through configuration combination and template expansion. Only the
> system configurations which take effect are included and will be tagged as
> “origin=intended” once being referenced in <running>. I believe there would
> be a need to get the pure system configurations regardless of whether being
> applied or not.*
>
>
>
> *[Qiufang Ma] I don’t think <system> would contain config=false nodes. A
> rewrite of the YANG validation architecture and a new version of YANG are
> not our
> intention.
> *
>
>
>
> Best Regards,
>
> Qiufang Ma
>
>
>
>
>
> Andy
>
>
>
>
>
> > 2.       Some instances are generated by system, but these instances can
> be referenced by other config nodes. The validation must be fail if these
> instances are not be recreated by user explicitly now.
>
> Yes, a configuration datastore is self-contained. If a client wants to
> configure the interface x, it has to define the interface x in the
> configuration. Note that this should not be confused with a system
> generating an interface x after probing the hardware. Note the difference
> between operational state, applied config, and running config.
>
> Frank: for example , predefined policies are provided by system, user
> configuration can reference these predefined policy directly. but because
> predefined policies are not configured by user explicitly, the validation
> will be fail. If you want to get a successful validation , user MUST have
> to redefined the system predefined policies.
> Let me add that the underlying model is that the client(s) have control
> over the configuration. A system making ad-hoc changes to the config (even
> with the best intentions) will be surprising. In this model, the only way
> to inject config into running on system boot is to have a client making
> changes to running following the normal procedures - at least conceptually.
> This means that conceptually the other clients need to be aware that there
> is a system client injecting configuration.
>
> If you follow this logic, it seems wrong to define a system datastore that
> is somehow magically merged into running - and it is not needed.
>
> Frank: system configuration not only includes some instances generated on
> reboot time, but also includes the configuration when hardware is plugged
> in, or a function is enabled (for example, in huawei's implementation, when
> QOS function is enabled, many qos predefined policies are created by
> system) or a user-created list instance is created, some leafs' value will
> be created by system. So , we need a system datastore to hold these
> configuration.
>
> > 3.       User may need know what the original system configuration is,
> if we get data from <operational>, you may get the modified system
> configuration.(for example, user modify or template is expanded, or only
> active instances)
>
> If you have multiple clients managing shared configuration, then yes it is
> good if they are aware of what is going on. I am not sure yet that exposing
> other clients intentions via additional datastores and defining merge
> mechanisms and semantics is the way to go.
>
> > I don’t care about whether system datastore is imported to running or
> intended datastores. But I think if a config node reference a system node,
> the validation (running or intended datastores) will be successful even if
> the system node is not configured by user explicitly.
>
> I am concerned about having to define what "is imported" means precisely
> and whether moving to a model multiple datastores have to be merged before
> validation is the way to go. We already acknowledged that there are
> template expansions in some implementations without working out how they
> work.
>
> > Especially on the client side,  if a client need validate all data
> retrieved from server, the validation SHOULD be successful. If system
> configuration are not imported to running, at a minimum, the client needs
> to know what the original system configuration is. Another way is adding
> with-system-used parameter to get-config operation to retrieved all user
> configuration and system configuration referenced by user configuration.
>
> Let me repeat, in the original model, the running configuration datastore
> is self-contained and can be validated without knowing additional
> datastores.
>
> /js
>
> --

---

From: Kent Watsen <[email-masked]>
Date: Wed, 4 Aug 2021 12:34:45 +0000

I am confused by the confusion  ;)

You all know that JUNOS implemented this concept before YANG was even a thing, right?  

Admittedly, it’s not a “datastore“, but flexing the NMDA is where we can do better. 

A “with-system” mechanism could also work.  The only downside is the inability for a client to get only the system configuration, without the rest of <running>.   

Please stop stating/suggesting “config true” nodes are referencing “config false” nodes,  or that config is referencing operational state.  There is no intention to break either of these tenants here.  

I think that some folks just joined the conversation and may have missed out when we covered all this before.  

The draft needs to be updated to more clearly identify the goals. 

K.

---

From: Andy Bierman <[email-masked]>
Date: Wed, 4 Aug 2021 06:19:38 -0700

On Wed, Aug 4, 2021 at 5:34 AM Kent Watsen <[email-masked]> wrote:

> A “with-system” mechanism could also work.  The only downside is the
> inability for a client to get only the system configuration, without the
> rest of <running>.
>
> Please stop stating/suggesting “config true” nodes are referencing “config
> false” nodes,  or that config is referencing operational state.  There is
> no intention to break either of these tenants here.
>

I strongly agree the draft could better explain the problem and the value
of the solution, especially how
this datastore is used to represent system values that are not in use.
Since NMDA already
provides a solution to retrieve the "system-only" nodes that are in use,
there is no value in that part
of the system datastore at all.

Also some pointers to implementations of this system datastore would be
helpful.

I am confused because there was an assertion made that this system
datastore was needed
so that the nodes could be referenced in config XPath.  This is not
correct.  The solution that Balazs
outlined is quite common and much better and less disruptive than adding a
new datastore.
It is trivial to identify the user-write operations that are allowed (e.g.
modify but not create, delete,
or rename a physical interface).

Andy

---

From: Jürgen Schönwälder <[email-masked]>
Date: Wed, 4 Aug 2021 15:39:56 +0200

The figure in RFC 8342 section 5 documents what was agreed upon
before. System configuration flows into <operational> but not upwards
into <running>. Over the years, we discussed several corner cases
(including things like configuring a new user and the system
automatically assigns an unused uid, which afterwards needs to be kept
stable). While there are for sure tricky corner cases, I am not
convinced that the model defined in RFC 8342 for the general cases is
wrong and that merging a new system datastore into <running> is the
right approach. If people want to change the model documented in RFC
8342, then they should make an explicit statement about this and
provide strong reasons that the model is flawed or incomplete.

Note that the model does allow having a system client merging config
into <running> (ideally controlled by an ACM so that such a client can
be turned off if it leads to surprises).

/js

On Wed, Aug 04, 2021 at 12:34:45PM +0000, Kent Watsen wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Wed, 4 Aug 2021 06:58:55 -0700

> be turned off if it leads to surprises).

This is a solved problem in proprietary ways.  It is simple to treat system
config
as an access control issue.

I am quite concerned that NMDA is getting extended in ways that lead to
confusion and poor interoperability.  Adding a new datastore is very
serious.
IMO ANY new datastore (even factory default) should be standardized in
a new version of NMDA (replacing RFC 8342).

A datastore has a lot of baggage
   - YANG library
   - YANG XPath evaluation
   - subtree and XPath filtered retrieval
   - usage in RPC operations (ds:datastore data type parameter)

Every time a datastore is added, all the existing RPC operations that use
datastores need to be clarified wrt/ support for the new datastore.
(Of course this is never done, leading to lots of interoperability issues)

I am quite confused by the XPath discussions because XPath can only
access existing nodes (i.e. the "accessible tree")
https://datatracker.ietf.org/doc/html/rfc7950#section-6.4.1

So what does it mean for the system datastore to contain possible values
that
cannot be represented in <operational>? The accessible tree cannot include
these values, so XPath-based validation cannot use them.

Andy

> > Please stop stating/suggesting “config true” nodes are referencing
> “config false” nodes,  or that config is referencing operational state.
> There is no intention to break either of these tenants here.
> >

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Mon, 9 Aug 2021 21:51:42 +0000

Hi guys,

I'm late to the game on this thread but I did read through the entire thing (whew - can't promise I absorbed every nuance though). I am familiar with this issue. I've been dealing with it from the server implementation side (some clients *do* complain if there are references to missing "system config" list entries).

One of the pretty fundamental issues IMO is whether we want good ol' standard <running> to always be valid (including for an offline client or tool to be able to validate instance data retrieved from a server against the YANG models). Even without this "system config" topic, we already have a looming problem with config templates & expansion.  Servers with YANG models that document a lot of constraints (e.g. leafrefs, mandatory statements, etc) are more likely to hit this issue of an invalid running config (offline).

I'm doubtful that it is easy for a client to know how templates are expanded for all given servers. Unless that expansion is fully standardized (which would be challenging given multiple different shipping implementations in the industry), there can be some subtle corner cases and it can be complex (multiple levels of template application, exclusion rules, etc).

I agree there can be dynamically added system config (e.g. create a new qos policy, and some queue list entries are automatically created inside that policy).

Note that config added by the system could be deletable or non-deletable. For deletable items there were some previous discussions a few years ago that those list entries could be simply a default config that is populated if the startup datastore is empty at boot time.

The system config list entries could also have leafs that are modifiable and leafs that are immutable. For the modifiable objects I think this is basically the system merging the explicit config with the system config (with the explicit taking precedence).

Perhaps a lot of this system config is a legacy hold over from human-driven CLIs. For machine interfaces maybe it's reasonable to have clients explicitly define their config *if* they need offline validation of <running> to succeed (i.e. allow clients to explicitly configure any of the system config policies they are actually using & referencing).  In many cases, users want the "master" view of the config to live up in the client/OSS side and just push it down to the server (without having the server touch the contents of the running, i.e. a read-back should return identically what was sent).

It might be good to set up a virtual interim or call of some sort to discuss this one. It is pretty complex.

Jason

From: netmod <[email-masked]> On Behalf Of Andy Bierman
Sent: Wednesday, August 4, 2021 9:59 AM
To: Juergen Schoenwaelder <[email-masked]>; Kent Watsen <[email-masked]>; Andy Bierman <[email-masked]>; NetMod WG <[email-masked]>
Subject: Re: [netmod] system configuration sync mechanism

On Wed, Aug 4, 2021 at 6:39 AM Jürgen Schönwälder <[email-masked]<mailto:[email-masked]>> wrote:

This is a solved problem in proprietary ways.  It is simple to treat system config
as an access control issue.

I am quite concerned that NMDA is getting extended in ways that lead to
confusion and poor interoperability.  Adding a new datastore is very serious.
IMO ANY new datastore (even factory default) should be standardized in
a new version of NMDA (replacing RFC 8342).

So what does it mean for the system datastore to contain possible values that
cannot be represented in <operational>? The accessible tree cannot include
these values, so XPath-based validation cannot use them.

/js

Andy

On Wed, Aug 04, 2021 at 12:34:45PM +0000, Kent Watsen wrote:
> [email-masked]<mailto:[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 9 Aug 2021 15:23:14 -0700

On Mon, Aug 9, 2021 at 2:51 PM Sterne, Jason (Nokia - CA/Ottawa) <
[email-masked]> wrote:

> Hi guys,
>
>
>
> I'm late to the game on this thread but I did read through the entire
> thing (whew - can't promise I absorbed every nuance though). I am familiar
> with this issue. I've been dealing with it from the server implementation
> side (some clients **do** complain if there are references to missing
> "system config" list entries).
>
>
>
> One of the pretty fundamental issues IMO is whether we want good ol'
> standard <running> to always be valid (including for an offline client or
> tool to be able to validate instance data retrieved from a server against
> the YANG models). Even without this "system config" topic, we already have
> a looming problem with config templates & expansion.  Servers with YANG
> models that document a lot of constraints (e.g. leafrefs, mandatory
> statements, etc) are more likely to hit this issue of an invalid running
> config (offline).
>
>
>
> I'm doubtful that it is easy for a client to know how templates are
> expanded for all given servers. Unless that expansion is fully standardized
> (which would be challenging given multiple different shipping
> implementations in the industry), there can be some subtle corner cases and
> it can be complex (multiple levels of template application, exclusion
> rules, etc).
>
>
>
> I agree there can be dynamically added system config (e.g. create a new
> qos policy, and some queue list entries are automatically created inside
> that policy).
>
>
>
> Note that config added by the system could be deletable or non-deletable.
> For deletable items there were some previous discussions a few years ago
> that those list entries could be simply a default config that is populated
> if the startup datastore is empty at boot time.
>
>
>
> The system config list entries could also have leafs that are modifiable
> and leafs that are immutable. For the modifiable objects I think this is
> basically the system merging the explicit config with the system config
> (with the explicit taking precedence).
>
>
>
> Perhaps a lot of this system config is a legacy hold over from
> human-driven CLIs. For machine interfaces maybe it's reasonable to have
> clients explicitly define their config **if** they need offline
> validation of <running> to succeed (i.e. allow clients to explicitly
> configure any of the system config policies they are actually using &
> referencing).  In many cases, users want the "master" view of the config to
> live up in the client/OSS side and just push it down to the server (without
> having the server touch the contents of the running, i.e. a read-back
> should return identically what was sent).
>
>
>

I think Balasz captured the 2 use-cases very well.
For implementations that combine the system into <running>, undoing
this behavior would be quite disruptive and therefore not likely to be done.

It would help to have some common understanding of the contents of <system>.
Maybe start with the well-known "interface problem".  The system configures
almost empty physical interfaces.
The user is allowed to add, modify, and delete all descendants except the
"name" and "type" leaf,
which are set by the system.

In this case <running> will have the /interfaces/interface nodes in them
(or else the client-accessible
nodes could not be represented in NETCONF).  So any config=true XPath can
point at name or type leafs
in <running> or <operational>.

I have heard that <system> is needed because it has nodes in it that do not
exist
in <running> or <operational>, but could exist.
The premise is that the server could have created these nodes
but it did not for some reason.  And the client can inspect these nodes and
somehow
force the server to change the nodes it adds to <running>. (So what is the
real use-case then?)

Retrieving the system-created nodes with origin=system is already supported
by NMDA.

Andy

>
>
>
>
> *From:* netmod <[email-masked]> *On Behalf Of *Andy Bierman
> *Sent:* Wednesday, August 4, 2021 9:59 AM
> *To:* Juergen Schoenwaelder <[email-masked]>; Kent
> Watsen <[email-masked]>; Andy Bierman <[email-masked]>; NetMod WG <
> [email-masked]>
> *Subject:* Re: [netmod] system configuration sync mechanism
>
>
>
>
>
>
>

---

From: Kent Watsen <[email-masked]>
Date: Tue, 10 Aug 2021 01:58:53 +0000

There was a request for concrete use cases.  This email from before was good:

	https://mailarchive.ietf.org/arch/msg/netmod/v5cNLcC2F_OT8t-407F3Zj6Vws4/ <https://mailarchive.ietf.org/arch/msg/netmod/v5cNLcC2F_OT8t-407F3Zj6Vws4/>

More below:

> On Aug 9, 2021, at 6:23 PM, Andy Bierman <[email-masked]> wrote:
> 
> On Mon, Aug 9, 2021 at 2:51 PM Sterne, Jason (Nokia - CA/Ottawa) <[email-masked] <mailto:[email-masked]>> wrote:
> Hi guys,
> 
>  
> 

I think that it is already the case with JUNOS that <running> by itself cannot be validated by a client unaware of how JUNOS templates work…as the templates may supply mandatory nodes and/or values, or some validations only make sense to evaluate when the config is flattened (e.g., ensuring "max-elements" is not exceeded, ensuring ACLs always end with a terminal rules, etc.).

> I'm doubtful that it is easy for a client to know how templates are expanded for all given servers. Unless that expansion is fully standardized (which would be challenging given multiple different…
Agreed, not easy but…

Some [smart] clients may never need to grok how device-level templates are expanded.  For instance, the NMS I worked on in a previous life would always onboard a new device by asking the device to provide its config already expanded, and thereafter would never send templated-config to the device (note: the NMS had its own template mechanism).  

On the other end of the client-spectrum, other [dumb] clients may also not need to grok how device-level templates are expanded, as they mostly push simple updates (e.g., open a port) known to pass validation and/or rely on server-side validation.

Of course, there MAY be clients in-between that read/write templated-config and, if they wish to perform off-box validation, most likely will need to grok how templates are expanded.

> I agree there can be dynamically added system config (e.g. create a new qos policy, and some queue list entries are automatically created inside that policy).
I'm unsure how this relates.

> Note that config added by the system could be deletable or non-deletable. For deletable items there were some previous discussions a few years ago that those list entries could be simply a default…
Yes, but I that’s a different use-case, right?  (see the link to concrete use cases provided at top)

> The system config list entries could also have leafs that are modifiable and leafs that are immutable. For the modifiable objects I think this is basically the system merging the explicit config wi…

The “modifiable” leafs are in this use-case, assuming “modifiable" is the synonymous as “over-writable".  Some “immutable” leafs are more like those discussed in RFC 8342 Section 5, but others are like what Andy discusses below (note: these immutable nodes are not currently in scope, as I understand it).

> Perhaps a lot of this system config is a legacy hold over from human-driven CLIs. For machine interfaces maybe it's reasonable to have clients explicitly define their config *if* they need offline…
Agree that a read-back of <running> should always return identically what was sent, but that doesn’t mean that there isn’t system config in play.

> It might be good to set up a virtual interim or call of some sort to discuss this one. It is pretty complex.
Good idea, but I’d hope first to get past the “do we understand the requirements” part of the discussion.  Of course, if people prefer, we could schedule an interim to discuss the use-cases also.

> I think Balasz captured the 2 use-cases very well.

The use-cases in his 7/31 message are good, but neither of them are part this work’s scope, as I understand it.  [update: his use-case ‘A’ may be in scope, depending on how we set the scope]

> this behavior would be quite disruptive and therefore not likely to be done.

I do not support merging <system> into <running> (unless appropriately-annotated in a “with-system” <get-config> response).  I do support merging <system> into <intended>.

> Maybe start with the well-known "interface problem".  The system configures almost empty physical interfaces.  The user is allowed to add, modify, and delete all descendants except the "name" and "type" leaf, which are set by the system.

I agree that concrete use-cases are needed.  The link provided at top provides some, but not all, and definitely not the specific one you just mentioned (which regards into Balazs’s use-case ‘A’ and Jason’s “immutable” config).

> In this case <running> will have the /interfaces/interface nodes in them (or else the client-accessible nodes could not be represented in NETCONF).  So any config=true XPath can point at name or type leafs in <running> or <operational>.

Ack.

> I have heard that <system> is needed because it has nodes in it that do not exist
> in <running> or <operational>, but could exist.  The premise is that the server could have created these nodes but it did not for some reason.  And the client can inspect these nodes and somehow force the server to change the nodes it adds to <running>. (So what is the real use-case then?)

This is what the link provided at the top of this message regards.

> Retrieving the system-created nodes with origin=system is already supported by NMDA.

True, but this “system config” feels different, or at least parts of it does.  We may need to tease apart the “leafref-able system nodes” from the “resource-independent nodes” from the “resource-dependent nodes”.

K.

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 13 Aug 2021 21:02:31 +0000

Hi Andy,
Pls see inline.
Jason

From: Andy Bierman <[email-masked]>
Sent: Monday, August 9, 2021 6:23 PM
To: Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>
Cc: Juergen Schoenwaelder <[email-masked]>; Kent Watsen <[email-masked]>; NetMod WG <[email-masked]>
Subject: Re: [netmod] system configuration sync mechanism

On Mon, Aug 9, 2021 at 2:51 PM Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]<mailto:[email-masked]>> wrote:
Hi guys,

I think Balasz captured the 2 use-cases very well.
[>>JTS: ] I looked back at Balasz' email and I don't see clear use cases. Do you mean his P1, etc principles ?

[>>JTS: ] I'm not necessarily advocating for the existence of a <system> DS, but there are various implementations for this problem space and some don't put these system objects (list entries) into <running>. I'm doubtful about automatically putting system config into <running>. That means the client no longer really owns/controls <running>. Readback won't return what was sent. The definitive <running> would then have to sit on the server (i.e. can't have the client as the definitive view/ownership of running).

It would help to have some common understanding of the contents of <system>.
Maybe start with the well-known "interface problem".  The system configures almost empty physical interfaces.
The user is allowed to add, modify, and delete all descendants except the "name" and "type" leaf,
which are set by the system.

In this case <running> will have the /interfaces/interface nodes in them (or else the client-accessible
nodes could not be represented in NETCONF).  So any config=true XPath can point at name or type leafs
in <running> or <operational>.
[>>JTS: ] An alternative is that these interfaces are *not* added to <running> unless a client explicitly adds them. The server could potentially consider leafrefs to them valid even if they aren't explicitly added. But if a user has a workflow that does offline validation, then they could explicitly add the interfaces into <running>

The premise is that the server could have created these nodes
but it did not for some reason.  And the client can inspect these nodes and somehow
force the server to change the nodes it adds to <running>. (So what is the real use-case then?)

Retrieving the system-created nodes with origin=system is already supported by NMDA.
[>>JTS: ] I don't think reading these nodes from <operational> addresses the issue. Clients do a <get-config> from <running> (or maybe <intended>) and want to be able to validate that config against the YANG model (without having to do additional reads from operational).

Jason

Andy

From: netmod <[email-masked]<mailto:[email-masked]>> On Behalf Of Andy Bierman
Sent: Wednesday, August 4, 2021 9:59 AM
To: Juergen Schoenwaelder <[email-masked]<mailto:[email-masked]>>; Kent Watsen <[email-masked]<mailto:[email-masked]>>; Andy Bierman <[email-masked]<mailto:[email-masked]>>; NetMod WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] system configuration sync mechanism

/js

Andy

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 13 Aug 2021 21:45:16 +0000

Thx Kent. Please see inline.
Jason

From: Kent Watsen <[email-masked]>
Sent: Monday, August 9, 2021 9:59 PM
To: Andy Bierman <[email-masked]>
Cc: Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>; [email-masked]
Subject: Re: [netmod] system configuration sync mechanism

              https://mailarchive.ietf.org/arch/msg/netmod/v5cNLcC2F_OT8t-407F3Zj6Vws4/

[>>JTS: ] I saw the taxonomy of resource-dependant vs resource-independent, and the sub-categories of applied-immediately vs applied-after-reference. And then the concept of "conditional" system config. But I think there may also be two orthogonal concepts that need to be clarified:
1) deletable/removable vs non-removable
2) modifiable vs non-modifiable (i.e. can you modify elements in the list entry)

Let me give you an example use case of deletable+modifiable:
- user "admin"  (list entry)
- exists in <running> when the server boots up with no <startup> DS
- can be deleted by the client (and then a copy-config from <running> to <startup> would create a <startup> with the absence of the user "admin"
- I think this particular example would be "resource-independent", "applied-immediately", and "non-conditional" (but there could also be deletable+modifiable objects that are applied-after-reference or resource-dependant)
- but I also think that deletable entries like this can just be handled as system populated <running> config at boot time when there is no <startup> DS present

An example of non-deletable+modifiable:
- CPU protection policy number 255 (rate limit for extracted control traffic) that can be assigned to an interface (leafref from interface->protection policy)
- not shown in <running> unless the client explicitly modifies parameters of that protection policy
- an option ?: have this show up in <running> if the client simply explicitly creates the list entry

An example of non-deletable+non-modifiable:
- QoS policy 1 that is assigned to every interface by default
- The leafref from interface->qos policy 1 has a default value of "1" in the YANG model
- an option ?: have this show up in <running> if the client simply explicitly creates the list entry

More below:
[>>JTS: ] Please see inline below

On Aug 9, 2021, at 6:23 PM, Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:

I think that it is already the case with JUNOS that <running> by itself cannot be validated by a client unaware of how JUNOS templates work…as the templates may supply mandatory nodes and/or values, or some validations only make sense to evaluate when the config is flattened (e.g., ensuring "max-elements" is not exceeded, ensuring ACLs always end with a terminal rules, etc.).
[>>JTS: ] Clients may not hit this as much with JUNOS. The latest models I looked at didn't have any leafrefs and had very few mandatory statements. So I'm not sure what parts of the templates provide data that is required to validate running against the YANG models ?
But for models with leafrefs and mandatory statements, there is high likelihood that clients won't be able to do offline validation if templates are used.

Some [smart] clients may never need to grok how device-level templates are expanded.  For instance, the NMS I worked on in a previous life would always onboard a new device by asking the device to provide its config already expanded, and thereafter would never send templated-config to the device (note: the NMS had its own template mechanism).
[>>JTS: ] But that's moving the mastership/ownership (i.e. definitive view of the <running>) down to the server. Or abandoning templates as part of the operator-owned config.

On the other end of the client-spectrum, other [dumb] clients may also not need to grok how device-level templates are expanded, as they mostly push simple updates (e.g., open a port) known to pass validation and/or rely on server-side validation.
[>>JTS: ] If we only want to solve server-side validation then this gets easier. And maybe that is the answer. But some clients today do complain when offline validation fails.

Of course, there MAY be clients in-between that read/write templated-config and, if they wish to perform off-box validation, most likely will need to grok how templates are expanded.[>>JTS: ] Yes - I think this is the issue (and workflow) we're trying to address here.

I agree there can be dynamically added system config (e.g. create a new qos policy, and some queue list entries are automatically created inside that policy).
I'm unsure how this relates.[>>JTS: ] Qiufang was asking if conditional system config exists. It does in some implementations.

Note that config added by the system could be deletable or non-deletable. For deletable items there were some previous discussions a few years ago that those list entries could be simply a default config that is populated if the startup datastore is empty at boot time.
Yes, but I that’s a different use-case, right?  (see the link to concrete use cases provided at top)[>>JTS: ] Not really a different use-case. A different attribute of some of the use cases we're discussing. I think its another dimension we need to consider (along with resource-dependant vs resource-independent, applied-immediately vs applied-after-reference, conditional vs non-conditional).

The “modifiable” leafs are in this use-case, assuming “modifiable" is the synonymous as “over-writable".  Some “immutable” leafs are more like those discussed in RFC 8342 Section 5, but others are like what Andy discusses below (note: these immutable nodes are not currently in scope, as I understand it).[>>JTS: ] Yes - I'm thinking of modifiable as over-writable. Maybe this is a separate issue - I'm not sure yet.

Perhaps a lot of this system config is a legacy hold over from human-driven CLIs. For machine interfaces maybe it's reasonable to have clients explicitly define their config *if* they need offline validation of <running> to succeed (i.e. allow clients to explicitly configure any of the system config policies they are actually using & referencing).  In many cases, users want the "master" view of the config to live up in the client/OSS side and just push it down to the server (without having the server touch the contents of the running, i.e. a read-back should return identically what was sent).
Agree that a read-back of <running> should always return identically what was sent, but that doesn’t mean that there isn’t system config in play.[>>JTS: ] Not sure what you mean here. If the client sent config X, and in X there is a leafref to some system list entry, and the system list entry wasn't sent by the client, then I'm doubtful the readback should return the system list entry (in which case the config would be considered invalid by offline validation, e.g. analysis of the instance data against the YANG model with yangLint).

The use-cases in his 7/31 message are good, but neither of them are part this work’s scope, as I understand it.  [update: his use-case ‘A’ may be in scope, depending on how we set the scope]
[>>JTS: ] I'm confused about your and Andy's references to Balazs' use cases. Do you mean this email ?
https://mailarchive.ietf.org/arch/msg/netmod/ncgHfMzteHL3cnqCdpDoojw171s/
I see principles. But I'm not sure what you mean by use-case 'A' ?

I do not support merging <system> into <running> (unless appropriately-annotated in a “with-system” <get-config> response).  I do support merging <system> into <intended>.
[>>JTS: ] I'm also doubtful about merging into running (that means magic config appearing in running). Even with annotation, a client that doesn’t understand the annotation would just see this as config being added that they didn't add.
An explicit RPC to cause the addition might be OK, but then a client would have to continually issue that RPC (maybe after every edit-config).

Ack.

K.

---

From: Andy Bierman <[email-masked]>
Date: Sat, 14 Aug 2021 08:19:03 -0700

On Fri, Aug 13, 2021 at 2:02 PM Sterne, Jason (Nokia - CA/Ottawa) <
[email-masked]> wrote:

> Hi Andy,
>
> Pls see inline.
>
> Jason
>
>
>
> *From:* Andy Bierman <[email-masked]>
> *Sent:* Monday, August 9, 2021 6:23 PM
> *To:* Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>
> *Cc:* Juergen Schoenwaelder <[email-masked]>; Kent
> Watsen <[email-masked]>; NetMod WG <[email-masked]>
> *[>>JTS: ] I looked back at Balasz' email and I don't see clear use cases.
> Do you mean his P1, etc principles ?*
>
>
>

It was a different email I think proposing extensions instead of a
datastore.

I actually do not really understand the 4 principles he mentioned:

*P1) “There is a need to validate configuration data against data created
by the system.”*

*P2) We want the running datastore to contain exactly what the operator has
written there*

*P3) the running/intended configuration MAY become invalid if the
system-data changes.*

*P4) rfc8342#section-5.3.4 MUST always be valid*

What does P1 really mean?
Provide a complete example using XML represented datastores (not
proprietary CLI)

P2) Why? What problem does this solve? What if the system creates an
interface list?
How does the client add any child nodes if this interface is not allowed to
be represented
in <running>?  IMO P2 does not even make any sense.

The "origin" attribute identifies the nodes that came from the system.
This is not visible in <running> only because of CLRs in the RFC.
It is visible in <operational>,

P3) not sure what this means. This is not what RFC 8342 says about
<intended>

P4) I agree with-origin must be supported (but it is an optional feature)
It is not possible for <operational> to always be complete and correct,
as discussed in NMDA, 5.3.

The YANG module in this proposal simply adds an identity for a new
datastore.
There are no examples and the guidelines in NMDA, Appendix A are not
followed.

The reason <system> was left out of NMDA seems to be because the system is
completely proprietary and not constrained in any way wrt/ how config and
operational
data nodes are instantiated.

> *[>>JTS: ] I'm not necessarily advocating for the existence of a <system>
> DS, but there are various implementations for this problem space and some
> don't put these system objects (list entries) into <running>. I'm doubtful
> about automatically putting system config into <running>. That means the
> client no longer really owns/controls <running>. Readback won't return what
> was sent. The definitive <running> would then have to sit on the server
> (i.e. can't have the client as the definitive view/ownership of running).*
>
>
>
> Maybe start with the well-known "interface problem".  The system
> configures almost empty physical interfaces.
>
> *[>>JTS: ] An alternative is that these interfaces are *not* added to
> <running> unless a client explicitly adds them. The server could
> potentially consider leafrefs to them valid even if they aren't explicitly
> added. But if a user has a workflow that does offline validation, then they
> could explicitly add the interfaces into <running>*
>

I still have no idea what problem this solves.
Maybe it is to make NMDA as hard to adopt as possible,
by being as different from non-NMDA as possible?

> *[>>JTS: ] I don't think reading these nodes from <operational> addresses
> the issue. Clients do a <get-config> from <running> (or maybe <intended>)
> and want to be able to validate that config against the YANG model (without
> having to do additional reads from operational).*
>
>
>

Why does the client need to know the origin of the data in <intended>
in order to validate it?  (Again, origin is only restricted to
<operational> because
the RFC forbids anything else).

Andy

---

From: Kent Watsen <[email-masked]>
Date: Sun, 15 Aug 2021 19:49:16 +0000

> It was a different email I think proposing extensions instead of a datastore.

This email: https://mailarchive.ietf.org/arch/msg/netmod/SHRPSxHIDxsfF2t0GXXiyFHOnGw/ <https://mailarchive.ietf.org/arch/msg/netmod/SHRPSxHIDxsfF2t0GXXiyFHOnGw/>
K.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Aug 2021 12:20:41 -0700

On Sun, Aug 15, 2021 at 12:49 PM Kent Watsen <[email-masked]> wrote:

> This email:
> https://mailarchive.ietf.org/arch/msg/netmod/SHRPSxHIDxsfF2t0GXXiyFHOnGw/
>
>

What is the purpose of system-configuration

Use-case A)    The system sets some values because it knows what they shall
be. In this case the client must not be allowed to modify these values. We
want to check configuration data against these values.  E.g., AcmeHomeRouter
has 5 interfaces called eth0, eth1, eth,2, eth3 and WAN. The client should
not try to add or remove interfaces to this set.

Use-case B)    The system provides initial values for something that can be
configured in many ways. In this case the client is free to modify the
system-defined values. E.g., an initial set of NACM rules is provided. In
this case any constraints based on the system data are very weak, as the
user can change the system-data itself.

It is possible to support these use-cases with access-control conventions.

I re-read RFC 8808 and 8342 again.
IMO this draft overlaps the factory-default datastore.
Unfortunately, RFC 8808 does not document NMDA, Appendix A3 details
https://datatracker.ietf.org/doc/html/rfc8342#appendix-A.3

It does not say if <factory-default> datastore feeds into <running> or into
<intended>.
It is not clear how <system> would interact with other datastores.
It is not clear why it is even needed since <factory-default> contains only
system settings.

K.
Andy