Subject: [netmod] Should the origin="system" be required for system configurations copied/pasted into <running>?

From: "maqiufang (A)" <[email-masked]>
Date: Tue, 23 Nov 2021 02:57:54 +0000

Hi, all

There is still another issue which is about origin metadata annotation: should the origin="system" be required for system configurations copied/pasted into <running>?

Currently any system configuration explicitly declared in <running> in order to configure its descendant nodes or maintain <running> offline-valid will show up in <operational> with origin=intended.
The question behind this issue is whether we want a copied/pasted system defined data node to override and take precedence over <system>.

The choices and some considerations of this issue received so far:
o    Origin=system IS required for system configuration copied/pasted into <running>
?  I believe that "system" reflects the most accurate source in this case. And only in this way, a server can allow a read-only system configuration to be declared in <running>(e.g., in order to valid <running>) by the clients.
?  The challenge for this choice is on the server side. It MUST be able to recognize a particular data node which explicitly defined in <running> is actually a mirror of what is in <system>.
o    Origin=system is NOT required for system configuration copied/pasted into <running>
?  Good consistency. For all configurations explicitly defined in <running>, if they appear in <operational>, the origin value is "intended" with no exceptions.
o    Define a system-mode which is similar to with-defaults basic mode and allow a server to advertise a particular behavior
?  Does it mean we could get the Pros from both choices?
Any other thoughts?

---

From: Jürgen Schönwälder <[email-masked]>
Date: Tue, 23 Nov 2021 08:11:44 +0100

Do we mark config copied from stackexchange as origin='stackexchange'?
If joe pastes config into running, then its joe's config.

/js

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Martin Björklund <[email-masked]>
Date: Tue, 23 Nov 2021 08:38:50 +0100 (CET)

Hi,

"maqiufang \(A\)" <maqiufang1=[email-masked]> wrote:
> Hi, all
> 
> There is still another issue which is about origin metadata
> annotation: should the origin="system" be required for system
> configurations copied/pasted into <running>?

I think the question is "if a node is present both in <intended> and
in <system>, which origin does it have in <operational>"?

(NOTE: it doesn't matter if the value was "copy & pasted" from
<system> or entered in some other way.)

Obviously, if a leaf node is present in both, but its value differ,
the origin must indicate which datastore had precedence.

But suppose the node is a list entry (e.g., an interface) or a leaf
with the same value.  In this case, it is not clear which origin
should be used.  I think it would be ok to use "system" in this case.
(But also perhaps it doesn't matter much).

> Currently any system configuration explicitly declared in <running> in
> order to configure its descendant nodes or maintain <running>
> offline-valid will show up in <operational> with origin=intended.
> The question behind this issue is whether we want a copied/pasted
> system defined data node to override and take precedence over
> <system>.
> 
> ?  I believe that "system" reflects the most accurate source in this
> case. And only in this way, a server can allow a read-only system
> configuration to be declared in <running>(e.g., in order to valid
> <running>) by the clients.

What do you mean with "a read-only system configuration [...] be
declared in <running>"?  <system> is a separate datastore that clients
can read, right?

/martin

> ?  The challenge for this choice is on the server side. It MUST be
> able to recognize a particular data node which explicitly defined in
> <running> is actually a mirror of what is in <system>.
> ?  Good consistency. For all configurations explicitly defined in
> <running>, if they appear in <operational>, the origin value is
> "intended" with no exceptions.
> o Define a system-mode which is similar to with-defaults basic mode
> and allow a server to advertise a particular behavior

---

From: "maqiufang (A)" <[email-masked]>
Date: Wed, 24 Nov 2021 03:21:14 +0000

Hi, Martin,

Please see my reply inline.

-----Original Message-----
From: Martin Björklund [mailto:[email-masked]] 
Sent: Tuesday, November 23, 2021 3:39 PM
To: maqiufang (A) <[email-masked]>
Cc: [email-masked]
Subject: Re: [netmod] Should the origin="system" be required for system configurations copied/pasted into <running>?

Hi,

> configurations copied/pasted into <running>?

I think the question is "if a node is present both in <intended> and in <system>, which origin does it have in <operational>"?

(NOTE: it doesn't matter if the value was "copy & pasted" from <system> or entered in some other way.)
[Qiufang Ma] Noted, thanks.

Obviously, if a leaf node is present in both, but its value differ, the origin must indicate which datastore had precedence.
[Qiufang Ma] Yes, exactly. If a client tries to modify a system-defined data node(with a different value),  what has been defined in <running> should take precedence over system predefined nodes if the system configuration is modifiable. Thus the origin should be indicated as "intended", there is no question about that. 

But suppose the node is a list entry (e.g., an interface) or a leaf with the same value.  In this case, it is not clear which origin should be used.  I think it would be ok to use "system" in this case.
(But also perhaps it doesn't matter much).
[Qiufang Ma] "Copy&paste" just refers to this case(a leaf with a same value or an ancestor node like "interface"). IMO, it should not be "intended" if the system configuration is non-modifiable(e.g., interface type or name).

> o Origin=system IS required for system configuration copied/pasted 
> into <running> ?  I believe that "system" reflects the most accurate 
> source in this case. And only in this way, a server can allow a 
> read-only system configuration to be declared in <running>(e.g., in 
> order to valid

What do you mean with "a read-only system configuration [...] be declared in <running>"?  <system> is a separate datastore that clients can read, right?
[Qiufang Ma] Yes, it's related to an ongoing work (https://datatracker.ietf.org/doc/html/draft-ma-netmod-with-system-00.txt). There is a read-only "system" datastore which contains system configuration non-modifiable or not.
I think that non-modifiable system configuration should be allowed to write into <running>(with a same value), as far as the origin is "system". Writing a different value for non-modifiable system config in <running> should return an error.

Best Regards,
Qiufang Ma

/martin

> o Origin=system is NOT required for system configuration copied/pasted 
> into <running> ?  Good consistency. For all configurations explicitly 
> defined in <running>, if they appear in <operational>, the origin 
> value is "intended" with no exceptions.
> and allow a server to advertise a particular behavior ?  Does it mean 
> we could get the Pros from both choices?
> Any other thoughts?

---

From: Jürgen Schönwälder <[email-masked]>
Date: Wed, 24 Nov 2021 10:25:08 +0100

> But suppose the node is a list entry (e.g., an interface) or a leaf with the same value.  In this case, it is not clear which origin should be used.  I think it would be ok to use "system" in this…

For me, <running> is explicit config and hence it has precedence. The
precedence must be a function of how the datastores related, it should
not depend on which values a config leaf has.

/js

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 24 Nov 2021 10:16:53 +0000

+1 to Juergen's comment.

// As a contributor.

-----Original Message-----
From: netmod <[email-masked]> On Behalf Of Jürgen Schönwälder
Sent: 24 November 2021 09:25
To: maqiufang (A) <maqiufang1=[email-masked]>
Cc: [email-masked]
Subject: Re: [netmod] Should the origin="system" be required for system configurations copied/pasted into <running>?

/js

---

From: Martin Björklund <[email-masked]>
Date: Wed, 24 Nov 2021 11:43:40 +0100 (CET)

> not depend on which values a config leaf has.

Here's a simple example.

Suppose <system> has:

   <interface>
     <name>lo</name>
     <type>loopback</type>
     <description>added by system</description>
   </interface>

and <intended> has:

   <interface>
     <name>lo</name>
     <description>set by a client</description>
   </interface>

Now we follow the picture in RFC 8342:

                      +------------+
                      | <intended> | // subject to validation
                      | (ct, ro)   |
                      +------------+
                            |        // changes applied, subject to
                            |        // local factors, e.g., missing
                            |        // resources, delays
                            |
       dynamic              |   +-------- learned configuration
       configuration        |   +-------- system configuration
       datastores -----+    |   +-------- default configuration
                       |    |   |
                       v    v   v
                    +---------------+
                    | <operational> | <-- system state
                    | (ct + cf, ro) |
                    +---------------+

So now we merge intended and system into operational state.  First we
add system to get:

  <interface origin="system">
    <name>lo</name>
    <type>loopback</type>
    <description>added by system</description>
  </interface>

and then we add intended to arrive at:

  <interface origin="system">
    <name>lo</name>
    <type>loopback</type>
    <description origin="intended">set by a client</description>
  </interface>

Doesn't this make sense?

/martin

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 25 Nov 2021 09:45:56 +0000

Hi Martin,

I think that the proposal is that <system> should feed into <intended> rather than directly into <operational>.  The reasoning for this is to allow configuration to depend on system defined configuration during validation without requiring that configuration to be copied into <running>.  Clients would still be allowed to explicitly express the system configuration is running as well - e.g., if they wanted a full configuration that they can validate off box.
 
In your example below, I would probably mark the origin of the lo interface, the name leaf, and description leaf as "intended", but the type is "system".  I think that this would be similar to how I would expect a default value to be reported.  I.e., if the running config explicitly sets a leaf to its default value, I think that it is more informative to report that as origin "intended" rather than "origin" default.  But I don't think that RFC 8342 proscribes what is be used in these cases.

Regards,
Rob

// As a contributor

> From: netmod <[email-masked]> On Behalf Of Martin Björklund
> Sent: 24 November 2021 10:44
> To: [email-masked]
> Cc: maqiufang1=[email-masked]; [email-masked]
> Subject: Re: [netmod] Should the origin="system" be required for system
> > > But suppose the node is a list entry (e.g., an interface) or a leaf with the
> same value.  In this case, it is not clear which origin should be used.  I think it
> would be ok to use "system" in this case.
> >

---

From: Jürgen Schönwälder <[email-masked]>
Date: Thu, 25 Nov 2021 12:47:33 +0100

I personally believe this notion of a system datastore is actually a
bad idea. A loopback interface, for example, is system generated and
it exists in operational but usually not in intended. I think it is
wrong to think that a system datastore feeds into intended. After all,
system config also comes and goes at the will of the system. I am not
following this in detail but I fear this work likely creates more
damage than that is solves serious real-world problems.

/js

---

From: Martin Björklund <[email-masked]>
Date: Thu, 25 Nov 2021 13:01:55 +0100 (CET)

> damage than that is solves serious real-world problems.

I strongly agree.  I didn't understand that part of the proposal.  I
guess the discussion about origin system confused me; if system feeds
into intended then the origin will be intended.

/martin

---

From: Martin Björklund <[email-masked]>
Date: Thu, 25 Nov 2021 13:01:55 +0100 (CET)

/martin

---

From: "maqiufang (A)" <[email-masked]>
Date: Thu, 25 Nov 2021 12:54:28 +0000

Hi, Jurgen, Martin,

-----Original Message-----
From: netmod [mailto:[email-masked]] On Behalf Of Martin Bj?rklund
Sent: Thursday, November 25, 2021 8:02 PM
To: [email-masked]
Cc: maqiufang1=[email-masked]; [email-masked]
Subject: Re: [netmod] Should the origin="system" be required for system configurations copied/pasted into <running>?

> damage than that is solves serious real-world problems.
[Qiufang Ma] There is often a desire to reference a system configuration or configure a descendant node of system configuration. How to keep <running>/<intended> valid when leaf-ref a system configuration which is only present in <operational>?   Or do we think that explicitly configuration in <running> should always be the case?

I strongly agree.  I didn't understand that part of the proposal.  I guess the discussion about origin system confused me; if system feeds into intended then the origin will be intended.
[Qiufang Ma] If <system> feeds into <intended>, the server MUST remember a particular data node is from system or from intended, those previously present in <operational> with origin=system will keep the origin unchanged. This work does not change that behavior. 

Best Regards,
Qiufang Ma

/martin

> > > Subject: Re: [netmod] Should the origin="system" be required for 
> > > system configurations copied/pasted into <running>?
> > > 
> > > same value.  In this case, it is not clear which origin should be 
> > > used.  I think it would be ok to use "system" in this case.
> > > >
> > > > For me, <running> is explicit config and hence it has 
> > > > precedence. The precedence must be a function of how the 
> > > > datastores related, it should not depend on which values a config leaf has.
> > > 
> > > Here's a simple example.
> > > 
> > > Suppose <system> has:
> > > 
> > >    <interface>
> > >      <name>lo</name>
> > >      <type>loopback</type>
> > > So now we merge intended and system into operational state.  First 
> > > we add system to get:
> > >

---

From: Jürgen Schönwälder <[email-masked]>
Date: Thu, 25 Nov 2021 14:01:28 +0100

The way to resolve things is to use name bindings. I can configure the
interface "lo" in running by refering to this interface by name. If
there is no "lo" in the system, then the config gets ignored.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 25 Nov 2021 07:24:40 -0800

On Thu, Nov 25, 2021 at 5:02 AM Jürgen Schönwälder <
[email-masked]> wrote:

> > [Qiufang Ma] There is often a desire to reference a system configuration
> or configure a descendant node of system configuration. How to keep
> <running>/<intended> valid when leaf-ref a system configuration which is
> only present in <operational>?   Or do we think that explicitly
> configuration in <running> should always be the case?
> >

I agree with Juergen.

RFC 7950 rules about leafref validation are very clear.
Adding a new datastore to these rules requires a massive change to NMDA
and all implementations.

2) As Juergen points out, servers populate the system config in <running>
and the operator is
free to ignore it, reference it, or possibly change it.  The premise of
<system> seems to
be that this approach is not "pure enough" for NMDA and <running> MUST only
contain
operator-created configuration. Why? What real problems would this change
solve?

Andy

> guess the discussion about origin system confused me; if system feeds into
> intended then the origin will be intended.
> > [Qiufang Ma] If <system> feeds into <intended>, the server MUST remember
> a particular data node is from system or from intended, those previously
> present in <operational> with origin=system will keep the origin unchanged.
> This work does not change that behavior.
> >
> > Best Regards,
> > Qiufang Ma
> >
> >
> > /martin
> >
> >
> >
> > >
> > > /js
> > >
> > > > I think that the proposal is that <system> should feed into
> <intended> rather than directly into <operational>.  The reasoning for this
> is to allow configuration to depend on system defined configuration during
> validation without requiring that configuration to be copied into
> <running>.  Clients would still be allowed to explicitly express the system
> configuration is running as well - e.g., if they wanted a full
> configuration that they can validate off box.
> > > >
> > > > In your example below, I would probably mark the origin of the lo
> interface, the name leaf, and description leaf as "intended", but the type
> is "system".  I think that this would be similar to how I would expect a
> default value to be reported.  I.e., if the running config explicitly sets
> a leaf to its default value, I think that it is more informative to report
> that as origin "intended" rather than "origin" default.  But I don't think
> that RFC 8342 proscribes what is be used in these cases.
> > > >
> > > > Regards,
> > > > Rob
> > > >
> > > > // As a contributor
> > > >
> > > >

---

From: Kent Watsen <[email-masked]>
Date: Mon, 29 Nov 2021 22:49:35 +0000

Hi Andy,

> Adding a new datastore to these rules requires a massive change to NMDA

Not really or, rather, it seems like it would be just part of adding support for <system>, which implies adding support for <intended> (if not supported already) and doing the validation on <intended> instead.

> 2) As Juergen points out, servers populate the system config in <running> and the operator is
> free to ignore it, reference it, or possibly change it.  The premise of <system> seems to
> be that this approach is not "pure enough" for NMDA and <running> MUST only contain
> operator-created configuration. Why? What real problems would this change solve?

Servers populating <running> (beyond coping <startup> into <running>) has never been a supported idea.  We’ve always maintained that <running> should only contain client-provided config, right?

I’m unsure what the “change” in your last question points to.  But note that on JUNOS-based systems, there are many hundreds of system-defined nodes available to be referenced by config in <running>.  Putting all these nodes into <running> would clutter the config immensely.   

FWIW, JUNOS “solves” this by *hiding* these system-nodes, such that clients must use a special command just to discover them.   And, yes, if ever any of these are hidden-nodes are reference, offline-validation of <running> will fail.  The <system> datastore being discussed effectively mimics this aspect of the JUNOS solution.

There is a separate thread for if *offline* validation of <running> *alone* must succeed.  The soft (fallback) solution is to have clients copy/paste the referenced-subset of the system-defined nodes into <running>.  The theory being that, since it’s just a subset, it won’t clutter things up too badly and, as it is already the case that some system-defined nodes must be copy/pasted into <running> in order for clients to configure descendent nodes (e.g., some tunable for the system-defined “lo” interface), the water is already over the bridge, so to speak, on the copy/pasting debate.  The only holdout  for not already demanding client’s also copy/paste the subset of referenced system-defined nodes is that it is completely unnecessary, unless trying to ensure offline validation of <running> alone.

> Andy

Kent, as a contributor.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 29 Nov 2021 15:14:06 -0800

On Mon, Nov 29, 2021 at 2:49 PM Kent Watsen <[email-masked]> wrote:

> Not really or, rather, it seems like it would be just part of adding
> support for <system>, which implies adding support for <intended> (if not
> supported already) and doing the validation on <intended> instead.
>
>
> Servers populating <running> (beyond coping <startup> into <running>) has
> never been a supported idea.  We’ve always maintained that <running> should
> only contain client-provided config, right?
>
> I’m unsure what the “change” in your last question points to.  But note
> that on JUNOS-based systems, there are many hundreds of system-defined
> nodes available to be referenced by config in <running>.  Putting all these
> nodes into <running> would clutter the config immensely.
>
> FWIW, JUNOS “solves” this by *hiding* these system-nodes, such that
> clients must use a special command just to discover them.   And, yes, if
> ever any of these are hidden-nodes are reference, offline-validation of
> <running> will fail.  The <system> datastore being discussed effectively
> mimics this aspect of the JUNOS solution.
>
> There is a separate thread for if *offline* validation of <running>
> *alone* must succeed.  The soft (fallback) solution is to have clients
> copy/paste the referenced-subset of the system-defined nodes into
> <running>.  The theory being that, since it’s just a subset, it won’t
> clutter things up too badly and, as it is already the case that some
> system-defined nodes must be copy/pasted into <running> in order for
> clients to configure descendent nodes (e.g., some tunable for the
> system-defined “lo” interface), the water is already over the bridge, so to
> speak, on the copy/pasting debate.  The only holdout  for not already
> demanding client’s also copy/paste the subset of referenced system-defined
> nodes is that it is completely unnecessary, unless trying to ensure offline
> validation of <running> alone.
>
>
>
>

IMO the least disruptive solution possible should be used.
There is a use-case for adding "origin" support to the <running> datastore
in the <get-data> operation.
This allows an NMDA client to identify system config that is not being used
in <operational>.

All "resource" oriented data models should have some mechanism to require
the resources
to be utilized or enabled somehow, within the data model. Unfortunately
this cannot be standardized
with a 1-size-fits-all solution.

A vendor must somehow populate these data models with origin=system data
nodes.
The "hidden system" data uses proprietary logic to decide what nodes to add
(and when).
This data is probably not represented with YANG models.  Transforming the
hidden system data
to the appropriate YANG models in any datastore is an implementation
detail, out of scope
for standardization.

Andy

---

From: Jürgen Schönwälder <[email-masked]>
Date: Tue, 30 Nov 2021 00:49:41 +0100

> This allows an NMDA client to identify system config that is not being used

Looking at Figure 2 of RFC 8342, I wonder how system config gets into
running - other than a client writing it into running, at which point
the config becomes explicit config subject to the usual update rules
of <running>. Conceptually, you can have a system daemon acting as a
client updating <running> (perhaps even controllable by NACM). The
"origin" was not designed to track from which application config data
in <running> is originating, that's to be solved by other mechanisms.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 29 Nov 2021 16:44:28 -0800

> in <running> is originating, that's to be solved by other mechanisms.
I see your point about the intent of "origin". The I2RS "owner" concept is
different.

The usage of the <running> config to contain both system and client data
nodes
goes back many decades in some cases.  Usually the client data cannot even
exist without
the system ancestor nodes, so not sure how <running> would even work with
client-only data.
The more different the solution is from current practice, the less likely
it will get adopted, unless
it solves an important real problem. I am not sure there is an operational
problem being solved here.

I agree the system is just another client (exactly how our server is
implemented), so why
does it need special tracking and control at all?  Why does any client get
(or need) to inspect the
edits of another client before they are made, and control how and when the
edits are added?

/js

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 30 Nov 2021 02:24:52 +0000

Hi Juergen,

> On Nov 29, 2021, at 6:49 PM, Jürgen Schönwälder <[email-masked]> wrote:
> 

This idea is to mimic RFC6243’s “default” annotation and the "with-defaults" RPC.  Just like with the “explicit” mode server, if a client writes to <running> and default value w/o the annotation, the server assumes it is explicit config (not the default anymore).  Same here, if the client writes a “system” node w/o the annotation, the server assumes it is explicit config (not the <system>-defined node).  In both cases, if the client includes the appropriate annotation, and the value/node matches the default/system value, then the server assumes that it’s actually the default/system value/node.

FWIW, about 5 months ago (June 29), I offered this modified version of Figure 2  (Note the <system> box on the left).  The “underlay” comment is meant to imply that <running> always overrides <system> when they’re merged:

     +-------------+                 +-----------+
     | <candidate> |                 | <startup> |
     |  (ct, rw)   |<---+       +--->| (ct, rw)  |
     +-------------+    |       |    +-----------+
            |           |       |           |
            |         +-----------+         |
            +-------->| <running> |<--------+
                      | (ct, rw)  |
                      +-----------+
                            |
 +----------+               |        // configuration transformations,
 | <system> |-------------->|        // e.g., removal of nodes marked as
 | (ct, ro) | // underlay   |        // "inactive", expansion of
 +----------+               |        // templates
                            v
                      +------------+
                      | <intended> | // subject to validation
                      | (ct, ro)   |
                      +------------+
                            |        // changes applied, subject to
                            |        // local factors, e.g., missing
                            |        // resources, delays
                            |
       dynamic              |   +-------- learned configuration
       configuration        |   +-------- system configuration
       datastores -----+    |   +-------- default configuration
                       |    |   |
                       v    v   v
                    +---------------+
                    | <operational> | <-- system state
                    | (ct + cf, ro) |
                    +---------------+

K.

---

From: Jürgen Schönwälder <[email-masked]>
Date: Tue, 30 Nov 2021 09:00:18 +0100

> FWIW, about 5 months ago (June 29), I offered this modified version of Figure 2  (Note the <system> box on the left).  The “underlay” comment is meant to imply that <running> always overrides <syst…

Your first paragraph does not match the figure. In your figure, system
config does not go into <running> instead it goes into a magic merge
box which is not shown (there is just an arror pointing to an arrow)
and then the result goes into intended.

/js