Subject: Re: [netmod] system configuration sync mechanism

From: Kent Watsen <[email-masked]>
Date: Mon, 2 Aug 2021 22:31:56 +0000

Hi Balazs, Andy, Quifang,

> I agree a new datastore will just add complexity without any value.
> Your solution approach is better, but I think it would require a new YANG version
> to allow config node XPath to reference non-config nodes.

In no case is there a need for a config Xpath to ref non-config.

> Another solution is to model the referenced node as config=true, but setup
> automatic NACM rules so no user editing is allowed. This works well for
> setting up an initial config that gets saved and not changed unless a reset is done.

I don’t like the NACM dependency.

> What if <intended> is what is NV-stored?  When that occurs, the config changes from system to user config.
> Routers have been saving the combined config for decades. IMO the standards
> intentionally avoid discussing the conversion of a datastore to/from NV-storage.

I’m unsure about this point, but a don’t see any dependency on an NV-store needing to use a particular encoding or format.

> As I see it this is the same problem that we discussed in https://github.com/netmod-wg/yang-next/issues/41 <https://github.com/netmod-wg/yang-next/issues/41>.
> 
> I know this is a radical change, but I think using a new YANG extension to create a read-only config=true datatype is a much cleaner solution. One which some companies have already implemented.
> 
> +1
> 
> Actually, separate running and system would be a radical change, not this.
> Cleaner and less disruptive. 

I disagree with the need to resolve issue #41 here, and I disagree a <system> datastore would be a radical change, or even complicated.

That said, something like RFC 6243 (with-defaults) but called “with-system” could work.  That is, the “system" config is like <running> config that is hidden until asked to be revealed. It would be interesting to discuss if “with-defaults" returns all of the system config, or just the parts that are used, assuming we define “used” appropriately.   Presumably system-defined ordered-list entries cannot be ordered by user, or used as before/after pivots.

We could even do both: have a read-only <system> datastore *and* a "with-system" interaction API.  Each complimenting the other.

FWIW, a "with-system <running>” still may not pass validation if, e.g., templates need to be expanded.  Of course, any controller/orchestrator can expand the templates themselves to resolve that concern.  Such a system could also fetch <system> without skipping a beat.

Thought exercise: a controller is managing 1000 endpoints all running the same software version.  Is it better for the controller to get "with-system <running>” 1000 times, or get <system> once and have knowledge for how its merged in?

K.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 2 Aug 2021 15:49:54 -0700

On Mon, Aug 2, 2021 at 3:31 PM Kent Watsen <[email-masked]> wrote:

> I’m unsure about this point, but a don’t see any dependency on an NV-store
> needing to use a particular encoding or format.
>
> As I see it this is the same problem that we discussed in
>> https://github.com/netmod-wg/yang-next/issues/41.
>>
>> I know this is a radical change, but I think using a new YANG extension
>> to create a read-only config=true datatype is a much cleaner solution. One
>> which some companies have already implemented.
>>
> +1
>
> I disagree with the need to resolve issue #41 here, and I disagree a
> <system> datastore would be a radical change, or even complicated.
>
> That said, something like RFC 6243 (with-defaults) but called
> “with-system” could work.  That is, the “system" config is like <running>
> config that is hidden until asked to be revealed. It would be interesting
> to discuss if “with-defaults" returns all of the system config, or just
> the parts that are used, assuming we define “used” appropriately.
>   Presumably system-defined ordered-list entries cannot be ordered by user,
> or used as before/after pivots.
>
> We could even do both: have a read-only <system> datastore *and* a
> "with-system" interaction API.  Each complimenting the other.
>

I prefer neither.
I think NMDA <get-data> with an origin-filter and/or with-origin parameter
already solves this problem.

Andy

> FWIW, a "with-system <running>” still may not pass validation if, e.g.,
> templates need to be expanded.  Of course, any controller/orchestrator can
> expand the templates themselves to resolve that concern.  Such a system
> could also fetch <system> without skipping a beat.
>
> Thought exercise: a controller is managing 1000 endpoints all running the
> same software version.  Is it better for the controller to get "with-system
> <running>” 1000 times, or get <system> once and have knowledge for how
> its merged in?
>
> K.
>
>
>
>
>

---

From: Kent Watsen <[email-masked]>
Date: Tue, 3 Aug 2021 01:21:06 +0000

> I prefer neither.
> I think NMDA <get-data> with an origin-filter and/or with-origin parameter already solves this problem. 

Some of the nodes defined in <system> may be referenced by nodes defined in <running>.  There is a desire for the <system> nodes to not be defined only in <operational>, for online/offline validation.

K.

---

From: "Fengchong (frank)" <[email-masked]>
Date: Tue, 3 Aug 2021 01:45:40 +0000

Hi andy and all.

I don’t think get-data with origin can solve the issues below:

1.       Some leafs like interface type CAN NOT be modified by user, but can be referenced by other config nodes(e.g. using leafref or occur in when/must). The validation will be fail if these leafs are not be configured by user now explicitly (we assume these leafs are optional and no default value).

2.       Some instances are generated by system, but these instances can be referenced by other config nodes. The validation must be fail if these instances are not be recreated by user explicitly now.

3.       User may need know what the original system configuration is, if we get data from <operational>, you may get the modified system configuration.(for example, user modify or template is expanded, or only active instances)

I don’t care about whether system datastore is imported to running or intended datastores. But I think if a config node reference a system node, the validation (running or intended datastores) will be successful even if the system node is not configured by user explicitly.

Especially on the client side,  if a client need validate all data retrieved from server, the validation SHOULD be successful. If system configuration are not imported to running, at a minimum, the client needs to know what the original system configuration is. Another way is adding with-system-used parameter to get-config operation to retrieved all user configuration and system configuration referenced by user configuration.

发件人: netmod [mailto:[email-masked]] 代表 Andy Bierman
发送时间: 2021年8月3日 6:50
收件人: Kent Watsen <[email-masked]>
抄送: Balázs Lengyel <balazs.lengyel=[email-masked]>; [email-masked]
主题: Re: [netmod] system configuration sync mechanism

On Mon, Aug 2, 2021 at 3:31 PM Kent Watsen <[email-masked]<mailto:[email-masked]>> wrote:

As I see it this is the same problem that we discussed in https://github.com/netmod-wg/yang-next/issues/41.
I know this is a radical change, but I think using a new YANG extension to create a read-only config=true datatype is a much cleaner solution. One which some companies have already implemented.
+1

I prefer neither.

Andy

K.

---

From: Jürgen Schönwälder <[email-masked]>
Date: Tue, 3 Aug 2021 06:15:23 +0200

> 1.       Some leafs like interface type CAN NOT be modified by user, but can be referenced by other config nodes(e.g. using leafref or occur in when/must). The validation will be fail if these leaf…

The interface type in the RFC 8343 module is config true. YANG does
not allow you to refer to config false nodes in constraints that apply
to config true nodes. A core principle is that the content of a
configuration datastore can be validated without knowing the actual
state of the system.
 
> 2.       Some instances are generated by system, but these instances can be referenced by other config nodes. The validation must be fail if these instances are not be recreated by user explicitly…

Yes, a configuration datastore is self-contained. If a client wants to
configure the interface x, it has to define the interface x in the
configuration. Note that this should not be confused with a system
generating an interface x after probing the hardware. Note the
difference between operational state, applied config, and running
config.

Let me add that the underlying model is that the client(s) have
control over the configuration. A system making ad-hoc changes to the
config (even with the best intentions) will be surprising. In this
model, the only way to inject config into running on system boot is to
have a client making changes to running following the normal
procedures - at least conceptually. This means that conceptually the
other clients need to be aware that there is a system client injecting
configuration.

If you follow this logic, it seems wrong to define a system datastore
that is somehow magically merged into running - and it is not needed.

> 3.       User may need know what the original system configuration is, if we get data from <operational>, you may get the modified system configuration.(for example, user modify or template is expa…

If you have multiple clients managing shared configuration, then yes
it is good if they are aware of what is going on. I am not sure yet
that exposing other clients intentions via additional datastores and
defining merge mechanisms and semantics is the way to go.

> I don’t care about whether system datastore is imported to running or intended datastores. But I think if a config node reference a system node, the validation (running or intended datastores) will…

I am concerned about having to define what "is imported" means
precisely and whether moving to a model multiple datastores have to be
merged before validation is the way to go. We already acknowledged
that there are template expansions in some implementations without
working out how they work.

> Especially on the client side,  if a client need validate all data retrieved from server, the validation SHOULD be successful. If system configuration are not imported to running, at a minimum, the…

Let me repeat, in the original model, the running configuration
datastore is self-contained and can be validated without knowing
additional datastores.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: "Fengchong (frank)" <[email-masked]>
Date: Tue, 3 Aug 2021 07:34:12 +0000

Hi juergen,
  Please see my comments inline.

  /frank

-----邮件原件-----
发件人: Jürgen Schönwälder [mailto:[email-masked]] 
发送时间: 2021年8月3日 12:15
收件人: Fengchong (frank) <[email-masked]>
抄送: Andy Bierman <[email-masked]>; Kent Watsen <[email-masked]>; Balázs Lengyel <balazs.lengyel=[email-masked]>; [email-masked]
主题: Re: [netmod] 答复: system configuration sync mechanism

> 1.       Some leafs like interface type CAN NOT be modified by user, but can be referenced by other config nodes(e.g. using leafref or occur in when/must). The validation will be fail if these leaf…

The interface type in the RFC 8343 module is config true. YANG does not allow you to refer to config false nodes in constraints that apply to config true nodes. A core principle is that the content of a configuration datastore can be validated without knowing the actual state of the system.
 
Frank: yes, interface type is a config node, but it isn't allow to be modified. If a config node reference if-type, in order to validate data successfully ,user must have to config if-type with determined value again. For example, user must config if-type with 'ethernet' for ethernet0/0/1. I don't think it makes sense. The if-type has been created by system, when running config is merged to <operational>, it can work and the validation is successful. If you think running datastore is self-contained, I will think the system configuration should be imported to running datastore automatically to ensure successful validation

> 2.       Some instances are generated by system, but these instances can be referenced by other config nodes. The validation must be fail if these instances are not be recreated by user explicitly…

Yes, a configuration datastore is self-contained. If a client wants to configure the interface x, it has to define the interface x in the configuration. Note that this should not be confused with a system generating an interface x after probing the hardware. Note the difference between operational state, applied config, and running config.

Frank: for example , predefined policies are provided by system, user configuration can reference these predefined policy directly. but because predefined policies are not configured by user explicitly, the validation will be fail. If you want to get a successful validation , user MUST have to redefined the system predefined policies.
So, system predefined data are useless. I think it is unreasonable.

Let me add that the underlying model is that the client(s) have control over the configuration. A system making ad-hoc changes to the config (even with the best intentions) will be surprising. In this model, the only way to inject config into running on system boot is to have a client making changes to running following the normal procedures - at least conceptually. This means that conceptually the other clients need to be aware that there is a system client injecting configuration.

If you follow this logic, it seems wrong to define a system datastore that is somehow magically merged into running - and it is not needed.

Frank: system configuration not only includes some instances generated on reboot time, but also includes the configuration when hardware is plugged in, or a function is enabled (for example, in huawei's implementation, when QOS function is enabled, many qos predefined policies are created by system) or a user-created list instance is created, some leafs' value will be created by system. So , we need a system datastore to hold these configuration.

> 3.       User may need know what the original system configuration is, if we get data from <operational>, you may get the modified system configuration.(for example, user modify or template is expa…

If you have multiple clients managing shared configuration, then yes it is good if they are aware of what is going on. I am not sure yet that exposing other clients intentions via additional datastores and defining merge mechanisms and semantics is the way to go.

> I don’t care about whether system datastore is imported to running or intended datastores. But I think if a config node reference a system node, the validation (running or intended datastores) will…

I am concerned about having to define what "is imported" means precisely and whether moving to a model multiple datastores have to be merged before validation is the way to go. We already acknowledged that there are template expansions in some implementations without working out how they work.

> Especially on the client side,  if a client need validate all data retrieved from server, the validation SHOULD be successful. If system configuration are not imported to running, at a minimum, the…

Let me repeat, in the original model, the running configuration datastore is self-contained and can be validated without knowing additional datastores.

/js

---

From: Jürgen Schönwälder <[email-masked]>
Date: Tue, 3 Aug 2021 14:33:21 +0200

> Frank: yes, interface type is a config node, but it isn't allow to be modified. If a config node reference if-type, in order to validate data successfully ,user must have to config if-type with det…

You are writing about system configuration and operational state as if
they are the same. I am not sure they are. Also note that interfaces
are not static by any means, they come and go on a modular system. Do
you consider plugging in an interface a system config change??
 
> Frank: for example , predefined policies are provided by system, user configuration can reference these predefined policy directly. but because predefined policies are not configured by user explic…

Referential integrity of a configuration does have benefits and hence
the config has to be there somehow. There was a strong desire to avoid
dangling references that may resolve or not.

> Frank: system configuration not only includes some instances generated on reboot time, but also includes the configuration when hardware is plugged in, or a function is enabled (for example, in hua…

Or you have a system client that is going to merge config into running.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 3 Aug 2021 10:11:16 -0700

On Tue, Aug 3, 2021 at 12:34 AM Fengchong (frank) <
[email-masked]> wrote:

> 抄送: Andy Bierman <[email-masked]>; Kent Watsen <[email-masked]>;
> Balázs Lengyel <balazs.lengyel=[email-masked]>;
> [email-masked]
> > 1.       Some leafs like interface type CAN NOT be modified by user, but
> can be referenced by other config nodes(e.g. using leafref or occur in
> when/must). The validation will be fail if these leafs are not be
> configured by user now explicitly (we assume these leafs are optional and
> no default value).
>
> The interface type in the RFC 8343 module is config true. YANG does not
> allow you to refer to config false nodes in constraints that apply to
> config true nodes. A core principle is that the content of a configuration
> datastore can be validated without knowing the actual state of the system.
>
> Frank: yes, interface type is a config node, but it isn't allow to be
> modified. If a config node reference if-type, in order to validate data
> successfully ,user must have to config if-type with determined value again.
> For example, user must config if-type with 'ethernet' for ethernet0/0/1. I
> don't think it makes sense. The if-type has been created by system, when
> running config is merged to <operational>, it can work and the validation
> is successful. If you think running datastore is self-contained, I will
> think the system configuration should be imported to running datastore
> automatically to ensure successful validation
>
>

I doubt another rewrite of ietf-interfaces would gain much support.

There are many objects that have user access restrictions that are not
standardized
with machine-readable statements.  I am interested in the 2 use cases that
Balazs described.
The system does its own access control (not NACM) wrt/ these objects. The
server just
rejects the edit requests, even though the standard YANG indicates it
should be writable.
Some standard extensions to tag the data would help applications better
predict server behavior.

There is no way in YANG 1.1 to have a config=true node reference
config=false in the XPath.
(Well, we implement it but the developer has to enable it).

I do not see the point of the system datastore if it contains config=true
nodes,
just so that XPath can reference it.  I do not agree that it is useful or
easy to implement,
especially since the contents of <system> are visible in <intended>.
Representing the possible
instances is non-trivial (e.g. pattern allowed, not a fixed name, resulting
in 6 million possible values).

If the system datastore contains config=false nodes, then you are talking
about a rewrite
of the YANG validation architecture and a new version of YANG.  IMO there
is no need for that.

Andy

> 2.       Some instances are generated by system, but these instances can
> be referenced by other config nodes. The validation must be fail if these
> instances are not be recreated by user explicitly now.
>
> generating an interface x after probing the hardware. Note the difference
> between operational state, applied config, and running config.
>
> Frank: for example , predefined policies are provided by system, user
> configuration can reference these predefined policy directly. but because
> predefined policies are not configured by user explicitly, the validation
> will be fail. If you want to get a successful validation , user MUST have
> to redefined the system predefined policies.
> Let me add that the underlying model is that the client(s) have control
> over the configuration. A system making ad-hoc changes to the config (even
> with the best intentions) will be surprising. In this model, the only way
> to inject config into running on system boot is to have a client making
> changes to running following the normal procedures - at least conceptually.
> This means that conceptually the other clients need to be aware that there
> is a system client injecting configuration.
>
> If you follow this logic, it seems wrong to define a system datastore that
> is somehow magically merged into running - and it is not needed.
>
> Frank: system configuration not only includes some instances generated on
> reboot time, but also includes the configuration when hardware is plugged
> in, or a function is enabled (for example, in huawei's implementation, when
> QOS function is enabled, many qos predefined policies are created by
> system) or a user-created list instance is created, some leafs' value will
> be created by system. So , we need a system datastore to hold these
> configuration.
>
> > 3.       User may need know what the original system configuration is,
> if we get data from <operational>, you may get the modified system
> configuration.(for example, user modify or template is expanded, or only
> active instances)
>
> If you have multiple clients managing shared configuration, then yes it is
> good if they are aware of what is going on. I am not sure yet that exposing
> other clients intentions via additional datastores and defining merge
> mechanisms and semantics is the way to go.
>
> > I don’t care about whether system datastore is imported to running or
> intended datastores. But I think if a config node reference a system node,
> the validation (running or intended datastores) will be successful even if
> the system node is not configured by user explicitly.
>
> I am concerned about having to define what "is imported" means precisely
> and whether moving to a model multiple datastores have to be merged before
> validation is the way to go. We already acknowledged that there are
> template expansions in some implementations without working out how they
> work.
>
> > Especially on the client side,  if a client need validate all data
> retrieved from server, the validation SHOULD be successful. If system
> configuration are not imported to running, at a minimum, the client needs
> to know what the original system configuration is. Another way is adding
> with-system-used parameter to get-config operation to retrieved all user
> configuration and system configuration referenced by user configuration.
>
> Let me repeat, in the original model, the running configuration datastore
> is self-contained and can be validated without knowing additional
> datastores.
>
> /js
>
> --

---

From: "Fengchong (frank)" <[email-masked]>
Date: Wed, 4 Aug 2021 02:54:08 +0000

Hi,andy,

发件人: Andy Bierman [mailto:[email-masked]]
发送时间: 2021年8月4日 1:11
收件人: Fengchong (frank) <[email-masked]>
抄送: Juergen Schoenwaelder <[email-masked]>; Kent Watsen <[email-masked]>; Balázs Lengyel <balazs.lengyel=[email-masked]>; [email-masked]
主题: Re: [netmod] 答复: system configuration sync mechanism

On Tue, Aug 3, 2021 at 12:34 AM Fengchong (frank) <[email-masked]<mailto:[email-masked]>> wrote:
Hi juergen,
  Please see my comments inline.

  /frank

-----邮件原件-----
发件人: Jürgen Schönwälder [mailto:[email-masked]<mailto:[email-masked]>]
发送时间: 2021年8月3日 12:15
收件人: Fengchong (frank) <[email-masked]<mailto:[email-masked]>>
抄送: Andy Bierman <[email-masked]<mailto:[email-masked]>>; Kent Watsen <[email-masked]<mailto:[email-masked]>>; Balázs Lengyel <balazs.lengyel=[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
主题: Re: [netmod] 答复: system configuration sync mechanism

Frank:I don’t know what’s the another rewrite of ietf-interfaces.

There are many objects that have user access restrictions that are not standardized
with machine-readable statements.  I am interested in the 2 use cases that Balazs described.
The system does its own access control (not NACM) wrt/ these objects. The server just
rejects the edit requests, even though the standard YANG indicates it should be writable.
Some standard extensions to tag the data would help applications better predict server behavior.

Frank: yes. I agree with you. But in some cases, only part of the same list’s instances are read only, we can not place a tag on YANG node.

There is no way in YANG 1.1 to have a config=true node reference config=false in the XPath.
(Well, we implement it but the developer has to enable it).

Frank: I don’t think it’s a correct way.

I do not see the point of the system datastore if it contains config=true nodes,
just so that XPath can reference it.  I do not agree that it is useful or easy to implement,
especially since the contents of <system> are visible in <intended>.  Representing the possible
instances is non-trivial (e.g. pattern allowed, not a fixed name, resulting in 6 million possible values).

Frank: my object is solving the issues that when we want to reference a system predefined instance we must recreate it.

In RFC7950 sec 6.4.1, the context of XPATH also include default value and NP-container, even though they are not exist in datastore.
It’s maybe a good idea that the context of XPATH can include running configuration and the system configuration and we don’t need to import system configuration to running datastore.
In this case, <system> datastore is necessary for client, because the client may need  system configuration for validation.

If the system datastore contains config=false nodes, then you are talking about a rewrite
of the YANG validation architecture and a new version of YANG.  IMO there is no need for that.

Frank: I think it is not correct way.

Andy

/js