Subject: [Netconf] magic leaf 'type' in IETF interfaces

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 14 Dec 2018 17:52:56 +0000

Hi all,

Cross posting because I'm not sure whether this is more of interest to NETCONF clients or a general YANG & datastore question.

In IETF Interfaces there is a leaf 'type'. Here is a snippet of the model:

      leaf type {
        type identityref {
          base interface-type;
        }
        mandatory true;
        description
          "The type of the interface.

           When an interface entry is created, a server MAY
           initialize the type leaf with a valid value

This says that although the leaf is mandatory, a NETCONF client creating a new interface does not have to specify/provide that leaf. That strikes me as the first unusual point.

Secondly -> this also means that a NETCONF client may send a config with elements X, Y, and Z, but then later read back the config to see X, Y, Z and T  (e.g. type). But they never configured the type leaf.

Shouldn't most clients generally assume that what they write, they read back (unless there are 'choice' or 'when' statements involved of course, but that are part of the YANG and any auto-clearing behavior from those would be expected)?

Or does 'anything go' / 'market decides' when it comes to behavior like this explained in 'description' statements?

Is it just fine that some NETCONF servers auto-magically create some (extra) data nodes inside a list entry that a client created? (would like to see opinions from multiple people on this - especially client developers).

I would think that each and every magic creation/deletion/changes done by a server (i.e. that aren't part of the YANG, except perhaps part of a human-readable (non-machine parsable) 'description' statement) would require some special handling code on the client (or app above the client) side.

I can imagine several alternatives to the way it was modelled above:
1) NMDA approach: make the leaf optional. If the operator doesn't set that leaf, then reading the conventional datastores doesn't return that leaf. But reading the operational DS could return the actual system selected value.
2) separate config & state leafs for 'type':  make the leaf optional. If the operator doesn't set that leaf, then reading the conventional datastores doesn't return that leaf. But have another state leaf called 'oper-type'.

I'm not proposing to re-open the IETF Interface model. Just using it to ask questions about server created config data and explore alternatives.

Jason

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Dec 2018 09:15:05 +0100 (CET)

Hi,

"Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]> wrote:
> Hi all,
> 
> Cross posting because I'm not sure whether this is more of interest to
> NETCONF clients or a general YANG & datastore question.
> 
> This says that although the leaf is mandatory, a NETCONF client
> creating a new interface does not have to specify/provide that
> leaf. That strikes me as the first unusual point.

Yes, unusual, but "safe" in the sense that a client that doesn't have
code for this special case will just see the "mandatory true" and
provide a value for the type as well.

> Secondly -> this also means that a NETCONF client may send a config
> with elements X, Y, and Z, but then later read back the config to see
> X, Y, Z and T (e.g. type). But they never configured the type leaf.

Right, but again this only happens for clients that know this special
rule.

> Shouldn't most clients generally assume that what they write, they
> read back (unless there are 'choice' or 'when' statements involved of
> course, but that are part of the YANG and any auto-clearing behavior
> from those would be expected)?
> 
> Or does 'anything go' / 'market decides' when it comes to behavior
> like this explained in 'description' statements?
> 
> Is it just fine that some NETCONF servers auto-magically create some
> (extra) data nodes inside a list entry that a client created? (would
> like to see opinions from multiple people on this - especially client
> developers).

In general, my reply is "no, that is not fine".

The reason for this unusual design is that most existing systems code
type-info into the name of an interface ("ethX", "ge-X/Y/Z",
"GigabitEthernet-X/Y" etc), and having to provide the type in the name
and in a special leaf would just be redundant and error prone.  The
thinking at the time was that the interface model that we designed had
to adapt to how existing systems behave.

> I would think that each and every magic creation/deletion/changes done
> by a server (i.e. that aren't part of the YANG, except perhaps part of
> a human-readable (non-machine parsable) 'description' statement) would
> require some special handling code on the client (or app above the
> client) side.

Yes.  But as long as this special handling is optional in the client I
think it is ok.

> 1) NMDA approach: make the leaf optional. If the operator doesn't set
> that leaf, then reading the conventional datastores doesn't return
> that leaf. But reading the operational DS could return the actual
> system selected value.
>
> 2) separate config & state leafs for 'type': make the leaf
> optional. If the operator doesn't set that leaf, then reading the
> conventional datastores doesn't return that leaf. But have another
> state leaf called 'oper-type'.

Both these cases assume that the type can be derived from the name.

> I'm not proposing to re-open the IETF Interface model. Just using it
> to ask questions about server created config data and explore
> alternatives.

The rule of thumb is to avoid server created config as much as
possible.

/martin

---

From: Balázs Lengyel <[email-masked]>
Date: Mon, 17 Dec 2018 09:05:59 +0000

Hello,

While I agree with Martin, in our systems we have a number of
places, where the system does create configuration in running, due
to

different levels of automation and autonomous algorithms
kick-in

the created config needs to be possible to be further modified
by the operator

the created config needs to be referenced from operator
created config

the created config is not always ephemeral, it might need to
be part of backup/restore

regards Balazs

On 2018. 12. 17. 9:15, Martin Bjorklund
wrote:

/martin

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Jan Lindblad <[email-masked]>
Date: Tue, 18 Dec 2018 14:36:44 +0100

Hi,
> While I agree with Martin, in our systems we have a number of places, where the system does create configuration in running, due to
> 
> different levels of automation and autonomous algorithms kick-in
> the created config needs to be possible to be further modified by the operator
> the created config needs to be referenced from operator created config
> the created config is not always ephemeral, it might need to be part of backup/restore
This is only a sampling from "the list of excuses". I have heard many more. The road to hell is paved with good intentions, however. If we want to build automation based on sound theory, clearly separating the orders from managers from a system's own operational view is key, IMO. Reliability, security, accountability are growing in importance, and they all play in this direction.

We may not need to standardize rules to outlaw the above; the market will take care of that. What we need to ensure is that it is possible to be standards compliant without having to implement design excuses like these.

Best Regards,
/jan

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Dec 2018 18:24:28 +0100

Jan Lindblad <[email-masked]> writes:

> We may not need to standardize rules to outlaw the above; the market
> will take care of that. What we need to ensure is that it is possible
> to be standards compliant without having to implement design excuses
> like these.

I agree. Now that we have NMDA, things like this shouldn't be necessary.

Lada

>
> Best Regards,
> /jan
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Dec 2018 10:14:36 -0800

On Tue, Dec 18, 2018 at 5:37 AM Jan Lindblad <[email-masked]> wrote:

> Hi,
>
> While I agree with Martin, in our systems we have a number of places,
> where the system does create configuration in running, due to
>
>    - different levels of automation and autonomous algorithms kick-in
>    - the created config needs to be possible to be further modified by
>    the operator
>    - the created config needs to be referenced from operator created
>    config
>    - the created config is not always ephemeral, it might need to be part
>    of backup/restore
>
> This is only a sampling from "the list of excuses". I have heard many
> more. The road to hell is paved with good intentions, however. If we want
> to build automation based on sound theory, clearly separating the orders
> from managers from a system's own operational view is key, IMO.
> Reliability, security, accountability are growing in importance, and they
> all play in this direction.
>
> We may not need to standardize rules to outlaw the above; the market will
> take care of that. What we need to ensure is that it is possible to be
> standards compliant without having to implement design excuses like these.
>
>
NMDA has a lot of room for proprietary mechanisms for converting <running>
to <intended>.
Many times the features desired by engineers exceed the capabilities of
YANG, such as
a dynamic default leaf.  YANG allows a simple constant, and no business
logic to pick the default.
This is a very valid use of "server auto-magic".

Maybe a future version of YANG can improve the client visibility into this
"auto-magic"

Best Regards,
> /jan

Andy

---

From: Jan Lindblad <[email-masked]>
Date: Wed, 19 Dec 2018 10:30:46 +0100

Hi,
> NMDA has a lot of room for proprietary mechanisms for converting <running> to <intended>.
> Many times the features desired by engineers exceed the capabilities of YANG, such as
> a dynamic default leaf.  YANG allows a simple constant, and no business logic to pick the default.
> Maybe a future version of YANG can improve the client visibility into this "auto-magic"

As you say, this is not uncommon. I usually recommend to leave out any default statement, and write in the description what happens if this leaf isn't set. The operator can then override the default by giving a value. 

While some more advanced features for default values may be of some utility, the simplicity of YANG is also important. We don't want to make the YANG models -- the interface contracts -- the new place for all business logic.

/jan

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Dec 2018 15:16:10 +0100

Jan Lindblad <[email-masked]> writes:

> As you say, this is not uncommon. I usually recommend to leave out any
> default statement, and write in the description what happens if this
> leaf isn't set. The operator can then override the default by giving a
> value.

Anyway, this is not a case where the server writes something on its own
to a configuration datastore.

>
> While some more advanced features for default values may be of some
> utility, the simplicity of YANG is also important. We don't want to
> make the YANG models -- the interface contracts -- the new place for
> all business logic.

Absolutely.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Dec 2018 10:46:29 -0800

On Wed, Dec 19, 2018 at 6:16 AM Ladislav Lhotka <[email-masked]> wrote:

> will take care of that. What we need to ensure is that it is possible to be

I don't think it is a problem if NMDA or non-NMDA servers write to
<running>.
Just part of the complexity that is baked in -- NMDA does nothing to help
the client know
why <running> is different than <intended> anyway.

> > make the YANG models -- the interface contracts -- the new place for
I am not proposing YANG needs a new default-stmt. There is a
description-stmt
and vendors can add their own extensions to flag auto-magic data nodes.

Andy

---

From: Qin Wu <[email-masked]>
Date: Thu, 20 Dec 2018 00:48:42 +0000

Hi:
发件人: netmod [mailto:[email-masked]] 代表 Jan Lindblad
发送时间: 2018年12月18日 21:37
收件人: Balázs Lengyel
抄送: [email-masked]; [email-masked]
主题: Re: [netmod] [Netconf] magic leaf 'type' in IETF interfaces

Hi,

  *   different levels of automation and autonomous algorithms kick-in
  *   the created config needs to be possible to be further modified by the operator

  *   the created config needs to be referenced from operator created config
[Qin]: We have a new draft that has just been posted to discuss some of these issues which allow system creation configuration in running
https://tools.ietf.org/html/draft-wu-netconf-nmda-compatibility-00

  *   the created config is not always ephemeral, it might need to be part of backup/restore
This is only a sampling from "the list of excuses". I have heard many more. The road to hell is paved with good intentions, however. If we want to build automation based on sound theory, clearly separating the orders from managers from a system's own operational view is key, IMO. Reliability, security, accountability are growing in importance, and they all play in this direction.

We may not need to standardize rules to outlaw the above; the market will take care of that. What we need to ensure is that it is possible to be standards compliant without having to implement design excuses like these.
      [Qin]: I would support standardizing some of guidance rules or design rules on how to deal with these cases.

Best Regards,
/jan

---

From: Qin Wu <[email-masked]>
Date: Thu, 20 Dec 2018 00:57:40 +0000

发件人: Netconf [mailto:[email-masked]] 代表 Andy Bierman
发送时间: 2018年12月20日 2:46
收件人: Jan Lindblad; Andy Bierman; [email-masked]; [email-masked]
主题: Re: [Netconf] [netmod] magic leaf 'type' in IETF interfaces

On Wed, Dec 19, 2018 at 6:16 AM Ladislav Lhotka <[email-masked]<mailto:[email-masked]>> wrote:
Jan Lindblad <[email-masked]<mailto:[email-masked]>> writes:

>> This is only a sampling from "the list of excuses". I have heard many more. The road to hell is paved with good intentions, however.. If we want to build automation based on sound theory, clearly separating the orders from managers from a system's own operational view is key, IMO. Reliability, security, accountability are growing in importance, and they all play in this direction.
>>

I don't think it is a problem if NMDA or non-NMDA servers write to <running>.
Just part of the complexity that is baked in -- NMDA does nothing to help the client know
why <running> is different than <intended> anyway.

[Qin]: I think it is important to make NMDA server can use existing operation to

   return the same results to Non-NMDA client with <rpc-reply> as non-NMDA server

   does. We call this Server Backwards-Compatibility in

   https://tools.ietf.org/html/draft-wu-netconf-nmda-compatibility-00#section-2.3

Absolutely.

I am not proposing YANG needs a new default-stmt. There is a description-stmt
and vendors can add their own extensions to flag auto-magic data nodes.

Lada

Andy

> [email-masked]<mailto:[email-masked]>

---

From: "Belotti, Sergio (Nokia - IT/Vimercate)" <[email-masked]>
Date: Thu, 20 Dec 2018 09:08:23 +0000

Hi,

SB>> But RFC 8342 says :

“It represents the configuration after all configuration transformations to <running> are performed (e.g.,template expansion, removal of inactive configuration)”

So my understanding is that by definition “intended” can be different from “running”.
Am I missed something?

Absolutely.

Lada

Andy

Thanks
Sergio

Sergio Belotti
Senior System Engineer and Standardization Architect
IP/Optical Networks, Optics BU
Nokia
M: +39-335761776
Via Energy Park, 20871 Vimercate (MB) , Italy
[email-masked]<mailto:[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 20 Dec 2018 07:31:43 -0800

On Thu, Dec 20, 2018 at 1:08 AM Belotti, Sergio (Nokia - IT/Vimercate) <
[email-masked]> wrote:

> more. The road to hell is paved with good intentions, however.. If we want
> “It represents the configuration after all configuration transformations
> to <running> are performed (e.g.,template expansion, removal of inactive
> configuration)”
>
>
>

The client can see WHAT changed because of server-auto-magic but not WHY
a particular server implementation converts <foo> into <bar> if <foo> is
edited.
A client (using plain YANG) cannot predict at all what will happen if <foo>
is edited.

The issue remains: how does a client know that a mandatory leaf is not
actually required in <running>?  I guess vague auto-magic description-text
is good enough

Andy

> [email-masked]