Subject: [netmod] new YANG 1.1 Issue Y61: I2RS support

From: Andy Bierman <[email-masked]>
Date: Sat, 25 Jul 2015 17:17:11 -0700

Hi,

I would like to open another issue for YANG 1.1,
because I don't want to have 1.1 and then 1.2 right away.
The NETMOD WG should evaluate the different ways to
support ephemeral state, based on Jeff's draft.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 26 Jul 2015 09:16:23 +0200

> support ephemeral state, based on Jeff's draft.

The NETMOD WG did spent almost a full day face-to-face interim meeting
time in September 2014 on this and now we have a requirements I-D plus
a solution proposal that does not directly match what was discussed
back then. It is my understanding that the solution discussed in
September 2014 does not require changes to YANG 1.1.

I2RS was aware of the YANG 1.1 timeline from the very beginning. YANG
1.1 is gating other specifications and I am not interested to hold
everything off (including RESTCONF) because of I2RS. There are many
other customers of YANG 1.1 beyond I2RS.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 26 Jul 2015 12:53:41 +0200

> On 26 Jul 2015, at 02:17, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 

Rather than implementing ephemeral data, I’d actually prefer to consider it a subproblem of making YANG less NETCONF-specific: YANG would deal only with data trees, and it would be up to each management protocol profile to define what trees the protocol uses and what is their semantics.

What might be a new task for YANG is to define general syntax for identifying different trees and inter-tree references.

Lada 

> 
> 
> Andy
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 26 Jul 2015 12:56:16 +0200

> What might be a new task for YANG is to define general syntax for identifying different trees and inter-tree references.

This is not the time to add new features to YANG 1.1.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sun, 26 Jul 2015 04:17:26 -0700

> other customers of YANG 1.1 beyond I2RS.

Yeah, it's too bad so many drafts are waiting on YANG.
Support for I2RS got started but never finished.
I care more about the costs of deploying tools and the extra complexity
on readers, who need to know all the versions of YANG.
The proposed solution changes one of the most important statments
in YANG.  Harding something to do as an afterthought.

It should not take that long because the NETCONF WG (same people)
have been spending almost all f2f and virtual interim time on I2RS.
The YANG doctors concluded that ephemeral state
was a general feature and not NETCONF specific.

The problem with using YANG extensions for important protocol features
is that the YANG spec says these statements MAY be completely skipped
by a tool implementation.  This is not acceptable for ephemeral state
(or operational state either).

/js

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 26 Jul 2015 13:47:15 +0200

> in YANG.  Harding something to do as an afterthought.

I do not think there is agreement on any solution yet. I heard also
about vendors implementing ephemeral state solutions that do not
require any changes to YANG (and that seem to be more inline with what
was discussed in September 2014).

> (or operational state either).

I do not know what is to be addressed for operational state.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sun, 26 Jul 2015 11:17:19 -0700

Hi,

I agree it should be a WG (maybe IESG) decision whether YANG 1.1
should be published ASAP and a new version started right away to update it.
The RFC publication process is not that hard to solve. The tool and user
confusion caused by all these versions is another matter.

more inline...

> was discussed in September 2014).

I know there is not agreement on the solution yet.
That is why it is an open issue. I2RS does not have to present NETMOD WG
with a solution (even though they did present an acceptable solution).

In order for I2RS to have the YANG validation rules that meet its
use-cases, new text is needed. This text needs to be associated
with data-nodes.

I like the proposed solution of config=true,ephemeral,false
because I know none of the existing text inadvertently applies
to ephemeral nodes.

I do not think YANG is fully specified wrt/ datastores because
operational state and ephemeral nodes are not separated.
An ephemeral datastore (and perhaps operational datastore)
are needed to solve this problem.

> I do not know what is to be addressed for operational state.

Then openconfig issues for starters.
That does not have to be solved at the same time as ephemeral data.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 27 Jul 2015 09:17:26 +0200

> I know there is not agreement on the solution yet.

Exactly. And hence talking about YANG 1.2 is simply confusing.

> with a solution (even though they did present an acceptable solution).

If you mean duplication of data models and data model specific merge
rules then I have trouble to believe this is desriable nor does it
seem to reflect what other proprietary ephemeral datastore
implementations do.

> because I know none of the existing text inadvertently applies

I would prefer to have ephemeral datastore support as an extension;
not all uses of YANG (perhaps even a big majority) require to describe
ephemeral datastores.

> are needed to solve this problem.

But depending on the solution selected, this may not require any
changes to YANG 1.1.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 28 Jul 2015 11:42:01 +0200 (CEST)

Hi,

> > > support ephemeral state, based on Jeff's draft.

[...]

> (or operational state either).

I don't agree that this is a problem.  If i2rs defines an extension,
then i2rs implementations will have to support that extension.  This
is the whole idea behind extensions - we should not have to revise
YANG everytime we need a new statement.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 28 Jul 2015 11:56:18 +0200

> On 28 Jul 2015, at 11:42, Martin Bjorklund <[email-masked]> wrote:
> 
> Hi,
> 

Yes, it could work in this case as long as modules containing this extension are never advertised to regular NETCONF/RESTCONF clients.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 28 Jul 2015 11:58:52 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> Yes, it could work in this case as long as modules containing this
> extension are never advertised to regular NETCONF/RESTCONF clients.

I think it would.  Such nodes would just be seen as config false
nodes.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2015 06:26:57 -0700

> extension are never advertised to regular NETCONF/RESTCONF clients.

This restriction is unacceptable.
It must be possible to put I2RS data in any YANG module.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2015 06:44:23 -0700

> I think it would.  Such nodes would just be seen as config false
I do not agree that YANG extensions are appropriate for defining standard
protocols.
They are fine for extra tools outside the scope of any protocol.
The "get filter" hack in RFC 6241 does not even work since
any tool is allowed to ignore the extension.

The solution proposed by I2RS changed the config-stmt.
IMO this is a better solution than defining an extension
that every YANG tool MAY ignore.

Ephemeral data can be defined in any YANG module,
not in special hack modules that are not allowed to be shared
by NETCONF or RESTCONF in any way,
That would be a terrible solution for ephemeral data.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 28 Jul 2015 16:13:16 +0200

> On 28 Jul 2015, at 15:44, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> I do not agree that YANG extensions are appropriate for defining standard protocols.

I am not sure the ephemeral property required by I2RS only means that a given parameter is is stored in volatile memory. In the meeting with Jeff and Alia on Thursday afternoon we (again) discussed the overlay property: they apparently want to be able to override normal config values with a new (ephemeral) value installed by an I2RS client. This would mean that the same data node exists in two instances - one permanent and one ephemeral, and config true/false/ephemeral is then clearly insufficient.

That’s why I asked during the session for clarification of the interactions between ephemeral and standard config values.

It also has implications for validation - the running config certainly has to be always valid but what about validity of running + ephemeral data? Jeff’s response was that no semantic validation is done in this case, and it is up to the consuming server application to somehow cope with invalid data and do whatever is the most appropriate action.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2015 07:38:05 -0700

On Tue, Jul 28, 2015 at 7:13 AM, Ladislav Lhotka <[email-masked]> wrote:

> I am not sure the ephemeral property required by I2RS only means that a
> given parameter is is stored in volatile memory. In the meeting with Jeff
> and Alia on Thursday afternoon we (again) discussed the overlay property:
> they apparently want to be able to override normal config values with a new
> (ephemeral) value installed by an I2RS client. This would mean that the
> same data node exists in two instances - one permanent and one ephemeral,
> and config true/false/ephemeral is then clearly insufficient.
>
> That’s why I asked during the session for clarification of the
> interactions between ephemeral and standard config values.
>
> It also has implications for validation - the running config certainly has
> to be always valid but what about validity of running + ephemeral data?
> Jeff’s response was that no semantic validation is done in this case, and
> it is up to the consuming server application to somehow cope with invalid
> data and do whatever is the most appropriate action.
>
>

There are 2 issues:

   1) overlay: how does I2RS access config=true data in the ephemeral
datastore

    A: Define a new <ephemeral /> leaf that can be used as a target
datastore
        for datastore parameters
        Define new operations like <edit-ephemeral>
        Enhance <get> to filter for ephemeral data

   2) state-only: how does I2RS access ephemeral data in the ephemeral
datastore
       This is special data that is not a config=true data model.  This
data is allowed
       to access operational state (e.g., reference system or protocol
assigned values),
      which is not allowed for config=true validation

  A:  Define ephemeral-stmt or modify config-stmt to identify
ephemeral-only data

Config-stmt approach
-----------------------------

Config or Ephemeral Data:
     config true;

Ephemeral-only data:
     config ephemeral;

Operational data:
      config false;

New statement approach
-----------------------------------

Config or Ephemeral Data:
     config true;

Ephemeral-only data:
     config false;
     ephemeral true;

Operational data:
      config false;
      ephemeral false;

Lada

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 28 Jul 2015 23:45:35 +0200 (CEST)

> that every YANG tool MAY ignore.

6020(bis) says that a tool that doesn't understand the extension
statement MAY ignore it.  My point is that if I2RS defines an
extension estatement, I2RS-compliant tools will have to implement this
statement and they can of course not simply ignore it.

Again, this is the whole point of having extensions - the language can
be extended without having to revise the base specification.

> by NETCONF or RESTCONF in any way,

I agree that get-filter-element-attributes is a hack.  But that hack
really just defines the NETCONF *protocol*.  It is not applicable to
RESTCONF.

I do not agree that a module that defines an extension is a "special
hack module".

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2015 16:57:25 -0700

Hi,

I do not agree that ephemeral data should be outside the scope of the
standard.
I do not agree that YANG extensions should be used for IETF standards track
modules.
It was a mistake to define the "get-filter" hack in the first place.
IMO it is a mistake to define Lada's "annotation" statement as an extension.

I don't think you are correct in your interpretation of "MAY ignore".
My reading of this text is that no YANG tool anywhere (or ever)
MUST or even SHOULD understand any YANG extension.
I don't see how YANG conformance applies to usage of extension
statements.  It only applies to usage of YANG statements.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 09:34:17 +0200 (CEST)

Hi,

> I do not agree that ephemeral data should be outside the scope of the

I don't think anyone has said that.

> I do not agree that YANG extensions should be used for IETF standards track

I strongly disagree.  This was one of the two original ideas behind
extension statements (the other was that vendors could add their own
stuff).

> statements.  It only applies to usage of YANG statements.

Then I think we need to clarify the text in 6020bis.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 11:35:54 +0200

> MUST or even SHOULD understand any YANG extension.

This was my reading, too. The text actually talks about "compiler" which
is even less clear - it may apply to server or client software as well.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 11:37:48 +0200 (CEST)

> IMO it is a mistake to define Lada's "annotation" statement as an extension.

We also have extensions defined in RFC 6536 (NACM).

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 12:25:11 +0200

> extension statements (the other was that vendors could add their own

An extension defines certain semantics. For example:

  extension default-deny-write {
    description
          "Used to indicate that the data model node
           represents a sensitive security system parameter.

           If present, and the NACM module is enabled (i.e.,
           /nacm/enable-nacm object equals 'true'), the NETCONF server
           will only allow the designated 'recovery session' to have
           write access to the node.  An explicit access control rule is
           required for all other users.

           The 'default-deny-write' extension MAY appear within a data
           definition statement.  It is ignored otherwise.";
  }

If a data model writer uses an extension, then the semantics
associated with the extension are embedded in the data model. The
extension can be seen as a shorthand for copying the semantics
directly into the description clause. In other words, an
implementation has to implement the semantics no matter which
tool is used.

Whether tools understand the semantics of an extension statement or
not is a tool implementation issue. If a YANG tool does not support
default-deny-write, an implementation still needs to provide the
behaviour called out for if a data model uses this extension.

Given this interpretation of extensions, I do not really understand
the discussion here.

/js

PS: Perhaps we should change

       If a YANG compiler does not support a particular extension, which
       appears in a YANG module as an unknown-statement (see Section 13),
       the entire unknown-statement MAY be ignored by the compiler.

    to 

       If a YANG parser does not support a particular extension, which
       appears in a YANG module as an unknown-statement (see Section 13),
       the entire unknown-statement MAY be ignored by the parser. Note
       that even in this case the semantics associated with the extension
       still apply (as if they were part of a description statement).

    and consistently use YANG parser everywhere instead of sometimes YANG
    parser and sometimes YANG compiler.

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 12:57:09 +0200

> On 29 Jul 2015, at 12:25, Juergen Schoenwaelder <[email-masked]> wrote:
> 

The difference is that YANG spec doesn’t say descriptions may be ignored.

> implementation has to implement the semantics no matter which

I agree, but then I think the text you have in PS should be removed entirely. There is again the issue of old client support that IMO doesn’t hold water anyway: a client that doesn’t fully understand the semantics of the data model cannot be guaranteed to work, and is in fact dangerous.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 13:47:47 +0200

> The difference is that YANG spec doesn’t say descriptions may be ignored.

I do not get it, please help me.
 
> I agree, but then I think the text you have in PS should be removed entirely. There is again the issue of old client support that IMO doesn’t hold water anyway: a client that doesn’t fully understa…

Again, I do not get it. There are non-machine readable semantics
anyway. A decent client better understands the semantics - otherwise
it is just a clueless yang browser. So why is there a problem with
extension statements (which are just a shorthand for semantics that
happen to be machine readable for those tools that are programmed to
read and understand them).

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 14:24:03 +0200

> On 29 Jul 2015, at 13:47, Juergen Schoenwaelder <[email-masked]> wrote:
> 

A client that ignores constraints expressed in a description statement should be considered broken. I am not sure it is also the case for a client that ignores an extension and the implied semantics, because sec. 6.3.1 seems to allow it.

In fact, I think it can be applied to the server, too: An implementor may take a module, remove extensions he doesn’t want to implement (or let a “compiler” remove them) and implement the modified form of the module.

> read and understand them).

As I said, it is unclear (to me at least) from the text in 6.3.1 whether extensions are an integral part of the contract expressed by the module. If I had a contract with an insurance company saying “Anybody who cannot read fine print may ignore it”, I would get quite nervous.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 14:57:57 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> 
> > On 29 Jul 2015, at 13:47, Juergen Schoenwaelder
> > <[email-masked]> wrote:
> > 
> >>> On 29 Jul 2015, at 12:25, Juergen Schoenwaelder
> >>> <[email-masked]> wrote:
> >>> 
> A client that ignores constraints expressed in a description statement
> should be considered broken. I am not sure it is also the case for a
> client that ignores an extension and the implied semantics, because
> sec. 6.3.1 seems to allow it.

The reason that the text about MAY ignore is there was to allow a tool
like pyang to function even if it finds an extension like yuma:root in
a model.  pyang doesn't "understand" this extension, yet it can
process the module.

I like Juergens analogy with text in the description statement - you
can write any rules in the description statement, or you can choose to
formalize it in an extension.  Sometimes such rules are directed to a
client, sometimes to a server, sometimes to the YANG compiler
(e.g., code generation directives), and sometimes to humans reading
the module.

/martin

> In fact, I think it can be applied to the server, too: An implementor
> may take a module, remove extensions he doesn’t want to implement (or
> let a “compiler” remove them) and implement the modified form of the
> module.
> 
> > 
> >> I agree, but then I think the text you have in PS should be removed
> >> entirely. There is again the issue of old client support that IMO
> >> doesn’t hold water anyway: a client that doesn’t fully understand the
> >> semantics of the data model cannot be guaranteed to work, and is in
> >> fact dangerous.
> >> 
> > 
> As I said, it is unclear (to me at least) from the text in 6.3.1
> whether extensions are an integral part of the contract expressed by
> the module. If I had a contract with an insurance company saying
> “Anybody who cannot read fine print may ignore it”, I would get quite
> nervous.
> 
> Lada
> 
> > 
> > /js
> > 
> > --

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 15:46:27 +0200

> On 29 Jul 2015, at 14:57, Martin Bjorklund <[email-masked]> wrote:
> 

I think this is unnecessary. After all, I can write a tool that ignores everything in a module except the module name, so what? Is that tool illegal? RFC 2119 keywords should be primarily used where it matters to interoperability, i.e. client/server contract.

> (e.g., code generation directives), and sometimes to humans reading

But in order to determine whether a given extension is directed to a piece of software, I need to know the semantics of that extension. If I have no idea what yuma:root means, it may not be safe to ignore it.

I think it is important to understand that all extensions in the data model advertised by the server are an integral part of the model. If an implementation or tool ignores an extension because it’s of no interest to it, then OK but it’s the implementor’s responsibility.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 16:14:03 +0200

> In fact, I think it can be applied to the server, too: An implementor may take a module, remove extensions he doesn’t want to implement (or let a “compiler” remove them) and implement the modified…

As I tried to explain, I believe there is confusion between a tool
skipping over an extension statement and the semantics that are
applied to the data model. The text in 6.3.1 talks about the tool
(the 'compiler').

> As I said, it is unclear (to me at least) from the text in 6.3.1 whether extensions are an integral part of the contract expressed by the module. If I had a contract with an insurance company sayin…

I believe this is a mis-interpretation of the text in 6.3.1. and I
tried to propose new text that hopefully clarifies this. You agree
with the proposed text?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 16:38:41 +0200

> On 29 Jul 2015, at 16:14, Juergen Schoenwaelder <[email-masked]> wrote:
> 

The first part about parser behaviour still doesn’t make much sense to me and may be misinterpreted. I think that even a parser that does understand an extension may ignore it. It all depends on what the parser output is used for.

I would just say (somewhere) that descriptions and extensions are integral and binding parts of the data model. Unless I missed something, it is not clearly stated for descriptions either.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 16:44:46 +0200

> The first part about parser behaviour still doesn’t make much sense to me and may be misinterpreted. I think that even a parser that does understand an extension may ignore it. It all depends on wh…

Oh boy - very high level of nitpicking.

> I would just say (somewhere) that descriptions and extensions are integral and binding parts of the data model. Unless I missed something, it is not clearly stated for descriptions either.

I tried to say so. Looking forward to your improved proposal.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 16:55:03 +0200

> On 29 Jul 2015, at 16:44, Juergen Schoenwaelder <[email-masked]> wrote:
> 

OK, here it is:

Sec. 7.21.3
-----------

Add this paragraph at the end:

Constraints and rules stated in the text of a “description” statement are an integral and binding part of the data model.

Sec. 6.3.1
----------

OLD

If a YANG compiler does not support a particular extension, which appears in a YANG module as an unknown-statement (see Section 13), the entire unknown-statement MAY be ignored by the compiler.

NEW

Extensions and their semantics as defined by the corresponding “extension” statement are an integral and binding part of the data model.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Jul 2015 08:00:20 -0700

Hi,

So why don't we make all the new YANG 1.1 statements like "action"
into extensions? This is just as good, right?  It seems like real keywords
are for feature you like, and extensions are for features you don't like.
IMO ephemeral state is much more fundamental than the action-stmt.
Every YANG tool MUST understand any statements added for this purpose.

Andy

On Wed, Jul 29, 2015 at 7:14 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > > On 29 Jul 2015, at 13:47, Juergen Schoenwaelder <
> > >>> On 29 Jul 2015, at 12:25, Juergen Schoenwaelder <
> should be considered broken. I am not sure it is also the case for a client
> that ignores an extension and the implied semantics, because sec. 6.3.1
> seems to allow it.
> >
> may take a module, remove extensions he doesn’t want to implement (or let a
> “compiler” remove them) and implement the modified form of the module.
> >
>
> entirely. There is again the issue of old client support that IMO doesn’t
> hold water anyway: a client that doesn’t fully understand the semantics of
> the data model cannot be guaranteed to work, and is in fact dangerous.
> > >>
> > >
> > As I said, it is unclear (to me at least) from the text in 6.3.1 whether
> extensions are an integral part of the contract expressed by the module. If
> I had a contract with an insurance company saying “Anybody who cannot read
> fine print may ignore it”, I would get quite nervous.
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 17:02:28 +0200

> > I tried to say so. Looking forward to your improved proposal.

Well, this leaves it open whether a tool can skip over extension
statements. I liked to have the difference explained...

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 29 Jul 2015 17:09:35 +0200

> are for feature you like, and extensions are for features you don't like.

I guess we leave constructive communication here.

> Every YANG tool MUST understand any statements added for this purpose.

Again, it remains unclear to me whether any changes to YANG are needed
to support ephemeral state and as long as this is case (there is no
agreement on the solution for ephemeral state) I am not that much
interested to further delay YANG 1.1. There was agreement when we
started work on YANG 1.1 that we want to finish this in a reasonable
amount of time (~ 1 year) and I hope this agreement still holds true
since this is what I am trying to achieve.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 17:11:33 +0200

> On 29 Jul 2015, at 17:02, Juergen Schoenwaelder <[email-masked]> wrote:
> 
> statements. I liked to have the difference explained…

Martin just said that tools normally skip extensions that are not directed to them. Tools can do whatever they are designed to do, and skip over ANY parts that are not important for the given task.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2015 17:19:28 +0200

> On 29 Jul 2015, at 17:09, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Actually, Andy has a point. We could have saved a massive amount of time by leaving action as an extension. Balazs could have already written a draft about it, and it could be used with YANG 1.0, too.

I really thought this wasn’t possible because of the non-binding character of extensions.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 17:50:53 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> 
> > On 29 Jul 2015, at 17:09, Juergen Schoenwaelder
> Actually, Andy has a point. We could have saved a massive amount of
> time by leaving action as an extension. Balazs could have already
> written a draft about it, and it could be used with YANG 1.0, too.

I am not sure how interesting this is, but yes, action could have
been done as an extension statement.  In fact, that's what we (and
others) have been doing since 6020 was published.

The difference here is that action has been around and used for
several years, whereas we don't even have agreement that "config
ephemeral" is the right solution to the problem.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Jul 2015 08:58:39 -0700

> ephemeral" is the right solution to the problem.
This is somewhat irrelevant since the I2RS work is still in progress,
so the fact that there are still open issues is to be expected.
There are brand new statements in YANG 1.1 (like complex if-feature logic)
that have never been implemented or proven in deployment.

The real difference is that extensions can be ignored by all
YANG tools and real statements cannot be ignored.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 19:44:31 +0200 (CEST)

> YANG tools and real statements cannot be ignored.

Are you saying that a server that advertises both ietf-system and nacm
is free to ignore the nacm statements in ietf-system, and for example
by default provide read-access to
/system/radius/server/udp/shared-secret?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Jul 2015 10:49:30 -0700

Hi,

I understand the intent is that an implementation of NACM
has to understand these NACM extensions.  I agree with Lada
that the YANG text about MAY ignore extensions casts doubt whether
this sort of NACM rule is enforceable or specified correctly.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2015 19:51:51 +0200 (CEST)

> this sort of NACM rule is enforceable or specified correctly.

So do you agree that it would be a good idea to clarify this
according to Juergen's suggestion?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Jul 2015 16:12:03 -0700

> according to Juergen's suggestion?
Not really.
Pretending the extension is just another description-stmt
does not really fix anything.

A real YANG statement like config-stmt or a new statement
called ephemeral-stmt can be modified with refine-stmt
or deviate-stmt.   This can never happen for
an external statement.

IMO ephemeral data support needs to be a real statement
that can be used with refine-stmt and  deviate-stmt.
It is a real property of a data node.

/martin
 Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 30 Jul 2015 11:30:00 +0200

> On 30 Jul 2015, at 01:12, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

In fact, generic tools like pyang ignore what’s written in descriptions.

Lada

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 30 Jul 2015 13:31:01 +0200

> In fact, generic tools like pyang ignore what’s written in descriptions.

Where does RFC6020 say that description-stmt may be used for defining 
additional semantics? The only instance where I can find "description" 
and "semantics" or "meaning" in the same sentence, is in the section 
that describes module updates. This is what a YANG description is:

    The "description" statement takes as an argument a string that
    contains a human-readable textual description of this definition.
    The text is provided in a language (or languages) chosen by the
    module developer; for the sake of interoperability, it is RECOMMENDED
    to choose a language that is widely understood among the community of
    network administrators who will use the module.

A textual description for humans. A docstring. I don't see semantics 
being mentioned anywhere, so where is all this coming from?

Jernej

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 30 Jul 2015 13:35:40 +0200

> On 30 Jul 2015, at 13:31, Jernej Tuljak <[email-masked]> wrote:
> 
> Where does RFC6020 say that description-stmt may be used for defining additional semantics? The only instance where I can find

Nowhere. That’s why I also proposed to add the following sentence to the section about “description” statement:

Lada

>  "description" and "semantics" or "meaning" in the same sentence, is in the section that describes module updates. This is what a YANG description is:
> 
> A textual description for humans. A docstring. I don't see semantics being mentioned anywhere, so where is all this coming from?
> 
> Jernej
> 
>> 
>> Lada
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 30 Jul 2015 13:38:22 +0200

> being mentioned anywhere, so where is all this coming from?

Seriously? Perhaps we have a different understanding of the word
'semantics'. Anyway, description statements (or DESCRIPTION clauses
back in SMIv2) have always been used to provide semantics that are not
expressable in machine readable form.

Example: The ietf-yang-types:counter32 definition defines a counter
type because of the text in the description statement. I would say
there is lots of semantics in the description statement. Without
the description statement, the ietf-yang-types:counter32 would not
at all be a counter.

I am absolutely surprised lately how little we seem to have a common
understanding about basic YANG concepts.

/js

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 30 Jul 2015 13:41:52 +0200

> Constraints and rules stated in the text of a “description” statement are an integral and binding part of the data model.

Wait, what? This would make a client broken if it chokes after receiving 
a message, where a container data node instance has a text value set and 
the container's description in the module claims: "This container MUST 
be treated as a leaf under X Y conditions".

It is in this WG interest for YANG modules to be interpreted without 
human presence, right? Automation? If it isn't, it would be best to drop 
YANG and write human readable specifications instead. Models that cannot 
be consumed by machines are pointless, IMHO.

Jernej

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 30 Jul 2015 13:50:35 +0200

> On 30 Jul 2015, at 13:38, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Then it has to be mentioned in 6020bis, or perhaps added as an erratum to 6020, too.

And it also goes without saying that generic tools aren’t able to figure out the semantics contained in descriptions.

> understanding about basic YANG concepts.

This is hardly surprising. Those with extensive experience in SNMP and SMI have a different background than those who came to NETCONF and YANG from the XML camp (like me).

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 30 Jul 2015 13:56:47 +0200

> On 30 Jul 2015, at 13:41, Jernej Tuljak <[email-masked]> wrote:
> 
> Wait, what? This would make a client broken if it chokes after receiving a message, where a container data node instance has a text value set and the container's description in the module claims: "This container MUST be treated as a leaf under X Y conditions”.

Or this:

description “This leaf is only valid on Friday”;

My understanding is that it is only the server that’s supposed to validate data (BTW, I don’t like this assumption), and the server software has to be coded so that constraints expressed in descriptions are checked, too.

Lada

> 
> It is in this WG interest for YANG modules to be interpreted without human presence, right? Automation? If it isn't, it would be best to drop YANG and write human readable specifications instead. Models that cannot be consumed by machines are pointless, IMHO.
> 
> Jernej
> 
>> 
>> Lada
>>

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 30 Jul 2015 16:20:56 +0200

> the description statement, the ietf-yang-types:counter32 would not

This does not change "semantics" as I understand them or as Ladislav 
implied them. I was thinking "YANG language semantics", which is what 
extensions are used for - extending YANG language with new keywords that 
hold special meaning for a YANG processor (humans included).

In a NETCONF message, ietf-yang-types:counter32 value will always be an 
integer on the wire, regardless of what the description text says, 
right? I also cannot magically turn all containers into leaves using 
text in a module description, saying "All container definitions in this 
module are actually leaf definitions", right? An extension could, 
however, be used for such a thing - a sort of YANG pre-processor 
directive (which any sensible YANG tool would ignore).

I disagree with the interpretation that an extension is just a 
formalized description statement.

I also disagree with the notion that a description text may alter, 
augment or define YANG language semantics, unless it describes an 
extension keyword. This is of key importance for generic tools, so that 
at the very least they can support standard extensions, such as 
"annotation". Having to parse human text would render such tools 
completely useless.

Jernej

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 30 Jul 2015 17:01:27 +0200

> directive (which any sensible YANG tool would ignore).

There are many ways to write nonsense definitions. 

typedef foo {
  type int;
  description "contains one of the strings 'one' or 'uno'"
}

Obvious nonsense written in plain YANG. I fail to see why extensions
are in any way different or more problematic.
 
> formalized description statement.

Well, at the end every YANG language construct is a "formalized
description statement". If I were to define a meta language to define
YANG, it would not have much else than this

  statement <id> {
    description ""
  }

All we do is agreeing on certain constructs to be generally useful and
so we can write shortcuts that have well defined meanings. This is why
we have a language and do not write everything in prose. It is more
compact and more efficit. A welcome side effect is that tools can read
_some_ of the semantics and help us getting things well defined and
implemented.

> "annotation". Having to parse human text would render such tools 

As long as you are willing to accept that description text are part
of the semantics of a definition, we are likely in agreement.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Jul 2015 08:56:12 -0700

Hi,

The YANG RFC should be very clear that description-stmt
is normative text.  Consider the <lock> operation that says
a lock is not granted if one is already held.  This is clearly normative
text that cannot be expressed in YANG constraints.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Jul 2015 19:41:30 +0200 (CEST)

> or deviate-stmt.   This can never happen for

There are two separate issues here:

1) It seems we are in agreement that if a model uses an extension
   statement, it is normative (like ietf-system's usage of nacm:*).

   Should we somehow clarify this in 6020bis?

2) Extensions cannot be refined or deviated.

   Actually, the *syntax* rules allows things like:

     deviation /x:foo {
       deviate replace {
         i2rs:ephemeral true;
       }
     }

   I agree that it not clear what this means, but we could also
   clarify this in 6020bis.

> It is a real property of a data node.

Note: it is still not clear that such a statement (base or extension)
is needed at all.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Jul 2015 10:58:35 -0700

>    Should we somehow clarify this in 6020bis?
No -- I agree that was the intent, but it is not achievable
because tools MAY ignore extensions.

> Note: it is still not clear that such a statement (base or extension)

I do not understand why you are so opposed to using real YANG statements
to support ephemeral state.  Do you have a better solution that the
proposal from draft-haas-i2rs-ephermal-state-reqs-00?  I don't see it.
I am opposed to using extensions to ephemeral state because
they are poorly defined in YANG.  This is OK since they are for
defining statements outside the YANG standard.  Normal mechanisms
like uses/refine and deviate must be supported.

I do see any advantages whatsoever for using external statements
instead of YANG statements.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 30 Jul 2015 20:11:11 +0200

> because tools MAY ignore extensions.

Why do we keep on confusing what a statement in a module means and
what a tool is required to do? If I decorate a definition with
nacm:default-deny-write, then I assume that an implementation will
implement the semantics associated with nacm:default-deny-write. If
this would not be the case, then why did we define this at all?  It
does not matter whether a tool knows how to implement
nacm:default-deny-write or whether this must be coded by a human.

Sorry for repeating myself.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Jul 2015 20:19:12 +0200 (CEST)

> because tools MAY ignore extensions.

But as have been said several times, this was clearly not the
intention.  So we should clarify this in 6020bis.

> to support ephemeral state.

If this solution was already well defined and agreed upon by everyone,
then it could be done as a YANG statement.  But I am worried that
adding this will take quite some time, and will thus delay YANG 1.1;
and since IMO extensions can be used just as well, there is no real
reason for adding this delay.

> Do you have a better solution that the
> proposal from draft-haas-i2rs-ephermal-state-reqs-00?

Use a separate ephemeral datastore.

> I don't see it.
> they are poorly defined in YANG.

Then we should fix this.

/martin

> This is OK since they are for

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Jul 2015 11:30:26 -0700

Hi,

I don't think I2RS is slowing down YANG 1.1.
The ephemeral state issue has been known to the NETMOD WG
for over a year.  I don't care if YANG 1.1 is delayed a month because
the solution details need to be worked out.

An ephemeral datastore would be useful for describing the collection of
YANG data nodes that share the ephemeral property.
A YANG statement is needed so ephemeral only data can
have its own validation rules.  I strongly object to using
extensions because they are not part of YANG.  They
cannot be used properly in refine-stmt or deviate-stmt.
They can be ignored by YANG tools.
Extensions are fine for proprietary usage, but not standards.

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Jul 2015 20:43:52 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> I strongly object to using

Putting the ephemeral question aside for a moment, and focusing on
extensions.

The original intention with the extension statement was that they
could be used for normative semantics, which is also evident from NACM
and ietf-system etc.

So if the text is unclear in this regard, we should clarify this.
Juergen already provided text.

> Extensions are fine for proprietary usage, but not standards.

6020 says:

   YANG is an extensible language, allowing extension statements to be
   defined by standards bodies, vendors, and individuals.

It is pretty clear that the intention was not that this is for vendors
only.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Jul 2015 12:05:56 -0700

>    I agree that it not clear what this means, but we could also

But this will take even longer than just defining the statements that
are needed for ephemeral state, so no point in doing that.

The text in  7.18.3.2 clearly does not support the example above at all.
Only one of the keywords listed in the table are supported.

   The substatement's keyword MUST match a corresponding
   keyword in the target node, and the argument's string MUST be equal
   to the corresponding keyword's argument string in the target node.

                       The deviates's Substatements

                 +--------------+---------+-------------+
                 | substatement | section | cardinality |
                 +--------------+---------+-------------+
                 | config       | 7.19.1  | 0..1        |
                 | default      | 7.6.4   | 0..1        |
                 | mandatory    | 7.6.5   | 0..1        |
                 | max-elements | 7.7.4   | 0..1        |
                 | min-elements | 7.7.3   | 0..1        |
                 | must         | 7.5.3   | 0..n        |
                 | type         | 7.4     | 0..1        |
                 | unique       | 7.8.3   | 0..n        |
                 | units        | 7.3.3   | 0..1        |
                 +--------------+---------+-------------+

> Note: it is still not clear that such a statement (base or extension)

Maybe not to you, but according to the I2RS ephemeral requirements,
it is needed, so unless you have a better plan, this is the default.

Explain how ephemeral state can be identified within a data node.
It must be possible to use groupings to define such data nodes.
If must be possible to refine these groupings.
It must be possible for a vendor to write deviation statements.
All YANG tools must recognize the ephemeral data property.
It should not be isolated to a particular YANG module, which would
create a dependency in all future YANG modules.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Jul 2015 21:16:29 +0200 (CEST)

> The text in  7.18.3.2 clearly does not support the example above at all.

The grammar allows extensions everywhere, so the example is (as I
wrote) syntactically valid.

> Only one of the keywords listed in the table are supported.

The text doesn't say so, and anyway my point was that - regardless of
the outcome of the ephemeral dicussion - it might be a good a idea to
clarify that extensions can be deviated as well.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 31 Jul 2015 10:06:20 +0200

>    Should we somehow clarify this in 6020bis?

Yes, and I made a proposal. The text should clarify what's important for
interoperability. Behaviour of tools, compilers and parsers is IMO
irrelevant because tools have different purposes.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 31 Jul 2015 10:12:21 +0200

> clarify that extensions can be deviated as well.

+1

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 31 Jul 2015 10:40:03 +0200

>    statement, it is normative (like ietf-system's usage of nacm:*).

What about the problem with "old clients"? Let's say a new version of a
device advertises a module that contains an extension (such as
"annotation") that changes server behaviour, datastore semantics, things
like that. So far the theory has been that a client may ignore such a
module, which however means ignoring the changes in semantics that may
be crucial for interoperability and/or security.

I think a robust and safe old client support is in fact very difficult
to achieve and may be a security hole. This is my favorite example:

augment "/if:interfaces" {
    leaf launch-missiles {
        type boolean;
        default true;
    }
}

IMO old clients may try to perform the same operations they used to do
with an old version without knowing the current datastore schema and
semantics, but no guarantees can be given - it may work or fail
miserably.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Jul 2015 05:40:05 -0700

> > clarify that extensions can be deviated as well.

So you are saying that a tool MAY ignore any extension, except
inside a deviation-stmt?  Then it becomes mandatory to support?

Or do you mean:

Yes you can put extension-stmts anywhere, but also, yes the tool
MAY ignore every single extension (everywhere, including inside a
deviation-stmt)

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 31 Jul 2015 14:45:18 +0200

> On 31 Jul 2015, at 14:40, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

I understood Martin’s proposal so that a device that doesn’t support an extension used in modules it advertises will have to put it in a “not-supported” deviation.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Jul 2015 07:54:02 -0700

On Fri, Jul 31, 2015 at 5:45 AM, Ladislav Lhotka <[email-masked]> wrote:

> > >> > > On Wed, Jul 29, 2015 at 10:51 AM, Martin Bjorklund <
> [email-masked]>
> > >> > wrote:
> > >> > >
> I understood Martin’s proposal so that a device that doesn’t support an
> extension used in modules it advertises will have to put it in a
> “not-supported” deviation.
>
>
So features are all "not-supported" by default. but extensions,
which a tool MAY ignore, are all supported by default?
A server would be required to identify and remember all
extensions used?  And what about nested extensions:

    acme:foo test1 {
       acme:foo2 test2;
    }

The server would be required to completely parse all extensions,
and not skip over them? Just to list them as deviations?

I think extensions should be like features and the server should
advertise them if they are supported.

I don't agree that "external-stmt" should be under YANG conformance
at all.  That is why MAY skip over is in the text.  A tool that
understands the extension is purely optional.

Lada
Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 31 Jul 2015 17:37:48 +0200

> On 31 Jul 2015, at 16:54, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

I proposed to remove this “MAY ignore” text.

> extensions used?  And what about nested extensions:

Yes.

> advertise them if they are supported.

My idea was that all extensions has to be supported, and those that aren’t will be declared as not-supported in deviations.

> understands the extension is purely optional.

But then we run into problems if extensions change semantics on the server side, as with nacm.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 31 Jul 2015 09:15:23 -0700

> I proposed to remove this “MAY ignore” text.

I do not support this change

> My idea was that all extensions has to be supported, and those that aren’t
> will be declared as not-supported in deviations.
>
>
I do not support this change.
Extensions are used to define external statements
which are, in fact, not part of the YANG language.
(hence the name "external")

IMO it is a mistake to try to make them part of YANG and
YANG conformance.

> But then we run into problems if extensions change semantics on the server

Some people have asked if they can just support the NACM extensions
and not the NACM module. A: no. Only the NACM module is required
to understand these extensions.  They do not affect the
syntax or semantics of the tagged objects at all.
They only affect access control for those objects.
IMO the NACM extensions should be real YANG statements,
and then the concepts of 'default-deny-write' and 'default-deny-all'
could be implemented without NACM.

Lada

Andy