Subject: [netmod] config+defaults wrt/ i-i and leafref

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 06:21:18 -0700

Hi,

The YANG definition of a database (config + defaults)
does not seem to interact well with the definition
of require-instance:

   leaf a {
      type int32;
      default 10;
   }

   leaf b {
      type leafref { path /a; }
   }

   leaf-list c {
      type instance-identifier;
   }

   container np-con {
     leaf d {
       type int32;
       default 42;
     }
   }

Here is the database:

  <config>
     // <a>10</a>
     <b>10</b>
     <c>/a</c>
     <c>/np-con</c>
     // <np-con>
     // <d>42</d>
     // </np-con>
  </config>

I am having a hard time understanding this design wrt/ require-instance:

  P1) The require-instance property for /b or /c is set
      to true, so an instance of /a MUST exist for /b or /c[1] to
      exist in a valid database.  Ditto for /np-con existing
      for /c[2] to be valid.

  P2) Leaf /a does not exist if its value is the YANG default.

  P3) NP-container /np-con does not exist if its child <d>
      contains the value '42'.

  C1) A leafref can never contain the YANG default value
      for the target leaf data-type.

  C2) An instance-identifier can never point to a leaf
      that currently contains the YANG default value.

  C3) An instance-identifier can not reliably point to
      a non-presence container if it contains only leaf(s)
      that may be set to a default value.

  * Sec. 9.9 currently does not include default leafs.
  * Sec. 9.13 currently does not include default leafs.

I do not think this database architecture is very helpful
or worth using.  The existence of a node should not depend
on its value, or the value of its descendant nodes.

IMO, the config+defaults definition of a database is
a horrible design decision, but if it is the WG consensus,
then at least clean up the definitions in 9.9 and 9.13.

How about some text in the intro explaining *why* a
leaf that happens to contain the default value exists
for every aspect of protocol operations except retrieval?
Be sure to explain how this *constant* is replaced by dynamic
meta-data, or undisclosed and unknown to the client for any of
the reasons already identified on the mailing list.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 16 Sep 2009 09:57:58 -0400

>does not seem to interact well with the definition

Can you please point me to where "config + defaults" is defined as
a database in YANG?  "config" is the database.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 08:21:15 -0700

> a database in YANG?  "config" is the database.

There is text that says a leaf that contains the
YANG default value is not part of the
database.  It has been pointed out many times.

Like this text on pg. 107:

   o  The accessible tree is made up of all nodes in the data tree, and
      all leafs with default values.

To any reader, this statement implies that leafs with default
values are not nodes in the data tree.  Yet, 9.9 and 9.13
only allow the target to be a node in the data tree.
If that is by design, it is horrible, but I think it a a bug.
I thought we were trying to debug this draft, and send
it to the IESG with as few defects as possible.

Lada, Tom, and I (I think) agree that this exclusion of
default leafs from the data tree confuses the database model.
You and Juergen seem to think it helps.  That looks like
2 for, 3 against, and 50 don't-care.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2009 18:10:20 +0200

> 2 for, 3 against, and 50 don't-care.

One way out of this would be to treat the default statement for leaf
"foo" as a declaration of equivalence of the configuration without "foo"
and with "foo" containing the default value. The advantage is that the
server then can use internally any of the three modes (report-all, trim,
explicit) and the only requirement is that both the server and client
always respect the fact that the two configurations are equivalent.

Lada
  
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 09:26:16 -0700

Ladislav Lhotka wrote:
> and with "foo" containing the default value. 

I thought that is what we were claiming the client gets
from YANG default, except for all the ways that isn't
quite true.

The current draft does not provide anything close to a constant,
so your idea would require lots of edits.  I can only conclude
that the WG consensus is that it is not important if a client
can conclusively determine the meaning of a missing leaf.
Close enough for rock and roll, or something to that effect.

The advantage is that the
> always respect the fact that the two configurations are equivalent.

Except NETCONF clients need to deal with modules that
change over time, and multiple concurrent servers that implement
different revisions during the module lifecycle.
Losing the ability to conclusively identity server data contents
across the lifecycle is actually important, even if the WG does not
think so now.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Wed, 16 Sep 2009 10:38:10 -0700

> it to the IESG with as few defects as possible.

I did not read it this way.  My understanding is that only values explicitly set by operators are part of the "config", whether or not those values are the same as the default value.  It is important to track explicitly set values because a firmware update may change a device default value...

Thanks,
Kent

--
Kent Watsen
Network Management
Juniper Networks

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 16 Sep 2009 21:13:29 +0200 (CEST)

> can conclusively determine the meaning of a missing leaf.

No, Andy, this is not the intention.  The intention is that if a
leaf has a default statement, and that leaf is missing, then a
standards compliant server "operationally uses" the default value.

A non-compliant server might use some other value, who knows.  If the
non-compliant server does that, it might have published a deviation
module, but we can't require that (since it is non-compliant).

I think you are saying that if a leaf is missing, there are some cases
when a client cannot tell which value a compliant server is using
internally.  Can you provide an example?  (I don't think you have sent
such an example, but if you have, I apologize for missing it).

If you think the spec will be better w/o the words re. how a client
should interpret missing leafs, then let's remove those words.  I
think the current text makes it easier, not harder, to understand, but
maybe that's just me.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 16 Sep 2009 22:43:52 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> or worth using.

I hate to say it, but this seems to be a garbage-in corner case.  Why
would you have a config leafreaf to a leaf w/ a default value?
Remember the original keyref did not have this corner case.

And why would you specify a config instance-identifier with
require-instance which points to a np-container?  I suspect that most
usages of instance-identifier in config will restrict the valid
instances (in description clause for now).

> for every aspect of protocol operations except retrieval?

It's not that it happens to contain the default - a leaf that is set
to its default value exists in the config.

And they are taken into account in two cases - must and when
expressions.  It would be very confusing if a must expression was true
when the default value was in effect, but false if the leaf was
explicitly set to the same value (or the other way around).

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 13:46:26 -0700

> standards compliant server "operationally uses" the default value.

Then why not write:

   The accessible tree is made up of all nodes in the data tree.

Instead of:

 The accessible tree is made up of all nodes in the data tree, and
 all leafs with default values.

> such an example, but if you have, I apologize for missing it).

I already did that, but 1 last time to practice typing:

  * A module without a revision
  * A changed module referenced by an import without a revision date
  * A leaf from a grouping changed by a refine, but the refine
    is not known because it is from a groupings-only module that
    the server shooses not to advertise
  * A default from a typedef from a module the server chooses
    not to advertise.

> think the current text makes it easier, not harder, to understand, but

I'm done with this spec for awhile.
I'll check the IESG version.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 13:48:55 -0700

> I did not read it this way.  My understanding is that only values explicitly set by operators are part of the "config", whether or not those values are the same as the default value.  It is importa…

Where does it say that?
I thought we agreed that server-created leafs exist, if
they do not contain the YANG default value.

So what is a data node that is created by the server?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 16 Sep 2009 23:10:35 +0200 (CEST)

>  all leafs with default values.

But this text is for how the must and when xpath expressions are
evaluated.

>   * A module without a revision

Ok, the problem here is if v1 does not have a default, but v2 adds a
default stmt, and the client uses v2 and the server v1.  The client
now thinks the server is using the default value, but it is not.

This can be solved in two ways:

  - do not allow default to be added in new revisions
OR
  - require a revision statement when a module has been updated.

>   * A changed module referenced by an import without a revision date

This is e.g. if a grouping in v1 does not have a default, but v2 adds
a default, and the client uses v2 and the server v1.  Same situation
as above.

This can be solved with:

  - do not allow default to be added in new revisions

>     the server shooses not to advertise

I don't get this.  If a module A imports B, and the client understands
A it must also understand B.  And also, the refinement is always in
the module that uses the grouping, so changing a default cannot break
anything in other modules.

>   * A default from a typedef from a module the server chooses

Same as above.  If the client understands module A with the leaf, it
also understands B with the typedef.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 18 Sep 2009 11:43:24 -0700

Martin Bjorklund wrote:
 ...
> Remember the original keyref did not have this corner case.

I disagree.

I think it is bad if leafs and subtrees disappear from
the data node tree for retrieval and leafref/i-i
validation, but NOT for XPath validation (must/when),
and NOT for min/max/unique validation.

Consider an i-i that is pointing at a leaf,
or a leafref containing the value of that leaf,
for years in a config file -- then a YANG default
is added, and suddenly, in the new version,
the running config is invalid and the server shuts down,
just because the leafref happens to contain a value
that is now the default, and the i-i is pointing at 'nothing'.
I do not understand why this is a feature.

Since a server can just patch in any default they want
with a deviation, a client can *never* safely set a
leafref or i-i to any value, based on the
defaults specified in the YANG data model(s) .

> explicitly set to the same value (or the other way around).

IMO, what the draft has now wrt/ defaults and deviations is not usable.
It is a 1-legged stool.

As a data modeler, I ask myself:
"Can I write a YANG data model using any valid
YANG expression, and it will behave the same way
on every compliant NETCONF server implementation?"  No.
For some valid YANG, it is OK for a server
not to accept it or do different things (e.g., mandatory,
default, module revisions, import-by-revision).

As an application developer I ask myself:
"Can I code to the YANG data model instead of
a data silo of platform-specific data?"  No.
Deviations and possibly incomplete meta-data prevent that.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 21 Sep 2009 10:24:28 +0200 (CEST)

> that is now the default, and the i-i is pointing at 'nothing'.

This is not what will happen.  Let's examplify:

   leaf a {
      type int32;
   }
   leaf b {
      type leafref {
        path "/a";  // warning: stupid model for illustration purpose only
      }
   }

And config file:

  <a>42</a>
  <b>42</b>

Now in the new version, leaf a gets a default statement with value
'42'.  The config file / datastore does not change; it will contain
both a and b.  So the leaf that b points to still exists.  This is
because a is "explicitly set".  

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 21 Sep 2009 03:14:28 -0700

> because a is "explicitly set".  

But none of the text on validation (maybe 30 places) says
anything about how the node was created.  They all refer
to 'the data node', or something to that effect.  There is
no text that says a unique-stmt applies to explicitly set
leafs but not server created leafs.

Whether or not the server actually contains a data structure
for a default value which is somehow different to the data
structure for a non-default leaf, is totally irrelevant
to the XML representation of that database in NETCONF PDUs.
Too bad you don't understand that.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 21 Sep 2009 12:49:51 +0200 (CEST)

> leafs but not server created leafs.

They apply to explicitly set leafs and server created leafs.  But a
default value is used by the server when there is no node in the data
store.  The new proposed not-yet-published section "The leaf's default
value" (see
e.g. http://www.ietf.org/mail-archive/web/netmod/current/msg03754.html)
starts with:

  The default value of a leaf is the value that the server uses if no
  value has been explicitly set. 

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 21 Sep 2009 04:08:41 -0700

>   value has been explicitly set. 

A few people (including me) do not
accept the premise that a node only exists
if the client creates it. Instead, a node exists
if its behavior is affecting system operation.
Your definition of default is focused on syntax
instead of semantics, and therefore is an inferior
modeling mechanism.

Besides, what you are saying totally contradicts
your previous comments that default leafs exist
for validation purposes.  If the 'value' exists
when it is time to generate a 'must-violation'
(for example), then there must be a node there.

The fact that the validation rules apply to the default leafs
is proof that they exist in the conceptual database model.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2009 14:23:13 +0200

> instead of semantics, and therefore is an inferior

As you know, I am proponent of separating operational state from
config state. So we do disagree here.
 
> (for example), then there must be a node there.

I think you have a point here and I think must statement expressions
on config data nodes must be validated on config data nodes and not
implicitely generated nodes derived solely for validation purposes.
So I am actually with you that the current text is somewhat odd.

> is proof that they exist in the conceptual database model.

I believe this is a sign of an inconsistency that needs fixing. How we
fix it needs to be worked out.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 21 Sep 2009 05:51:20 -0700

> config state. So we do disagree here.

I do not object to additional data retrieval filtering
mechanisms over time, but the database as conceptual
model has all the instructions in it, not just client instructions.

Whether the server decides the interface MTU is 1500
or the client decides it is 1500 does not
change the fact that there is a configuration instruction for
the system to limit packet size on that interface
to 1500 bytes.

>> (for example), then there must be a node there.

  list fubar {
     key x;
     must "count(y) > 0";
     must "count(y) <= 10";
     min-elements 1;
     max-elements 10;
     leaf x { type string; }
     leaf y {
        type string;
        default 'foo';
     }
  }

IMO, if the min-elements/max-elements do not produce the
same runtime evaluation results as the must-stmts, then
the validation model is confusing and broken.

(Note that min-elements and max-elements are redundant
because of the count() function in XPath -- e.g. count(keyleaf)
better equal number of elements in min/max test.)

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2009 15:59:30 +0200

> the system to limit packet size on that interface

There is no configuration, there is operational state. As I wrote,
we simply disagree on this. 
 
> the validation model is confusing and broken.

I did not suggest that.

A crucial point here is that constraints on config data elements are
not the same as contraints on operational data. Our confusion here (if
there is any) results from our previous disagreement.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 21 Sep 2009 07:43:06 -0700

> we simply disagree on this. 

We do not even agree on the basic components of
the managed system.  IMO, the NETCONF server is the
configuration broker between the outside world
and the managed device.  The mtu=1500 configuration
instruction is given from the NETCONF server to the
managed system.  The NETCONF server itself has no
interfaces to limit packet size on -- only the managed system
has interfaces to operate.

But what matters is that the NETCONF standard does not
make any distinction between instructions given by the
operator and by the server, so your proposal will take
several years before it will be available in real devices,
since it will require new protocol operations.

Also, the NETCONF WG agreed that the 3 forms of defaults
were all valid within a NETCONF server.  This means that
report-all is a valid defaults model and must be supported.

>> the validation model is confusing and broken.

It is up to each individual server to decide if it
wants a leaf with a default to exist or not.
This means that database validation will work differently
on every server, but it should be consistent
within a single server. (IMO, a totally worthless standard,
but pretty much what developers get now with proprietary CLI).

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 22 Sep 2009 10:17:19 +0200

> (for example), then there must be a node there.

Yes, XPath relies on the notion of "XML document" and for our purposes
this document must contain the leafs with default values, otherwise the
XPath expressions in 'must' and elsewhere won't work properly. Do we
agree on this?

If so, I think the cleanest and least confusing approach is to have the
"conceptual data tree" in YANG the same as this "XML document" for
XPath. This of course doesn't prevent implementors from using the
"explicit" or "trim" approaches in the database and PDUs, they only have
to keep in mind that all statements in the YANG language standard refer
to the "report-all" data tree. The current situation where "conceptual
data tree" may mean different things to different people is IMO very
problematic.

Lada

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Sep 2009 06:30:22 -0700

> data tree" may mean different things to different people is IMO very

Yes, and it is not just XPath constraints that apply here.
Unique-stmt and other 'plain' YANG constructs are impacted.
It has been stated that the 'full view' of the data tree
is only needed for XPath, but this is false.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Sep 2009 10:17:54 -0700

Hi -

> From: "Ladislav Lhotka" <[email-masked]>
> To: "Andy Bierman" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, September 22, 2009 1:17 AM
> Subject: Re: [netmod] config+defaults wrt/ i-i and leafref
...
...

I totally agree.

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 22 Sep 2009 20:46:14 +0200

> > data tree" may mean different things to different people is IMO very

And I disagree. I am fine with evaluating constraints on the
conceptual data tree and have the isomorph to the XML document but I
have a problem to agree that this implies "report-all" for the data
tree.

/js

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Sep 2009 12:04:28 -0700

Hi -

> From: "Juergen Schoenwaelder" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, September 22, 2009 11:46 AM

I think we're in violent agreement.  Lada wrote "refer to the 'report-all' data
tree'.  I read that as meaning what you described as "isomorph to the
XML document."  That doesn't mean that responses to requests are
necessarily "report-all" - that's a problem for the protocol to nail down.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Sep 2009 12:21:50 -0700

> necessarily "report-all" - that's a problem for the protocol to nail down.

Exactly -- that is what I was trying to say 2 or 3 days ago.
Just because the conceptual model for database validation
is consistent and predictable, does not mean the <copy-config>,
<get> or <get-config> operations always return the defaults.

Although, if I have to know the exact values in use in
order to be able to edit the database (and I do), then retrieval
is critical as well -- in those cases where a hidden default is
causing the database to be invalid somehow.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 22 Sep 2009 21:25:36 +0200

> necessarily "report-all" - that's a problem for the protocol to nail down.

That's right. For example, the intuitive notion of "mandatory" has a
quite different meaning when applied to different but isomorphic - I'd
say  equivalent - views of the configuration database (hence the lengthy
discussions in this ML). IMO it would be considerably easier to choose
one of the views for specifying the standard. It is not that important
which of the views is chosen but then why not choose the view that's
natural for XPath expressions?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Sep 2009 12:49:07 -0700

> natural for XPath expressions?

I disagree that it is not important which model is chosen.
I have shown examples where client applications which work
fine, will suddenly break because the server decided it was time
to call that value a default (either new default or deviation).

Also, there are examples where the XPath and non-XPath
constraints need to be in synch

  unique x;
  // XPath omitted, but the default needs to cause an error
  // if a client-provided value is not used for the 2nd to Nth
  // entries, or a client value is the same as the server-selected default

OR

  max-elements 4;

  must "count(.) <= 4";

OR

  must "foo/bar";

  leaf X {
     mandatory true;
     type leafref {
        path ../foo/bar";
     }
  }

IMO, it is critical that the data modeler and application
developer get the same validation behavior on every compliant server
for every YANG database property, not just most of them.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Sep 2009 13:06:35 -0700

...

Also, default does not apply to leaf-list at all.
It is important that validation rules for leaf and leaf-list
work the same, especially if max-elements=1 (this should
be conceptually equivalent to a leaf for validation purposes).

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 23 Sep 2009 09:05:31 +0200

> to call that value a default (either new default or deviation).

Well, this depends on the definition of 'default' and 'mandatory' for a
given model - it should be different for each. But it's true that with
the "report-all" model the server has less degrees of freedom.

> constraints need to be in synch

Yes, you are right with these non-XPath constraints.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 05:48:25 -0700

> the "report-all" model the server has less degrees of freedom.

The 'standard' is mostly useless because there is
no standard definition of mandatory or default.

> Yes, you are right with these non-XPath constraints.

Notice that no XPath or 'unique' constraint example we have
ever written on this mailing list has even once made
a distinction between a server-assigned and a client-assigned
leaf value.

To me, this is rather important.
If there is a must-stmt:

   must "foo < (bar+baz+12)";

1) I want it to work the same on every server
2) I want it to work the same whether the client or the server
   created the leafs that are referenced in the expression
3) I want to be able to retrieve the leafs in the expression
   so I know when it is OK to attempt a commit

So far, YANG is what we baseball fans call an 'oh-fer'
on all these issues.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 23 Sep 2009 09:13:42 -0400

If the terms mandatory and default mean different things for different 
models then they effectively mean nothing.

If that is true, then they should both be removed from the language.

If we purely want syntax, with no semantics, then that is fine.
If we are trying to add some semantic content, then mandatory and 
default seem the obvious starting point.
And if so those terms MUST mean the same thing for all YANG models.

Yours,
Joel

Ladislav Lhotka wrote:
...
> given model - it should be different for each. 
....

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 23 Sep 2009 15:17:57 +0200 (CEST)

> 1) I want it to work the same on every server

On every compliant server, yes.

>    created the leafs that are referenced in the expression

... or if one leaf happens to have its default value.

>    so I know when it is OK to attempt a commit

Provided that you have the data models, or the server supports
with-defaults, you can do this.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 06:24:05 -0700

> And if so those terms MUST mean the same thing for all YANG models.

Well, at the core, we do not agree on what it means
for a data node to exist or not.  Pretty basic stuff.

We do not agree whether the NETCONF server is
the entire system or part of a larger system architecture.

We do not even agree that NETCONF has the right
protocol operations to access these nodes that we can't agree on.

We are better off with no standard content layer in NETCONF,
than a standard that can be implemented so many different ways
that it is useless to the modeler and the application developer.

My concern before YANG started was that it is not the end goal.
The end goal is standard NETCONF content.  It seems like we are still
5 years away from having our first standard configuration object,
if the protocol has to be redesigned to work with YANG.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 06:41:30 -0700

> On every compliant server, yes.

But every client is forced to use the deviations
from the server, so the constants specified in the
YANG data models are useless for 'coding to the standard'.

> ... or if one leaf happens to have its default value.

But, constraints such as must,
i-i, and leafref do not work the same at all on every server.
The server can have a different 'defaults scheme' for
every leaf, and every session if it wants.

> with-defaults, you can do this.

With-defaults is optional-to-implement and the meta-data
from the server is likely to be incomplete, so 'old server'
and 'new server' distinction algorithms will be impossible to execute.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 23 Sep 2009 16:04:00 +0200 (CEST)

> YANG data models are useless for 'coding to the standard'.

Deviations tells the client how a server is non-compliant.

Where does it say that the client is forced to use them?

We have had this conversation before.  Let's try to identify problems
in the text and fix them.

> With-defaults is optional-to-implement

Should we say that it is mandatory for servers which implements YANG
models?

> and the meta-data

What exactly is the problem?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 23 Sep 2009 16:18:32 +0200

> models then they effectively mean nothing.

This is a misunderstanding, by models we meant "database models", i.e.
equivalent views of the configuration datastore using the "report-all",
"trim" and "explicit" methods. I am sorry for the confusion.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 07:28:17 -0700

> Where does it say that the client is forced to use them?

The text in 4 or 5 places that says the client MUST
use the default value.  This text does not say
anything about deviations.

I want it to be clear where to point the blame when
a client and server do not interoperate.  This is
why specs matter in the IETF.  Vendor A won't
budge and vendor B claims vendor A is wrong
and won't change any code either.  If the standard
says they are both right, then the standard is
not that useful.

> Should we say that it is mandatory for servers which implements YANG

yes.
IMO, this is the most operationally friendly decision,
because forcing data modelers and servers to keep all
the meta-data 100% complete is not realistic.
It also allows the server to leave out defaults
except when explicitly requested by the client.
This is exactly the operational mode everybody
wants anyway.

> What exactly is the problem?

I'm not going to re-list all the
ways for the client to guess wrong about
the meaning of a missing leaf.  It's in the
ML archive if you care to read it.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 09:33:40 -0700

> have a problem to agree that this implies "report-all" for the data

Why?

When a data modeler writes a YANG module, they are trying
to specify the desired end state of the running system.
They are *not* focusing on the YANG syntax, or the external
representation of a YANG configuration database.
If they write some constraints (e.g. mtu value on an
interface), the constraint refers to the behavior
of the running system, not the syntax in an external
config file.

So we are a million miles apart on the reason for even
having a data modeling language, let alone the details.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 23 Sep 2009 18:52:13 -0400

At least as you have phrased this, I disagree.
The difference between report-all, trim, or explicit, should not (MUST 
NOT) actually drive any difference in what the "mandatory" or "default" 
statements mean or do.
Those reporting flags simply affect what portion of the contents of the 
theoretical data model are reported to the requestor.  If those are 
treated as corresponding to three actually distinct data models, we have 
a nightmare that noone will be able to understand, and I would be 
willing to bet on sufficiently different implementations that 
interoperability will be near-meaningless.

Yours,
Joel

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 24 Sep 2009 01:10:30 +0200

> of the running system, not the syntax in an external

So far, I thought we agree that constraints are validated at
config commit time and this meant to me that constraints apply to
a system's config and not to "the desired end state".
 
> having a data modeling language, let alone the details.

Yes, every email seems to increase our distance.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 16:32:06 -0700

> interoperability will be near-meaningless.

I agree 100%
I do not care if 'mtu' is called config, operational
state, or something else.  If 'mtu' is part of the
validation criteria for a managed system, then it applies
to the value of 'mtu', not whether the client set it
or the server did.  Having some (but not all) the constraints in
a YANG data model fail unless the client set the value
is worthless at every level, not just interoperability.

Andy

>>> If the terms mandatory and default mean different things for
>>> different models then they effectively mean nothing.
>>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Sep 2009 17:35:48 -0700

> a system's config and not to "the desired end state".

So you think that constraints should only work
if the client set the value in use, but if the
server set the value, then the constraint should
fail and the database is therefore invalid?  This is how
Martin claims leafref and i-i should work.
This applies to any server-set value, not just a YANG default.

Do you think all validation rules should fail if the
server sets a value instead of the client, or just some?
Should the the validation rule be ignored unless the
client set the value?

Since default leafs impact if NP containers are present
and which case is selected from a choice, are you
suggesting that if the server sets the value, the NP container
is not really there, or the choice is not really set.
This affects how mandatory, must, when constraints work.

   container fubar {
      leaf x { type string; }
      leaf y { type int8; default 1; }
   }

If the client creates /fubar/x, the server will create /fubar/y = 1.
Then the client deletes /fubar/x.  Does /fubar/y remain
or not?  I do not agree that all servers must
use the 'explicit' mode, so IMO it does not matter
how /fubar/y got created originally.
Once it does get created, it stays instantiated instead
of magically disappearing.  The client wanted it to
exist, or the delete would have been for /fubar, not /fubar/x.

I agree with Joel that if there is no
WG consensus on how a database property works on
every server, then it needs to be removed from YANG.
Only the constructs that have WG consensus should remain
in the standard.

> Yes, every email seems to increase our distance.

Imagine how much fun the IESG will have reviewing this draft.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 24 Sep 2009 07:06:31 +0200

On Thu, Sep 24, 2009 at 02:35:48AM +0200, Andy Bierman wrote:

> fail and the database is therefore invalid?

I said config is validated at commit time. There will be "assigned-by
server" config nodes (the uid example) and the question how a config
node came into existance does not matter for validation. This,
however, does not mean all operational state of the system becomes
config, that is I do not want to have the server populate hundreds of
config nodes just because there is operational state. This is where we
differ.

> Should the the validation rule be ignored unless the

see above
 
> This affects how mandatory, must, when constraints work.

see above
 
> exist, or the delete would have been for /fubar, not /fubar/x.

My preferred interpretation of "default 1;" is that this statement
creates a leaf "assigned-by server;" with a constant initial value -
so the config leaf would stay instantiated. Note that this does not
mean that all operational state a system has becomes config.

/js

---

From: WashamFan <[email-masked]>
Date: Thu, 24 Sep 2009 14:04:49 +0800

>  mean that all operational state a system has becomes config.

What I wanna say is, from the client's perspective, it doesn't matter
whether default leafs in internal config database or not, it doesn't
matter whether default leafs are meta-data or not. it doesn't matter
whether the default leafs are created by client or not. What it matters
is whether default leafs exist in "conceptual XML document" which can
be retrieved in 3 ways (report-all, trim, explicit). And all constraints are
applied to this co-called "conceptual XML documents".

washam

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 24 Sep 2009 08:08:37 +0200 (CEST)

> This applies to any server-set value, not just a YANG default.

No, I never meant that.  For the purpose of constraints, it doesn't
matter if the server or client created the value.  Once it is in the
db, it counts.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Sep 2009 08:53:36 +0200

> interoperability will be near-meaningless.

No, the three different views "report-all", "trim" and "explicit" are of
course equivalent, but nevertheless it is IMO important to fix the view
against which the YANG spec is written because it does make a difference
in quite a few places - XPath expressions, definitions of 'unique',
'mandatory' etc.

For example, in the "report-all" view the leafs with specified defaults
become in fact mandatory in that they are always present in a valid
configuration - either they are provided by the client or supplied from
the default. I think the current definition of 'mandatory' actually
supports the "trim" view - at least this is how it gets translated to
RELAX NG <optional> patterns.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 24 Sep 2009 03:58:18 -0700

> applied to this co-called "conceptual XML documents".

well said.

+1

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Sep 2009 14:38:08 +0200

> > applied to this co-called "conceptual XML documents".

Yes, this is exactly my position, too. The conceptual data tree also
needn't be the same as the contents of the configuration database in any
particular box, it is just a reference XML infoset on which YANG
semantics can be explained in the easiest way.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 24 Sep 2009 14:49:10 +0200 (CEST)

> semantics can be explained in the easiest way.

So we can keep the text that says:

  A NETCONF server that replies to a <get> or <get-config> request MAY
  choose not to send the leaf element if its value is the default
  value. 

?

I think that this could work as well.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Sep 2009 15:05:10 +0200

>   choose not to send the leaf element if its value is the default

Yes, or alternatively the text could refer to the with-defaults draft
and only declare that the default values specified by the 'default'
statement are exactly those that with-defaults procedures apply to.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 24 Sep 2009 07:00:05 -0700

> I think that this could work as well.

agreed.

A clarification: I do not support config=false nodes
in validation constraints against candidate.  There just isn't
any state data to access in candidate.

In other words, I run the commit-checks against the configuration
database, and config=false is never in the candidate database.
Maybe this is a 4741bis issue, but if I had to run the commit-tests
against running+state, instead of against candidate, then
different XPath results might occur.

So Juergen's original concern remains -- is there is a new
classification that is neither config or state?
What is being validated?  Just the client provided
instructions, or the combination of all server and client
instructions?  Is candidate validated on its own, or
is the state data (available with <get> only) part of
the validation criteria?

Andy

---

From: WashamFan <[email-masked]>
Date: Thu, 24 Sep 2009 22:38:03 +0800

> different XPath results might occur.

If YANG allows XPath to apply to all nodes which could be
specified by YANG. XPath should apply to both config=true and
config=false nodes. So I am afraid "conceptual XML document"
should contain both kind of nodes.

> is the state data (available with <get> only) part of

If validation applies to whatever the "third kind" nodes.
"conceptual XML document" should contain them.

washam

---

From: "tom.petch" <[email-masked]>
Date: Fri, 25 Sep 2009 16:17:56 +0200

---- Original Message ----- 
From: "Juergen Schoenwaelder" <[email-masked]>
Sent: Thursday, September 24, 2009 7:06 AM

> config nodes just because there is operational state. This is where we

Juergen

I think that validation depends on the datastore, as in 8.3.3, so an 
<edit-config> of <running/> may access operational state with a 
must statement, whereas a <commit> of <candidate/> may not.

We could end up with two different must statements for the two
different conditions.

Tom Petch

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Sep 2009 14:04:40 -0400

>must statement, whereas a <commit> of <candidate/> may not.

This is not true, since 7.19.5 details the nodes which are accessible
based on the the context node:

   o  If the context node represents configuration, the tree is the data
      in one of the NETCONF datastores.  The XPath root node has all
      top-level configuration data nodes in all modules as children.

   o  If the context node represents state data, the tree is all state
      data on the device, and the <running/> datastore.  The XPath root
      node has all top-level data nodes in all modules as children.

So even for <running>, only config=true nodes are available.

You could have two "twin" leafs, one of which represents config
data and the other operational data, each of which have related
constraints that are evaluated with different sets of available
nodes, but the same node's must constraints are always evaluated
in the same style.

Thanks,
 Phil

---

From: "tom.petch" <[email-masked]>
Date: Mon, 28 Sep 2009 16:30:29 +0200

----- Original Message ----- 
From: "Phil Shafer" <[email-masked]>
Sent: Friday, September 25, 2009 8:04 PM

Phil

Thanks for the correction; I overstated my case.

I think it still true, though, as I first said, that it is not just at <commit>
when validation occurs.

Tom  Petch

---

From: Phil Shafer <[email-masked]>
Date: Mon, 28 Sep 2009 11:33:27 -0400

>I think it still true, though, as I first said, that it is not just at <commit>

Okay, so now I'm confused.  What statement are you refering to as
"still true"?  That edit-config may access operational state with
a must statement?  If the must statement is on a "config=true"
node, the validation will not see the operational nodes.  If
the must statement is on a "config=false" node, then it cannot
be affected by <edit-config>.

Thanks,
 Phil

---

From: "tom.petch" <[email-masked]>
Date: Tue, 29 Sep 2009 10:06:01 +0200

----- Original Message -----
From: "Phil Shafer" <[email-masked]>
Sent: Monday, September 28, 2009 5:33 PM

Phil

The original statement, to which I was responding, said
'config is validated at commit time'
so my initial comment was that it was not only at <commit> time
that validation would occur but also at <edit-config> time
and it would then be <running/>  that was being validated and
not <candidate/>.

I was then thinking what else might be in <running/> and
not in <candidate/> and the answer is quite a lot.  I was wrong
to consider state, as you have pointed out, but anything that
has been previously edited in <running/> could be different.

Will there be cases where such editing creates validation that
works for <candidate/> and fails for <running/> or vice versa?
I cannot think of a realistic example (yet:-) but am still mulling
that one over.

Tom Petch

---

From: Phil Shafer <[email-masked]>
Date: Tue, 29 Sep 2009 08:01:48 -0400

>and it would then be <running/>  that was being validated and

Ahh, so you're talking about the rules in section 8.3
(esp 8.3.3) about when differing constraints are enforced.

Thanks,
 Phil