Subject: [netmod] agent-default -- new 3rd state

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Aug 2009 07:37:05 -0700

Hi,

We know that we are trying to represent 3 states
with a boolean -- this has been brought in several
times in the past.

I do not want to replace 'mandatory' with 'agent-provided'.
Why not solve the problem instead of pretending it is
an agent implementation choice?  Because it isn't.
It is a data model designer's choice.

1) mandatory true means the manager must provide a value

2) 2 forms of default are needed:

OLD:

     default-stmt        = default-keyword sep string stmtend

NEW:

     default-stmt        = (agent-provided-keyword /
                            default-keyword sep string) stmtend

     agent-provided-keyword = 'agent-provided'

EXAMPLE:

  leaf A {
     description
       "manager must provide a value";
     type string;
     mandatory true;
  }

  leaf B {
     description
       "agent must provide a value if the manager does
        not provide one";
     type string;
     agent-default;
  }

  leaf C {
     description
         "agent must provide the specified default if the
          manager does not provide a value.";
     type string;
     default 'fred';
  }

IMO, this would solve the problem better than combining A and B.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 17 Aug 2009 17:09:51 +0200

> It is a data model designer's choice.

Another and IMO simpler option is to decouple mandatory/optional and
default:

1. A leaf is mandatory if it must be present in a valid configuration,
unless it is contained in a container with presence or a choice's case
(these two cases require special handling).

2. Default values defined in the data model is an indication for the
server about which values are to be suppressed when returning the
configuration in the mode with-defaults=false.

So if the factory setup of a device assumes valid configuration, all
mandatory leafs must be provided, either to a default specified in a
data model or to something else. Therefore, 'mandatory' and 'default'
wouldn't be mutually exclusive.

Lada
 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 17 Aug 2009 10:55:25 -0400

>I do not want to replace 'mandatory' with 'agent-provided'.

s/agent/server/ or /system/

Using the term "agent" is SNMP-specific, and does not work with
folks who are not aware of the odd way this protocol used this term.
It is needlessly confusing and we are trying to move to more modern
terms.  In normal CS terminology, an agent is a semi-independent
piece of software that acts on behalf of a user, which is very
different from the "server" role that the netconf server performs.

The existing proposal for "assigned-by system" solves the problem
you are refering to.

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Aug 2009 17:42:26 +0200

> configuration in the mode with-defaults=false.

Sounds like we are designing something completely new now...

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 17 Aug 2009 20:06:56 +0200

> Sounds like we are designing something completely new now...

I don't think so, I am just trying to give a precise meaning to the
'default' statement because YANG spec does it in a hand-waving way:

The "default" statement, which is optional, takes as an argument a
string which contains a default value for the leaf.

But apparently there are at least two types of default values and one of
them has nothing to do with the 'default' statement, as you said in
another mail.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Aug 2009 20:30:13 +0200

> string which contains a default value for the leaf.

It has been clear to us I thought up until now that the YANG default
statement is like the SMIv2 DEFAULT clause except that the value is
binding. And here is another quote from YANG:

   If a leaf has a "default" statement, the leaf's default value is set
   to the value of the "default" statement.  Otherwise, if the leaf's
   type has a default value, and the leaf is not mandatory, then the
   leaf's default value is set to the type's default value.  In all
   other cases, the leaf does not have a default value.

   If the leaf has a default value, the server MUST use this value
   internally if no value is provided by the NETCONF client when the
   instance is created.

I think this is a much better quote than what you did select.

> them has nothing to do with the 'default' statement, as you said in

There are system assigned values and yes a system assigned value is
not the same as a default. The MTU of an interface is system assigned
based on the system's knowledge of the hardware.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 17 Aug 2009 21:03:53 +0200

> I think this is a much better quote than what you did select.

The problem is with "the NETCONF client". If I wanted to work around any
defaults specified in the data model, I would equip my device with a
script declared as "built-in NETCONF client" that automatically creates
the instances a sets their value to anything within the limits of the
datatypes. This satisfies the above requirement but from the human
user's point of view the defaults specified in the data model mean
nothing.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Aug 2009 21:14:42 +0200

On Mon, Aug 17, 2009 at 09:03:53PM +0200, Ladislav Lhotka wrote:
 
> The problem is with "the NETCONF client". If I wanted to work around
> any defaults specified in the data model, I would equip my device
> with a script declared as "built-in NETCONF client" that
> automatically creates the instances a sets their value to anything
> within the limits of the datatypes. This satisfies the above
> requirement but from the human user's point of view the defaults
> specified in the data model mean nothing.

You are having a problem that can't be fixed with a specification.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 21:26:15 +0200 (CEST)

> It is a data model designer's choice.

While I agree with all this, I think the consensus was to not include
a specific statement for this in YANG 1.0.  See the ML thread starting
at http://www.ietf.org/mail-archive/web/netmod/current/msg01622.html.

I think the approach taken by Gerhard in the ipfix module is what can
be done in YANG 1.0 - i.e. make the leaf optional and describe the
behavior in the description clause.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 17 Aug 2009 21:36:04 +0200

> You are having a problem that can't be fixed with a specification.

I don't have any problem. The question is:

1. Do we want to enforce the defaults in the data model on vendors? In
my view, this is hopeless, see above.

2. Do we want to use defaults in the data model as a mean for
simplifying configurations presented to clients? This will work fine: if
the value is missing in a get-config reply, ANY client will know that
the default value is implied.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Aug 2009 22:15:35 +0200

On Mon, Aug 17, 2009 at 09:36:04PM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v Po 17. 08. 2009 v 21:14 +0200:
> I don't have any problem.

Good to hear.

> The question is:

As hopeless as a SHOULD or MUST - everything can be ignored. The power
of specifications has clear limits and if an implementation argues to
have an embedded manager that does surprising things to defaults, well
let the market decide.

Yes, the default statement is a very sharp tool as are must and when
statements. I am sure we will have to pay some price while learning
how to use these tools and perhaps there will be a BCP after a few
years explaining when it is safe to use these tools and when not.

> the default value is implied.

This is for me a completely new definition of the default statement
and it only makes sense to some interpretations what configuration
data really is. So I am not in favour - I would rather drop the
default statement altogether instead of using it as a filter.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Aug 2009 10:56:59 +0200

> default statement altogether instead of using it as a filter.

Hmm, for me the main purpose of the 'default' statement has been so far
that it makes certain parameters available to the with-defaults
capability. If it is not so, I am seriously confused.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Aug 2009 11:41:42 +0200

On Tue, Aug 18, 2009 at 10:56:59AM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v Po 17. 08. 2009 v 22:15 +0200:
> > 

We need to discuss what is in the document and not interpretations of
it.  Which text in draft-ietf-netmod-yang-07 makes you believe the
default statement acts as a magic filter for :with-defaults? Section
7.6. I think spells out what a default does to a leaf:

It does not talk about filtering. And section 1.2 of
draft-ietf-netconf-with-defaults-03 explains the different ways server
deal with default config data - trim is only one out of three options.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Aug 2009 13:16:03 +0200

> default statement acts as a magic filter for :with-defaults? Section

draft-ietf-netconf-with-defaults-03: "Default values SHOULD be specified
in documents describing the data models supported by the NETCONF
server."

It is IMO a mistake that YANG draft completely ignores this function.
Without it, the 'default' statement is pretty much useless and serves
only as a source for confusion.

>    internally if no value is provided by the NETCONF client when the

Again, this sentence is carefully formulated to avoid any reference to a
particular information model and is therefore ambiguous. What "use
internally" means in terms of configuration datastores? Is it OK if a
device in the factory default state has the DM-default value in the
startup datastore and another value in running? Or the other way around?
In both cases the default value is "used internally". What about the
"built-in NETCONF client" trick, or another form of
NETCONF-client-in-the-middle?
 
As a result, the sentence provides actually no guarantees that the
default value will really appear in any particular datastore when the
device completes the boot procedure.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Aug 2009 13:51:55 +0200

> only as a source for confusion.

Well, we do not have to agree.
 
> device completes the boot procedure.

The server MUST use this value internally if no value is provided by
the NETCONF client when the instance is created. You can always use
the "built-in NETCONF client" trick to arbitrary things and I do not
think a specification can prevent this. In Phils interpretation,
however, such a "built-in NETCONF client" would leave explicit config
around - so things become visible.

I do not think we make progress so I stop here.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 05:33:26 -0700

> device completes the boot procedure.

I share your concern.

The standard gives the impression that there is going to be
some sort of consistent behavior across implementations,
yet nothing could be further from the truth.

I can't believe with all the little esoteric machine-clauses
in YANG, that the best this WG can do is leave this issue
up to the description clauses.  There are 3 agreed upon
states, but only 2 of them need machine-readable clauses?
Which means that 2 of the 3 modes are ambiguous and worthless
to managers.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 05:47:08 -0700

> different from the "server" role that the netconf server performs.

I do not agree with this terminology rule.

Manager and agent are network management specific, not SNMP specific.

I do not see any value forcing the use of the generic 'client/server' terms.

The NETCONF RFC says manager <--> client and agent <--> server,
so there is no need to change any text.  We can use either terminology.
Both are acceptable.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 06:24:53 -0700

> I do not think we make progress so I stop here.

We are confusing implementations of data models
and specification of data models.

The data model designer needs a way to indicate
that the model does not allow for any implementation
of the model to use an unspecified value.

  create-interface(name)
  ping(host-addr)

I don't care what the statement is called.
It is useful for automation tools to know
that the agent MUST NOT provide a value for a mandatory leaf.

The ifMtu object would have a default-stmt if there
was a static default.  Calling this object mandatory
is wrong.  Calling it optional is wrong.
What about calling it 'dynamic-default'?
(Then mandatory means manager-must-set.)
This is an important use-case that is going to
show up  a lot.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 18 Aug 2009 15:41:25 +0200 (CEST)

> of the model to use an unspecified value.

Absolutely.  It is called 'mandatory'.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 18 Aug 2009 09:56:18 -0400

I have been watching this debate, which appears to consist of people 
talking past each other.

Maybe my addition will just add confusion, although I beleive I agree 
with Andy and Juergen.  (As a caveat, I use "means" below to indicate 
what I think the YANG text explicitly requires.)

1) Mandatory means that if a manager is writing that part of the tree, 
the manager MUST provide a value for that item.

2) Default means that if no manager has not set the item, the item will 
have the explicit value given in the default clause in the data model.

So the question is what to do with a value that must have an item, but 
which the system will generate a value if one is not provided, and which 
the data model writer can not define that value in advance?

To remove a corner case, if the manager is not allowed to set the item, 
then it is not config data, and this whole debate is irrelevant.  We are 
talking only about config data.

As far as I can tell, when the mandatory and default clauses were 
defined, we did not think about this case.  (Or, if some folks thought 
about it, it was not discussed and articulated.)  As a result, it is 
simply not covered.

There are, as far as I can tell, a couple of ways out, of varying 
attractiveness.

1) We can follow Andy's suggestion and create a keyword to represent 
this case.

2) We can add words to say that this case will NOT use Mandatory, will 
not use Default, and that instead the description clause will tell the 
human being what value is likely to be found here.  As far as I can 
tell, nothing in the YANG document prevents the system from setting 
values that have not been set by managers and which are not Mandatory. 
Since the system will provide a value, this would work.  It means that 
from an automaton perspective, the model does not distinguish between 
things the system sets and things that are truly optional.

3) We could do nothing and say nothing in the document.  This would seem 
to be a mistake, because different model writers and different manager / 
agent implementors will do different things.  Interoperability is likely 
to fail.  (Given that we have differing interpretations already 
demonstrated on the list, I think "likely" is an understatement.)

Personally, I would go with 1.  But I can live with 2.

Yours,
Joel

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Aug 2009 16:05:08 +0200

> so there is no need to change any text.  We can use either terminology.

Which page of RFC 4741 do I have to look at?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Aug 2009 16:36:26 +0200

> demonstrated on the list, I think "likely" is an understatement.)

I think there is yet another possibility:

4) Using the rules for "mandatory" in Sec. 3.1, the device MUST
determine the minimum valid configuration and make sure all mandatory
leaves in the "running" configuration are initialized with values that
are valid according to the data model. These system-assigned values
SHOULD be described in system documentation. 

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 10:24:10 -0400

>I have been watching this debate, which appears to consist of people 

We have a rich history of such debates.

>with Andy and Juergen.

My understanding was that Andy and Juergen did not agree here.

>the manager MUST provide a value for that item.

Sort of.  With candidate, there's no validation until the commit
operation, so a valid configuration MUST contain a value, but
when and how it appears is flexible (meaning that I can do one
edit-config operation to make that part of the tree, and then
another operation to populate some missing mandatory values.

Running is more straight-forward, since the configuration must
be valid after each operation.

>have the explicit value given in the default clause in the data model.

And this issue here is what "the item will have" means.  Will this
node appear in the database or not?  Is the default implicit or
does it become part of the configuration?  If you didn't set "mtu"
but there's a default, will a get return that default?

>about it, it was not discussed and articulated.)  As a result, it is 

I believe it was articulated in the "assigned-by system" proposal:

http://trac.tools.ietf.org/wg/netmod/trac/wiki/Issues_yang#issue-01622

I am personally in favor of this, but the concensus was not.  The
issue is still marked open, so perhaps it's time for another concensus
call?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 18 Aug 2009 16:54:56 +0200 (CEST)

> issue is still marked open, so perhaps it's time for another concensus

I'm also in favor of this, but I'm not sure I want to re-open the
concensus call.  See the email
http://www.ietf.org/mail-archive/web/netmod/current/msg01622.html for
some open issues.  My main concern is that if we add this it will
delay the process.  This is a known issue, and the WG decided to not
add any more features.  This issue was known at the time.  There are
more nice to haves that could be added, but at some point we have to
stop.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 10:37:53 -0400

>so there is no need to change any text.  We can use either terminology.

In 4741, the term "agent" appears only one, in the section on filters
that you wrote.  I think this is a typo that Rob missed.  By contrast,
the term "client" appears dozens of times.  The YANG draft avoids
the term "agent" completely.

Googling for "network management agent" yields snmp-related pages,
as well as many references to "agents" in the modern sense:

    An Agent-Based Network Management System
       This paper describes ongoing research work to build a network
       management system from autonomous software agents, using the
       BT Zeus Agent Building Toolkit.

So even if the NM world, the term is confusing.

I firmly believe this is an SNMP-specific term, perhaps reanimated
from GDMO, that is confusing to non-snmp folks.  We should bury it.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 10:45:36 -0400

>SHOULD be described in system documentation. 

This is a completely different concept of "mandatory" than what
YANG has.  YANG's mandatory is a contraint on the contents of the
configuration database.

Your concept is rather the opposite, telling the device to create
values when none is provided.   I think this is a non-starter, since
the instances where I want the system to assign a value are very
very rare.

If we adopted your concept, how would you represent the current
"mandatory", where the device cannot determine a valid value?

If I made "first-name" mandatory under a list of <user>s, how
would the value be determined?

Thanks,
 Phil

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 18 Aug 2009 11:03:59 -0400

I could certainly live with adding "agent-supplied value", or 
"assigned-by-system."
I understand Martin's argument for not adding a new feature.

But if we are not going to add a feature, then I think we MUST (in the 
ineroperability sense) add some words that explain how this is supposed 
to work.  It is quite clear that people understand it differently. 
(Ladislav's understanding of how one could use Mandatory for this is 
just one example.  I do not read the text as allowing the behavior he 
understand in good faith to be correct and appropraite.)

Yours,
Joel

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Aug 2009 18:59:06 +0200

> YANG has.  YANG's mandatory is a contraint on the contents of the

Right, from sec. 3.1 and 7.6.4 it follows that all mandatory leafs MUST
exist (and I understand this as "MUST exist for the configuration to be
valid"). Therefore, the initial configuration can be valid only if all
mandatory leafs (and, transitively, all mandatory nodes) exist. Why is
my formulation "a completely different concept"? Do you want to allow an
initial configuration that is invalid?

> values when none is provided.   I think this is a non-starter, since

No, this is only relevant for the mandatory leafs in the minimum valid
configuration, essentialy those that are either top-level or reachable
from the root node through non-presence containers or lists with
min-elements>0. For everything else that is added later via edit-config
operations, the client must always supply all mandatory nodes so that
the resulting configuration is again valid.

> the instances where I want the system to assign a value are very

Sure, the minimum valid configuration can also be empty.
 
> "mandatory", where the device cannot determine a valid value?

Either as optional without any default specified, or, if the device
really cannot operate without it, it would be defined as mandatory, but
then the device must not start the NETCONF subsystem before such a value
is provided by other means, perhaps by editing a file or via an initial
configuration dialog.

> would the value be determined?

If the list has min-elements=0, the device will start with an empty list
(which is valid configuration), and then of course every edit-config
operation that adds an entry must provide "first-name" for the entry. If
the data model specifies min-elements=1, then the device should add "Joe
Admin" or start an initial setup dialog the fills the first entry.

But in my view every device must have a valid running configuration as
soon as it starts accepting NETCONF sessions and so all mandatory nodes
must be present at that time.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 13:10:36 -0400

Ladislav Lhotka writes:
>... Why is

Sorry.  My bad.  I completely missed that you are talking about
initial configurations.  The comment "the device MUST ... make sure
all mandatory leaves ... are initialized with values" confused me,
since this sounds like the device must fill in those values.  You
are just saying that <running> should be valid when the box boots
up, and I agree with this.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 12:19:32 -0700

> understand in good faith to be correct and appropraite.)

The only value YANG provides to the world at this time is as
the description of contract between the application and the device.
(Not manager and agent -- that's not what 4741 calls it -- sorry.)

It is only the extra details that make a NETCONF database different
than an XML instance document, than gives YANG any reason to exist.
Otherwise we would just use RelaxNG and not bother with trying
to characterize network configuration at all.

Martin's proposal for assigned-by is almost perfect.
The only problem is that 'assigned-by' (system/user)
seems too much like 'ordered-by' (system/user).

The real semantics of assigned-by is (user/all) [default: all].

Assigned-by user can be used with mandatory (true/false).

Here is the full set of values with 2 independent variables:

  leaf A {
      type string;
      mandatory true;
      assigned-by all;
  }

  leaf B {
      type string;
      mandatory false;
      assigned-by all;
  }

  leaf C {
      type string;
      mandatory true;
      assigned-by user;
  }

  leaf D {
      type string;
      mandatory false;
      assigned-by user;
  }

The CLR is that assigned-by user and default-stmt do
not make sense together.

  A: current ambiguous mandatory=true in yang-07; agent may provide a value
  B: current ambiguous optional leaf (may get created or not)
  C: user must provide a value; server will not provide any default
  D: user may provide a value; server will not provide any default

Does this help or make things worse?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Aug 2009 21:25:00 +0200

> up, and I agree with this.

Yes, only initial configuration is special. Other mandatory leafs should
be set explicitly by a client.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 15:46:26 -0400

>The CLR is that assigned-by user and default-stmt do

"assigned-by user" is the normal (aka default ;^) behavior, so
mixing this with the default statement is fine.

>  A: current ambiguous mandatory=true in yang-07; agent may provide a value

I don't think mandatory is ambiguous in yang.  Mandatory means
the value must appear in a valid configuration.

>  B: current ambiguous optional leaf (may get created or not)

"leaf" is ambiguous?  When did this happen?

I think there are eight cases, with three axis:
  statement      value                code
  mandatory      (true | false)       mf | mt
  assigned-by    (system | user)      as | au
  default        (given | none)       dg | dn

Using a stupid two-letter code for each axis gives:

  mt-as-dg: error: invalid combination (mandatory plus assigned-by system)
  mt-as-dn: error: " "
  mt-au-dg: error: invalid combination (mandatory plus default)
  mt-au-dn: value must be present
  mf-as-dg: error: invalid combination (assigned-by system plus default)
  mf-as-dn: system will assign a value if none is present
  mf-au-dg: default will be implicitly used if none is configured
  mf-au-dn: no value will be used if none is configured

So the CLRs are:
- mandatory cannot be true if assigned-by system or a default value is given
- assigned-by system cannot be set if a default value is given

But the WG concensus was against assigned-by, so we should let this
topic drop.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 13:35:05 -0700

> mixing this with the default statement is fine.

no -- you have it backwards.
YANG does not support 'assigned-by user' at all,
except in description statements.

> But the WG concensus was against assigned-by, so we should let this

The current meaning of mandatory=true is not clear.
There is no way in the YANG data model for the application
developer to get satisfactory answers question for a few simple questions:

  leaf foo {
     type string;
  }

  "If I do not provide the <foo> leaf, will the agent pick some value,
   or not?"

  "If the agent does not support with-defaults=report-all, then how would
   I even find out if this is the source of the config problem?"

  leaf bar {
     mandatory true;
     type string;
  }

  "If I do not provide the <bar> leaf, will the agent pick some value,
   or will I eventually get back a 'data-missing' error?"

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 14:04:59 -0700

Andy Bierman wrote:
...
>    or will I eventually get back a 'data-missing' error?"

  choice fubar {
    mandatory true;
    case a {
      leaf X { type string; }
      leaf Y { type string; mandatory true; }
    }
    leaf b { type string; }
  }

If the device hides all defaults, then how do I know
if the candidate has /X or /b or neither?

The answer also has an impact on NP containers.
A mandatory child leaf within an NP container is
only mandatory if any of its siblings exist.

The very fuzzy definition of a default value in YANG
makes it impossible to know if that mandatory leaf
needs to be created or not.  A leaf without a default
can also exist because the agent created it, so
the issue is not related to the YANG default-stmt.

This is a very important application-centric question.
Every single developer of every single YANG module ever written
is going to ask the same question "Do I have to provide that leaf,
and therefore write some code to do that, or not?"

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 18:10:14 -0400

>and therefore write some code to do that, or not?"

If mandatory is true, the configuration is not valid if the leaf
does not have a value.  Baring miracles, the application should
understand that providing that value is part of its job.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Aug 2009 18:17:54 -0400

>except in description statements.

All config=true leafs in YANG can be assigned by the user.

>  "If I do not provide the <foo> leaf, will the agent pick some value,

This is the hole that the lack of "assigned-by system" leaves.  The
WG concensus is that we are skipping it.  Move along.  These aren't
the 'droids you're looking for.....

>   I even find out if this is the source of the config problem?"

The leaf is not mandatory.  There is no default.  It's not
the source of your config problem.

>   or will I eventually get back a 'data-missing' error?"

No, the agent will not pick some value.  The 'mandatory true'
means that if you make this hierarchy in which this leaf belongs
and you don't make this leaf, your configuration is not valid.
When you validate it, you will get an error.  No clue which error.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 16:18:56 -0700

> understand that providing that value is part of its job.

I don't know why you would reach that conclusion.
The mandatory-stmt does not mean the application must supply
a value.  It has no utility at all to an application.

On the device-side, if I had some data model
like the ping control table in ping.yang,
then I never want the device to fill in the
host address to ping.  If I commit the candidate,
I want it to fail, not start pinging a random
address because the agent filled in a mandatory
leaf (because all mandatory means is that some
value must be present).

Ironically, the only thing an application can
count on is the default-stmt, if no leaf is provided.
But since the device MUST provide the exact static value,
it is very likely that WGs and vendors will not be able
to agree on 1 static value in every possible scenario.

So most leafs will have undocumented defaults (where
defaults='trim').  We can't agree on what a default
even means, so on devices where default='explicit',
you have no idea how to even discover the default values.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 17:29:43 -0700

...

 My main concern is that if we add this it will
> delay the process.  

If it is the IETF WG process you mean,
than I don't see how that is possible.

So what happens if we stop discussions now because
we are out of time?  We move forward with a WG Last Call,
and ask "does anybody have any issues with this draft?"
Hurrying up so we can get to WGLC only delays a resolution;
it does not avoid it.

This is a known issue, and the WG decided to not
> more nice to haves that could be added, but at some point we have to

I find the leafref to be much more esoteric and corner-case nice-to-have
than properties as basic as 'default' and 'mandatory'.  A lot more effort went
into leafref, and these database properties are much more important
than cross references.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 18 Aug 2009 17:40:53 -0700

> understand that providing that value is part of its job.

But you say that the ifMtu object should not be mantatory, right?

  leaf if-mtu {
     type uint32;
     mandatory true;
  }

  leaf ping-target {
     type inet:ip-address;
     mandatory true;
  }

Every implementation must have these 2 objects in a valid
configuration.  In one case, the device is going to pick
a suitable default every time, and the data modeler wants
that to happen.  In the other case, the data modeler does
not ever want a default value picked by the device.

So how does a YANG tool tell these 2 very different use-cases apart?
Humans may guess by the leaf name, but the tool can't.

I do not agree with you that an application developer does
not need to know the difference between these two leafs
that have identical machine-readable properties.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 09:01:33 +0200

> understand in good faith to be correct and appropraite.)

I also have to apologize for contributing to the confusion. After
reading Phil's later message I realized the misunderstanding was mainly
due to the lack of distinction between the assumptions on the initial
configuration on one side and subsequent edits on the other.

Some explanation of the former is needed in the sense that the server
must provide certain minimum set of mandatory nodes in order to make the
initial configuration valid - these nodes are thus implicitly
"assigned-by system". Otherwise, the rules seem to be OK if we assume
that any other mandatory nodes are up to the client to fill in.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 09:13:43 +0200 (CEST)

> Hurrying up so we can get to WGLC only delays a resolution;

There are many things we could add to YANG in order to make more
precise data models and to help the tools automation process.  We will
never be able to have formal statements for everything, so we know
that we will also have to rely on description clauses.  At some point
we will have to be done with version 1, publish it and get
experience.  In fact, we already decided that we have reached this
point.  The features we have are good enough for version 1.

These are my concerns.  But at the same time, if we agree that what we
have does not work without a new statement, then we will have to fix
it.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 09:39:00 +0200 (CEST)

> a value.  It has no utility at all to an application.

This is not true.  If the client creates data in which there is a
mandatory leaf, it must provide a value for that leaf to make the
config valid.

This means that there are some use cases which cannot be perfectly
defined today.  For example, a leaf which is initialized by the server
if it is not provided.  One use case that has been described before is
the 'uid' of a user.  If a 'user' is created without a value for the
'uid', the server will assign a value for this leaf.  This is not a
(dynamic) default value; it is really present in the config.  A client
can read it and change it.  In one sense such a leaf is mandatory,
since it is always present in the config.  But just marking it as
'mandatory true' is not correct, since this will make the client think
it MUST provide a value.  Currently, such a leaf must be defined as
'mandatory false', and the behavior defined in the description clause.
The statement 'assigned-by system' was an attempt to support this use
case.

Another use case that cannot be perfectly defined is the case of
dynamic default values.  These comes in many flavors; here are three:

  o  The default value depends on some state data, e.g. the default
     mtu depends on the card type.

  o  The default value depends on some other config data, e.g. the
     listen-port for a server depends on the protocol (smtp port 25,
     http port 80 etc).

  o  The default value is the value of another leaf (this has been
     called default ref in past discussions).  E.g. there is a global
     idle-timeout leaf which is used unless a specific idle-timeout
     leaf is defined for a server.

I'm sure there are more variants.  Currently, these leafs must be
defined as 'mandatory false', and the behavior defined in description
clauses (and/or with YANG extensions).

Do we add formal YANG statements for all these use cases?  As I wrote
in some other thread, in general I am in favor for more precise models
and more support for automation, but at some point we have to stop.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Aug 2009 10:02:01 +0200

> not ever want a default value picked by the device.

I think the MTU value that pops up when I initialize an interface with
no explicit configuration is operational state and not config state.
So the above definition of if-mtu is likely a data modeling error.

And perhaps most parameters a system uses if you switch it on are just
operational state and not really config data. A home router might boot
up with a default non-global IP address and this can again be
considered operational state since there is no config data yet. I
guess I start to like the model where stuff that has not be
explicitely configured really is not configuration and thus not part
of a configuration datastore.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 11:26:48 +0200

> considered operational state since there is no config data yet. I

It may write this address to configuration data as well, as it is also
explicitly configured (not obtained from DHCP). I just reset my web
camera to the factory default state and the auto-assigned address
appeared in the configuration form. So it can work both ways.

> of a configuration datastore.

In this case the link between the IP address as state data and the
corresponding configuration slot is also important. If the IP address is
manually configured, the state data item is essentially a leafref.

But is it reasonable to continue working with both interpretations and
allow both modelling approaches? The simple question whether "mtu"
should be defined as mandatory or not then has no simple answer.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 02:59:45 -0700

> mandatory leaf, it must provide a value for that leaf to make the

I'll drop this thread, but none of you actually seem to
understand the difference between MUST and SHOULD.

> clauses (and/or with YANG extensions).

These account for lots of data model objects.
The static default-stmt may not get used a lot
by WGs or vendors.

> and more support for automation, but at some point we have to stop.

what kind of strawman is this?

I am talking about 1 bit of information.

What happens on every device if a mandatory leaf is not provided?
Will this be silently accepted, or rejected with an error?
The official answer is "who cares", but I was looking for "yes" or "no".

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 03:15:45 -0700

> So the above definition of if-mtu is likely a data modeling error.

You would be right about this except for one
tiny detail.  If an MTU is configured, then
the interface will use the supplied value
and not pick its own value.

That makes it configuration data, not operational state.

> of a configuration datastore.

My home DSL modem has an override in the WEB pages for MTU.
You can configure it use 1500 and NOT to pick its own value.
Just because this is rarely done does not change
the leaf to config=false.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 12:36:57 +0200 (CEST)

> The official answer is "who cares", but I was looking for "yes" or "no".

No, that's not what I said, and that's not what Phil said.  At
validation time (i.e. <validate> of candidate, <commit> or
<edit-config> of running), you will get an error if a mandatory leaf
is not provided.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Aug 2009 12:39:32 +0200

> That makes it configuration data, not operational state.

Yes, but my point was that as long as the MTU is not configured
explicitly, it is not config data and there is no need to have boxes
generate config data upon startup since this blurs the line between
operational state and config data.

> the leaf to config=false.

I fail to see the point. I am not saying there should not be a config
leaf. My point was that (a) making it mandatory is questionable and
(b) a box that boots up and has no further information might have an
MTU but it is not config.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 13:28:26 +0200

> explicitly, it is not config data and there is no need to have boxes

The simple fact is that the MTU parameter is absolutely essential for
the operation of an Ethernet interface, so any implementation must make
sure it is set to a valid value - which can be achieved in multiple ways
- and IMO this information should somehow be contained in its data
model.

In the data model for an L3 interface, do you want to have the IP
address parameter optional? So what if I add a new network card that NV
memory has no data for?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Aug 2009 13:38:46 +0200

On Wed, Aug 19, 2009 at 01:28:26PM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v St 19. 08. 2009 v 12:39 +0200:
> 

The IP interface won't work until it is configured or there is
operational state (e.g. obtained via DHCP) that provides a suitable
address. For me, it does not matter whether a protocol like DHCP is
involved or the kernel or firmware does the job in some other way.
The result in all cases is that the system did generate operational
state that might make a device do something useful but it is not
really configuration data in my admittedly strict interpretation what
config data really is.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 14:05:10 +0200

> really configuration data in my admittedly strict interpretation what

I am not against having the state data where the actual address (from
DHCP or manually assigned) is recorded, like in the /proc filesystem.

But you said that your home router has a static preconfigured address,
so let's forget about DHCP for the moment. I understood that since the
preconfigured address is state data, the data model for the interface
*configuration* will have IP address defined as optional, to allow for a
valid configuration where this parameter is not present. However, this
data model is wrong for an interface on the extension card because there
is no preconfiguration available and the configuration will be declared
valid even though an essential parameter may be missing. So you would
have to have another model for the other interface with mandatory IP
address. It is IMO much better to have one data model for both
interfaces and just accept that for the built-in interface the IP
address is configured by the device itself.

With DHCP it is the same, there will be a choice with one case for DHCP
and another for manual address, but the choice must have
"mandatory=true".

Lada 
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Aug 2009 14:39:14 +0200

> address is configured by the device itself.

Sorry, but you are interpreting too much into the concept of
"valid". Valid means consistent with the data model. It does not mean
correct and usable.

I am saying a data model which requires an IP address for a valid
configuration is broken since I can run for example DHCP. If you want
to formally express notions like "this leaf is mandatory if some other
condition is true" we will likely need years to come up with a correct
data model.

> With DHCP it is the same, there will be a choice with one case for
> DHCP and another for manual address, but the choice must have
> "mandatory=true".

I choose to disagree. I might very well simply choose to not use IP at
all since all my box is doing is sniffing packets. If we go over board
with constraints in data models, we will kill ourself. Authors of YANG
data models applicable to a wide range of boxes better we very careful
with their usage of must, when, mandatory, default - these are sharp
tools to hurt yourself.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 15:00:14 +0200

> configuration is broken since I can run for example DHCP. If you want

See below for DHCP.

> with their usage of must, when, mandatory, default - these are sharp

This way we can just get rid of "mandatory" completely. I could argue
for any parameter that it is optional because I might choose to use the
box as paperweight. 

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Aug 2009 15:04:24 +0200

> for any parameter that it is optional because I might choose to use the

We are really making progress here.

/js

---

From: "tom.petch" <[email-masked]>
Date: Wed, 19 Aug 2009 11:23:09 +0200

----- Original Message ----- 
From: "Joel M. Halpern" <[email-masked]>
To: "Martin Bjorklund" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, August 18, 2009 3:56 PM

Joel

Nice summary, except that I want a different third state, that is one 
where the item must have a value and the data model writer can define
one in advance.  For me, that is the overwhelming case, when I look
at a device configuration, see all the things I might have specified but
never have, but equally can see that the system needs them (timers,
minimum and maximum values, thresholds, ...)

As the I-D stands, I cannot have mandatory and default, so I agree with
Lada, 
"'mandatory' and 'default' wouldn't be mutually exclusive".

(But I can also see, as Lada points out, that this also depends on a definition
of datastore and that without that, we are unlikely to agree).

Perhaps we are trying to be too clever and should allow for some intelligence
to reside in data-modellers, expect them not to provide a default if there
is no meaningful default.

And yes, this general issue has recurred on this list, many, many times.
At one time, we had three parameters and a possible eight states.

Tom Petch

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 15:24:20 +0200

> We are really making progress here.

OK, my point just is that if there is any parameter at all that is
absolutely essential for a device or protocol, it should be declared as
mandatory in the generic data model. The fact that some implementations
choose to set it from NVRAM or even have it hardwired doesn't make it
optional and/or state data.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 09:42:43 -0400

The problem however remains for later additions.
The user, via the management application (manager), adds a new piece of 
configuration.
That piece of configuration has a piece of information in it that must 
exist for the system to work.
Currently, if we label that piece of information "mandatory", then the 
user / manager must include that piece of information in the provided 
config.
If, instead, we tag that piece as having a default, the current 
definitions require that the model writer define the default value, and 
that the node have that value.

Thus, if the system is capable of setting the value, but the user is 
allowed to set it, the model must leave off both the mandatory and the 
default statements.
But this piece of information has significant semantic differences from 
a fully optional element.

If we do not want to define a new statement to represent this kind of 
configuration information, then we need text to reduce the human 
astonishment.  If the user does not provide a value, and the system does 
provide a value, does that magically appear when the user does a get of 
that configuration?  Or does it appear only for with-defaults even 
though the item has no default statement?

Yours,
Joel

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 16:05:39 +0200

> a fully optional element.

The current "mandatory" clause addresses the static state of a datastore
and I would prefer to keep it that way because the DSDL mapping relies
on it. The specification whether a particular leaf's value may/must be
set by the client, server or both is a new orthogonal feature and should
be handled separately, but probably not now, as Martin said.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 10:29:33 -0400

In one sense, I follow your reasoning.
"Mandatory" means "must exist when validation occurs" (which allows for 
the differences between running and other configs as to when validation 
occurs.)

Therefore, you can argue that if the system provides the value, then the 
mandatory value "exists" at the right time, even though the managing 
application did not provide such a value, even for things created during 
system operation.

However, from the point of view of a managing application trying to use 
the data-model, that appears to make the "mandatory" flag irrelevant.

Under the model in which the managing application had to provide all 
mandatory elements before verification would pass, a smart human 
interface application could build up the model as things were defined, 
and could warn the human being before he hits the verify/commit button 
that critical things are missing.

Another potential use for "mandatory" would be to have a generalized 
validator that runs on the managed device to check that configs match 
the data model before they get committed.  (In some designs, that could 
reduce later error checks.)  However, as best I can imagine, for that to 
have any utility it would have to run before any system processes that 
fill in special values.  So it can not complain about missing mandatory 
elements, since the system may decide "oh, that's okay, I know what to 
do about that missing value."  Again, "mandatory" becomes part of the 
description, telling humans that if they don't provide it, the system will.

Under the interpretation whereby the system might choose to fill it in, 
the mandatory tag might as well be part of the description clause for 
all the good I can understand.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 10:38:26 -0400

This is an interesting idea.

Given that the data model already requires that the modeller figure out 
what is mandatory and what isn't (and that providing for system-provided 
values requires the modeler to understand what the system can or can not 
provide), I think the variation in situations (in Lada's comments on 
this) merely indicates that one model does not cover all cases.  (I 
presume that careful use of choice would allow one to cover more cases, 
but probably not all.)

Allow me to paraphrase to see if I understand you correctly.
For a piece of configurable information which the system will create a 
value (essentially a more complex default), one can model this with a 
pair of pieces of information:
A state variable, for example in-use-mtu
A configuration variable, for example configured-mtu.

The configured variable would not be mandatory.  If it is not provided, 
it is not present.  If a manager wants to know what value is being used, 
he can always read the state variable.  (There is no leafref, since the 
values of the two are loosely coupled.)

While there is a bit of a loss of semantic content, this is useable.

If we decide this is how we think this aspect should be handled, we need 
to tell people in the model itself.  Otherwise, folks will do all sorts 
of different things, and management applications will have no clue what 
is going on.

Yours,
Joel

..

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 09:15:47 -0700

> A configuration variable, for example configured-mtu.

We are drifting into data model design issues,
but that is good, because the if-mtu object
is the poster child for confusion about
interactions between the mandatory and default statements.

There seems to be 3 ways to model this object (so far):

A)
   if-mtu {
     type uint32;
     mandatory true;
   }

B)
   if-mtu {
     type uint32;
     // assigned-by system
   }

C)
   if-mtu {
     type uint32;
     config false;
   }
   desired-if-mtu {
     type uint32;
   }

> While there is a bit of a loss of semantic content, this is useable.

I am still concerned about the lack of any mechanism
in YANG for the data modeler to declare a parameter
that MUST NOT have a device-assigned value.
This applies to /rpc/input, not just data nodes.
It will be left to description statements and extensions.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 18:41:37 +0200 (CEST)

> There seems to be 3 ways to model this object (so far):

D)
   leaf if-mtu {
     type uint32;
     // dynamic-default;
   }

You might argue that D isn't very good, b/c it relies on the
with-defaults capability for a client to learn what the value is.

B isn't perhaps very common, since it will actually create a something
in the config data store.

> that MUST NOT have a device-assigned value.

You keep saying that, and I keep saying that this is what 'mandatory'
does.

I think the original concern was with mandatory top-level nodes which
by definition cannot be created by a NETCONF client, but can be
created as part of initial pre-loading.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 12:53:59 -0400

That (D) seems confusing to me.  Continuing below.

Martin Bjorklund wrote:
...
> with-defaults capability for a client to learn what the value is.

If we are going to say that something that is declared without a default 
clause may
a) have a value
b) not appear in response to a normal get
c) appear in response to a get with-defaults

then we better document that VERY clearly.
It would seem to me that at that point the managing application would 
have to do a get with-defaults all the time, because otherwise it would 
not know what values things have, or whether things have a value.

If we were willing to have feature creep, and wanted to have a way to 
represent this, I suppose we could have a default syntax along the lines 
of "default-dynamic" as an actual statement with no parameters, which 
would at least tell the application what is happening.

(Personally, I don't care what syntax is used to represent this case 
assigned-by, dynamic-default, ...  I can even live with significant 
explanatory text.  But I would not bet on itneroperability as things 
stand right now.)

Yours,
Joel

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 10:08:30 -0700

> with-defaults capability for a client to learn what the value is.

no -- it is good.
If the device saved the if-mtu leaf in NV-storage, it
would definitely be (A).

If not, then it is still ambiguous, because of the
way defaults work in YANG.  I think (B) and (D)
are going to be far more prevalent than we would like,
which is not good news for automation tool developers.

We can see how things work out, and if any proprietary
extensions do a good job of addressing the issues.

> You keep saying that, and I keep saying that this is what 'mandatory'

No.  The definition in the draft just refers to
a valid configuration. It says nothing about
how the value got there.

This issue applies to /rpc/input and data nodes.

  rpc ping {
    input {
      leaf host {
         type inet:ip-address;
         mandatory true;
      }
    }
  }

A device is allowed to provide its own value for mandatory=true leafs.
That isn't what I want in this data model design.  Just because
192.168.0.1 is a valid address, does not mean that would be a good
address to ping, in case <host> is missing.  It MUST be a data-missing
error in all implementations.

Is that what this YANG specifies?
Or does it say the opposite -- that an agent MAY provide a value
in this case?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 19:27:56 +0200 (CEST)

> a valid configuration. It says nothing about

What text would you like to see added?

> Is that what this YANG specifies?

That's the intention.

> Or does it say the opposite -- that an agent MAY provide a value

No.

> > created as part of initial pre-loading.

But at the same time I want to allow devices to do initial pre-loading
of the data store.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 10:38:58 -0700

> What text would you like to see added?

7.13.2, para 2:

OLD:
   If a leaf in the input tree has a "mandatory" statement with the
   value "true", the leaf MUST be present in a NETCONF RPC invocation.

NEW:
   If a leaf in the input tree has a "mandatory" statement with the
   value "true", the leaf MUST be present in a NETCONF RPC invocation.
   Otherwise, a device MUST return a 'data-missing' error.

Since there is ambiguity about how an existing data node
can become present in the database, we should make it clear
there is no ambiguity about rpc/input.

I'm sorry it took so many emails to get to the 1 sentence
that I would like to see added to the draft.

>> Or does it say the opposite -- that an agent MAY provide a value

good

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Aug 2009 14:54:31 -0400

>extensions do a good job of addressing the issues.

Sounds good.  Let's let this issue drop.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Aug 2009 14:58:59 -0400

>optional and/or state data.

This is where the quality and flexibility of the model will come
into play.  If the model is tightly drawn, the number of deviations
will be high.  If the model is more flexible, then the need for
deviations should be low.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 12:32:03 -0700

> deviations should be low.

This would only prove the standard specification was too vague.
Dynamic defaults are a good use of deviations, to fully document
the device behavior.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Aug 2009 15:30:16 -0400

>Dynamic defaults are a good use of deviations, to fully document

I guess this depends on the style of modeling.  The JUNOS models
don't include a default MTU, with the expectation that each interface
will choose a reasonable one.  IMHO forcing this issue (on which
reasonable implementations disagree) means you are drawing the
model too tightly.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 21:55:15 +0200 (CEST)

>    Otherwise, a device MUST return a 'data-missing' error.

Fixed.  (with s/a device/the server/).

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 15:57:30 -0400

My concern is that given a model, it appears from the discussion that 
different device implementors will do different things, thinking that 
they are doing exactly what the model tells them.  Similarly, different 
management applications will draw different interpretations.

That does not sound like a recipe for interoperability.

Yours,
Joel

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Aug 2009 22:21:21 +0200

> > There seems to be 3 ways to model this object (so far):

So let me add one more:

E)
    leaf if-mtu {
        type uint32;
        default 1500;
    }

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 19 Aug 2009 16:29:18 -0400

If that were what the data model said, then presumably an application 
which knew that no one had modified the object (for example, it had jsut 
created the object), could reasoanbly assume that the mtu would be 1500, 
and does not need to read it to confirm that.

If that is not the case, then the value in the default statement appears 
to be without meaning.

Yours,
Joel

Ladislav Lhotka wrote:
...
...

---

From: WashamFan <[email-masked]>
Date: Thu, 20 Aug 2009 10:36:59 +0800

Hi,
 
>  created as part of initial pre-loading.

mandatory top-level nodes are alway kept in my mind. Those nodes could
not be set via <edit-config>, they should be set before NETCONF is started.
So how to interpret 'mandatory' for top-level nodes?

washam

---

From: WashamFan <[email-masked]>
Date: Thu, 20 Aug 2009 11:05:37 +0800

Hi,

> Why do you say that they (mandatory top level nodes) can not be set by 
> 
>  edit-config.
>  While it is true that they must exist in all configs, that does not 
>  prevent an edit-config operation from changing the value.

You are right, mandatory top level nodes can be MODIFIED by edit-config
definitely. Sorry for that and thank you for pointing it out.

I meant, (a) some mandatory nodes should be created when the hierarchy (which
is not the whole module) is created, and (b) some mandatory nodes (including
top-level mandatory) should be created when the module is loaded. In the
latter case, the mandatory nodes should be CREATED before <edit-config>
can be used.

Since the prevalent interpretation of  'mandatory' is the value should be
supplied by manager. My concern is how to do that in the case (b) with
NETCONF.

Thanks,
washam

>  WashamFan wrote:

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 20:31:21 -0700

> supplied by manager. My concern is how to do that in the case (b) with

Since the client does not know if the server is going to
create a mandatory leaf at module-load-time, the only
safe thing to do is never define a top-level mandatory node
in a YANG module.

That is what yang-usage-01 specifies.

Andy

---

From: WashamFan <[email-masked]>
Date: Thu, 20 Aug 2009 11:46:55 +0800

Hi,

>  safe thing to do is never define a top-level mandatory node

Shouldn't mandatory nodes be created only by the client? Why
the client care about the behavior of the server? 'mandatory'
means the value MUST exist and MUST be supplied by the client,
does it? I am still confused.

Thanks,
washam

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Aug 2009 21:07:26 -0700

> does it? I am still confused.

no. that is only for /rpc/input.
for data nodes, mandatory=true just means the
node MUST exist in the running database.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Aug 2009 09:24:21 +0200 (CEST)

> Fixed.  (with s/a device/the server/).

I was too quick.  The error message should be 'missing-element'.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Aug 2009 11:36:20 +0200

> So how to interpret 'mandatory' for top-level nodes?

My proposal is to require the device to set all mandatory nodes in the
"minimum valid configuration" (which may also include leafs that are not
top-level).

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 20 Aug 2009 10:32:43 -0400

I am not sure I follow what you are suggesting.
In the initial config, that the box comes up with, all configuration 
level mandatory nodes must exist.  Whether this is done by the 
manufacturer preloading such a config, by magic, or by the device 
creating likely useful vvalues for these does not change them.

However, when a netconf application uses netconf to create a piece of 
configuration (for example, the top level of a module), then that 
application (and / or the user behind it) is responsible for providing 
all top level mandatory pieces of information.  That can be done in 
steps, for example in :candidate before a commit.  But the user is 
responsible.

I think we are talking about the strange case where the component is 
defined in a module, but is separate from the basic thing the user is 
trying to create.
As far as I can tell, it is still the users responsbility to create it, 
because there is no time point in time before the commit when the system 
can create it / them for him.  And at the commit what the user has 
provided has to be valid.

Admittedly, this is pretty awkward.  As such, Andy's recommendation in 
the usage guidelines is probably the right answer.  "Don't do that."

Yours,
Joel

---

From: "tom.petch" <[email-masked]>
Date: Thu, 20 Aug 2009 11:43:15 +0200

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
To: "Ladislav Lhotka" <[email-masked]>; "NETMOD Working Group"
<[email-masked]>
Sent: Wednesday, August 19, 2009 10:29 PM

Joel

The meaning I give it is that the writer of the data model knows best,
perhaps because that is the WG in the SDO that developed the
protocol and has evolved it over the years.

So this is telling the application that unless the user of the application
comes up with a better idea, then this is the value that should be
set in the device.  The manufacturer of the device may have different ideas,
which no longer reflect best practice.  (Plenty of examples in IPv6
of that).

Alternatively, we could say that the manufacturer MUST ship the box
with values as per data model, but I prefer the first approach.

Of course, the value in question may be critical to the operation of
the box and cannot be omitted but, of course, the leaf cannot be defined
as 'mandatory'.

Tom Petch

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 20 Aug 2009 14:13:10 -0400

To paraphrase, you are looking at the default statement as a SHOULD 
strength statement about what the value ought to be in a newly created item.

As far as I can tell, that means that if a managing entity cares about 
that value, it better provide it even if the value it would provide is 
the same as the default.  Becuase it can not count on the machinery 
actually ending up with that value if it does not set it.

That is defensible, but it sure isn't what I thought the definition of 
the statement said.

Yours,
Joel

---

From: Andy Bierman <[email-masked]>
Date: Thu, 20 Aug 2009 11:27:12 -0700

> strength statement about what the value ought to be in a newly created

that got my attention...

We must not go back to the worthless SMIv2 DEFVAL design.
Let's keep 'default-stmt' mandatory-to-use by the device.
I have found many instances of writing YANG module
that the static default-stmt could be used.

I think a consensus has emerged that more machine-readable
standard language statements would be more precise than
the current 'mandatory=true' property.   However, there
is no consensus on the solution, so it should be left
out of YANG 1.0.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 20 Aug 2009 14:15:46 -0400

>strength statement about what the value ought to be in a newly created item.

The default is a MUST strength statement:

    If the leaf has a default value, the server MUST use this value
    internally if no value is provided by the NETCONF client when
    the instance is created.

>actually ending up with that value if it does not set it.

Neither is true.  The managing entity can know that the device
will behave according to the "contract" that the YANG module
represents, and can count on the default value being used
internally when a value is not present in the configuration.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 21 Aug 2009 16:13:09 +0200

> level mandatory nodes must exist.  Whether this is done by the

Not all of them, for example, mandatory nodes inside containers with
presence needn't be created, they become mandatory only when the
container is created. The idea simply is that the device has to start
the NETCONF session with a valid configuration.
 
> steps, for example in :candidate before a commit.  But the user is 

Yes, at least for configuration data, rpcs and notifications are
special.

> defined in a module, but is separate from the basic thing the user is 

I don't get what you mean here.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 21 Aug 2009 10:22:39 -0400

I am trying to explore what your suggestion means.
Suppose that we have a Module, with two top level items, A and B.
The module defines B as "mandatory".  (That seems to be just a mistake, 
but it is allowed by our definitions.)

Now, a user wants to create an A, as defined by the module.
You seem to be asserting that the system can (and should / will?) fill 
in the "mandatory" B.  I don't see how.
Assuming that the user is using a configuration with deferred 
validation, when that validation finally occurs, the B must be there. 
In that config.

In order for the system to add the "B" by itself, some very interesting 
manipulation of the validation step would have to occur.  Manipulation 
that is not called for by the document.

As such, even if some implementation can magically cause the B to 
appear, the management system that is creating the configuration, and 
presumably has logic to assist the user in enforcing the rules, will 
have to assume that it is the user / management system that is 
responsible for creating "B".

This is different from mandatory top level nodes in the base config, 
since those were presumably created before any management interactions 
occurred.

Yours,
Joel

...

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 21 Aug 2009 16:38:05 +0200

> with values as per data model, but I prefer the first approach.

Me too. IMO, defaults cannot be enforced. In fact, if the leaf's data
type permits a certain set of values, it is reasonable to assume that
none of them is a priori wrong, so why should one value be prescribed?

> the box and cannot be omitted but, of course, the leaf cannot be defined

As a matter of fact, with the "explicit" strategy of defaults handling,
such values are mandatory meaning that they can never be missing in the
configuration. They are either set explicitly or have the default value.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 21 Aug 2009 16:43:14 +0200 (CEST)

"Joel M. Halpern" <[email-masked]> wrote:
> The module defines B as "mandatory".  (That seems to be just a mistake, but it
> is allowed by our definitions.)
> 
> You seem to be asserting that the system can (and should / will?) fill in the
> "mandatory" B.  I don't see how.

When the NETCONF server advertises the namespace of Module in the
hello message, it guarantees that running contains valid data for
Module.  (If there are no mandatory top-level nodes, this "valid data"
may be empty).  Thus, the server needs to make sure that B has a value
when the NETCONF service starts.  How this is done is implementation
specific, but can be done through some initial CLI, by pre-loading the
data base with some factory data, etc.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 21 Aug 2009 16:46:06 +0200

>     internally if no value is provided by the NETCONF client when

What does "use internally" mean?

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 21 Aug 2009 10:47:24 -0400

That at least has the advantage of putting mandatory module components 
on the same basis as mandatory basic configuration components.
I can buy that.  This looks like something that should be qualified, but 
if we want to allow it at all, this provides a reasonable model for what 
it means to have a mandatory top level item in a module.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 21 Aug 2009 10:56:25 -0400

I understand your and Tom's reasoning.

But that is not what the YANG document says "default" means.  The YANG 
document says that "[I]f the leaf has a default value, the server MUST 
use this value internally if no value is provided by the NETCONF client 
when the instance is created."

Not "SHOULD use".  "MUST use".  It sure seems to me that if you 
implement what you and Tom are asking for, without getting a spec 
change, then what you are doing is not what the spec says.  And the 
application attempting to set things is likely to be very confused 
because it can reasonably depend upon the behavior that is called for in 
the spec.

If you argue that the system changing the value from the default is no 
different from some other human changing the value, and if you can 
persuade the rest of the working group to agree with you, then I would 
ask that we remove the sentence in question from the spec, as it is 
meaningless.  And we instead replace it with something that says what 
you think default actually means.
Note, that at least as far as I can tell, there is significant 
disagreement with your interpretation.  But it is not my call to make.

Yours,
Joel

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 21 Aug 2009 17:35:18 +0200

> when the instance is created."

But I am questioning here what "use internally" means. For example, in
Linux, the hotplug subsystem allows for user-space hooks that are run
after the component is initialized. So even if the kernel driver sets a
default value ("uses it internally") for a parameter, it may be changed
before the user can do anything about it, and various distributions
indeed make their choices.

> Not "SHOULD use".  "MUST use".  It sure seems to me that if you 

IMO, SHOULD has effectively the same strength as MUST in this case.

> you think default actually means.

I agree. I can see how defaults may be useful as a way for controlled
simplification of configurations presented to the clients, but they
really have nothing to do with validity of configurations.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Fri, 21 Aug 2009 11:16:51 -0400

>What does "use internally" mean?

Are you truly not following, or just wanting better words?

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 21 Aug 2009 22:04:10 +0200

On Fri, Aug 21, 2009 at 05:35:18PM +0200, Ladislav Lhotka wrote:
> Joel M. Halpern p????e v P?? 21. 08. 2009 v 10:56 -0400:

Are we still talking about config or already about operational state?

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 10:00:27 +0200

> Are you truly not following, or just wanting better words?

Both. Could it be reformulated like this?

    "An edit-config operation that creates the instance but does not 
     provide a value for such a leaf MUST have the same effect as if
     the default value for that leaf was explicitly specified."

But neither this nor the original sentence covers the case where no
NETCONF client/operation is involved, i.e. when the instance is a part
of the initial configuration or is created automatically after a new
piece of hardware is plugged in. Are there any assumptions for these
cases?

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "tom.petch" <[email-masked]>
Date: Mon, 24 Aug 2009 11:23:17 +0200

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
To: "Ladislav Lhotka" <[email-masked]>
Cc: "tom.petch" <[email-masked]>; "NETMOD Working Group"
<[email-masked]>
Sent: Friday, August 21, 2009 4:56 PM

> disagreement with your interpretation.  But it is not my call to make.

I agree that what I want is not what the spec currently says.  What I see as
a common use case is not covered.

Also, I have a problem with the spec.  I see information coming from four
sources; device-manufacturer, model-writer (could be SDO), manager-implementer
and user, and any one of the four may know best in a given situation, nor do
I think we have the skills to prescribe any particular precedence.

I expect manufacturers to implement best practice at the time the box is
built, but what do they do as best practice (and the SDO-provided data model)
moves on?  either create non-conformance statements, or ship upgrades to change
their values to current best practice.

I don't see either as realistic; what we ask for seems, well, unrealistic, and
so
likely to be ignored.  Given that, I think we can find a better use for the
default substatement.

Tom Petch

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Aug 2009 13:48:39 +0200

On Mon, Aug 24, 2009 at 11:23:17AM +0200, tom.petch wrote:

[...]
 
> I don't see either as realistic; what we ask for seems, well,
> unrealistic, and so likely to be ignored.  Given that, I think we
> can find a better use for the default substatement.

It is difficult to speculate here. All we know is that DEFVALs have
been used quite a bit in SNMP land. Now, the SMIv2 DEFVAL statement
was advisory but not a MUST. It would be an exercise to go over all
DEFVALs in MIB modules in order to determine which of the DEFVAL uses
are actually hard defaults and which ones not. I am sure there are a
number of DEFVALs where there is probably no dispute that they should
be hard DEFVALs.

You want the default statement to have a very different meaning. So we
seem to have three proposals on the table:

a) A default statement defines a value that MUST be assigned when the
   manager does not provide a value.

b) A default statement defines a value that SHOULD be assigned when
   the manager does not provide a value.

c) A default statement defines a value that can be assumed if config
   is retrieved with an option to supress defaults.

Right now, the YANG specification says a). If my list above is
correct, perhaps the chairs can do a concensus call and we move on.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 13:48:58 +0200 (CEST)

> a common use case is not covered.

It seems you are saying that you don't think the current default
statement is very useful.  If this is a correct interpretation, I can
respect that, but at the same time I would like to know if you see any
technical problems with the current text?

> likely to be ignored.  Given that, I think we can find a better use for the

Given the discussions we have had about this, and that we don't have a
concrete proposal to discuss, shouldn't this wait?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 14:22:29 +0200

> number of DEFVALs where there is probably no dispute that they should

If the default is only advisory, it is not a big issue if different
implementors interpret the default statement differently since a
reasonable manager should always check that the default value is really
in the expected place. A MUST requires a precise specification,
otherwise  interoperability problems are much more likely than in the
advisory case.

Lada 

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Aug 2009 14:32:26 +0200

On Mon, Aug 24, 2009 at 02:22:29PM +0200, Ladislav Lhotka wrote:
 
> in the expected place.

Which means the default statement is pretty pointless. It is easier to
simply send the default value in this case instead of checking what
got assigned and fixing things.

> A MUST requires a precise specification, otherwise interoperability
> problems are much more likely than in the advisory case.

The default clause is IMHO as precise as you can make it. Not sure
what is lacking. Perhaps you can post suggested text so I can better
understand.

Note that implementors can use a deviation to handle the case where
they choose to not comply to the specification. So a manager needs to
be able to parse deviations to understand the support implemented on
the device.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 14:55:58 +0200

> got assigned and fixing things.

We are in a perfect agreement here.

> what is lacking. Perhaps you can post suggested text so I can better

I sent an alternative formulation earlier today in this thread but I
suspect it will be found too weak. Given the differences in datastore
models, possible interactions of operational and configuration
parameters etc., it is really difficult to tell what the implications of
the default statement are.

For example, with the following module:

module foo {
    namespace "http://example.org/foo";
    prefix foo;
    leaf bar {
        type uint8;
        default 42;
    }
}

Now, if I unwrap the device implementing this module and start the first
NETCONF session, what can I rely on regarding the value of "bar", if
anything at all?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 08:48:10 -0400

>correct, perhaps the chairs can do a concensus call and we move on.

I disagree with the words "be assigned" in (a).  The current spec
says something between (a) if "be assigned" were changed to "be
used internally" and (c) with no mention of the "suppress defaults"
option.

    If a leaf has a "default" statement, the leaf's default value is set
    to the value of the "default" statement.  Otherwise, if the leaf's
    type has a default value, and the leaf is not mandatory, then the
    leaf's default value is set to the type's default value.  In all other
    cases, the leaf does not have a default value.

I think Lada's "behave as if" words are much more descriptive than
"use internally".

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Aug 2009 15:05:37 +0200

> NETCONF session, what can I rely on regarding the value of "bar", if

Unless there are other semantics in description statements, there will
be no <bar> instance.

According to 7.6, default values become relevant "when the instance is
created" and "no value is provided by the NETCONF client".

Since the only way to create this instance is by writing a value, the
default statement is somewhat pointless - unless there are other
semantics that can lead to the instantiation of "bar" without writing
an explicit value (so I think it is not wrong to allow for this
example).

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Aug 2009 15:12:37 +0200

> I think Lada's "behave as if" words are much more descriptive than

I agree; sorry for the sloppy wording. But we need to put this
discussion somehow to an end.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 15:25:48 +0200

> Unless there are other semantics in description statements, there will

But what about

<nc:get-config>
  <nc:source>running</nc:source>
  <nwd:with-defaults>report-all</nwd:with-defaults>
</nc:get-config>

?
> an explicit value (so I think it is not wrong to allow for this

Right, so let's expand it a bit:

module foo {
    namespace "http://example.org/foo";
    prefix foo;
    container wrap {
        presence "I mean something";
        leaf bar {
            type uint8;
            default 42;
        }
    }

Does the default statement then mean that the following two operations
are equivalent (and nothing else)?

<nc:edit-config>
  <nc:target>running</nc:target>
  <nc:config>
    <foo:wrap>
      <foo:bar>42</foo:bar>
    </foo:wrap>
  </nc:config>
</nc:edit-config>

<nc:edit-config>
  <nc:target>running</nc:target>
  <nc:config>
    <foo:wrap/>
  </nc:config>
</nc:edit-config>

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Per Hedeland <[email-masked]>
Date: Mon, 24 Aug 2009 15:37:28 +0200

> I think Lada's "behave as if" words are much more descriptive than

Well, the text Lada suggested, unless I have missed some message, was:

I think this is not acceptable, since "the same effect" would mean that
it *was* actually "assigned". The description needs to capture that the
device must function as if that value had been assigned, even though it
hasn't actually been - i.e. it cannot imply that the value should be
assigned in the data store. "behave as if" may be better than "use
internally", but it would have to be qualified as to which behavior it
is talking about - personally I'm fine with "use internally", since it
is free from implications about data store contents.

--Per Hedeland

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Aug 2009 15:42:50 +0200

>   <nc:target>running</nc:target>

The answer likely depends on how you define "equivalent". In terms of
the behaviour of the device, they are equivalent. The presence
property of wrap does not change anything related to the instantiation
of bar as far as I understand things.

My preferred model, as stated before, is the one where the server
makes a distinction between config values that have been explicitely
set and values that are operationally used but not explicitely
configured and thus not part of a configuration datastore. So in both
cases, according to my model, the box is going to use the value 42 if
it needs a value for bar and hence the behaviour of the device should
be the same in both cases. The first edit-config, however, makes the
value 42 explicit configuration.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 15:46:21 +0200 (CEST)

>   <nc:target>running</nc:target>

The behavior of the application in the server will be equivalent, but
the difference is that in the first example, /wrap/bar exists in the
config data store with a value of 42.  In the second example,
/wrap/bar does not exist in the data store.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 09:32:01 -0400

>We are in a perfect agreement here.

But you are in disagreement with what operators want.  There's an
understandable difference between what applications want and what
human users want, and IMHO NETCONF/YANG cannot succeed if they
achieve programmatic access at the cost of human access.

In particular, the human wants configuration to be brief, simple,
and readable.  They want to find the important bits, with the
asssumption that making these bits more obvious means a simpler
time finding and correcting errors.  Populating a real config with
default values is a non-starter.

Given that we want to be better than existing schema langauges, are
there any examples of existing schema langauges that _don't_ have
the concept of default values?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 09:33:06 -0400

Juergen Schoenwaelder writes:
>we need to put this

+1

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 09:37:23 -0400

>piece of hardware is plugged in. Are there any assumptions for these

The assumption is that running is always valid, so if any client
of any mechanism changes the contents of running, it must be left
in a valid state.  Otherwise, the contract implied by the YANG
module is not being followed.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 15:57:01 +0200

> /wrap/bar does not exist in the data store.

This again depends on the datastore model. But in any case, an immediate
get-config with with-defaults set to "report-all" must return the same
output, right?

What's the answer to the first question in my previous mail?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 16:02:22 +0200 (CEST)

>      the default value for that leaf was explicitly specified."

'default' affects not only edit-config, but other operations and rpc
parameters as well.

And what you say here is not completely correct - it does not have
"the same effect" on the data store, but it does have the same effect
on the "managed application".

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 16:04:15 +0200 (CEST)

>   <nwd:with-defaults>report-all</nwd:with-defaults>

In this case you will get <bar> with the value 42 in the reply.

/martin

---

From: Per Hedeland <[email-masked]>
Date: Mon, 24 Aug 2009 16:06:58 +0200

On 2009-08-24 15:25, Ladislav Lhotka wrote:
> Juergen Schoenwaelder píae v Po 24. 08. 2009 v 15:05 +0200:

Hm, I thought that it had finally been established that a device can
have configuration before the first NETCONF session is started - e.g.
preloaded before delivery (a.k.a. "factory default", where "default" of
course has a very different meaning than the Yang default-statement)
and/or via an initial CLI setup session.

As far as I can see, such an initial configuration can be anything - a
Yang data model has no way to specify restrictions on it beyond that it
must be valid, and this seems perfectly reasonable to me. I also don't
really see the problem - why would a NETCONF client need or want to
treat a "freshly unwrapped" device any different than one that has been
in service for a while (maybe in another network so it's "new" in its
current deployment)?

So, my answer would be "nothing at all".

>> Unless there are other semantics in description statements, there will

Which should mean that if there is some part of the device's operation
that is dependent on the value of <bar>, it should use the default value
- shouldn't it?

>   <nwd:with-defaults>report-all</nwd:with-defaults>

If there is no <bar> instance, it should return <bar>42</bar>.

>> created" and "no value is provided by the NETCONF client".

Actually I think this text is not so good - "the instance" seems to
refer to the leaf with the default value, but the "you can only create

>   <nc:target>running</nc:target>

This creates the <wrap> container, but it doesn't create the <bar> leaf
within it - yet the consensus (+1) is that the default value of <bar>
must now "be in effect". IMO, once <wrap> is created, this is exactly
the same case as a toplevel <bar> with a default value.

--Per Hedeland

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 16:18:06 +0200 (CEST)

> In this case you will get <bar> with the value 42 in the reply.

After reading Per's reply (which I fully agree with), I should add
that I read the question: With this data model, and if nothing has
been set in the data store (no preloaded data etc), then you get the
value 42.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 07:41:21 -0700

>> discussion somehow to an end.

^^2

We need to leave the default-stmt to be MUST implement.
Otherwise it has no value at all to the application
developer.

It seems to me the entire purpose of a default
would be (as Phil suggests) to allow the application
to rely on the default, so that leaf can be
effectively ignored by the operator.

If the meaning of a default leaf changes from MUST to SHOULD,
then operators will never be able to rely on it.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 17:14:01 +0200

> In this case you will get <bar> with the value 42 in the reply.

Does this follow from the statement in Sec. 7.6?

My interpretation is that the MUST only applies if the premise is
satisfied, i.e., if the NETCONF client creates the instance but does not
provide any value.

BTW, the term "instance" is used quite often but AFAIK never defined in
the draft, is it the same as "data node"?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 17:16:11 +0200

> >> discussion somehow to an end.

I am sorry, but since this has implications for the DSDL mapping draft,
I want to make sure I understand it correctly.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 09:21:39 -0700

> satisfied, i.e., if the NETCONF client creates the instance but does not

I do not interpret this text that way at all.
The client is never allowed to provide invalid values
in an edit-config or copy-config.  The server will
reject the leaf with an invalid-value (or whatever duplicate
error-tag the server may pick).

The problem (IMO) is that YANG won't go as far as to say
that leafs with an agent-supplied value actually exist.
This is confusing and perhaps harmful to interoperability.

The server can supply missing configuration nodes at boot-time.
That much is clear (I hope).

   If the leaf has a default value, the server MUST use this value
   internally if no instance of the leaf is provided when an
   instance of the parent data node is created.  A top-level
   leaf node with a default statement is always present.

IMO, this is closer to what we mean.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 12:41:58 -0400

>I want to make sure I understand it correctly.

So let's work backwards:  how different is the a:defaultValue in
RNG from the default statement in YANG?  What are the implications
of the current definition?

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 19:06:41 +0200

> error-tag the server may pick).

Where do I say that the value can ever be invalid?

> That much is clear (I hope).

Yes, but the question is whether the supplied value can differ from the
default value, if one is specified by the data model. My take is that it
can.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 10:12:50 -0700

No.

If there is a default-stmt that applies (via typedef or leaf),
then that is the exact value that MUST be used by the server.

If there is no default-stmt in affect, then the server MAY
create a leaf instance with any valid value for that leaf.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 19:25:41 +0200

> of the current definition?

The annotation attribute is nma:default and is used essentially for
changing the infoset of the instance document by adding the annotated
element with the default value if it is are missing. This must be done
before Schematron validation.

However, if the default statement in YANG had another meaning
(especially one expressed with MUST), the resulting DSDL schemas most
likely won't be able to capture this semantics.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 10:40:18 -0700

> likely won't be able to capture this semantics.

But isn't the problem with mandatory (wrt/ DSDL) independent
of the default-stmt?  Mandatory=true means the contents of a valid
database will contain the mandatory node.  The WG agrees that is
what mandatory=true means.  This is needed to account for
factory initialization and dynamically selected server defaults.

So any schema language that describes the NETCONF PDUs instead of
the contents of the conceptual NETCONF database is not going
to be able to capture this semantics.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 19:49:08 +0200

> create a leaf instance with any valid value for that leaf.

So if the data model is

module foo {

and my friendly vendor preloads the device with configuration

<bar xmlns="http://example.org/foo">13</bar>

the device will be considered non-compliant?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 20:04:58 +0200

> factory initialization and dynamically selected server defaults.

Yes, it is pretty much independent. Mandatory/optional is handled in
RELAX NG and simply checks whether all non-optional elements are present
in a given document. Whether an element is optional or not also depends
on the document type that is being validated.

A problem would be to capture semantics of assigned-by.

> to be able to capture this semantics.

A reply to get-config without any filter will be invalid if any of the
mandatory nodes are missing.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 13:50:14 -0400

>the device will be considered non-compliant?

This is a gray area, since the your friendly vendor could say that
there's a boot time client that did an operation that set the value
of bar explicitly.  If you get the configuration, you will see
<bar>13</bar> so to your app, there's nothing strange afoot.

But if you have delete <bar>, it should not reappear
spontaneous.  Or if you have:

    list foo {
        key name;
        leaf name { ... }
        lear bar { ... default 42; }
        ...
    }

and your app makes a new <foo> and it's <bar> suddenly
becomes set to 13, something it definitely broken.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Aug 2009 20:33:53 +0200

> <bar>13</bar> so to your app, there's nothing strange afoot.

Exactly! And if I plug in a new piece of hardware, the vendor can say
there's a hotplug client that did the operation. But then in practice
such default statements are not binding for the vendor.

> spontaneous.  Or if you have:

A datastore implementation that has the defaults explicitly represented
("report-all" style) will have to re-create <bar> with the default
value.

> becomes set to 13, something it definitely broken.

I agree here.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Aug 2009 20:40:41 +0200 (CEST)

> such default statements are not binding for the vendor.

It is, b/c if you do a get-config, and you do NOT get a value for this
leaf, you KNOW that the server is internally using the value 42.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 11:56:21 -0700

> such default statements are not binding for the vendor.

It is binding.
Phil is pointing out the possibility that
the operator may be able to provide CLI parameters
to the server that cause it to change from the assigned
mandatory default to the desired value.

So in this case, the mandatory default was conceptually
assigned, and then re-assigned to the CLI value.

In the absence of any initial configuration, the default
statement value MUST be the value used by the server.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Mon, 24 Aug 2009 20:54:33 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Monday, August 24, 2009 1:48 PM

> technical problems with the current text?

Martin

Weell, look at how the description starts:

'If a leaf has a "default" statement, the leaf's default value is set
   to the value of the "default" statement.  '  :-)

which is not exactly a definition, and as you see, I think of it as
a SHOULD, the best available wisdom of the data modeller, and
see that as more important than the issues of a manager knowing
what will be set in advance, what it will get on a retrieval with
various different options etc etc.  These are valid considerations,
problems to be solved, but, for me, secondary ones to be
solved in other ways.

(And yes, I do realise that there is more in 7.6 from which I
quote, but that is the first 'definition' of default:-)

The text probably should be strengthened once we have reached
a consensus:
eg  insert before the sentence I quote
  The default value is the value that the server MUST use
   internally if no value is provided by the NETCONF client when the
   instance is created.
and then go on to say where this default comes from,
pace agreement about 'internally' which, like Lada, I am unclear about

Tom Petch

---

From: "tom.petch" <[email-masked]>
Date: Mon, 24 Aug 2009 20:55:15 +0200

----- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "Ladislav Lhotka" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 24, 2009 7:12 PM

> create a leaf instance with any valid value for that leaf.

I have no problem with having this capability in YANG. It meets
a use case and so has a reason to exist, so that manager-implementers
have a ready way to know what is there.

What then does not exist is what Joel so elegantly described
as a SHOULD ie the model writer wants to say
'This value represents our recommended wisdom" so passing this
on to device manufacturers, manager-implementers and all
sorts of users without forcing them to modify products to suit,
eg with code upgrades or non-compliance statements etc.

Rather, this use case has to go in the 'description' substatement.

As Juergen pointed out,
" If we go over board
with constraints in data models, we will kill ourself."
so I am cautious about the (perhaps over-)use of MUST.

Tom Petch

---

From: "tom.petch" <[email-masked]>
Date: Mon, 24 Aug 2009 20:54:51 +0200

----- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "Ladislav Lhotka" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 24, 2009 6:21 PM
Subject: Re: [netmod] meaning of default

Andy

When I read Juergen saying

'My preferred model, as stated before, is the one where the server
makes a distinction between config values that have been explicitely
set and values that are operationally used but not explicitely
configured and thus not part of a configuration datastore'

I read something different, that what is supplied by the server
at boot time is not  then part of configuration - ever.

Tom Petch

---

From: Per Hedeland <[email-masked]>
Date: Mon, 24 Aug 2009 22:19:18 +0200

On 2009-08-24 19:50, Phil Shafer wrote:
> of bar explicitly.

Why would he say such a silly thing, instead of simply explaining that
the device is preloaded with a configuration that (among other,
hopefully specified, values) has the value 13 for the <bar> leaf? Do you
consider the concept of a preloaded configuration to be somehow a
violation of any YANG data model? Or just a preloaded configuration where
some leafs have configured values other than the defaults specified in
the data model?

Yang describes how the operations of a NETCONF client affect the
configuration, and other sources of configuration or configuration
changes are outside its scope - but taking this to mean that no
configuration other than that which has been provided by a NETCONF
client can exist on the device, which this discussion seems to keep
circling around to, has no basis either in the draft or in reality IMO.

I do think the specification of when the server must use the default
value is less than ideal though - not the "use internally", but the
rest. I would like to suggest a change like the one below, which mirrors
the specification of what "mandatory" means. But perhaps this is
overkill, and it could end with "... internally." The (presumably
intended meaning of the) original text is just a special case of "no
value has been set".

OLD:

NEW:

   If the leaf has a default value, and no value has been set, the
   server MUST use the default value internally in the following cases,
   depending on the type of the leaf's closest ancestor node in the
   schema tree which is not a non-presence container:

   o If no such ancestor exists.

   o Otherwise, if this ancestor is a case node, if any node from the
     case exists.

   o Otherwise, if the ancestor node exists.

--Per Hedeland

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 13:27:07 -0700

tom.petch wrote:
....
> at boot time is not  then part of configuration - ever.

huh? what page in the draft says this?

If the client issues a <get-config> operation
on running and with-defaults=report-all, then
the server-supplied values will be there,
so why would you think otherwise?

The use of static default-stmt values are
limited to those cases where there is agreement
on the exact value.  If not, then the description
statement is (by design) where information about
dynamic defaults, or preferred defaults, or
discouraged defaults, will be found.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 25 Aug 2009 00:17:21 +0200

> so why would you think otherwise?

Lets step back for a while and look at config files and what I like
about some of them. Here is what I really like about some config
files: They contain comments indicating settable objects and the
default value that applies. Something simple like this:

# bar=42

The config file tells me that there is a bar and what its default
value is. If I want, I can copy the line and turn this to

# bar=42
bar=42

and bar=42 becomes explicit part of my config. Of course, I can choose
a different value, e.g.

# bar=42
bar=24

and a software update might even lead to this (by updating the
commented default):

# bar=24
bar=24

I do not know about others, but I really love these kind of config
files. They seem to make my life easier.

Emulating this with NETCONF/YANG requires that there is a distinction
between operationally used (default) values and config data. And,
obviously, config data that happens to match the current default value
should not cause any special processing or behaviour - the fact that
the implemented default changed in the example above should not change
the config snippet bar=24 at all.

>From this perspective, the width-defaults draft kind of goes into the
wrong direction. I would need instead a way to retrieve the
operationally used values that are not config data. And it does not
matter at all whether the YANG data model defines a machine readable
default value - what I am interested is what the device operationally
uses; and this in particular applies to objects that do not have a
stable static default value in the data model.

The width-defaults ID assumes that all parameters ultimately are part
of the config and that only some of them happen to be magically
hidden.  I am not convinced this is the right model; I am a big fan of
the explicit config model where stuff not explicitely configured is
simply not part of the config datastore. In other words, the
operationally used values that are not configured live somewhere else
but not as part of the config. In the config file example shown above,
comment characters are used to indicate exactly that.

/js

---

From: WashamFan <[email-masked]>
Date: Tue, 25 Aug 2009 10:09:22 +0800

Hi,
 
>     o Otherwise, if the ancestor node exists.

Or

     If the leaf has a default value, the server MUST use this value
     internally if no value is provided by the NETCONF client when the
     instance is created implicitly.
                               ^^^^^
for simplicity?

washam

---

From: Phil Shafer <[email-masked]>
Date: Mon, 24 Aug 2009 21:57:53 -0400

>comment characters are used to indicate exactly that.

Amen.  One can look at this as two distinct problems:

(a) Before I make a list item, what can I say about the
    leafs that I am not creating?
    ex: If I make an interface, but don't set the duplex,
    will it default to full, half, or auto?

(b) After I create the list item, what can I learn
    about the leafs of the real instance of this item?
    ex: Now that I've made an interface, what is the
    duplex at this point in time?

There are questions about giving feedback to the user of my application
before they create the interface, but there are questions about the
state of the interface that was defined in the configuration data.
The interaction isn't as easy as the with-defaults ID makes it
sound.

With-defaults doesn't address (a) or (b).  I can't do a with-defaults
on a non-existant instance, so the source for (a) has to be the
YANG module.  And doing a with-defaults report-all for (b) will
tell me the YANG default of "auto", not the current value of "full".
"auto" may be interesting to know, but given that I had to write
the code to find it for (a), finding it for (b) is trivial.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Aug 2009 19:50:57 -0700

> the code to find it for (a), finding it for (b) is trivial.

You could design a duplex-admin leaf and
a duplex-operational leaf if you cared so much
about this little corner case.

For every other type of leaf, (B) works fine.
(A) is the same result you get now, without with-defaults.

The advantage of with-defaults=report-all is that it
clearly shows the actual values in use
on a specific agent at a specific time.
That can be really useful in debugging.

Before making changes to the config, it is
often useful to know the current contents of the config.
After making changes to the config, it is
often useful to know exactly what happened.

I agree with Juergen that the best way to really
make progress is to have interoperability test events,
and see if most implementations do things a certain
way, and only 1 or 2 implementations are off in the weeds,
wrt/ defaults.

  leaf foo {
    type string;
  }

I am concerned that there will be many leafs of the type
above.  Telling customers to just look up the CLI documentation
for all the server-provided leafs is fine, if you hate your customers.

Using with-defaults=report-all is a million times less work
for them.  There is also no substitute for a direct answer to
a direct question, at the time the information is needed.

Andy

---

From: Per Hedeland <[email-masked]>
Date: Tue, 25 Aug 2009 08:57:34 +0200

>      instance is created implicitly.

I disagree, on two counts. First, the leaf isn't created at all in the
relevant case, neither implicitly nor explicitly. It is created by
giving it a value, at which point the default value becomes irrelevant.
Second, I specifically wanted to avoid the "operational" description
that seems to be interpreted to mean that the usage of the default value
is dependent on who created what and how. Whether the default value MUST
be used can always be determined by observing the existing
configuration, in a manner that is very similar to how it is determined
that a leaf marked "mandatory" MUST exist.

If a simpler version is wanted, I still vote for:

     If the leaf has a default value, and no value has been set, the
     server MUST use the default value internally.

But that requires the reader to be able to figure out that the value,
default or otherwise, of leafs in non-existing presence containers, list
entries, or choice cases, is irrelevant. I don't know if that is a
realistic requirement.

--Per Hedeland

---

From: Per Hedeland <[email-masked]>
Date: Tue, 25 Aug 2009 09:25:27 +0200

On 2009-08-25 03:57, Phil Shafer wrote:
> With-defaults doesn't address (a) or (b).

It does address (b).

>  I can't do a with-defaults

No, of course report-all will not report the default value for a leaf
that has a different value. IMO this is obvious from the very motivation
of the capability in the introduction section, the need to provide data
that may otherwise be suppressed. Values that are different from the
default value are never suppressed, hence the capability is irrelevant
for those. If still in doubt, see the example.

--Per Hedeland

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Aug 2009 11:43:47 +0200 (CEST)

Hi,

I think this discussion has found three issues that need to be addressed:

  o The presentation issue reported by Tom; first describe what a
    default value is, then how it is defined.

  o We need a more precise definition of *what* it is, i.e. what "use
    internally" means.  
  
  o We need a more precise definition of *when* the default value is
    "used internally".

The *how* text can be kept more or less as is, I think.

The problem is that the description of all these three are
intertwined.  Here's an attempt.  I made a new subsection for this.

7.6.1.  The leaf's default value

   The default value of a leaf is the value that the server uses if no
   value has been explicitly set.  The usage of the default value
   depends on the type of the leaf's closest ancestor node in the schema
   tree which is not a non-presence container:

   o  If no such ancestor exists, the default value MUST be used.

   o  Otherwise, if this ancestor is a case node, the default value MUST
      be used if any node from the case exists.

   o  Otherwise, the default value MUST be used if the ancestor node
      exists.

   In the cases where the default value MUST be used, the server MUST
   operationally behave as is if the leaf was present in the data tree with
   the default value as its value.

   If a leaf has a "default" statement, the leaf's default value is the
   value of the "default" statement.  Otherwise, if the leaf's type has
   a default value, and the leaf is not mandatory, then the leaf's
   default value is the type's default value.  In all other cases, the
   leaf does not have a default value.

And also modify section 7.13.2 (and 7.13.3 and 7.14 similarly)

   If a leaf in the input tree has a default value, the NETCONF server
   MUST use this value in the same cases as described in Section
   7.6.1.  In these cases, the server MUST operationally behave as if
   the leaf was present in the NETCONF RPC invocation with the default
   value as its value.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Aug 2009 14:13:26 +0200

>    o Otherwise, if the ancestor node exists.
Actually, draft-ietf-netmod-dsdl-map-03 defines in Sec. 8.1.2 the term
"implicit node" which tries to capture these conditions, although for
the case node the formulation is different. An implicit node is a leaf
with specified default, a non-presence container containing at least one
such leaf as its child, etc. Using this term, YANG draft could perhaps
formulate the requirement for defaults as follows:

A valid configuration in which an implicit node is missing MUST be
equivalent to another configuration which is identical except that the
implicit node is present and, if it is a leaf, has the default value.

This says for example that for the module

module foo {

the following configurations are equivalent:

1. empty configuration

2. <wrap xmlns="http://example.org/foo"/>

3. <wrap xmlns="http://example.org/foo">
     <bar>42</bar>
   </wrap>

Could this work?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: WashamFan <[email-masked]>
Date: Tue, 25 Aug 2009 21:46:40 +0800

Hi,

----- Original Message -----
From: Per Hedeland <[email-masked]>
Date: Tuesday, August 25, 2009 2:58 pm
Subject: Re: [netmod] meaning of default
To: WashamFan <[email-masked]>
Cc: [email-masked]

> the
> the

OK. That is related the issue whether default leaves belong to the
config. Since majority does not think default leaves belong to the
config. i.e., no default leaves exist in data tree, i.e., no instance at all.
I am fine with that.
 
>  that a leaf marked "mandatory" MUST exist.

But the usage of the default value MUST be defined, I think Martin's 
proposal in another email is good.

washam

---

From: WashamFan <[email-masked]>
Date: Tue, 25 Aug 2009 22:03:33 +0800

Hi,

----- Original Message -----
From: Ladislav Lhotka <[email-masked]>
Date: Tuesday, August 25, 2009 8:17 pm
Subject: Re: [netmod] meaning of default
To: Per Hedeland <[email-masked]>
Cc: [email-masked]

>  formulate the requirement for defaults as follows:

I assume the implicit node refers to node in schema tree, right?
I read the sec 8.1.2, but it did not say how about default leaf defined in 
grouping, is it a implicit node in this case?

>  implicit node is present and, if it is a leaf, has the default value.

I am not sure expressing in this way is legal or not, since default leaf
might never present in the config in most people's POV.

>  2. <wrap xmlns="

I am not sure it is legal or not? Can you explicit a NP container
without setting its children?

washam

>  3. <wrap xmlns="

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Aug 2009 16:22:14 +0200

> I assume the implicit node refers to node in schema tree, right?

No, they are nodes in the data tree that are filled in, if they are
missing, when generating the configuration in "report-all" style. So to
be precise, the last bullet should actually read

o  As an exception to the above two rules, a leaf or container node
   that is defined inside a case of a choice can be implicit only if that
   case is declared as default by using the 'default' statement, see
   Section 7.9.3 in [5].

> grouping, is it a implicit node in this case?

No, only nodes in the data tree come into play, grouping definitions
don't contribute to the data tree.

> might never present in the config in most people's POV.

It doesn't matter, I think. It just says that everything should work
like when the node is present. I know, equivalence of configurations is
also a bit vague, but not worse than "uses internally". The advantage is
that it is expressed in terms of configurations, i.e., statically.

> without setting its children?

It is legal, see Sec. 7.5.7:

A NETCONF server that replies to a <get> or <get-config> request MAY
choose not to send a container element if the container node does not
have the "presence" statement and no child nodes exist.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 07:38:08 -0700

> config. i.e., no default leaves exist in data tree, i.e., no instance at all.

I am not convinced this is the WG consensus.
Will the co-Chairs please confirm.

I believe the WG consensus is that server-supplied values are
actually present.  The server is not required to return
those leafs unless the :with-defaults capability is supported.

> proposal in another email is good.

We are never going to agree that mandatory=true
means the client must set the leaf, because we cannot
agree on a 3rd state like 'server-assigned'.

You cannot observe the current config and know for
certain what server-supplied values will be added
if a new subtree is added to the config.

As Phil pointed out, you can only retrieve data from
the server that is already there.  You have to read
the YANG file, and it may or may not have default-stmt
or description-stmt information that will help.

SMIv2 is not nearly as precise as YANG
in this area, so we should be happy with a big improvement.
YANG has the default and description statements, and
both are optional-to-use.  It is up to the data model
writer to make sure all relevant information is documented
in the YANG module.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 07:45:38 -0700

Andy Bierman wrote:
...
> or description-stmt information that will help.

This is not true if the server supports :candidate
and :with-defaults capabilities.  Then the client
can lock the candidate, create any portion of the config,
and retrieve it with-defaults=report-all.
The operator does not have to carefully read a
stack of YANG modules in this case, just to
find out what the server will do.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Aug 2009 16:50:00 +0200 (CEST)

> I am not convinced this is the WG consensus.

Wait!  The text is written so that it allows the server to handle
defaults in any of the three ways described in the with-defaults I-D.

Do any of you want to change this text now?

> actually present.

server-supplied values is something different, with no formal YANG
support. 

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 08:08:54 -0700

> Do any of you want to change this text now?

There is lots of text scattered throughout the draft.
There is text that says (for must and when expressions)
the set of defaults are included in the server data set.

To me, the litmus test is the expected behavior for
this example:

 container top {
   leaf foo {
     type string;
   }
   leaf bar {
     type string;
     default 'barney';
   }
   leaf baz {
      type string;
      mandatory true;
   }
 }

Client Creates:

  <config>
       <top>
          <baz>wilma</baz>
       </top>
   </config>

Server Contains:
(i.e., this is returned if with-defaults=report-all)

   <config>
       <top>
          <foo>fred</foo>
          <bar>barney</bar>
          <baz>wilma</baz>
       </top>
   </config>

What happens when a <get-config> for a server-supplied
leaf is issued, without any :with-defaults support?

   <get-config>
      <source><running/></source>
      <filter type="subtree">
         <top>
            <foo/>
            <bar/>
         </top>
      </filter>
   </get-config>

We know that that the server can skip over defaults if
the request is for an ancestor node of the default leaf,
but what if the request is directly for the specified leaf?
I think Lada brought up this issue before.

IMO, all servers MUST return the requested data for this filter.

> server-supplied values is something different, with no formal YANG

So both NETCONF and YANG are silent about whether or not
all servers will respond in a consistent manner for this example?
Why is this considered an acceptable level of interoperability?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Aug 2009 17:20:27 +0200 (CEST)

> To me, the litmus test is the expected behavior for

So 'foo' is "server assigned"?

>       <source><running/></source>

I can only speak for <bar/> - if the server does not store defaults
(with-defaults basic mode 'explicit'), bar will not be returned.

As for foo, personally I would like it to be included, i.e. a server
assigned value will always be part of the config, but it all depends
on if/when the WG do 'server-assigned'.

> I think Lada brought up this issue before.

It will not be returned.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 08:34:48 -0700

> So 'foo' is "server assigned"?

yes -- on this server, at this moment.
It is undocumented, so you cannot assume
anything about past or future instances
of the <foo> leaf.

>> I think Lada brought up this issue before.

I do not think this meets the minimum level
of interoperability that an application developer needs
to utilize this standard.  I wonder what the IESG will think.

We never had this behavior with SMIv2 and SNMP.
I can't even imagine how much harder it would
have been on the NMS developers, if get-next
randomly skipped over nodes the user was authorized
to retrieve.

When 4741 was written, the concept that some vendors
would omit portions of the data never came up.
It was assumed the only filters were those defined
in the standard.

The idea that the server can contain configuration data which affects
device behavior, but the client never needs to see it, is incorrect,
and harmful to interoperability.

It may be a goal of the NETCONF and NETMOD WGs to reflect
industry practice and just expose the native API.  That is
fine, but it does not take precedence over the IETF's
goal to achieve real protocol interoperability.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 12:49:15 -0400

>and retrieve it with-defaults=report-all.

Translate that use case into real-world activity.  If my application
wants to display the default value for <mtu> to some GUI user while
he's building an ethernet interface, I need to:

- lock the database
- fake an entry
- re-fetch w/defaults
- discard changes
- unlock

Imho this is unworkable.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 12:52:46 -0400

>It will not be returned.

I agree with Martin, and agree that this is an issue that was decided
long ago.  Let's stop redesigning at last call.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 10:30:38 -0700

> he's building an ethernet interface, I need to:

IMO this is simple, easy to use, deterministic, and already
fully supported when :candidate and :with-defaults
are enabled in the server.  I can't imagine server code
that would know the user was faking an entry, vs.
creating a real scratchpad entry.

This takes an operator all of a few minutes to complete.
Looking up all the relevant vendor documentation,
especially for leafs without YANG default statements,
may not even be possible.  Even if it is found, there
could be a bug in the server, so the documentation is wrong.

Retrieving the actual values in use at a particular moment
is the best way for an operator to discover all the configuration
parameter values affecting server behavior.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 13:19:32 -0400

Andy Bierman writes:
>I wonder what the IESG will think.

My hope is that they will understand what the word "default" means.
It doesn't mean spontaneously appearing values, but values that are
used when no value is given.  If no value is given, then the non-value
isn't part of the configuration, but the system will use the default
value instead of failing.

When I use "ping" with the default packet size, it doesn't rewrite
my command line with that value.  It uses the default value internally
instead of failing with an error saying that it needs a packet size.

When I edit /etc/rc.conf, I only enter the vital information, not
page after page of values I do not care about.  When the system
boots up, no one rewrites /etc/rc.conf to include all the default
values which I do not care about.

When someone configures an interface in JUNOS, they don't need to
specify values for the hundred or so configurables on that interface
which they do not care about.  When they view the configuration, I
show them only what they have configured, not the hundred or so
leafs that they did not.  If I suggested that showing them this
noise was a feature, I would be laughed off their network.

I can't tell you what a bad idea populating default values into the
configuration would be.   I'm shocked we are still on this rat hole.

>and harmful to interoperability.

The client can learn it from the YANG module.  We are expecting
clients to handle this, since the module will also tell the other
vital information like keys, mandatory, ranges, and datatypes.  If
a client is not interested in learning the rules of the road as
spelled out in the YANG module, I don't want that client managing
my box.

Do you believe we are writing YANG with the expectation
that clients will be unwilling to parse and digest the
information in YANG modules?

>goal to achieve real protocol interoperability.

I see no conflict between default values and interoperability.  I
see large conflict between bloated configurations and working
networks.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 10:37:24 -0700

> long ago.  Let's stop redesigning at last call.

I understood the decision incorrectly.
I thought it applied to retrieval of ancestor nodes,
not to explicit retrieval requests.

NETCONF has no explicit 'get' operation like SNMP,
which is an interoperability problem.

I thought an explicit request for a specific leaf instance
would return the value in use, even if it was not set in
some special way that makes a difference to this server's
meaning of the term 'default'.

Otherwise the standards value to an application developer
is (IMO) too low to be approved by the IESG.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 13:29:36 -0400

>This takes an operator all of a few minutes to complete.

This isn't going anywhere, eh?  As Randy Bush was fond of saying
"I invite my competitors to architect their solution this way".

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 10:50:08 -0700

> see large conflict between bloated configurations and working

We are in strong disagreement here.
If the operator explicitly selects a leaf in the <filter>,
or uses with-defaults=report-all, then the operator knows best,
not you (the vendor).

I am not referring to other types of retrieval requests.

IMO, there MUST be a way for an operator to retrieve all
the nodes, even those that have some strange definition
of a default value.  The only way to do that in NETCONF
is to construct a subtree or XPath filter that explicitly
selects the desired leaf.  This is not nearly as good
as the SNMP Get PDU, but oh well.

I don't not think the solution to let every server decide
what a default means, and what nodes are going to be
retrievable, has any standards value to an application
developer attempting to use the YANG and NETCONF standards.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 10:52:12 -0700

> "I invite my competitors to architect their solution this way".

you don't understand that :candidate+:with-defaults
already supports what you call unworkable?

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 13:38:31 -0400

>meaning of the term 'default'.

"specific leaf instance" and "in use" imply that there's something
there, but this is configuration data, not state data.

Try this: Consider that configuration data is instructions from the
user to the system that tell it what to do.  The default value is
the value that is used when a piece of the instructions are not
given by the user.

If I ask the server for a piece of the instructions, I'm asking it
to tell me what I told it to do.  If I didn't ask it to do something,
then there are no instructions and it should tell me that I didn't
tell it anything.  It should not tell me what it might do since I
didn't give it specific instructions.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 11:13:04 -0700

> didn't give it specific instructions.

We do not view the problem space the same way.

When I am debugging, I assume nothing, and trust nothing
until it is verified.  Normally, I don't want to look at
the server-supplied config=true leafs.
But normally, everything works.

By definition, if I'm debugging, then something doesn't work.
If I just assume that every possible setting in the server
is just fine, then I don't know much about debugging,
and could be at it forever, assuming only my settings
could possibly be responsible for the current network problem.

The WG consensus is that this is an optional server
feature (:with-defaults).  So I guess the market will
decide if servers without this debugging feature
are useful or not.  I can't imagine any 3rd party
application developer bothering with NETCONF/YANG at all,
but YMMV.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 13:58:08 -0400

>already supports what you call unworkable?

I think expecting to touch production boxes in non-maintenance
windows to learn default values that are available in YANG modules
would get me a thrashing by my customers.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 25 Aug 2009 14:04:41 -0400

>or uses with-defaults=report-all, then the operator knows best,

When the user explicitly adds a leaf, they are adding
it to their set of instructions.  I treat it as a piece
of vital data.

>developer attempting to use the YANG and NETCONF standards.

I don't recall anyone suggesting that every server can
decide a default means.

Thanks,
 Phil

---

From: "tom.petch" <[email-masked]>
Date: Tue, 25 Aug 2009 19:21:43 +0200

----- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "WashamFan" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, August 25, 2009 4:38 PM
Subject: Re: [netmod] meaning of default

> those leafs unless the :with-defaults capability is supported.

To quote a much-quoted phrase,
"configuration data: The set of writable data that is required to
transform a system from its initial default state into its current
state "

'writable' tells me capable of being written to, even if noone in
the world ever actually writes it.

The alternative view, which keeps popping up, is
'The set of having been written data that has transformed the
system from its initial default state into its current state "

I think that the former prevails, but I am not too sure; I do think
that this will not go away until we do something about it.

I see server-generated values, which are writable, as
part of configuration, perhaps even a writable leaf for which
the server has not generated a value.

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 11:34:39 -0700

> would get me a thrashing by my customers.

1) Leafs without YANG default statements may not be documented
   in the YANG file.

2) This has no impact on the running network whatsoever,
   if the :candidate capability is implemented correctly.

3) It would be the network operator working for your
   customer spending 60 seconds instead of all day discovering
   this information.

4) Using some memory, then discarding the 'changes',
   is an operator decision, not a vendor decision.

I guess this is going nowhere.
I guess I'm the only one who thinks letting every server
define its own undocumented meaning of a default, and therefore its
own set of retrievable nodes for each YANG module,
will impair interoperability for application developers.

Andy

---

From: WashamFan <[email-masked]>
Date: Wed, 26 Aug 2009 02:52:32 +0800

Hi,

>  developer attempting to use the YANG and NETCONF standards.

Isn't that what :with-defaults is for? :with-defaults is introduced
to manage defaults in a consistent way by users regardless of
how servers handle them internally, IMO. Do you suggest
:with-default should be mandatory-to-implement?

washam

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 11:59:46 -0700

> the server has not generated a value.

agreed.
According to the YANG spec, if they were not writable,
then config=false.

Although the default-stmt is allowed with config=false,
so this 'magic filtering' problem has nothing to do
with the config-stmt.

The quote above is a normative reference to RFC 4741.
We agreed that YANG is not redefining RFC 4741, so
we are not going to change the YANG draft to use
its own definition of configuration data (I hope).

I cannot find any definition of a server-supplied default
anywhere in the YANG document.  According to the YANG draft,
only leafs with a default (or typedef-inherited default)
actually have a default value.  (type1 below).

Any other server-supplied leaf (type2 and type3 below) is undocumented.
A server MAY create them, but the YANG spec does not actually say
such nodes exist or not.  The WG agreed that a server
MAY create them, but it is still ignored in the spec.

  leaf type1 {
     type string;
     mandatory false;
     config true/false;
     default 'exact';
  }

  leaf type2 {
     type string;
     mandatory false;
     config true/false;
  }

  leaf type3 {
     type string;
     mandatory true;
     config true/false;
  }

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Aug 2009 22:42:16 +0200 (CEST)

> the world ever actually writes it.

"its current state" tells me that this "set of writable data" changes
depending on the device's current state.  So what the term
"configuration data" means depends on the current state of the box
(which presumably depends on how it is configured...)

I am not sure what exactly you want to read into the quote above.

> part of configuration

So do I.  If we had a statement 'assigned-by system', the server would
actually add the value to the config.  As such, it would always be
reported.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 14:09:47 -0700

> actually add the value to the config.  As such, it would always be

this is much more than an existentialist discussion on server
created leafs.  It has an impact on the protocol as I (and others) have
pointed out:

  - Do must statements apply to server-created leafs?
  - Do min-elements and max-elements constraints apply to
    server created leafs?
  - Do unique statement constraints apply to server-created leafs?
  - Are leafrefs allowed to contain values that exist only
    within a server-created leaf or leaf-list?
  - Are instance-identifiers (where require-instance=true) allowed
    to reference server-created leafs?
  - Do server-created leafs count as being a sibling node when deciding if
    a mandatory leaf is missing?
  - How does a client know which case is selected (if any)
    in an optional or mandatory choice, if only server-created
    nodes exist?
  - Does the nc:operation='create' fail if the client issues
    such an edit-config operation on a server-created leaf?
  - Does the nc:operation='delete' succeed if the client issues
    such an edit-config operation on a server-created leaf?
  - Does an XPath expression (must/when/select) encounter a
    missing-node or some value, if a server-created leaf is
    referenced?
  - Does a subtree filter with a select-node or a content-match
    node for a server-created leaf return any data, or not?

Leaving all these questions unanswered does not meet the
level of interoperability expectations I had for YANG.

Andy

---

From: WashamFan <[email-masked]>
Date: Wed, 26 Aug 2009 10:11:26 +0800

Hi,

----- Original Message -----
From: Andy Bierman <[email-masked]>
Date: Tuesday, August 25, 2009 10:52 am
Subject: Re: [netmod] Configuration nodes was Re: meaning of defau
To: Phil Shafer <[email-masked]>
Cc: "[email-masked]" <[email-masked]>

>  a direct question, at the time the information is needed.

Is the kinda server-supplied data returned when with-defaults=report-all or
explicit or trim? How does :with-defaults see server-supplied data? defaults
or config?

washam

---

From: WashamFan <[email-masked]>
Date: Wed, 26 Aug 2009 10:36:54 +0800

Hi,

----- Original Message -----
From: Andy Bierman <[email-masked]>
Date: Wednesday, August 26, 2009 5:11 am
Subject: Re: [netmod] meaning of config really was meaning of default
To: Martin Bjorklund <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]

>  level of interoperability expectations I had for YANG.

Why these questions are relevant to server-supplied nodes? These
questions are relevant to instances existing in data tree, imo, and the
rough consensus seems server-supplied nodes are in data tree.

washam

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Aug 2009 19:40:04 -0700

> explicit or trim? How does :with-defaults see server-supplied data? defaults

The report-all enum is the only mandatory-to-implement option.
This option returns everything the server contains, and does not filter
out any nodes based on any definition of the term 'default'.

The definition of the term 'all' in 'report-all' is consistent with
the Merriam Webster definition of that word:

  http://www.merriam-webster.com/dictionary/all

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 08:36:38 +0200

> That can be really useful in debugging.

We are in agreement that the retrieval of the actual values in use is
important. But "actual values in use" means operational state not
necessarily config state. In fact, we will have both situations, and
we need to support both.

The first example concerns values that are operationally used like an
interface MTU or the detected speed of an interface. In most cases,
these parameters are not configuration data and we would IMHO make a
mistake if we claim they are config data. On the other hand, we will
have config objects where a manager creates say a list entry and
additional config values will be assigned by the server. A good
example here is creation of a user account where you let the server
pick an unused uid if you do not provide one while setting up the list
entry. But once the uid has been assigned by the server, it pretty
much becomes config data (and so it show up when I do a get-config,
regardless of any with-defaults options).

How we model the distinction with YANG statements needs to be worked
out. But in my preferred approach, I pretty much need a way to
retrieve operational state that I can easily map to config state and
it seems with-defaults is not really getting me there.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 08:43:44 +0200

> he's building an ethernet interface, I need to:

Nice to see that you prefer to write an explicit discard-changes in
example code instead of relying on the unlock side effect. :-)

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 08:43:45 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> out.

Do you believe we need to work this out for YANG 1.0?

Will the introduction of 'assigned-by server' solve the issues people
are seeing?

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 09:31:46 +0200

> Do you believe we need to work this out for YANG 1.0?

Given the intensive discussion here, I think we better work this out
now.
 
> Will the introduction of 'assigned-by server' solve the issues people

The addition of 'assigned-by server' helps to make an important
distinction in the YANG data model and I am in favour of this
approach; I actually believe that marking config nodes the server can
be expected to fill in is much more important that the default
statement itself, which only handles static values and for many use
cases you have to ue the description statement anyway.

I am more concerned about the bigger picture:

* This includes the with-defaults document - I would prefer we can
  settle on the 'explicit' model plus automatically 'assigned-by
  server' leafs - but then I personally would not even need the
  with-defaults extension at all since a get-config always gives me
  the entire config anyway.

* The question how I can effectively retrieve operationally used state
  and relate it _easily_ to config state and how we deal with
  differences in value sets between config state and operational state
  requires some thought. So far, in both the NETMOD/NETCONF WGs, we
  have pretty much ignored this point and perhaps this has been OK so
  far but we are reaching a point where we need to spend some cycles
  on this.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 01:40:34 -0700

>   far but we are reaching a point where we need to spend some cycles

I am concerned about consistent answers to the 11 questions
I posted yesterday.  It seems to me how a particular leaf
got its value is a red herring and totally irrelevant.
All that matters is whether server-supplied values
count as an instance of a leaf (or leaf-list) or not.

It is irrelevant whether we call it config, operational node,
or something in between, if some servers behave as if there is no
leaf present at all, and others behave as if there is a leaf present.

These 'phantom' leafs may count towards a max-elements error (or not).
They may cause a unique-error (or not).  11 different error conditions
that may or may not be reported -- all supposedly from compliant servers.

IMO, the complete lack of any predictable server behavior for these 11
conditions is a problem for interoperability.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 11:34:48 +0200

> got its value is a red herring and totally irrelevant.

I do not think I said this.

> count as an instance of a leaf (or leaf-list) or not.

There are two cases to distinguish, namely the case of a box using a
certain value without the value being part of the explicit
configuration and a server generating explicit configuration as a side
effect of creating say a list instance. Once we area clear about this
distinction, I think everything falls out naturally.

> leaf present at all, and others behave as if there is a leaf present.

As I said, my preference is to adopt the explicit model and adding
something to YANG to make the behaviour one can expect an explicit
data model design decision.

> that may or may not be reported -- all supposedly from compliant servers.

There are no phantom leafs in my preferred approach.

> conditions is a problem for interoperability.

I agree that we should agree on a common way to looking at
configuration and my preference is the explicit approach with server
generated config nodes marked explicitely in the data model (and
perhaps even an explicit description that details how the server is
generating config leafs).

Sure, we will then have WGs discussing when to use this in data models
but this is IMHO better than leaving it to the implementor's choice
since interoperability is our goal.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 11:57:15 +0200 (CEST)

> since interoperability is our goal.

Interoperability is our goal, but another goal is to get good data
models.

If I understand this correctly, you envision something like:

  leaf foo {
    type int32;
    server-assigned "some text";
  }

And then also mandate the explicit mode for defaults.

My concern with this is that people in the explicit camp will do:

  leaf foo {
    type int32;
    default 42;
  }

and people in the report-all camp will do:

  leaf foo {
    type int32;
    server-assigned "MUST set to: 42";
  }

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 12:42:46 +0200

>     server-assigned "MUST set to: 42";

So what are you concerned about - that one uses server-assigned while
the other does not? I agree that this is a design decision that then
needs to be taken for each object.

Or that one does have a default statement and the other does have text
in a textual clause? I am even willing to even give up on default,
especially as when it is used to define default values that are used
operationally but are not part of config (your first example above).

I am not sure what the right syntax of server-assigned or assigned-by
would be - the statement ideally reflects that server assignment only
happens if there is no explicit value configured. So this would be
more like if-server-assigned...

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 13:07:02 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> the other does not?

Yes.  People will ask what the difference is between these two ways of
doing seemingly the same thing.  From a data modelling perspective,
there is no difference, so the data modeller will use one of them,
depending on if he likes the explicit or report-all mode for defaults.
This data model design decision has impacts on the protocol level.

> I am even willing to even give up on default,

This seems to be completely the opposite to what you wrote in 
http://www.ietf.org/mail-archive/web/netmod/current/msg03594.html?

I do not want to remove default.

> more like if-server-assigned...

I think server-assigned or assigned-by system is correct - it means
that the server will assign a value, and then of course the user can
modify it.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 14:48:54 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> On Wed, Aug 26, 2009 at 01:07:02PM +0200, Martin Bjorklund wrote:
> I do not see the impact on the protocol level. I also think the two
> snippets do different things if I assume 'explicit'.

And that is the problem - on some level they do different things, but
for the guy with the domain knowledge writing his YANG module, the
difference is subtle.

Another way of looking at it is that when I write data models, I will
always use the former style b/c I like the explicit mode, but my
friend Balazs who likes report-all, will always use the latter style.
Our models might look exactly the same in all other regards, but this.

> I am all in favour of being able to learn what is actually used like I
> type ifconfig if I have debug something. But still, my interface
> config does not include all the stuff ifconfig shows - and for a good
> reason. I trust implementors to choose sensible defaults.
> 
> Assuming the explicit config model, the default statement in 
> 
>    leaf foo {
>      type int32;
>      default 42;
>    }
> 
> tells me that if there is no config, 42 can be assumed for foo. And it
> only covers the case where there is a fixed stable agreeable value,
> which might be pretty rare. So I can give up on this YANG feature
> since what I want really goes beyond that - I like to retrieve what
> the box uses if I do not specify a value also in cases where there is
> no fixed stable agreeable value or where the value is actually a
> function of some other config/state of the device. I think we do not
> have a good plan how to do this yet.

I mentioned a couple of cases in 
http://www.ietf.org/mail-archive/web/netmod/current/msg03461.html

But just b/c there are more cases, I do not want to give up the static
default value we currently have.  I do think it will be used.

If we look at our favorite ipfix module, they have 13 static defaults,
4 server-assigned (and maybe 2 could be characterized as "dynamic
defaults").

> But I can create a user by sending down a uid to the server when I
> create the user, no?

Yes.  Remember that save / load backups must work.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 26 Aug 2009 09:17:38 -0400

I think that there are two issues mixed together here.

Firstly, there is the question of how one should compose a data model. 
What kinds of things should the operational (non-config) data in a 
netmod data model reflect.  What kinds of knobs show such a model have, 
and how should they be organize, typed, etc.  When should a model use 
groupings..  ad almost infinitum.
This is a topic on which the YANG document ought not speak, in my 
opinion.   In fact, it is probably a topic on which we do not have 
enough information to speak authoritatively.

One aspect of modeling is that one can use an operational field to 
represent "the actual value of X, as determined by the system".  This is 
particularly useful when the actual value used may be different from the 
configured value, but one would want to retain the configured value. 
There almost certainly are other cases where this is useful.
You raise the issue of representing the semantic relationship between 
the operational information item and the configuration information item.
At this time, I don't think we should try to do that.
Firstly, it can clearly be represented in the descriptive text.
Secondly, it is not clear what relationships (because there can be a 
range of these) it is useful to represent programmatically.  The 
difficulty lies in knowing what a client (a managing application) will 
want to do based on this association.

Yours,
Joel

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 14:07:27 +0200

I do not see the impact on the protocol level. I also think the two
snippets do different things if I assume 'explicit'. The first tells
me, you can assume the value 42 is used if there is no explicit
config. The second says if there is no explicit config, the server
will create a config leaf with the value set to 42.

I do not see the impact on the protocol level. The protocol in both
cases does what the data modeler wanted (assuming he knows what he is
doing).

   leaf foo {
     type int32;
     default 42;
   }

But I can create a user by sending down a uid to the server when I
create the user, no? Or is the idea that this keyword would render a
write during creating a failure since only the server can assign or
create this leaf?

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 26 Aug 2009 14:59:58 +0200

> Our models might look exactly the same in all other regards, but this.

This is the point. We either settle on one model and stick to it or we
will have to accept diversity. I am in favour of at least trying to
settle on one model. If we manage to agree on what is config and what
not, we will be able to solve the other questions more easily. If we
fail to settle on one approach, we will keep arguing about default
statements and end up with carefully crafted wordings that allow
different interpretations that will cause long term pain.

> I do not want to remove default.

Fine. I am fine keeping default. What I was trying to get across is
that finding agreement on the config model is for me far more worth
than keeping default.

> Yes.  Remember that save / load backups must work.

But then server-assigned or assigned-by system is not telling the
truth since I can assign by client as well... No, I do not have a good
proposal (yet).

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 15:45:29 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> settle on one model.

If you're talking about if default values are part of the config or
not -- we've been there before.  I don't think we can force one model
onto devices.  If we pick one way, all other devices will look at this
and say they can't implement it.

/martin

---

From: WashamFan <[email-masked]>
Date: Wed, 26 Aug 2009 21:56:18 +0800

Hi,

>  truth since I can assign by client as well... No, I do not have a good

If I understand correctly, mandatory=true nodes MUST be created by
clients except top-level ones. server-assigned nodes could be created
by servers, of cource, clients can always create and modify all config
nodes. defaults nodes have never been created either by clients or servers,
but have values which could be used internally by servers.

washam

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 07:58:18 -0700

> distinction, I think everything falls out naturally.

I disagree.
I don't care if some other operator who worked
here a year ago set some parameter explicitly,
or if the device picked the value on its own.

All that matters is the configuration parameter behavior
in affect when I'm trying to debug a network outage.

Anything I *can* set is configuration,
not anything I've *already* set.
IMO this is a big difference and the source of
the problem with the standard.

Making this an arbitrary choice within the YANG architecture
is really hurting the simplicity and standards value of YANG.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 09:11:07 -0700

> and say they can't implement it.

I think the problem is with-defaults basic behavior = explicit.
All other variables really don't matter.  basic=trim is fine because
the YANG default-stmt serves as a marker to the client that the leaf
instance is really there.

I'm not even sure everyone agrees that all the YANG validation
rules still apply to invisible leafs if the server uses
with-defaults basic=explicit behavior.

IMO, the concept that a leaf instance only exists if the
client provides it is totally broken wrt/ YANG database validation
concepts and mechanisms.

The YANG validation rules need to apply to the entire configuration
database.  If we cannot agree on what 'entire' means, I don't think we have
a workable standard.

If the client cannot retrieve the entire configuration
(except leafs set to the YANG default), then the client
cannot use edit-config, validate, or commit effectively.
The term 'flying blind' best describes this mode of database editing.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 12:43:36 -0400

>instance is really there.

But with a default statement, the leaf instance is not really there.

>with-defaults basic=explicit behavior.

The YANG spec is pretty clear on this point.  Constraints that
refer to nodes will use their default value if the leaf does not
exist.

>client provides it is totally broken wrt/ YANG database validation

I strongly disagree.

>database.  If we cannot agree on what 'entire' means, I don't think we have

"entire" isn't the problem.  The issue you are having is that
you want default values to be populated in the configuration
database as real leaf instances.  I strongly oppose this.

>cannot use edit-config, validate, or commit effectively.

As mentioned many times, the client can use the YANG modules
to learn default values.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 10:09:02 -0700

> As mentioned many times, the client can use the YANG modules

basic=explicit has nothing to do with the YANG default-stmt.
That's the problem.  You are describing basic=trim, and I said in the
first sentence there is no problem there at all.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 12:56:35 -0400

>truth since I can assign by client as well... No, I do not have a good

"assigned-by system" means that the value for this leaf _can_ be
assigned by the system, if one is not provided.  If one is provided,
the system will not assign one and the leaf will be treated like
normal configuration data.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 13:13:55 -0400

>basic=explicit has nothing to do with the YANG default-stmt.

Can you explain this sentence?  My understanding was that
"explicit" meant you were attempting to fetch the config
with a full set of default values.  How can this have
nothing to do with the YANG default statement?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 10:44:14 -0700

> nothing to do with the YANG default statement?

basic=explicit means that the server considers any leaf
not explicitly provided by client to be a default.

basic=trim means the server considers any leaf set
to the schema default value to be a default,
even if the client explicitly set the leaf to the
schema-defined default value.

I consider this inconsistency to be an interoperability problem.

A default is a schema-assigned value when the element
is missing in a valid configuration.  The standard should
decide what a default is, not every server implementation.

There is no way to even discover what the server will do,
unless :with-defaults is supported, in which case report-all
is available, so there is no problem.

Even worse, there is no guarantee that the server will
use the same unspecified behavior across platforms,
across versions, or even across sub-agents within
the same implementation.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 21:45:52 +0200 (CEST)

> I consider this inconsistency to be an interoperability problem.

What exactly is the inconsistency, and how is it an interoperability
problem?

> across versions, or even across sub-agents within

Look at this leaf from the ipfix module:

    leaf sourceIpAddress {
      type inet:ip-address;
      must "../transportProtocol='udp'";
      description "Sets source IP address if UDP is transport
        protocol. If not configured, the IP address assigned to the
        outgoing interface is used.";
    }

Do you want to make this illegal?  Do you think a new YANG statement
is needed to cover this case?   server-assigned would not be
appropriate to use, since it would fix this value, even if the IP
address of the outgoing interface is used.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 21:55:15 +0200 (CEST)

Phil Shafer <[email-masked]> wrote:
> My understanding was that
> with a full set of default values. 

report-all gives you all default values.  explicit gives you only the
values actually configured (i.e. your sw would advertise
basic=explicit)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 13:05:49 -0700

> values actually configured (i.e. your sw would advertise

The with-defaults terms need to be more precice:

OLD:
  o  report-all: All default data is always reported.

NEW:
  o  report-all: All requested server data is always reported.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 13:07:09 -0700

> address of the outgoing interface is used.

I do not want to make any YANG illegal.

I want to be able to retrieve all nodes on all servers
in a simple and consistent manner (i.e., with-defaults=report-all).
I never said I want this to be the only mode, or the default mode;
just an available mode on all servers.

If the server contains nodes which are being considered in any of
the must/when/unique/mandatory/min/max/which-case-is-present/
create-allowed/delete-allowed/select type of database decisions,
then the client needs to be able to retrieve them,
in order to effectively use the NETCONF protocol.

I think the NETMOD WG is trying to codify existing CLI practice
too closely, and that is not entirely standards-worthy.

Sometimes optional means nothing, like when we made all
9 groups of RMON-1 optional, but every RMON vendor implemented
every group anyway.   3rd party developers vote with their feet,
and that takes some time to realize.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 13:17:20 -0700

> What exactly is the inconsistency, and how is it an interoperability

The NETMOD WG does not have a clear definition of
server behavior for server created leafs which are not
the YANG default-stmt value.  The NETCONF server behavior
for configuration retrieval is inconsistent across server
implementations.

This violates the spirit and intent of the NETCONF protocol,
as specified in sec. 1, para 1, sentence 3:

   Applications can use this
   straightforward API to send and receive full and partial
   configuration data sets.

Note the phrase "receive full".
IMO, the NETCONF WG has failed to deliver on
this objective, because every server -- not the standard --
decides what "receive a full configuration data set" means.

I am unclear on the exact solution proposal (assigned-by-stmt?)
that is being proposed.  There have been many partially defined
solutions over several months.  Is there a complete definition
somewhere for review?

Andy

---

From: "tom.petch" <[email-masked]>
Date: Wed, 26 Aug 2009 21:30:16 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Tuesday, August 25, 2009 10:42 PM

> I am not sure what exactly you want to read into the quote above.

Apologies that I am not around to reply more promptly; my modus operandi,
as you will have seen, means I only get to post a small number of times a day or
sometimes only one.

So I have read Andy's posts eg
<AB>

</AB>

and that is a view I agree with.  My quote could be read that way
but perhaps, more likely, it will be read as config data is only
that which would be needed at this instant in time to create
current state from 'default initial state' (which of course has nothing
to do with the 'default' substatement), whether the value comes from
server or client, so what I will actually get as part of config data, or what
will
be used for constraint checking, millisecond by millisecond, is unpredictable as
the server instantiates values or whatever the opposite of instantiate is.

That seems difficult operationally, compared with eg the SMI approach
where there is an OID, which eg, I can fire a get at and get a response.

I like Andy's idea that config data is anything I <can> set, although that might
be a stretch given the wording.

> actually add the value to the config.  As such, it would always be

This leaves me unclear where you stand.  Agreeing with me seems to say
that server-generated values count as config data so who needs
assigned-by-system
to achieve that?  A server-generated value which is part of the
configuration means to me that the server has added the value to the config.

Tom Petch

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 16:16:57 -0400

>  o  report-all: All requested server data is always reported.

But what does "server data" mean?  As opposed to non-server data?

Hmm... on second reading, the definition is considerably worse,
since default values are not configuration data.  How about:

    report-defaults:  Default values are incorporated with the
    data returned.

Hmmm...  is "reported" the wrong term, since APIs return
data, not report it.  Should we use "return-defaults" or
"include-defaults"?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 16:29:25 -0400

>decides what "receive a full configuration data set" means.

The NETCONF and YANG WGs have worked hard to give some flexibility
in terms of how defaults are handled.  I do not see this as a failure
but as a successful effort to deal with the world as it exists,
without trying to force a single world view on every device.

At the end of the day, a client can learn and deal with different
devices in a reasonable manner.  Devices can implement servers and
modules with minimal deviation.  Life is good.

>solutions over several months.  Is there a complete definition

The last WG concensus was that we did not want to include this work
in the current YANG work.  Perhaps a few of us should write this
up as an I-D and see if the concensus still holds.

But IMHO "assigned-by system" is a rare and uninteresting corner
case.  Pretending this casts the entire effort as a failure is
sillyness.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 14:11:22 -0700

> without trying to force a single world view on every device.

The true measure of a standard is interoperability, not flexibility.

> case.  Pretending this casts the entire effort as a failure is

It is not a failure, but IMO it is not that suitable for use
by 3rd party developers who thought they were getting
something really better than before.  We're trading in
the SMIv2 data silos for YANG data silos.  We're trading in
SMIv2 DESCRIPTION clauses for YANG description statements.

It is obvious that automation-based tools are only going
to work with servers that supply them with sufficient
machine-readable and device-retrievable data.

The IETF standards do not need to specify any automation
requirements, or mandate that a server support any sort
of automation. Even the description-stmt is optional in
YANG, so the requirements for specifying data model semantics
are as low as they can go.  Every machine-readable statement
except 'type-stmt' is optional-to-use.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 26 Aug 2009 23:32:59 +0200 (CEST)

"tom.petch" <[email-masked]> wrote:
> that server-generated values count as config data

Exactly.  See also my mail on terminology
(http://www.ietf.org/mail-archive/web/netmod/current/msg03665.html). 

> so who needs assigned-by-system to achieve that?

It has been argued
(e.g. in http://www.ietf.org/mail-archive/web/netmod/current/msg03437.html)
that it is needed in order for the server to behave predictable.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 14:42:08 -0700

> that it is needed in order for the server to behave predictable.

But Joel is confused along with the rest of us:

.....
Maybe my addition will just add confusion, although I beleive I agree with Andy and Juergen. (As a
caveat, I use "means" below to indicate what I think the YANG text explicitly requires.)

1) Mandatory means that if a manager is writing that part of the tree, the manager MUST provide a
value for that item.

2) Default means that if no manager has not set the item, the item will have the explicit value
given in the default clause in the data model.
.....

Neither one of these assertions is correct.
Mandatory means the node MUST exist in a valid database,
and says nothing about client vs. server created.

I'm not sure the WG has a real definition of a default.
I know there are a lot of circular references that end up
saying that a default is whatever the server decides it means.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 15:59:57 -0700

Phil Shafer wrote:
...
> modules with minimal deviation.  Life is good.

Really? The client can learn the operational values that are
in use on the server?  By client, you mean a human who
knows where to look up YANG modules and other documentation?
That counts as retrieving information through the NETCONF protocol?

Imagine an independent NETCONF application that is
started up in a new network.  The application wants to
examine what configuration is already in place for the 'foo'
module, to do its job.

But the server has decided that "the" client already knows
what configuration settings "the" client
has already provided, so it refuses to return those leafs.
Except there isn't just one client, and
it is absurd to expect all applications to keep in synch, instead of
simply returning data the client asked for in a get request.
That's how useful basic=explicit is to client applications.

Let's look at basic=trim now.
Instead of simply answering a direct get request
with the current value of a leaf, the server can decide
never to allow the client to retrieve this leaf,
because the server-selected value MAY be documented
in an off-line file somewhere.  The server wants to
make sure the client does not waste bandwidth.
It is much better if the client just deduces
the current operational state from available documentation.

BTW, there is no mandatory standard way to ever know what a
given server implementation considers to be a node which
is not available for retrieval.  There is a circular definition:
if the server thinks the leaf is a default, then it is a default.

Hey SNMP application developers! Can you imagine if
this this was the way get, get-next and get-bulk PDUs worked?

Andy

---

From: Jon Saperia <[email-masked]>
Date: Wed, 26 Aug 2009 19:18:58 -0400

> the current operational state from available documentation.

If I understand Andy's point here, I think this is an issue of concern  
- to the extent one wants to write effective applications..  If one  
wants to write applications that are effective, they need to know, for  
any system they manage:

What the defaults are for any values/parameters there are in the system.
Are the defaults in use or where they changed as part of the creation,  
or after.
Have the defaults changed.
Knowing if the system supplied the information, it was provided by a  
CLI interaction or by another application is also important.
The 'servers' in this context, not behaving consistently is a concern.

I may have misunderstood, but not meeting these requirements would  
make applications more difficult.

While it is possible to write software to read documentation if  
consistently and well written, but...  It seems like an unnecessary  
and costly complication for the application developer.

/jon

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 17:17:34 -0700

Jon Saperia wrote:
> If I understand Andy's point here, I think this is an issue of concern -
> to the extent one wants to write effective applications..  If one wants
> to write applications that are effective, they need to know, for any
> system they manage:
> 
>    1. What the defaults are for any values/parameters there are in the
>       system.
>    2. Are the defaults in use or where they changed as part of the
>       creation, or after.
>    3. Have the defaults changed.
>    4. Knowing if the system supplied the information, it was provided by
>       a CLI interaction or by another application is also important.
>    5. The 'servers' in this context, not behaving consistently is a
>       concern. 
> 
> 

I don't really understand the relative importance of all
these features.

Which definition of the word 'default' are you using?

> I may have misunderstood, but not meeting these requirements would make
> applications more difficult.
> 
> consistently and well written, but...  It seems like an unnecessary and
> costly complication for the application developer.   
> 

Seems rather costly to me.
Software that finds and reads all the relevant description statements
and vendor documentation and then does useful work without
any human intervention -- sounds expensive.

I don't understand why it is considered a burden
on NETCONF servers to return all the operational data.
This was never considered a burden for SNMP agents, and
managed device resources have increased 20 or 100 fold
since SNMP made that standards decision.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 20:39:29 -0400

>The true measure of a standard is interoperability, not flexibility.

The true way to interoperability is thru flexibility and
capability discovery, Grasshopper.

>machine-readable and device-retrievable data.

YANG supplies them with these.  Yes, there are odd corner cases
that are not included in this first release, but they can be added
via extensions.  YANG offers a large set of descriptive modeling
features in a reasonable small language.

Let's publish and learn where the awkward points are.  I'm willing
to bet that "assigned-by system" is not on the Top Ten list.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 20:49:47 -0400

Andy Bierman writes:
>in use on the server?

Nope.  We dealing with configuration models, not operational
data.  I don't think we're anywhere near messing with
operational data.

>By client, you mean a human who

Clients manipulating data in YANG-based models should know how to
look up data in a YANG module.  This is the minimum bar.  We
are not making YANG so modelers can write models that no one
reads.

>That's how useful basic=explicit is to client applications.

Here we simply disagree, and I'm fairly certain that you aren't
seeing it.  We've stewed on this topic for long enough.  I think
we have concensus.  I think we've hard concensus a number of times.
Let's move on.

Worse case: you are right, the working group is wrong, and the
marketplace will mandate the implementation of the with-defaults
capability.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 18:07:32 -0700

> capability discovery, Grasshopper.

spoken like a true server developer.

> to bet that "assigned-by system" is not on the Top Ten list.

At first, I accepted the premise that
the "server does not store the defaults",
so they cannot be returned.

After a lot more thought and implementation
experience, I think that is utter nonsense.

If the server does not store the default, then it
must know the value some other way, perhaps some
is_value_default() callback.  If the server does not
know the leaf is a default, then it cannot know to skip it
when it is requested in a <get> or <get-config> request.
The sever MUST know the leaf exists if the 'default' leaf is being
considered in YANG validation procedures, and we all agree
that is the case.

So the server is always capable of returning all leafs that
it is using.   All the complexity created for the client
due to 'server flexibility' is unwarranted.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 20:59:20 -0400

>and costly complication for the application developer.

Given the investment in the YANG module, and the valuable content
already there (hierarchy, types, mandatory, keys, unique, must,
etc), the additional cost of learning default values is near zero.

The application needs to know if it needs to make a leaf, needs to
know its parent hierachy, its type definition, and a litany of
constraints put on it by the modeler.  The default value is the
least of the application's concern.  Mixing it (or any meta-data)
into the configuration data is a mistake on par with decorating
the leaf with attributes containing it's other meta-data:

<foo type="string" max-length="64" unique="true" must="not(../goo)">42</foo>

Worse, since you can't differential a default from something
a human user explicitly set.  I count clear up the above by
discarding all attributes, but I don't know enough to discard
uninteresting defaults without discarding human input.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 18:25:26 -0700

> are not making YANG so modelers can write models that no one

Sure -- why use up 10 milli-seconds of the server's precious
resources when you can hire sysadmins that can
spend hours looking up the same information manually.
What a great plan.

> marketplace will mandate the implementation of the with-defaults

perhaps.
IMO, the way YANG and NETCONF deal with defaults is very complicated,
and makes the entire protocol harder to learn and understand.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 21:14:04 -0400

>due to 'server flexibility' is unwarranted.

You are missing my point.  When a user looks at configuration, they
don't want to see the 50 or 100 knobs.  They want to see the ones
they care about.  Interfaces in JUNOS have literally hundreds of
knobs:

% egrep 'attribute .* {' ddl/input/dcd.cnf.dd | wc -l 
     803

(attribute is the JUNOS equivalent of YANG's leaf; dcd is the
daemon that handle interfaces; ".cnf.dd" are the configuration
data model files)

On a typical interface, less than 10 are set.  Throwing hundreds
of defaults for thousands of interfaces is simply not smart.  It's
not a server thing; I can certainly generate a couple of hundred
thousand xml elements.  But it's shipping this much data with values
that are uninteresting to anyone is, well, not smart.

Think about config archival; do I want to archive uninteresting
default data?  Think about user display; do I want to show the
user a hundred knobs they've never heard of?

Default values are not data; they are meta-data.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 26 Aug 2009 21:15:04 -0400

>spend hours looking up the same information manually.

It's in the YANG module.  The application can look it
up in less time than the server.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 26 Aug 2009 19:12:08 -0700

> they care about.  Interfaces in JUNOS have literally hundreds of

no -- we went through this all before.

It is up to the operator to decide
what the operator wants, not the server.
If I send an XPath select="/this/exact/leaf",
then it's because I want that leaf.

If with-defaults was mandatory-to-implement,
then the operator would have the ability
to force the server to return the data it wants,
and let the server set its own default handling
to 1 of 3 specific modes.  The client would be
able to determine the default handling in
every <hello>.  This would be much better
than the current situation.

But I don't know if that would be worth
a new protocol version.  IMO, there should
be 1 'bundled' capability for all new 4741-bis features,
and servers will advertise 4741bis plus 4741,
but 4741bis capability would still be optional.

> Default values are not data; they are meta-data.

disagree on that one too -- IMO, there is nothing
that special about leaf that happens to conform
to some particular definition of a default.
There are just chucks of config that can be diffed
edited, retrieved, etc.

Andy

---

From: WashamFan <[email-masked]>
Date: Thu, 27 Aug 2009 11:46:26 +0800

Hi,
 
> with Andy and Juergen. (As a

But my impression is the majority talks about mandatory things based
on 1) above. 
 
>  saying that a default is whatever the server decides it means.

My perception is, default leafs are not explicitly set, so they are not in
config, but they have the same effect as they were presented in config.

>From a server's view, data could be categorised into 
1) the data that is explicitly set and the value doesn't equal to defaults 
if defaults is provided 
2) the data that is explicitly set to the default values and defaults is provided
3) the data that is not explicitly set and defaults is provided, in this case,
the value equals to defaults, but the data is not in config.

>From a client's view, they can always retrieve 1), they can retrieve
2) if with-defaults=explict/report-all, they can retrieve 3) if with-defaults
=report-all

I can issue with-defaults=trim <get-config>, then get data A
I can issue with-defaults=explicit <get-config>, then  get data B
I can issue with-defaults=report-all <get-config>, then get data C
Well, A is data within 1), B-A is data within 2), C-B is data within 3).

washam

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 09:40:59 +0200 (CEST)

> no -- we went through this all before.

Indeed we did.  And we realized that we will never be able to agree on
one model for default handling.  (The current discussion is evidence
of that).  So the NETCONF WG is chartered to work on how clients can
learn how a server handles defaults.  I don't think we should drop all
this just b/c you now favor the 'report-all' mode.

> then it's because I want that leaf.

And if the operator is used to the 'explicit' mode, he does not want
to see stuff he did not set.  This is no argument for either side.  If
anything it is an argument for :with-defaults which gives the operator
the flexibility to choose.

> a new protocol version.

I still think we *could* make :with-defaults mandatory to implement if
the server implements YANG models.  But I really do not think it is
necessary for interoperability.  The YANG spec is clear that if a
particular leaf is not present in a reply, and it has a default value,
the default value is operationally used by the server.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 10:24:03 +0200 (CEST)

> But my impression is the majority talks about mandatory things based

I agree.  This is what mandatory means.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 27 Aug 2009 11:04:35 +0200

+1

> leaf present at all, and others behave as if there is a leaf present.

YANG actually requires that XPath expressions be evaluated as if the
leaf is present.

> conditions is a problem for interoperability.

Yes, and also the long threads we are now enjoying in this mailing list
prove that this ambiguity is really confusing.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 11:29:52 +0200 (CEST)

[...]

> prove that this ambiguity is really confusing.

I will answer this with the terminolgy I introduced earlier, b/c I
think much of the confusion comes from us using the same words for
different things.

If we talk about assigned-by-system values, there is no ambiguity.
They are proper instances, so the answers to these questions falls out
naturally.  See Phil's email.  The proposal we had for an "assigned-by
system/user" statement formalized them.

If we talk about dynamic defaults, I agree that it is unclear how they
should work in general.  But NOTE -- we don't have any dynamic
defaults in YANG!  However, people are free to define leafs like
"sourceIpAddress" in the ipfix model.  What is the problem with this?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 27 Aug 2009 11:32:43 +0200

> and say they can't implement it.

This is not necessary. In an analogy to the MVC paradigm, YANG spec can
define a single model, and implementations may work with different views
of this model. One view can be identical to the model and such an
implementation has no extra work. An implementor using a different view
will have to (mentally) move back and forth between the view and the
model but the mapping between them is not that difficult.

I guess the situation is similar to two physicists, one of them using SI
units and the other CGS. They can still talk together, read papers
employing either system etc. They only need to exercise extra care.

My preference is that the YANG model be of the report-all style because
this is what XPath expressions assume, but I could live with another
model, too, if it is precisely defined.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 27 Aug 2009 11:53:15 +0200

> think much of the confusion comes from us using the same words for

Yup, RFC 4741 talks about "initial default state", with-defaults draft
has its defaults and YANG as well, but they are all different.
 
> system/user" statement formalized them.

Both Phil and Andy said at different moments - or at least I understood
it so - that if there is a default specified for an assigned-by-system
leaf, the system MUST assign this value. Independently of the fate of
'assigned-by', I would prefer if the 'default' statement was orthogonal
and just postulated that certain configurations mean the same thing.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 11:56:45 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> leaf, the system MUST assign this value.

I'm not sure they did, but I disagree with this.

> Independently of the fate of

I agree.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 27 Aug 2009 12:53:03 +0200

> I'm not sure they did, but I disagree with this.

http://www.ietf.org/mail-archive/web/netmod/current/msg03410.html
http://www.ietf.org/mail-archive/web/netmod/current/msg03579.html

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 08:23:44 -0400

>http://www.ietf.org/mail-archive/web/netmod/current/msg03410.html

  Depending on if the 'default' statement appears inside the model.
  If the model has a default statement and the configuration was
  made using this model, then the device should honor that model's
  default value.

This is very different than the meaning you've given my words above.
My meaning was this: if I make configuration for an interface using
the ACME Interface Model and that model has an "mtu" leaf with a
default of 1024, an interface made using that model should use that
model's default value instead of my normal system model default of,
say, 1500.  Since you used the ACME model, I need to honor the
default value that is defined inside that model, since your application
can reasonably expect that behaviour.

This is completely orthogonal to the "assigned-by system" issue.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 08:26:31 -0400

>"sourceIpAddress" in the ipfix model.  What is the problem with this?

In particular, dynamic defaults are not configuration data, so no
mechanism that fetches configuration data should ever return them.
They should be completely outside this discussion.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 06:08:22 -0700

> this just b/c you now favor the 'report-all' mode.

fine -- I invite application developers to
write code to read description clauses,
because 1 or 2 server vendors refuse to let
them use the <get> and <get-config> code
they already have.  The market will decide
on this one -- rather easily in fact.

IMO, NETCONF is unusable in this 'offline lookup mode',
but if others think it's a good use of their time
to support this sort of device, then good luck with that.

> the flexibility to choose.

You do understand that there is just one bit
devotes to 'set-explicit', right?
You don't get to see just "your changes".
You see whatever explicit operations all operators
have made since the device was purchased.

We used to let the NMS handle this sort of
nonsense -- but there are still some SNMP agents
that reset their counters in the device,
instead of the NMS, because "the" operator
wants the counters reset.

> the default value is operationally used by the server.

I think prospects for interoperability in NETCONF/YANG are extremely
low.

Your client cannot even tell what the server isn't
letting it retrieve without with-defaults.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 27 Aug 2009 15:10:54 +0200

Hmm, so let me also paste in Juergens text that you replied to:

>... If you plug a PCMCIA network card in your notebook, you will see
>a suitable MTU - you did not configure it nor is it going to be the
>same for all possible network cards.

My reading is that if the "mtu" leaf had

  assigned-by system; 
  default 1500;

"should honor" means that I could rely on MTU being configured to 1500
(and get-config and <with-defaults>report-all</with-defaults> should
report the same value).

If this is what you meant, I don't agree.

Lada
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 06:30:28 -0700

> YANG actually requires that XPath expressions be evaluated as if the

I find the YANG concepts to be more confusing because
the rules are not specified as if there exists
a static database representation (as XPath expects).  The
NETCONF protocol is also more complicated because there
is not a 'whole' database available on all devices.

I find the problem premises to be absurd, and contrary to
everything we learned over 20 years of monitoring with SNMP:

  1) the server should decide what the client wants to retrieve

  2) retrieving data that can be deduced from studying
     documentation is a waste of bandwidth

I do not accept these assertions at all, even if the WG does.
The with-defaults capability is a hack, piled on top of more hacks,
to fix a kludged architecture.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 16:30:03 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> fine

Good!  So I guess we can move on now.

Leaving :with-defaults behind us, what is the result from this long
discussion?  Are there any issues around "server-assigned" or "dynamic
defaults" that we need to cover now?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 07:49:31 -0700

> Good!  So I guess we can move on now.

To the next review round anyway.
IMO the NETCONF protocol is not usable on servers
which refuse to allow the client to retrieve the full
configuration datastore, as specified in RFC 4741.

IMO, such servers are not even compliant because RFC 4741
never says anywhere such filtering is permitted.
The NETCONF RFC only recognizes the existence of
a config=true/false filter, and subtree or XPath filters
to select a subset of the whole datastore.  IMO, the NETMOD WG
is not producing a standard consistent with the existing NETCONF standard.

I don't think the NETCONF protocol as you want it to be
is an improvement over SNMP monitoring.  It was never
too much of a burden for SNMP agents to return the
data the manager requested.  It is harmful to interoperability
to support such low-performance, minimal features in NETCONF monitoring.

There is a huge difference between a default mode
of pruning response data, that the client can override,
and a NETCONF server with an arbitrary set of 'write-only'
nodes in its database.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 08:57:43 -0700

> Good!  So I guess we can move on now.

The reason this is a show-stopper for me is that I incorrectly
assumed that a 'get-exact' (e.g., filter for exact leaf) would
return a value.  It was just another annoying YANG-ism I didn't
like before that.

The justification for this filtering is fatally flawed.
The notion that the operator can look up documentation
that describes what the server *should* be doing is somehow
equivalent to using a monitoring mechanism to find out what
the server *is* doing -- is just incorrect.

If this filtering was just limited to YANG default-stmts,
then it would be fine, but it is not.

  leaf ifMtu {
    // this counts as documentation
    type uint32;
  }

  leaf ifMtu {
     type uint32;
     description
        "Server defaults:
           on the x3100 platform this is set to 1500.
           on the x4100 platform this is set to 1514.
           on the x9100 platform this is set to 1518, unless
           the serial number begins with X222, then the
           default is 9000.  If VLAN encapsulation is
           enabled on the interface, then add the appropriate
           number of bytes (refer to user manual) on these platforms.
           On all other platforms, refer to the chart at
           http://www.example.com/defaults/ifMtu";
   }

Imagine the client needs to get the ifMtu value for
every server, as part of its work.  (If you incorrectly
insist that ifMtu is state data, then imagine a
'config false;' statement in these leafs.)

In NETCONF, every server can decide "I have that information
in my vendor documentation so I am not returning it to you".
Or decide "that leaf isn't one of the leafs a previous
session already set, so you don't need to know its value."

So instead of a simple consistent request-response that
works the same on all implementations of the standard,
there is no standard at all.  The client has no idea what the server
will decide to hide and no way to find out.  To the client code,
the server probably appears not to support that leaf at all.

IMO, the easiest solution is to make with-defaults (report-all)
mandatory-to-implement as part of 4741bis.  RFC 4741 should
be deprecated as soon as 4741-bis is published.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 18:11:33 +0200 (CEST)

> then it would be fine, but it is not.

I am just talking about static defaults, i.e. as defined by the
"default" statement.  You are talking about "dynamic defaults", and -
again - YANG does not have any support for dynamic defaults.  The only
thing YANG specifies is that *if* a leaf has the "default" statement
(static defaults), the server may choose to not send the default.

I don't know what makes you say that it also applies to dynamic
defaults defined in description clauses.  AFAIK we haven't really
discussed this before.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 09:27:12 -0700

> defaults defined in description clauses.  AFAIK we haven't really

The draft DOES support any notion of a default
any server implementor thinks up.  I completely disagree
that 'any notion of a default' is somehow meta-data.
The draft DOES NOT limit a server to filtering
just leafs with YANG default-stmts.  That's why
it has no standards value for monitoring.

If YANG ignores this topic -- RFC 4741 does not.
It clearly states that the NETCONF server must be able
to return a full configuration database.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 27 Aug 2009 19:10:02 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> any server implementor thinks up.

Can you provide a reference?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 10:35:54 -0700

> Can you provide a reference?

no -- because there is no text in the draft that
limits the server to filtering only leafs that
contain the YANG default-stmt value.

If that is how YANG is proposing to alter the NETCONF
protocol, then I approve.

But it isn't.  The proposal is to be silent in the standard
and let the server do whatever it wants.

*******************************************************************
Re-work of a previous solution proposal:

    Let's make 'trim' (*) the default behavior, and force servers
    to implement an optional extension (with-defaults or 4741bis)
    in order to use 'explicit' or even mandatory 'report-all'.

    (*) == where 'trim' only applies to leafs containing
           the YANG default-stmt value
********************************************************************

That is what the 'least-astonishment principle' says to do.
The only leaf type that is safe to omit in all conforming
implementations is one with a YANG default.

If the server advertises ':with-defaults?basic=explicit',
then I have no problem with keeping 'explicit', even though
it is not multi-user friendly.   At least a new client
will be warned and an old client is no worse off than it was before.

But allowing the server to pick some unspecified filtering scheme that
the client cannot discover through the protocol
violates even Phil's First Principles of protocol design.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Thu, 27 Aug 2009 20:53:40 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, August 27, 2009 10:24 AM
Subject: Re: [netmod] meaning of config really was meaning of default

> I agree.  This is what mandatory means.

So we do not agree.

Martin, Washam, I think you are saying that a writable leaf with mandatory
substatement
can only have a value set by a client, never by the server/manufacturer side
AND such a leaf need not exist in the data tree,
(Does it have to be NETCONF protocol or would CLI do? fudge that for the moment)
this being the consensus arrived at recently but not yet in the I-D. And, what
the I-D does say, is that such a leaf cannot have a default substatement (and
yes there are also caveats about containers and case)

Andy, I think you are saying that if a writable leaf has a mandatory
substatement
the leaf MUST exist in the data tree, which is how I read the I-D. Also, as the
I-D says, such a leaf cannot have a default substatement (but a value can be set
by server or client).

Lada, I recall you saying you would like to decouple default and mandatory

"Another and IMO simpler option is to decouple mandatory/optional and
default:

1. A leaf is mandatory if it must be present in a valid configuration,
unless it is contained in a container with presence or a choice's case
(these two cases require special handling)."

which sounds similar to Andy.

Tom, I agree with Lada on mandatory (I am not yet into default).

Phil, Juergen, Joel, Per, Balazs, Jon, David, ... I am uncertain where you stand

but I do not see a consensus that mandatory means the first option,
ie that the client/manager must provide the value and that the leaf need not
exist in the data tree.

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 14:25:40 -0700

>> I agree.  This is what mandatory means.

This is because the standard is too loosely defined.
Some may say poorly-specified.  Some prefer
the term 'under-specified'.

A good standard is precisely defined, such that
the MUST, SHOULD, MAY semantics as expressed in RFC 21119
can be easily identified by independent implementors.

I think the 100s of emails on these mandatory and default
topics is clear evidence the draft is not standards track
quality yet.

The requirements for mandatory are in 7.6.4 and they
say only that a mandatory leaf MUST exist.  Period.
There is no normative recognition of client-created vs.
server-created anywhere in yang-07 or 4741bis.

Every single reference to a default in yang-07 refers
to these paragraphs (7.6, para 6 & 7)

There are no other variants of a default anywhere in yang-07 or 4741bis.

There is normative text in 4741 (and 4741bis) that says a server must
be capable of returning a full configuration (note lowercase must though).

I cannot find any normative text in YANG or NETCONF which would lead
anyone to conclude that any other forms of defaults processing
are recognized by the standard.  That means that every and any other form
of defaults filtering is NON-COMPLIANT, AND NOT 'OK'.

The conformance to the :with-defaults capability cannot possibly
alter the contract specified by the 4741 base capability.  That capability
has no support at all for anything other than get/get-config
of subtree filtering. Therefore, all RFC 4741 and 4741bis servers
MUST ONLY use 'with-defaults=report-all'.  The protocol standard does not
contain anything at all that would suggest otherwise.

Perhaps the IESG will be the only ones able to settle this.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 16:25:50 -0400

>If this is what you meant, I don't agree.

My "depending in the .. model" applies to Juergen's "not is it
going to be the same" comment.  If the model has a default value,
then that default should be honored.  This is a distinct issue
from 'assigned-by system'.  In fact, mixing defaults (which tell
you how the system will behave if you don't give a value) with
'a-by system' (which tells you the system will assign a value
if you don't) is pointless.  If you don't assign a value, the
default value tell both you and the sytem how it will behave.
No assignment is needed.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 13:32:45 -0400

>protocol, then I approve.

YANG is not proposing altering the NETCONF protocol.

>That is what the 'least-astonishment principle' says to do.

The "trim" behaviour means you can set a leaf, fetch that leaf, and
get nothing back.  This is hardly least astonishment.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 12:18:42 -0400

>           http://www.example.com/defaults/ifMtu";

Follow you case one step deeper:

    description "defaults:
                    on a type I pic: 1500
                    on a type II pic: 1514
                    on a type III pic: 4096"

Now if I pull a type I pic and insert a type II pic, did my
configuration change?  Heck, no.  This isn't configuration data.
The default value does not belong in the configuration database.
Hot plugging FRUs in my chassis does not change configuration data.

Dynamic defaults are not configuration data.  Can we please move on?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 12:13:28 -0400

>never says anywhere such filtering is permitted.

You are including default values in your definition of "the full
configuration datastore".  YANG does not.  YANG-based servers are
not refusing to allow any configuration data, but they are not
forcing implementations to consider default values as configuration
data.  Claiming this violates 4741 is baseless.

>is an improvement over SNMP monitoring.

NETCONF is meant for configuration, so claiming that we are not an
improvement over SNMP monitoring is pointless.

>It was never
>data the manager requested.

IMHO this was a weakness of SNMP; real data could not be adequately
separated from fake data.  Would you have archived your configuration
by saving a mibwalk of your snmp tree?  SNMP gave two tools
(get,get-next) and one data structure (key=value), which gave
designer and modelers too few choices.  NETCONF and YANG give far
better tools, mechanisms, and features.  Trying to say "but that's
the way we did it in snmp" won't fly.  The "NG" in "YANG" is
supposed to mean something.

>It is harmful to interoperability

Not putting well-known, static default values in the configuration
database is harmful to monitoring?  I can't see any credibility to
this claim.

>and a NETCONF server with an arbitrary set of 'write-only'

Please point me at the discussion of "an arbitrary set of 'write-only'
nodes".   I don't think anyone has mentioned such.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 12:29:41 -0400

>to return a full configuration database.

I'm puzzled that you are mixing monitoring with configuration.  How
do you see them overlapping?  Apps that monitor configuration?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 11:51:22 -0400

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 17:25:51 -0400

>the term 'under-specified'.

Google for "why specs matter".

>topics is clear evidence the draft is not standards track

I can work with Martin to add suitable words to the draft to eliminate
this confusion.  We thought the wording was straight forward and
direct, but if you believe that we were not sufficiently clear,
that can be corrected.

We can also work up a format proposal for "assigned-by system",
since that seems to be the feature you are lobbying for.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 14:41:27 -0700

> Dynamic defaults are not configuration data.  Can we please move on?

You are only proving that the client needs to retrieve this
leaf from the server, whether its config or not.
Any standard that pretends looking this up in the vendor
documentation is better than just returning the stupid number
is a joke.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 14:49:20 -0700

> data.  Claiming this violates 4741 is baseless.

It's OK if we keep the leafs that contain the YANG default-stmt
value out of the database.

Then you agree that every other config leaf is part of the database.
That is what the draft says -- we all agree on that.

The only definition YANG has for a configuration node
is if the config-stmt value is set to true.
We all agree on that.

So, all YANG nodes where the config-stmt is set to true
are part of the database.  Yet we don't seem to agree on that.

Therefore, all leafs that contain a value other than the
YANG default-stmt value MUST be returned by the server.
We don't seem to agree on that either.

I'd like you to point to the normative text in YANG-07 that
supports any other server behavior for defaults besides
omitting leafs containing their YANG default-stmt value.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 17:48:54 -0400

>are part of the database.  Yet we don't seem to agree on that.

"YANG nodes" are not part of the database.  YANG nodes are metadata.
They describe the data that can appear in the database and the
constraints placed against the data.

If my YANG modules says the following:

   leaf foo {
       type string;
    }

and I perform the following operation:

    <edit-config>
      ...
      <config>
         ...
         <foo>yes<foo>
   ...

then I have just added foo to the database.  foo is in the database
with the value of "yes".  If I fetch it, I will get the value "yes".

If I then perform the following operation:

    <edit-config>
      ...
      <config>
         ...
         <foo delete="delete">yes<foo>
   ...

then I have deleted foo from the database.  If is no longer
part of the database.  If I fetch it, it will not be there.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 15:22:40 -0700

> If my YANG modules says the following:

Let's focus on what the standard says.
It clearly says that any instance of a leaf
where config-stmt=true, in which the value
is not set to the YANG-specified default
value, is part of the database.

This has to be true because all these nodes are
part of the database validation procedures.

YANG has no concept of a node which is
config=false if the server sets it but config=true
if the client sets it.  That is non-standard.

I strongly disagree with this interpretation of
a standard NETCONF configuration database.
The draft clearly says that any node statically
defined as config=true is retrievable with
the get-config operation (unless it is set to the YANG default).

I don't see any support in the standard for the
:explicit with-defaults filtering scheme at all.
YANG only recognizes with-defaults=trim.

The with-defaults draft should just add :report-all,
and drop :explicit.  It is not part of the standard
and should stay that way.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 27 Aug 2009 23:16:32 -0400

>YANG only recognizes with-defaults=trim.

The draft says:

    A NETCONF server that replies to a <get> or <get-config> request MAY
    choose not to send the leaf element if its value is the default
    value.  Thus, a client that receives an <rpc-reply> for a <get> or
    <get-config> request, MUST be prepared to handle the case that a leaf
    node with a default value is not present in the XML.  In this case,
    the value used by the server is known to be the default value.

These words are meant to allow both the "explicit" and the "trim"
styles, since the server's removal of data nodes whose value is the
default value is optional.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 27 Aug 2009 21:11:58 -0700

> default value is optional.

This is where I strongly disagree.
The NETCONF WG deferred specification of the content
layer to the NETMOD WG.  The NETMOD WG is developing
that YANG language, which has a specific definition
of a default (7.6, para 5).  That means the standard
NETCONF content layer for YANG modules has a specific definition
of a default.

It also has a specific definition of a mandatory node
and a configuration node.
OK -- now you got me typing ASCII art...

            +------------------------------------------------------+
            |                 <NETCONF server>                     |
            |   +----------------------------------------------+   |
            |   | <config database for validation and XPath>   |   |
            |   |                                              |   |
            |   |  +--------------------+   +---------------+  |   |
            |   |  | <config database>  |   |  <default>    |  |   |
            |   |  |  config true;      |   |  config true; |  |   |
            |   |  |  no default-stmt   |   | value set to  |  |   |
            |   |  |  or value != to    |   | default-stmt  |  |   |
            |   |  |  default-stmt      |   | per sec. 7.6  |  |   |
            |   |  +--------------------+   +---------------+  |   |
            |   |                                              |   |
            |   +----------------------------------------------+   |
            |                                                      |
            |   +------------------+   +---------------------+     |
            |   |  <state data>    |   | <notification data> |     |
            |   |  config false;   |   |   <rpc operations>  |     |
            |   +------------------+   *---------------------+     |
            |                                                      |
            +------------------------------------------------------+

IMO, this is what the standard has specified in it today.
If this doesn't work for some reason, then the correct
action in a SDO is to make the standard say what you mean,
not what you just kinda hope it means.  Stuff not specified in the
standard is non-standard.

If the server can synthesize the value of a leaf for the purposes
of validation and XPath, that it can do the same for the
purposes of retrieval.  How the server returns the
entire <config database> portion is just another implementation
detail that is irrelevant to our standards discussion.

This issue also has nothing to do with monitoring.
It has to do with operator requirements:

>From RFC 3535, section 3:

   2.  It is necessary to make a clear distinction between configuration
       data, data that describes operational state and statistics.  Some
       devices make it very hard to determine which parameters were
       administratively configured and which were obtained via other
       mechanisms such as routing protocols.

   3.  It is required to be able to fetch separately configuration data,
       operational state data, and statistics from devices, and to be
       able to compare these between devices.

   6.  Given configuration A and configuration B, it should be possible
       to generate the operations necessary to get from A to B with
       minimal state changes and effects on network and systems.  It is
       important to minimize the impact caused by configuration changes.

I do not think pruning by any other means than leaf=default-stmt-value
is clearly defined in the YANG draft, and the NETMOD WG is failing
to meet these 3 operator requirements, if the standard permits anything
else.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 01:10:51 -0400

Andy Bierman writes:
>purposes of retrieval.

We've traveled this road before.  The issue isn't that the server
cannot supply default values, it's that it should not supply them.
They are not interesting to most clients, nor are they worth the
effort or bandwidth.  Then you claim that this is needed for clients
that can't read the YANG module, and I reply that we are not writing
YANG modules so that clients can ignore them, and that there are
other important bits of meta-data in the module that any client
will need (keys, mandatory, parents, unique, etc).  Eventually
we say that the with-default capability can handle this if people
want to implement it, so let's let the marketplace decide.

>From RFC 3535, section 3:

(2) is covered by "config true/false".
(3) is covered by <get-configuration>
(6) is covered by xml diff

In 3535, I read a very strong "don't do what snmp did" message.  We
aren't taking a GDMO-like approach.  We are following a simpler,
non-object-based, CLI-like approach.  I've used this approach for
years under JUNOS and it continues to perform well in real-world
scenarios with real-world customers.  I think we hit every point
on every list contained in 3535.

But again, this is a repeat of many previous emails.  We aren't
moving the discussion along and aren't sorting things out or
clarifying issues.  These should all be FAQs by now, and we should
be pointing at the previously published answers.

Thanks,
 Phil

---

From: WashamFan <[email-masked]>
Date: Fri, 28 Aug 2009 14:35:26 +0800

Hi,
 
>  AND such a leaf need not exist in the data tree,

no, that is not I was saying, such a leaf DOES exist in the data tree.

>  (Does it have to be NETCONF protocol or would CLI do? fudge that for 

Except top-level mandatory leafs, all mandatory leafs should be set by
client via NETCONF. top-level mandatory leafs might be pre-loaded or 
initialised via CLI. In this regard, the mandantory definition seems broken,
I admit.

>  yes there are also caveats about containers and case)

No default at all for mandatory leafs, they MUST be explicitly set whatever
server or client did it.

washam

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 28 Aug 2009 08:55:31 +0200

> normal configuration data.

I understand the intended semantics. My point was that the proposed
syntax can be easily misread.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 00:37:12 -0700

> cannot supply default values, it's that it should not supply them.

Again -- it is up to the client to decide what
the client wants to retrieve, not the server.

Again -- documentation for what the server should contain
is not the same as retrieval of the data that the server actually contains.
Documentation is purely optional in YANG anyway.

The operators at the IAB workshop clearly asked
for a retrieval mechanism for all types of NM data.
Nobody at that meeting asked for an opportunity
to practice looking up vendor documentation
as a substitute for a retrieval mechanism.

> (2) is covered by "config true/false".

then why do you think a node is config=false
if the server sets it but config=true if the
client set it?  That is not specified
anywhere in the draft.

> (6) is covered by xml diff

XML diff is incomplete unless you have the
whole database.

The draft has no text at all that defines
your notion of a NETCONF database.
It just has a static mandatory true/false
and a static default-stmt.

> on every list contained in 3535.

You want to take a huge step backward.
We figured out 25 years ago that a consistent
device data retrieval mechanism was far more useful
than "looking up what the number should be" in
some offline documentation.

That's not going back to SNMP.
It's going back to pre-SNMP, pre-SGMP, pre-historic.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 10:28:08 +0200 (CEST)

> contain the YANG default-stmt value.

But a dynamic default means that it is a value that the server
operationally uses if the leaf *is not present in the config*.  So
since it is not present in the config, the server will not return it
in a get-config.  This does not mean that the server filters it out -
how can it filter out something that isn't even there?

> protocol, then I approve.

YANG does not alter NETCONF.

> and let the server do whatever it wants.

Would it help if we add text that says that a server MUST send all
nodes that the client asks for, if he has access, except if the
'default' statement is present on a leaf?

The current text allows servers to do 'trim', 'explicit' or
'report-all'.  I do not want to change that.

> violates even Phil's First Principles of protocol design.

AFAIK, no one has suggested that we should allow servers to do unspecified
filtering mechanisms.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 10:45:20 +0200 (CEST)

> can only have a value set by a client, never by the

No, b/c I want to allow "pre-loaded" data, and other scenarios as well
where the server-side creates data.  What I mean might be easiest
explained with an example:  suppose the list 'foo' has a mandatory
leaf 'bar'.  Now, if the client creates a new foo entry, it MUST
provide a value for 'bar' to make the config valid.  If this is an
edit-config towards running, 'bar' MUST be present in the same
edit-config.  If it was towards the shared candidate, *someone* MUST
set 'bar' before commit.

Now, if we had 'assigned-by-server', and a leaf 'baz' was marked with
this, the client knows that it does not need to provide a value, since
the server will pick one if necessary.   If the server picked one, it
will be present in the config data store, just as if the client set
it.  (If we introduce 'assigned-by-server', we can debate over whether
the server MUST or MAY set a value...)

> AND such a leaf need not exist in the data tree,

Such a leaf absolutely exist in the data tree!

> (Does it have to be NETCONF protocol or would CLI do? fudge that for the

No it is important to get the words right here so we don't exclude
e.g. CLI users.

> yes there are also caveats about containers and case)

This still applies of course.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 04:38:09 -0700

> how can it filter out something that isn't even there?

There is no dynamic default in the YANG standard that I know about.
Please point out which page that is defined.

> YANG does not alter NETCONF.

BS

>> and let the server do whatever it wants.

Where is this specified in the standard.

> 'default' statement is present on a leaf?

yes

> 'report-all'.  I do not want to change that.

No it does not.

The draft defines a default in sec. 7.6.
The draft says everything except this default
is a node in the database (if config=true).

There is no mention anywhere in the standard about
anything related to server-created vs. client-created.
Please point out which section in the draft specifies
this behavior.

> AFAIK, no one has suggested that we should allow servers to do unspecified

I claim that the only behavior included in the
standard is that behavior actually specified in the standard.

You claim YANG supports 'explicit' type defaults filtering,
but there isn't any text in the draft to suggest this is true.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 05:26:03 -0700

Martin Bjorklund wrote:

I want to be precise:

If the draft clearly defined exactly how an 'explicit' filter
is identified in YANG, and exactly how it affects the NETCONF
protocol, then it is not a problem because then it would be specified
in the standard.

The operators at the NM workshop were fairly
clear about 3 types of data (config, state,
and statistics), although these terms are never defined in RFC 3535.

Our model has 2 states (config=true/false).

Maybe the model needs more work.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 05:43:52 -0700

> syntax can be easily misread.

I don't think I understand all the intended semantics.

When the modeling language clearly distinguishes and supports this list,
then it will be ready for prime-time:

   - pure config, such as an ACL
   - combined config and state, such as if-mtu
   - combined config and state, such as static and learned routes
   - pure statistics, such as ifInOctets

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 14:44:30 +0200 (CEST)

> Please point out which page that is defined.

This is leading nowhere.  But see (*) below.

> > YANG does not alter NETCONF.

If this really is what you believe, I suggest you start a new thread
where you identify which portion of NETCONF that YANG alters, and
how.

> Where is this specified in the standard.

Do you realize that you comment on your own statement?

> > 'default' statement is present on a leaf?

(*)  Ok.  If we add something like this, are there still any issues left
(regarding defaults)?

> > 'report-all'.  I do not want to change that.

Here is the proposed updated text on the leaf's default:

   o  If no such ancestor exists in the schema tree, the default value
      MUST be used.

   o  Otherwise, if this ancestor is a case node, the default value MUST
      be used if any node from the case exists in the data tree, or if
      the case node is the choice's default case, and no nodes from any
      other case exist in the data tree.

      exists in the data tree.

   In the cases where the default value MUST be used, the server MUST
   operationally behave as is if the leaf was present in the data tree
   with the default value as its value.

> but there isn't any text in the draft to suggest this is true.

The text above states that the default value is used if no value has
been set.  So 'explicit' is not filtering, since it just means that if
no value has been set, nothing is returned.

'trim' is supported by the text in 7.6.5:

> Please point out which section in the draft specifies

I have never said that the draft talks about server-created vs.
client-created.  Joel pointed out that we need to write text about it,
or introduce something like assigned-by-system.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 28 Aug 2009 15:17:52 +0200

> Maybe the model needs more work.

This is what I believe. Since we do not really know how to relate
operational state to config data, we are getting into trouble to deal
with things like the MTU of an interface - which in most cases is
really just operational state of an interface, but you can still
configure it if there is a need (and in most cases you do not care).
The same goes for the difference between /etc/sysctl.conf and sysctl
-a. For debugging, I need sysctl -a and not so much sysctl.conf, which
I like to keep short. (Sorry for those on Windows. ;-)

/js

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 09:52:56 -0400

(I am not sure how the text below from Martin relates to later notes 
from Martin, but I am sending my comments in case there is still an 
argument for non-instantiated dynamic defaults.)

Assuming that the value (the dynamic default value) under discussion 
matters,
defining a structure for the system in such a way that the managing 
application can not determine what the managed device is doing seems broken.

There are multiple ways to define things that avoid such brokenness.

We can define that the system just does not allow that (dynamic 
defaults).  (I will admit that this seems unlike to be a good choice.)

We can define that when the system uses such a value, it is effectively 
creating the node in the config.  Given that the system inherently does 
know what value it is using, it can provide it.

Or we can describe this "dynamic defaults" case, and say that it should 
be represented by having a pair of nodes, one config=true representing 
what has been configured and one config=false representing what the 
system has chosen to do.  We can leave the relationship between these 
two to the description clause for new.

Personally, it had never occurred to me that one would have invisible 
values in the controlling set of values.  I was disagreeing with Andy 
about some things because I simply could not get my head around the 
description Martin provides below of a value that is set by the system, 
depending upon system situation / characteristics, but is not present in 
the observable set of information.

Yours,
Joel M. Halpern

Martin Bjorklund wrote:
...
...

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 16:05:18 +0200 (CEST)

Hi Joel,

I think this is a good starting point for a discussion about dynamic
defaults. 

As input to the discussion, consider this leaf from the IPFIX module:

    leaf sourceIpAddress { 
      type inet:ip-address; 
      must "../transportProtocol=udp";
      description "Sets source IP address if UDP is transport 
        protocol. If omitted, the IP address assigned to the
        outgoing interface is used.";
    }

I think that this leaf matches my definition of dynamic default.  Do
we want to make this illegal in some way?  Andy previously said that
he did not want to make it illegal.

If we look at Andy's 11 generic questions, in this particular case,
they do not apply (it is not part of a must expression, no unique, and
so on), so it does not seem harmful to have this leaf legal.

/martin

"Joel M. Halpern" <[email-masked]> wrote:
> (I am not sure how the text below from Martin relates to later notes from
> Martin, but I am sending my comments in case there is still an argument for
> non-instantiated dynamic defaults.)
> 
> Assuming that the value (the dynamic default value) under discussion matters,
> defining a structure for the system in such a way that the managing application
> can not determine what the managed device is doing seems broken.
> 
> We can define that the system just does not allow that (dynamic defaults).  (I
> will admit that this seems unlike to be a good choice.)
> 
> creating the node in the config.  Given that the system inherently does know
> what value it is using, it can provide it.
> 
> Or we can describe this "dynamic defaults" case, and say that it should be
> represented by having a pair of nodes, one config=true representing what has
> been configured and one config=false representing what the system has chosen to
> do.  We can leave the relationship between these two to the description clause
> for new.
> 
> Personally, it had never occurred to me that one would have invisible values in
> the controlling set of values.  I was disagreeing with Andy about some things
> because I simply could not get my head around the description Martin provides
> below of a value that is set by the system, depending upon system situation /
> characteristics, but is not present in the observable set of information.
> 
> Yours,
> Joel M. Halpern
> 
> Martin Bjorklund wrote:
> ...

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 10:20:40 -0400

The example you cite is crisply on the edge of the knife.
I find the if_mtu example to be much clearer for understanding how to 
handle things.

But, to use this example, I probably end up saying that it is permitted, 
but for a different reason than you want.  I don't think that is 
actually a dynamic default at all.
As far as I can tell, what that says is that when the packet is going to 
be sent, when the system finally decides what outgoing interface will be 
used, it will pick a source IP address that is suitable.
The reason that that is not a dynamic default is that the semantics are 
such that I really can not ask the system "what value will you use for 
this" because the system really does not know.  There is no "default" 
computed dynamically.  There is an actual value used, when the time 
comes to use a value.

To try to generalize this example, there probably are information items 
which conceptually have a value, and where the value can be set by an 
operator (a management application, a client), but where it is not 
meaningful to ask the system "what is the value for this" when one has 
not been set.  That's fine.  That means the system works without setting 
that value, and other parts of the system will do their job.  (Heck, 
ping variants where the system is expected to pick a random value when 
it sends the message.  I would not expect to be able to ask in advance 
"what random value will you pick?")

In contrast, every interface really has to have an MTU.  If one is not 
provided by the operator, then the system will have to pick one.  Given 
that the system will pick one, the managing system has to be able to ask 
the question "what value did you pick."  If the value is not being 
defined by a default clause, then when the system picks a value it MUST 
appear in the configuration or operational state data.

Yours,
Joel

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 10:13:43 -0400

>protocol, then it is not a problem because then it would be specified

NETCONF does not have 'explicit' filters.  YANG does not add
them.  What YANG says is:

This allows the server to either set leaf elements if the value is
the default value (which the with-default draft calls explicit) or
to choose not to send such elements (which the with-defaults draft
calls trim).

Thanks,
 Phil

---

From: Jon Saperia <[email-masked]>
Date: Fri, 28 Aug 2009 10:35:49 -0400

On Aug 28, 2009, at 10:20 AM, Joel M. Halpern wrote:

> I find the if_mtu example to be much clearer for understanding how  
> to handle things.
>
> But, to use this example, I probably end up saying that it is  
> permitted, but for a different reason than you want.  I don't think  
> that is actually a dynamic default at all.
> As far as I can tell, what that says is that when the packet is  
> going to be sent, when the system finally decides what outgoing  
> interface will be used, it will pick a source IP address that is  
> suitable.
> The reason that that is not a dynamic default is that the semantics  
> are such that I really can not ask the system "what value will you  
> use for this" because the system really does not know.  There is no  
> "default" computed dynamically.  There is an actual value used, when  
> the time comes to use a value.
>
> To try to generalize this example, there probably are information  
> items which conceptually have a value, and where the value can be  
> set by an operator (a management application, a client), but where  
> it is not meaningful to ask the system "what is the value for this"  
> when one has not been set.  That's fine.  That means the system  
> works without setting that value, and other parts of the system will  
> do their job.  (Heck, ping variants where the system is expected to  
> pick a random value when it sends the message.  I would not expect  
> to be able to ask in advance "what random value will you pick?")
>
> In contrast, every interface really has to have an MTU.  If one is  
> not provided by the operator, then the system will have to pick  
> one.  Given that the system will pick one, the managing system has  
> to be able to ask the question "what value did you pick."  If the  
> value is not being defined by a default clause, then when the system  
> picks a value it MUST appear in the configuration or operational  
> state data.

Joel, this is an important part of what I was asking about.  I am not  
sure how people would call it but it [the MTU in your example] must be  
retrievable from the system.
/jon
>> defaults. As input to the discussion, consider this leaf from the  
>> IPFIX module:
>>    leaf sourceIpAddress {       type inet:ip-address;       must  
>> "../transportProtocol=udp";
>>      description "Sets source IP address if UDP is  
>> transport         protocol. If omitted, the IP address assigned to  
>> the
>>> defaults).  (I

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 16:46:23 +0200 (CEST)

Hi Joel,

Here's another example that we have used before:

    leaf hold-time {
        mandatory true;
        type int32;
    }
    list group {
        key 'name';
        leaf name {
            type string;
        }
        leaf hold-time {
            type int32;
            description
               "If not set, the value of ../../hold-time is used";
        }
    }

Would this (group/hold-time) be a dynamic default?  Would it be legal?

/martin

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 10:44:34 -0400

"Joel M. Halpern" writes:
>I would not expect to be able to ask in advance 

Nor after ("what random value did you pick for my last ping?").

>appear in the configuration or operational state data.

For MTU, the system chooses a value when the configuration does not
specify one and no default in specified in the model.  This value
is not configuration data, since it doesn't persist and the hot
plugging of a new interface may lead the system to choose a new
value.  So this "value in use" would clearly be operational data.

We've traditionally said that operational data was the realm of the
great and all-powerful <get> operation, and left it at that.
Currently, nothing is said about what <get> returns or how it works
except for the ominously all-powerful (and unwise) comment in NETCONF
7.7:

   Parameters:
      filter:
         This parameter specifies the portion of the system
         configuration and state data to retrieve.  If this parameter is
         empty, all the device configuration and state information is
         returned.

I hate to think of what such a <get> operation would do to a
fully-populated core router.

But maybe it's time to start defining how <get> works.  Or to create
a new (perhaps less contriversial) operation like <get-operational-data>
that retrieves operational data for YANG modules.

I would be against the "dual leaf" scheme, where "leaf mtu" requires
a config=false twin "mtu-right-now", since this means twice the
work with no relationship between the two leaves.

On the other hand, the set of values supported by config and
operational leafs may not be proper subsets.  "auto" is a fine value
for "duplex" for configuration data, but not for operational data.

My question would be does this issue need sorted out now, or can
we delay it and publish YANG as a solution for configuration data,
since configuration data puts the CONF in NETCONF?  Or are we
required to boil all the oceans at once?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 10:54:01 -0400

>client set it?  That is not specified

Please let me know where I've said anything like this so I can
correct myself.  I have no recollection of saying anything to silly.

>You want to take a huge step backward.

No, I've developed something that people are actually using for
automation in major ISPs every day.  This is a huge step forward,
and was not hindered at all by its CLI-like mechanism of handling
defaults.  This is the style that ISPs are used to.  They like it
and use it daily.  I'm still trying to find a vendor that fully
populates their user-visible configuration data with default values.
I can't imagine operators sitting still for this.

I work hard to make configuration data smaller, simpler, and better
organized, so that users can see the important bits and are not
distracted by the noise.  A config populated with default values
would be worthless.  Hundreds of lines of stuff I don't care about?
The first feature request will be to turn off default values.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 08:14:20 -0700

> to choose not to send such elements (which the with-defaults draft

I already agree that a server does not have to send a leaf
when it is set to its default value.   YANG defines that to
be to value in the default-stmt.

Now that we agree what YANG supports, we can move on right?

The text you quoted supports MY case, not yours.

The text above ONLY refers to the YANG default-stmt.

Leafs that are equal to the YANG default-stmt are defaults.

If we write YANG DEFAULT-STMT, will that be any clearer to you?

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 11:22:42 -0400

Yes it is legal.
I would suggest calling the second hold-time "configured-hold-time", 
largely to avoid naming confusion for humans (but clearly that naming 
change is irrelevant to the question at hand.)

There is an operational subtlety which affects what the model needs to 
represent.
At t0 the hold-time mandatory leaf is 10.
A group is created, an entry, with a name, and no hold time.
So it uses 10 as its hold time.

Now, if someone changes the mandatory leaf value to 20, does the hold 
time for the entry change?  If yes, then what we have is an 
unrepresented semantic relationship, which is inevitable some of the time.
If the answer is no, then we have different situation.  If the hold time 
for a group element is actually set by the system when the entry is 
created, and is set to the then-current mandatory hold-time, and does 
not track the other value, then the system is really creating the 
hold-time in the group list.  If so, when I get that entry later, it 
better come back with the actual value that the system is using for that 
entry.

And I can not tell from your description clause which of the two cases 
you intend.   Which relates to the problem that some folks will always 
write bad models, no matter what we tell them.  But we have to give them 
enough information in the standard that they have a hope of writing good 
ones.

Yes, I think we need to figure out what we mean, and say it clearly. 
Otherwise, the odds are that folks will do the wrong thing.

It is unfortunate but absolutely inevitable that sometimes tool writers 
will have to look at the real semantics, not just the semantics 
represented by the YANG, and write code for that.  We should not 
willfully create cases where that has to happen.  But we should not 
pretend we can completely avoid it either.

Yours,
Joel

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 17:32:09 +0200 (CEST)

> Now, if someone changes the mandatory leaf value to 20, does the hold time for
> the entry change?  If yes, then what we have is an unrepresented semantic
> relationship, which is inevitable some of the time.
> If the answer is no, then we have different situation.  If the hold time for a
> group element is actually set by the system when the entry is created, and is
> set to the then-current mandatory hold-time, and does not track the other
> value, then the system is really creating the hold-time in the group list.  If
> so, when I get that entry later, it better come back with the actual value that
> the system is using for that entry.
> 
> And I can not tell from your description clause which of the two cases you
> intend.

I intended the first case.

So would you call this a dynamic default?

Could you provide an example of something that you would like to make
illegal.

I am trying to understand what sort of changes - if any - we need to
do to the YANG draft.  From you latest emails, it seems as if the
problem is more on the designer of the data model.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 11:45:27 -0400

The case I would make illegal is any case where
the system uses a value for a node if one is not specified
that value is not the default value for the node
that value is not simply a dynamic reference to the current value of 
some other well defined node
and that value can not be accessed by fetching the node itself.

In other words, if the value is not set, and the system is using well 
defined value (not one generated at need and discarded), then there has 
to be a reasonable way to get that value.  Reasonable ways include the 
default statement, and a tractable, even if defined in the description 
clause, reference to another node.  Or getting that node itself.
(If we want to complicate our life, we could define a default-reference 
statement which had a leafref to where that linked default comes from. 
But that is something we can consider for later.)

So, the illegal list includes the second interpretation of your case. 
It also includes the if_mtu when the system sets an "appropriate" mtu if 
one is not provided.

Yours,
Joel

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 08:45:51 -0700

> I like to keep short. (Sorry for those on Windows. ;-)

There is no support in YANG for a node that
changes back and forth between config=true and false.
Since the get and get-config filters are tied to
the YANG config-stmt, this would impact the protocol
behavior.

YANG has no concept of a node that never needs to be
retrieved unless some client session in the past has
already explicitly set the node (possibly to the same
value it already had).

I think Joel, myself and others are saying,
"Ifa leaf instance has a value other than its YANG default-stmt value,
then it MUST be retrievable."

Working backwards a little bit, why do we want this
new classification?  What protocol operations are
we trying to support?  Is it just with-defaults=explicit
(returning only the leafs already set by some client)?

Because this violates the 'must-be-retrievable' requirement
for leafs not equal to the YANG default-stmt value.
What benefit does the YANG data modeler get out of a
new classification?  Same question for the operator?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 09:07:19 -0700

> But that is something we can consider for later.)

I find this to be much less operationally useful
than the standards-value level we have already achieved with
data retrieval through SNMP.

There is no way to ever enforce the concept: 'defined in the
description clause'.  As somebody who has spent so many years
authoring and reviewing MIB modules, I should be happy
with all the trust and faith you people are showing in
external out-of-band user documentation.  In my experience
of writing DESCRIPTION clauses and implementing SNMP agents,
the description clause is rarely complete, and often plain
wrong about some detail or another.

It sounds like you want standards-based application programming
programmers to stop relying on the operational data
the program receives via protocol retrieval from the exact server
instance that is being managed, and instead write software
that parses and analyzes description clauses, written to cover
every instance of every server implementation.

Even though the documentation may be wrong, incomplete,
and the server may have bugs as well.

I am not as trusting as you.
All non-trivial programs have bugs in them.
All non-trivial user manuals have bugs in them.
Just assuming this cannot happen is a non-starter.

> So, the illegal list includes the second interpretation of your case. It
> also includes the if_mtu when the system sets an "appropriate" mtu if
> one is not provided.
> 
> Yours,
> Joel
> 

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 09:28:53 -0700

Hi Joel,

The most important reason that data retrieval within
the protocol is so important is that server implementations
will vary from the documentation in unpredictable ways, over time.

Even for a vendor MIB, the longer a MIB module has been published,
the more platforms have implemented it in slightly different ways.
There may be valid reasons, such as more HW/system resources
to consider than the generalized DESCRIPTION clause considers,
or new HW/protocols/etc. that make the DESCRIPTION clause obsolete
for that platform with that HW installed.

Often, there is no valid reason, just lazy programming or an honest mistake.
But the motive for the bug makes no difference to the client.

Within no time at all, the DESCRIPTION clause is hopelessly incomplete,
and it will stay that way.

There is no reason to believe that NETCONF server implementations
and YANG description statements will be any different.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 28 Aug 2009 18:30:09 +0200

> the YANG config-stmt, this would impact the protocol

Did I say this anywhere? I said "we do not really know how to relate
operational state to config data". Some people believe this can be
solved by making all "relevant" operational state data config data.  I
doubt this works. Think of IP routing tables that contain configured
routes and OSPF routes - how does the data model for this look like?
Think about 802.1 bridges with learned and statically configured
forwarding base entries. We have this all over the place and right now
we have no receipe how to deal with this. While we could duplicate all
definitions (config=true and config=false), I doubt that this solution
will scale well.

> already explicitly set the node (possibly to the same

A config node that does not exist just does not exist. Still
operational state can exist that determines the behaviour of a
device. Why is it so complicated to see the difference?

> new classification?  Same question for the operator?

I find the

  if (value == default_value) skip;

logic not convincing. I am clearly with Phil here. If a client
configures a value, it is part of the config regardless of the value
configured.

But we are not making any progress here.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 09:57:13 -0700

> definitions (config=true and config=false), I doubt that this solution

The oper/admin status or static/learned route
semantics are done in pairs, to avoid the 3rd state you are trying
to formulate.  Or there is an ad-hoc boolean leaf in
the data model to tell them apart.

E.g., the static routes are clearly configuration, and
can be mixed with the learned routes added by the server.
The client and server know in an ad-hoc manner
which will be saved in NV-storage and which will not.

I think you are on the right track for a robust solution,
which is to codify this in a database property somehow.
IMO, this has to be a complete model, that does not
contain any gaps.

> device. Why is it so complicated to see the difference?

I agree with you, but I consider a configuration object
to be a node in which a client can directly alter the value.
An instance of a configuration object can be created by either
the server or the client.  But even if the server sets the
leaf first, it is still config.

I don't like the with-defaults=explicit at all,
because it assumes the description statement is complete and correct.

I also don't like any data retrieval filtering mechanism unless
the client initiates and controls the filtering.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 13:00:22 -0400

>>> the YANG config-stmt, this would impact the protocol

Joel pointed out that my words could have been confusing you.  I
in no way am talking about a leaf that "changes back and forth
between config=true and false".

Consider this example: the "mtu" leaf is configuration data.  If
it's set to a value, that value is part of the configuration data.
If it is not set, the system will use a value for that leaf, but
not set it in the configuration database.  We currently lack a
mechanism for retrieving this "mtu-right-now" value, but I am not
proposing the the "mtu" leaf fade in and out of "config=true"-ness.
 
>because it assumes the description statement is complete and correct.

I don't follow this linkage at all.  Keeping explicitly created
values for configuration data is not linked with description
statements.  This can be implemented for modules that don't
contain any description statements at all.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 13:04:58 -0400

>be to value in the default-stmt.

But YANG also allows me to record that a leaf was set to a value
and return that value even if it is the default value.

If the default value is "3" and the user sets it to "3", then I
will still have "3" in my configuration database and will return
it if an app fetches my configuration data.  If the value is not
set explicitly, then I will return nothing.  This behavior is
permitted under the YANG spec.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 10:38:02 -0700

> proposing the the "mtu" leaf fade in and out of "config=true"-ness.

You are just proposing that a node that is config=true
can be ignored by the <get-config> operation, even if the
client has no way to know the server is going to do this.

The reason that filter='trim' is OK is that it
is transparent to the contract implied by the YANG module.
This is a critical concept.

In essence, omitting a leaf from a retrieval request
that contains its YANG default-stmt value
(which MUST be 1 exact valid value for the data type),
is just a shorthand way of returning a leaf with
its YANG default-stmt value.

This is only possible because the YANG default-stmt is
a MUST implement for servers.

> contain any description statements at all.

exactly -- even worse.

We all agree that only actual instances of a leaf object
matter here.  The whole premise behind filter='explicit'
is that the client does not need to know the server
selected values of these instances because the exact
value the server has selected is described somehow in
the optional-to-use user documentation.

The client is still responsible for knowing all these values
when it comes to editing the database and conforming to all the
must/min/max/unique/create/delete constraints defined for the
instances that are not retrievable.

IMO, that's a big problem.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Fri, 28 Aug 2009 18:18:22 +0200

---- Original Message ----- 
From: "WashamFan" <[email-masked]>
Sent: Friday, August 28, 2009 8:35 AM

> initialised via CLI. In this regard, the mandantory definition seems broken,

I see a further problem if, as I think you are saying, there will be leafs
in the data tree with mandatory=true which only the client can set, to whit,

a) Mandatory true means (at least for some leafs) that the application 
MUST supply the values

b) Therefore the manufacturer cannot ship values for such a leaf in
the initial configuration

c) A device can only advertise, in 'hello', modules with valid values

d) But the device cannot set values for a mandatory leaf and so
a module with a mandatory leaf can never be valid at initial boot.

e) Therefore the device can never advertise a module with a mandatory
leaf in it, and the application can never know that the device supports it.

f) Hence the application can never set a mandatory leaf.

Um

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 10:59:26 -0700

> permitted under the YANG spec.

so what.
You don't have to use the shorthand if you don't want.

IMO, the safest approach is to use report-all with good filters.

It all comes down to how many assumptions the operator
wants to make about network conditions.

When everything is working great, then making lots of assumptions
about the current state of the system is probably OK.

But when reality starts conflicting with one or more assumptions,
you can either choose to pretend the system is still functioning
fine, or you can diagnose the problem, including verification of
your assumptions.  It doesn't matter if you run the HR dept.
at a large corporation or you are the only sysadmin for
your own little home network,  this "systems rule" still applies.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 14:12:24 -0400

Phil,
    I am missing something basic.
    Why, when the system fills in a value that is not the "default", do 
you think it is helpful and important that the value not appear as a 
node in the config tree?  Is your concern that this will result in an 
overcrowded config tree?  (I understand wanting a shorter tree that does 
not have explicit default values.  I understand that such a tree is MUCH 
larger than desired.)
    Yours,
    Joel

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 16:04:47 -0400

"Joel M. Halpern" writes:
>node in the config tree?

I want to make sure we're discussing the same issue, so let's
separate the "defaults are not config data" issue from the "system
fills in a value" issue.

If the system creates configuration data, then that leaf/value is
real configuration data, and it MUST be treated as such.  It will
exist in the configuration database until it is removed.

For example, if my model for users has a uid leaf that the system
will fill in when the application does not provide a value, then
that value will persist and will be there for all clients to see.
The following CLI snippet shows this behavior:

  [edit system login]
  phil@dent# set user mumble class super-user 

  [edit system login]
  phil@dent# show user mumble 
  class super-user;

  [edit system login]
  phil@dent# commit
  commit complete

  [edit system login]
  phil@dent# show user mumble    
  uid 2002;
  class super-user;

  [edit system login]
  phil@dent# 

Note that the uid leaf is not present as configuration data.  This
is the behavior indicated by the non-standardized "assigned-by
system" statement.  In all of JUNOS's hundreds of knobs, only the
uid field and an obscure IDP-related index number work this way.
I think this is a "a-by system" is a rare corner case.  YANG currently
does not handle these cases, but there are other odd corner cases
that YANG does not cover.  While I'd be fine adding "assigned-by
system", leaving it out is not too troubling.

But if I had a default password field of "*" (preventing login), I
do not create a 'password' leaf with a value of "*", because the
code that fetches it from the database will know the default is "*"
and uses that value if there is no password field in the database.
The distinction between an account that does not yet have a password
assigned, and an account that has been marked explicitly by a human
user with a password of "*" (to prevent login) are preserved.

Returning to the "mtu" example, if the mtu varies by the type of
PIC installed in a particular slot, I don't want the default mtu
value for that pic type to be added to my configuration for that
interface.  The default value should be used at run time (if no
value were explicitly configured) and if I change to a different
PIC, that PIC's default value for mtu should be used.  Upgrading
PIC should not require that I delete the previous PICs default
values.

>Is your concern that this will result in an 

Config data is fairly sparse.  Making it dense with default value
increases the signal-to-noise ratio for anyone who deals with
configuration.  It makes it impossible to split "data that I care
about" from "data that I'm uninterested in".

I'm wondering if this is just a chasm between the SNMP world and
the CLI world.  SNMP/GDMO deal with "objects" and fields inside
objects are populated with initial-values and default-values (even
though the default-value for snmp is vague and weak).  I can't
make a C++ object without putting some value into each field.

In constract, the CLI world (and the .conf file world) is more
comfortable with defaults.  There are no objects or structures,
just data.  If I don't provide a field, it defaults in a way that
is very natural, at least to the users in this world.

For example, if there's no "ssh_enable" variable in my /etc/rc.conf,
it defaults to "no".  If there's no "FallBackToRsh" in my .ssh/config,
it defaults to "no".  If "search" field in my /etc/resolv.conf, it
defaults to the domain value.  If I had to provide these default
values for each field in these files, it would be a nightmare.

I agree that there is an issue of learning the values in use "at
the moment", but I think this issue is distinct from both the
"default" issue and the "assigned-by system" one, in that I could
have configured a value and that configured value may be different
from the current operation value of the moment.

For example, I could set "duplex" to "auto" and fetching it would
give me no clue if the interface is current running at full or half
duplex.  This operational mode value may be derived from the
configured (or default) value, but it should not interfer with the
configuration settings.

I hope this helps clear up any confusion.

The important issue to me is that we talked about all this as
far back as the interim meeting in DC, and decided to give
servers the flexibility to suppress default values (as IOS and
IOS clone systems do), express explicit configured values even
if they are identical to the default value (as JUNOS and JUNOS
clones systems do), or fully publish all default values no matter
what (as Andy wants to do in his code).

But this closed issue is being resurrected in a way that is trying
to push all implementations to follow Andy's.  I believe including
defaults on every operation is a poor idea and pushing every
implementation to implement this poor idea will damage YANG's chances
of being accepted in the market place.

I would like to keep the current wording in YANG, add any clarifying
text, and move on.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 22:23:31 +0200 (CEST)

> the optional-to-use user documentation.

You keep saying this, and we keep saying that this is not the way it
works.  The premise is that the client knows the (static) default
value from the "default" statement (if it needs to know it at all).

/martin

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 16:08:49 -0400

>IMO, the safest approach is to use report-all with good filters.

That's your opinion, and you are free to implement this inside your
source code.  But it's not the WG opinion, which is reflected in
the current YANG spec.

YANG gives you the flexibility to take your own path, but it gives
the rest of the world the flexibility to not follow you.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 13:30:56 -0700

> value from the "default" statement (if it needs to know it at all).

if this is true, then why does the 'explicit' mode
of with-defaults just look at whether the leaf was
explicitly set and not the YANG default?

I assume 'explicitly set' means by the
client or the startup file.  If it included
the server, it would be kind of pointless.
A flag for 'set by gamma ray interference' ?

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 16:34:04 -0400

Where I get confused is with the distinction you have below.

You are indicating that the situation where the system uses a value for 
something, that it crafts is not the same as the situation where the 
system assigns a value to the object.  You use the UID example for the 
later, and the MTU example for the former.

The only difference I can see is that the algorithmic rules for 
calculating the values happen to be different.
Which leaves me very confused as to how the behavior required or 
expected by the standard can be different.
Unless we simply make the standard so weak as to be usesless in this area.

Yours,
Joel

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 22:35:45 +0200 (CEST)

"Joel M. Halpern" <[email-masked]> wrote:
> that value is not simply a dynamic reference to the current value of some other
> well defined node
> In other words, if the value is not set, and the system is using well defined
> value (not one generated at need and discarded), then there has to be a
> reasonable way to get that value.  Reasonable ways include the default
> statement, and a tractable, even if defined in the description clause,
> reference to another node.  Or getting that node itself.
> statement which had a leafref to where that linked default comes from. But that
> is something we can consider for later.)

[The example I has earlier actually looked like this in its original
form:

            leaf hold-time {
                type int32;
                tailf:default-ref '../../hold-time';
            }

We discussed adding default-ref to YANG (was it at the interim?) but
we decided to not do it now.  Since we had this functionality in our
old proprietary dml, we added it as an extension.  It is useful in a
few places, but not very common.]

> So, the illegal list includes the second interpretation of your case. It also
> includes the if_mtu when the system sets an "appropriate" mtu if one is not
> provided.

But do you think that we can add words that describe this, and further
that tools can actually verify this?  Is it better to try to state
this in the usage guidelines?

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 28 Aug 2009 22:40:08 +0200 (CEST)

> explicitly set and not the YANG default?

I didn't write that draft - you did ;)  But I assume that it is
supposed to be DML-agnostic?  For me it would be fine to make it
YANG-specific, or at least be very specific that *if* the data model
is written in YANG, it means that it applies to leafs with the
"default" statement that are not explicitly set.

> the server, it would be kind of pointless.

Yes.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 28 Aug 2009 16:44:07 -0400

Yes, I think the relevant distinction can be easily described.

I think the alternative, if it can not be reasonably described, is that 
the config node gets set with the proper value.

Yours,
Joel

---

From: "tom.petch" <[email-masked]>
Date: Fri, 28 Aug 2009 21:45:07 +0200

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
Sent: Monday, August 24, 2009 5:14 PM

Lada,

I never saw a follow up to this.  I think the answer no.  A data node is part of
the schema tree.  I do not see a defined term for a something in the data tree,
rather a reference to eg 'a node in the schema tree that can be instantiated in
a
data tree' so I take (most) references to instance to being such an
instantiation (object classes, object instances, well not really:-) in a data
tree.

A lot of what we are discussing is about this something, when does it come into
existence (something the I-D is very vague about), by what, and does it then
form part of the configuration data tree as opposed to another data tree.
(Andy's response to your post showed a different view to yours on these
matters).

One way forward would be to decouple this something from the value it takes so
that we have the static structure in the data tree that XPath needs but can
still say it has or has not got a value - as we already have 'not a number' in
places - and where that value came from; application, device, .....

Tom Petch

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 16:39:38 -0400

>Unless we simply make the standard so weak as to be usesless in this area.

The distinction is not algorithmic, but specified by the data modeler
when they write the YANG module.  It's about the nature of the leaf
and how the modeler wants it used.

In the case of the uid leaf, the key detail is that the uid leaf
persists in the configuration database, so if I archive and restore,
my users will have the same UIDs.  This works because the uid is
added to the configuration database as a leaf, so it follows the
normal data handling rules.

The mtu leaf will persist only if I set it, since the operational
value for the mtu of an interface is not configuration data.  Nor
would the operational value for the mtu of an interface ever appear
as the "mtu" leaf, since this leaf models the user configured value
for the mtu, not the operational value of the mtu.  This operational
value will be derived from configuration data (if it exists), but
will also use the PIC type (if it matters), platform details
(perhaps), and perhaps a final static default.  But it should not
show up as the value for the "mtu" leaf in the configuration data.

(This all assumes the modeler for mtu does not specify a default
value for the mtu leaf in the YANG module.)

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 16:45:27 -0400

>explicitly set and not the YANG default?

The explicit mode should have logic like:

   if (the user explicitly set it)
       return it

But really it works like:

   if (there is a value in the config database)
      return it  // since the user must have set it

In neither approach are default values inspected.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 28 Aug 2009 16:47:45 -0400

typo!!

Phil Shafer writes:
>Note that the uid leaf is not present as configuration data.

s/not present/now present/

The "uid" leaf is now in the configuration database, since
the system assigned a value to it (2002).

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 28 Aug 2009 23:19:07 +0200

> IMO, this has to be a complete model, that does not

I think our main difference is that I do make a clear distinction
between config data and operational state data. You still seem to
think in terms of "a database".

> leaf first, it is still config.

The problem is that we can both agree on the text but still likely
have read different things. For you, the MTU of my interface is a
config object. For me, it is operational state until I explicitely
configure it (which I never do on the systems I am responsible for;
none of my config files messes around with the MTU because I trust the
kernel to figure out a reasonable value - so I "configure" the system
to not have config data for the MTU).

> I don't like the with-defaults=explicit at all, because it assumes
> the description statement is complete and correct.

Repetition does not make this statement true.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 14:54:44 -0700

> think in terms of "a database".

Because the NETCONF protocol is written that way.
I'm trying to limit the standard behavior to
what is actually specified in the standard.

The YANG draft says that config=true means it works
on <get-config>.  So does 4741.
The protocol already makes this distinction, not me.

> to not have config data for the MTU).

As long as your implementation follows the rules
for config=true/false, then you can define your MTU
leaf any way you want.

If you want the <edit-config> or <copy-config>
operations to have any affect on your leaf, you better
make sure the config-stmt is set to true.

> Repetition does not make this statement true.

Fine.
In my experience, it is faster and less expensive to resolve debugging
issues based on data, rather than assumptions about that data, but YMMY.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 15:41:44 -0700

> In neither approach are default values inspected.

Precisely.
Therefore, it has no relation at all to
the model defined in the YANG draft, which is

  'config + yang-defaults = entire database for validation purposes'

In order for the server to provide the filtering
behavior you are describing, the server MUST support
the :with-defaults capability.

The YANG spec allows either 'trim' or 'report-all'
without the need to support the :with-defaults
capability.  As you pointed out,
the server MAY suppress YANG-defaults, but it does
not have to do that, so a NETCONF client MUST be
able do deal with either of these modes at any time.

If the :with-defaults capability is supported, then
the server can choose basic=explicit, but
it MUST also  honor 'report-all' retrieval requests,
if the client issues any.

This is what the drafts support now.
This is a very acceptable solution to me,
because it allows all available data to be
retrievable on all servers, and allows servers
to use basic=explicit if they want, even though the YANG and
NETCONF standards do not make any client vs. server
distinction.

I am a big fan of unix config files.
I just wish there weren't so many different
semi-structured variants of them.

One more detail...
Can we say that a server MUST (or SHOULD) store its
NV-storage data using the defaults scheme advertised in
its 'basic=xxx' capability?  I think that is assumed,
but it should be at least suggested.  If no with-defaults
support, then the server can choose 'report-all' or 'trim',
and not be bound to either choice (mixed-mode).

Andy

---

From: WashamFan <[email-masked]>
Date: Sat, 29 Aug 2009 11:13:50 +0800

Hi,

>  a module with a mandatory leaf can never be valid at initial boot.

If a mandatory leaf without value can invalidate the config before a client
can get a chance to set it via NETCONF, the leaf MUST be set by the
server or via non-NETCONF and exists in data tree. This is how top-level 
mandatory leafs work.

the mandatory leafs which MUST be set by a client is those ones whose
values MUST be provided when the client issues <edit-config>. e.g., I
<edit-config> to create a P container, then I MUST explicitly set any
mandatory leafs underneath it, otherwise, the <edit-config> fails. (Of
cource, that applied when you directly edit running, when you edit
candidate, you can supply the values till <commit>.)

So, My interpretation of mandatory seems, a client MUST supply the values
of mandatory leafs unless he/she can't get a chance to do it via NETCONF.

washam

---

From: Andy Bierman <[email-masked]>
Date: Fri, 28 Aug 2009 20:54:40 -0700

> server or via non-NETCONF and exists in data tree. This is how top-level 

No -- this is why mandatory top-level nodes don't always work.
The server MAY create a mandatory leaf.  That is 2119-speak
for "the client cannot rely on it".

Most likely, a mandatory leaf is not going filled
in the server, but there is no way to tell.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 29 Aug 2009 06:56:02 +0200

> issues based on data, rather than assumptions about that data, but YMMY.

Debugging requires to look at the operational state (consider a
routing problem when you run OSPF). If I do not understand what my
kernel does, I type sysctl -a. If I conclude I need to change a
setting, I edit /etc/sysctl.conf. And my /etc/sysctl.conf is rather
short.

/js

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 29 Aug 2009 00:58:41 -0400

"Specified by the data modeler" where?
Even the description clauses do not tell me which elements are created 
in the model by the system and which elements are filled in behind the 
scenes, and not recorded in the data model.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 29 Aug 2009 01:04:52 -0400

On the question of why / whether this non-representation is important, 
if I am reading your text correctly, the difference is very much 
dependent upon who this is for.
If the XML response is going to be read by a human being, then keeping 
it very sparse is quite helpful.
On the other hand, if it is going to be read by an automaton, then being 
verbose is actually an advantage, as the odds of the machinery being 
able to work with the content is much higher.

If we really need to allow this distinction as to whether the set values 
appear or not in normal responses (and I am not convinced we do), then I 
end up envisioning having a default statement that says "default 
by-system" and then it would not be returned when defaults are 
suppressed, but would be returned when defautls are delivered.  Ugly.

Given that in your model, these specific values do not appear when 
defaults are delivered, and do not appear when defaults are suppressed, 
it seems to me that this is a different issue from whether defaults must 
be delivered, or can be delivered, or must never be delivered, or are 
included to confuse everyone.

I am not trying to push everyone to follow any particular implementation 
strategy.  I am however trying to end up with something that I thing can 
be useful for management systems.

Yours,
Joel

...

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 29 Aug 2009 13:27:44 +0200

> instantiation (object classes, object instances, well not really:-) in

Yes, we run into it quite often and this "dual life" of nodes also fuels
the confusion. For example, last bullet in sec. 7.6.4 says

  o  Otherwise, the leaf MUST exist if the ancestor node exists.

but if fact it should read

  o Otherwise, the leaf MUST be instatiated if the ancestor is instantiated. 

which is quite awkward.

In other schema languages, this distinction is is much clearer, XSD
defines types ("element E is of type T") and RELAX NG defines patterns
("element E matches P").

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 29 Aug 2009 16:03:39 -0700

> "default" statement that are not explicitly set.

Yes, that is what I want to do:

If the server advertises a YANG module capability URI
for a particular portion of the configuration database,
then the <with-defaults> filtering is applied to
any instances of any objects defined in that portion
of the configuration database as follows:

   trim:      leaf instances set to their YANG default-stmt value
              are omitted in the <rpc-reply>
   explicit:  leaf instances that match the 'server-assigned data'
              definition are omitted in the <rpc-reply>
 report-all:  no instances are omitted in the <rpc-reply>

Also, the definition of a default in YANG may still too
simplistic because it does not account for deviation statements:

module X {

  leaf foo {
    type int32;
    default 10;
  }
}

module Y {
   import Y { prefix y; }

   deviation /Y:foo {
     deviate delete {
       default 10;
     }
   }
}

There is no 'default 10' for the /foo leaf on this server.
Since the deviations for /foo can appear anywhere,
the' deviations=Y' part of the module X capability URI
becomes a critical piece of data for the client.
For this leaf, on this server, the 'trim' mode does
not omit the /foo leaf, because this server does not
implement the default.

There are many places where NETCONF protocol behavior
is defined in terms of a YANG statement value.
I am not sure it is clear that this really means
the synthesis of the YANG statement plus any deviation
statements that apply to the same statement within
a specific YANG object definition.  I hope so.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Sun, 30 Aug 2009 18:35:35 +0200

----- Original Message -----
From: "Phil Shafer" <[email-masked]>
Sent: Friday, August 28, 2009 10:45 PM

Phil

I think that you are absolutely right that this mode should exist; I see it as
a major, if not the major, requirement to emerge from RFC3535. But I
disagree that it is the only requirement; I think that there is another one, an
operational one, I share certainly with Andy.

Enterprise systems could not exist without configuration management; they simply
have too many thousands of devices to configure by any other means.

One key part of this is a three part system, a central store of data for each
device, probably several versions for each, a pull protocol to get data from
a device and a push protocol, such that given a fresh from shrink-wrap,
manufacturer-supplied box, the push will create a replacement operational box.

Box A gets trashed, take a new one off the shelf, push the data to the new box
and the service is restored.

So the pull protocol must get everything that needs to be set to create the
operational box, regardless of which 'user' set it - CLI, SNMP, Netconf,..... -
or when.

I don't think YANG can help much here; it could have a substatement,
read-only, modified <true|false>, but really this is a Netconf protocol
requirement for the device manufacturer to support.

I think it wrong to equate this mode with the one and only meaning of words
such as config.

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Sun, 30 Aug 2009 13:49:29 -0700

> I think it wrong to equate this mode with the one and only meaning of words

I can agree with Phil that 'config' I did not set
is not really config -- if the YANG and with-defaults
draft specify the same thing correctly.

But if it is not config when an instance exists
because the server created it, then it must be
operational state.  So it MUST be retrievable,
in order to support applications (NETCONF was chartered
to get away from screen-scraping).

I am not willing to accept that any leaf instance
(defined as a leaf with a value != YANG default-stmt)
can ever be inaccessible for retrieval.

If the server has a value, and the client asks for it
correctly, it MUST be returned via <get>, <get-config>,
or <copy-config>.  That's what the standard says now, and
I agree with it.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 31 Aug 2009 09:41:01 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> operational state.

I think it is much simpler than this.  If there is a config leaf
ifMTU, and it is not set, but the device operationally uses some value
depending on hw, this does NOT mean that an instance of ifMTU was
created by the server.  It is a separate thing (which can be handled
today by having a separate ifMTU-oper leaf, as has been said before).

So "if an instance exists because the server created it" then it *is*
config.   The example we have used is the user's uid.  If it is not
provided, the server fills in a value, and this value is part of the
config data store, just as if the client set the value in the first
place.

> So it MUST be retrievable,
> or <copy-config>. 

But look at Phil's "duplex" example.  If it is set to 'auto', it
exists in the config, and this is the value that will be returned in a
reply to <get> or <get-config>, even though the operational value is
"half". 

To extend the example, suppose its default value is "auto", and it
does not exist in the config.  The operational value is "half", but
the best you can get in a reply is still "auto".

So I don't think that what you propose above actually solves the
problem with operational data.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 31 Aug 2009 01:16:58 -0700

> the best you can get in a reply is still "auto".

this corner case is a red herring.
One can always design the knob differently,
or use an oper/admin pair.

> problem with operational data.

I am not suggesting that the email I referenced in any way
represents a solution proposal.  I am suggesting that
config = 2 states is not complete, but it's what
the standard has right now.  I can live with that,
as long as all servers stick to the YANG definition
of a database.  There will be corner cases in which
the server will be forced to use 2 leafs (oper/admin)
or forced to return server-assigned leaf instances because
the server is actually using some value built
into the code somehow.

NETCONF/YANG is not useful enough for the client unless all the
standard operations apply to all the YANG clauses in
a tightly specified and deterministic manner.

It doesn't really matter that much what the proprietary CLI world
uses for humans to manage devices, because we need
a standard that supports an application programmatic interface,
and that is why NETCONF was chartered.

Andy

---

From: David Partain <[email-masked]>
Date: Mon, 31 Aug 2009 11:26:16 +0200

> >discussion somehow to an end.

I completely agree.  I'm trying to read through huge heaps of email to try to 
figure out what the right question to ask is.  Stay tuned.

David