Subject: [netmod] capabilities related edits

From: Andy Bierman <[email-masked]>
Date: Thu, 03 Sep 2009 19:38:04 -0700

Hi,

Here are the requested edits for yang-07, 5.6.4:

5.6.4.1:  (replace some text in entire section)

OLD:

   Devices indicate the names of supported modules via the <hello>
   message.  Module namespaces are encoded as the base URI in the
   capability string, and the module name is encoded as the "module"
   parameter to the base URI.

   Modules that do not contribute any data definitions, rpcs,
   notifications, or deviations to the device MAY be advertised in the
   <hello> message.

NEW:

   Devices indicate the names and revision dates of supported modules
   via the <hello> message.  The module namespace MUST be encoded
   as the base URI in the capability string.  The module name MUST be
   encoded as the "module" parameter, and the most recent revision-date
   MUST be encoded as the "revision" parameter, to the base URI.

   All revisions of all modules used by the server MUST be advertised
   in the <hello> message.

5.6.4.2, para 1: (add new last sentence)

OLD:

   Devices indicate the names of supported features via the <hello>
   message.  In hello messages, the features are encoded in the
   "features" parameter within the URI.  The value of this parameter is
   a comma-separated list of feature names which the device supports for
   the specific module.

NEW:

   Devices indicate the names of supported features via the <hello>
   message.  In hello messages, the features are encoded in the
   "features" parameter within the URI.  The value of this parameter is
   a comma-separated list of feature names which the device supports for
   the specific module.  All features supported by the server MUST be
   advertised in the <hello> message.

5.6.4.3, para 1: (add new last sentence)

OLD:

   Device deviations are announced via the "deviations" parameter.  The
   value of the deviations parameter is a comma-separated list of
   modules containing deviations from the capability's module.

NEW:

   Device deviations are announced via the "deviations" parameter.  The
   value of the deviations parameter is a comma-separated list of
   modules containing deviations from the capability's module.
   All deviations used by the server MUST be advertised in the
   <hello> message.

7.1.9, para 1: (s/SHOULD/MUST)

OLD:

   The "revision" statement specifies the editorial revision history of
   the module, including the initial revision.  A series of revisions
   statements detail the changes in the module's definition.  The
   argument is a date string in the format "YYYY-MM-DD", followed by a
   block of substatements that holds detailed revision information.  A
   module SHOULD have at least one initial "revision" statement.  For
   every published editorial change, a new one SHOULD be added in front
   of the revisions sequence, so that all revisions are in reverse
   chronological order.

NEW:

   The "revision" statement specifies the editorial revision history of
   the module, including the initial revision.  A series of revisions
   statements detail the changes in the module's definition.  The
   argument is a date string in the format "YYYY-MM-DD", followed by a
   block of substatements that holds detailed revision information.  A
   module MUST have at least one initial "revision" statement.  For
   every published editorial change, a new one SHOULD be added in front
   of the revisions sequence, so that all revisions are in reverse
   chronological order.

7.18.3, para 4: (s/SHOULD/MUST)

OLD:

   Device deviations are strongly discouraged and SHOULD only be used as
   a last resort.  Telling the application how a device fails to follow
   the standard is no substitute for implementing the standard
   correctly.

NEW:

   Device deviations are strongly discouraged and MUST only be used as
   a last resort.  Telling the application how a device fails to follow
   the standard is no substitute for implementing the standard
   correctly.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 4 Sep 2009 09:08:36 +0200

>    of the revisions sequence, so that all revisions are in reverse

Should "editorial change" not have been "non-editorial change"?
Affects OLD as well as NEW.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Phil Shafer <[email-masked]>
Date: Fri, 4 Sep 2009 02:57:59 -0400

>   All revisions of all modules used by the server MUST be advertised

-1, since advertising multiple copies of the same module which are
only need for other modules (groupings, types, etc) is confusing.

-1, since we support revision-less modules.

-1, since MUST should not be applied like paint.  "namespaces are
encoded" is sufficient.  2119 says "used with care and sparingly".

>   advertised in the <hello> message.

So if my code supports a feature that I don't want to advertise,
then I'm in violation of the spec?  Do we need to debate the term
"supports"?  Isn't the act of announcing I support feature X enough?
If I don't announce a feature, how can I claim to support it?

>   All deviations used by the server MUST be advertised in the

This one's odd, since the deviation means the server admits that
something is not implemented according to the contract of the module.
Requiring all deviations to be announced means that the servers
not announcing their broken-ness are broken.  True, but is it helpful?

>   of the revisions sequence, so that all revisions are in reverse

-1, since this issue was already decided by the WG.

>   the standard is no substitute for implementing the standard

"MUST" implied you are in violation of the YANG spec if you used a
deviation but it wasn't really your last resort.  How is this
helpful?  How will we know that this really wasn't your last resort?
Doesn't using "MUST" imply some sort of real rule that it at
least vaguely enforcible?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 00:33:18 -0700

Phil Shafer wrote:

Unless the replacement for data retrieval
of YANG defaults is deterministic, it is not
good enough to achieve interoperability,
or measure interoperability for standards-track advancement.

Even if the IESG does not care about the
complexity, they might care about the inability
to measure interoperability for RFC 2026 purposes.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 04 Sep 2009 09:59:57 +0200 (CEST)

> least vaguely enforcible?

I agree with Phil's reasoning, and further I think we should do
s/SHOULD/should/ - also SHOULD implies a rule that at least can be
verified.

/martin

---

From: Per Hedeland <[email-masked]>
Date: Fri, 04 Sep 2009 12:30:23 +0200

> encoded" is sufficient.  2119 says "used with care and sparingly".

+1 to that -1 (= -2?) Wholesale replacement of is/are with MUST in text
that amounts to basic description of a protocol only makes it harder to
read and dilutes the MUST. It's simply pointless if there isn't some
alternative that an implementation might reasonably choose if the MUST
is absent.

--Per Hedeland

---

From: Per Hedeland <[email-masked]>
Date: Fri, 04 Sep 2009 12:33:21 +0200

> s/SHOULD/should/ - also SHOULD implies a rule that at least can be

I agree with both of you, but offer s/SHOULD/must/ as a possible
improvement.

--Per Hedeland

---

From: David Partain <[email-masked]>
Date: Fri, 4 Sep 2009 13:54:58 +0200

> -1, since we support revision-less modules.

Would acceptance of 
http://www.ietf.org/mail-archive/web/netmod/current/msg03920.html
change that last -1?

David

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 04:51:06 -0700

> I agree with both of you, but offer s/SHOULD/must/ as a possible

No -- this is not going to solve the problem.
Avoiding 2119 language just adds more confusion.

I hope it is clear to the IESG that a deterministic
mechanism, such as direct data retrieval, is needed to
detect data model compliance.

There are about 10 ways that the 'rock-solid' YANG default-stmt
is not deterministic --> 'not reliable', --> 'not always correct',
--> 'sometimes wrong'.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 04 Sep 2009 14:21:30 +0200 (CEST)

> http://www.ietf.org/mail-archive/web/netmod/current/msg03920.html

No, but maybe acceptance of that + this text would work:

  Devices indicate the names and revision dates of supported modules
  via the <hello> message.  The module namespace MUST be encoded
  as the base URI in the capability string.  The module name MUST be
  encoded as the "module" parameter, and the most recent
  revision-date, if present, MUST be encoded as the "revision"
  parameter, to the base URI.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Fri, 4 Sep 2009 09:56:45 -0400

>or measure interoperability for standards-track advancement.

The behavior of YANG-based server is deterministic.  The rules are
spelled out in the YANG spec.  You didn't those rules, but that
does not stop them from being deterministic and interoperable.

The YANG server is not required to return default values for
configuration data.  This is deterministic.  If the server does not
return a value for a leaf of configuration data that has a default
value the client will know that the server is using the default
value.  This is deterministic.  Clients will know this and can be
written that behave according.  This will allow interoperability.

The fact that YANG allows servers to behave in a way that you dislike
does not make the YANG spec broken.  It just means the WG made a
decision that you do not agree with.  Since the WG has made decisions
that I do not agree with, I don't find this surprising.  I'm sure
this is a common experience.  But somehow other WG members are able
to move on rather than continue the debate after the issues have
been resolved.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 4 Sep 2009 10:01:28 -0400

>http://www.ietf.org/mail-archive/web/netmod/current/msg03920.html

The change in msg03920.html refers to the Change Rules, requiring
a revision if/when the module is republished.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 08:42:23 -0700

> does not stop them from being deterministic and interoperable.

There are about 10 ways that a server complying with the
spec can cause the client to guess 1 of N possible outcomes.
That is the definition of non-deterministic behavior.

> written that behave according.  This will allow interoperability.

Nonsense.
There are many ways the client cannot be sure what
the data or lack of a response really means.
The NETCONF protocol behavior that YANG specifies
be allowing values (even non-default via deviations).

> does not make the YANG spec broken.  

I plan to let the IESG decide if my dislike of
this aspect of YANG is based on subjective or objective
criteria.

It just means the WG made a
> decision that you do not agree with.  

There have been several people -- about 4 to 6,
that have insited that NETCONF/YANG MUST have
a mandatory-to-implement data retrieval menchanism.
I do think there is any WG consensus otherwise.
Maybe 5 to 4, or so, and that is not consensus.
Even so, the IESG gets to decide whether interoperability
and security and all other issues meet IETF requirements
or not, not this WG.

Since the WG has made decisions
> to move on rather than continue the debate after the issues have

The mechanism you proposed is full of holes.
I found about 10 of them in the first week.
Why didn't you find them before that?
How many more ways are left to discover
how this extremely fragile design can break?

The latest 'D_leaf' example I posted shows that
this plan is hopelessly and fatally flawed.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 09:25:02 -0700

>   parameter, to the base URI.

It should be rather clear that in order to get the
'assumed-defaults' mechanism to actually work,
that YANG has to be locked down so tight that
it will be unusable by humans.

If the YANG default was reliable as a constant, it would be usable.
At first I thought it was a constant, but within a couple days
realized it is almost a constant, and that it not good enough.
There are also several ways in which 'no-returned-value' is ambiguous,
that can occur during the YANG module lifecycle.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 4 Sep 2009 14:00:37 -0400

>spec can cause the client to guess 1 of N possible outcomes.

Please list them so we can resolve them.

>this aspect of YANG is based on subjective or objective

Excellent.  I expect that IESG members will have sufficient experience
and expertise to understand what the terms like "default value" means
in the world of router configuration.

>this plan is hopelessly and fatally flawed.

Nope, it illustrates a strong reason for modelers to use revisions.
If you don't use revisions, there are serious implications for
changes to your module.  We should understand and describe those
implications.

I personally cannot see the value of not having a revision statement,
but the WG decided that this allows for flexibility that is needed
during development ("work in progress") and for standards that won't
be revised (IIRC).  I'll avoid it by always using revisions.  I
expect the IETF guidelines will push the same onto ietf modules.
Past that, I'm less concerned.  I proposed three options, and none
of them seemed to generate much in the way of concensus.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 11:56:22 -0700

> Please list them so we can resolve them.
I've already posted emails to this list.
They are in the archive.

> in the world of router configuration.

yep.  The doc-shepherd can record a strong objection,
and the IESG can try to understand why it is such
a terrible burden for every server to be able to
return the value in use for every instance of
every leaf.  If it exists for validation, why
doesn't it exist for retrieval?

> changes to your module.  We should understand and describe those

I don't buy the argument that 1 data point is as good as 2.
Never will -- sorry.

The NETCONF client needs to be able to retrieve
all the data no matter what legal combination
of YANG mechanisms is selected by the data modeler.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 04 Sep 2009 23:26:24 +0200 (CEST)

> I've already posted emails to this list.

Andy, I really try to keep track of any unresolved issue.  I have two
issues that you brought up:

  require a new revision if a non-editorial change is made

  question if default should be allowed to be added in a new revision

So if there are 8 or more other issues, please send links to the
messages or resend them.  (It is difficult to keep track of issues
that are embedded deep in other, sometimes unrelated, threads).

I also do not understand what you mean with "1 of N possible
outcomes".  Could you elaborate?

> yep.  The doc-shepherd can record a strong objection,

What exactly do you strongly object to?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 14:49:14 -0700

>   question if default should be allowed to be added in a new revision

I will combine them into 4 issues:

  - deviations break contract;
    no evidence AGENT-CAPABILITIES used, so not likely
    deviation-stmts will be used either.

 - incomplete module capabilities allowed, so meta-data
   required by client can be inaccurate (9/3 capabilities related edits)

 - import-by-revision is optional (9/3 - import-no-revision breaks default)
   which can cause the meaning of a missing leaf to be ambiguous.

 - verification of all leaf instance values for interoperability
   purposes not the same as meta-data + deviations/

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 4 Sep 2009 15:12:39 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Martin Bjorklund" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Friday, September 04, 2009 2:49 PM
> Subject: Re: [netmod] capabilities related edits
...
>    purposes not the same as meta-data + deviations/
...

I think the real issue underlying all of these is this:

There is a three-way conflict between the need to be able to talk about
a management interface conforming to some "standard" model,
the need for that management interface to reflect what the
underlying gizmo actually does, and the need for "the" model to be
something both software and humans can grok.

If we start with the premise that one of the important uses of a model
is to serve as an interface contract, and if we acknowledge that "deviations"
are a fact of life, then I think we need to be honest about the fact that what a
"deviation" really does is to provide the instructions for creating a new contract
from one already known.

If deviations are not being documented, we're left with several possibilities:
   (1) Status quo: Implementations will need to take interface contracts
         with a big grain of salt. This limits interoperability.
   (2) There are market disincentives to documenting deviations.
   (3) There are technical reasons why deviations are not documented.

I suspect (2) is a factor, but I also think that the desire for flexibility coupled
with the awkwardness of the mechanisms for describing deviations lead
to (3).

This begs the question of whether deviations really should be modelled
any differently from any of the other kinds of "refinements" to models that
might be needed, and what a refined model needs to do in order to claim
conformance to a base model.  

Randy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 15:49:27 -0700

> with the awkwardness of the mechanisms for describing deviations lead

Trusting 100% in the documentation is just not reliable.
There will be deviations from the 'MUST use YANG default'.
That is obvious.  IMO, it is also obvious that server
vendors do not want to add 'deviations=x,y,z' ever,
and engineers writing the server code will not be encouraged
or even allowed to publish a deviations-stmt for any product.

The only reason SNMP even works in the real world is that
the actual variant values are always available to the NMS.
That way, the operator has the reported value, the documented value,
and the observed behavior, and can decide which (if any) to trust.

> conformance to a base model.  

I strongly object to making deviations part of the standard.
Vendor X can decide to change a instance-identifier to a boolean,
or whatever illegal change they want.  Sounds like free-for-all standards.

IMO, from the client developer POV, lots of different server
variants with an arbitrary, but large set of optional features,
is worse than a small set of mandatory features.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 4 Sep 2009 16:00:37 -0700

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Friday, September 04, 2009 3:49 PM
> Subject: Re: [netmod] generating a new contract from boilerplate
>
> Randy Presuhn wrote:
> > Hi -
> > 
...

I think you've missed the thrust of my argument: a deviation (or optional
feature set, for that matter) effectively defines A NEW MODEL.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 16:35:03 -0700

> feature set, for that matter) effectively defines A NEW MODEL.

I think I got that part.
That's why it sounds like free-for-all standards.

Will actual implementation of the standard data model
count as extra credit?

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 4 Sep 2009 18:22:49 -0700

Hi -

> Sent: Friday, September 04, 2009 4:35 PM
...
...

This is why being able to talk about the relationship between two models
is important.  It's fairly rare in my experience that someone implements
"the standard" with neither extensions nor restrictions or some kind.
Being able to look at interface contracts (whether as objects, aspects,
packages, signatures, or something else) in a modular, composable
manner is key to whether a modeling approach will help or hinder automation
if the stuff being managed is either flexible or diverse.

Part of the question is actually getting agreement on what it means
"to implement the standard data model".  I have a clear idea
what than means in GDMO-land or with the SMI.  I admit that it's
not that clear to me what that means with YANG, especially from
the perspective of configuration backup / restore and offline validation.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 18:37:14 -0700

> if the stuff being managed is either flexible or diverse.

Extensions (augment-stmt) are great. I'm a big fan.
Restrictions in which the syntax and semantics
are bounded together in the YANG construct are OK.
Not a fan, but this is part of reality, just like
the inability to trust a YANG default.

Deviations are really arbitrary patches to the syntax,
with no semantics attached at all.
The client code for a specific YANG object
isn't going to work if the server deviates too much.

The YANG spec says a client MUST be able to
use the YANG default as if it were a value,
but only says the server SHOULD or MAY provide
the meta-data needed to do that.  I hope the
IESG does not let that stand.  It is broken.

> the perspective of configuration backup / restore and offline validation.

I don't think there will be any 3rd party apps that
utilize standard YANG modules, that will work on
a server without report-all.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 05 Sep 2009 10:35:31 -0700

> feature set, for that matter) effectively defines A NEW MODEL.

If we make the deviations part of the standard,
such that clients are expected to be able to use
arbitrarily adjusted contract (as if that is
somehow a standard), then what is the point of
all the normative text in sec. 10?

Sec 10. has the strict riles to follow when updating
a module, unless you want to change anything you want
s by using deviations.

Why not delete sec. 10 if there really are no
update rules at all?  Tell the client developers the truth.
There is no contract implied by a standard YANG data model
at all.  The whole thing could be 'not-supported' or
completely altered so only the object naming remains.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 05 Sep 2009 10:29:15 -0700

> I agree with both of you, but offer s/SHOULD/must/ as a possible

Here's the problem...

7.6.5: para 3:

   A NETCONF server that replies to a <get> or <get-config> request MAY
   choose not to send the leaf element if its value is the default
   value.  Thus, a client that receives an <rpc-reply> for a <get> or
   <get-config> request, MUST be prepared to handle the case that a leaf
   node with a default value is not present in the XML.  In this case,
   the value used by the server is known to be the default value.

7.13.2, para 3:

   If a leaf in the input tree has a default value, the NETCONF server
   MUST internally use this default if the leaf is not present in a
   NETCONF RPC invocation.

7.13.3, para 3:

   If a leaf in the output tree has a default value, the NETCONF client
   MUST internally use this default if the leaf is not present in a
   NETCONF RPC reply.

7.14, para 4:

   If a leaf in the notification tree has a default value, the NETCONF
   client MUST internally use this default if the leaf is not present in
   a NETCONF notification.

None of the MUST requirements are implementable by any client -- ever.
The server has mostly MAY requirements, a few SHOULD requirements,
and (now proposed) a few MUST requirements.

The MAY on the RHS of the equation does not add up to the MUST in the LHS.

I'm not sure what it even means the change all these MUST to MAY
in the above sections.  The protocol MAY work or it MAY NOT work.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 5 Sep 2009 19:42:56 +0200

> all the normative text in sec. 10?

STD 58 has AGENT-CAPABILITIES and module update rules and they both
serve different purposes.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sat, 05 Sep 2009 11:01:34 -0700

> serve different purposes.

OK, but AGENT-CAPABILITIES are not part of the SNMP protocol.
There is no mechanism at all in SNMP that requires the manager
to synthesize data from the agent, based on AGENT-CAPABILITIES.

I am still strongly opposed to making deviations a mandatory
part of the contract.  It has no standards-value.  Vendors need
to use augment to provide different knobs than the
ones defined in a standard.

Application developers have it better with SNMP, where at least
the alterations are tightly controlled.  Anything else requires
a new OBJECT-TYPE or AUGMENT macro.

The NETCONF server MUST NOT be able to replace standard default-stmt
values with its own, and somehow the client is non-compliant if it does
not work properly with defaults altered from the standard.

I don't care what vendors do with their own YANG modules,
but the constants in the standard YANG modules MUST be binding,
such that a client can hardwire their value into code
and remain complaint with the standard YANG module.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 05 Sep 2009 14:12:32 -0400

It seems to me that a deviation statement is a statement that the 
managed device does not comply with the model.

Obviously, if a device does not comply with the model, a client, while 
it is likely to try its best, can not reliably manage such a device. 
THe deviation statements do not change this.  What they do is increase 
the odds that the management operation will come sufficiently close. 
And they give a clean way for the vendor to tell the humans what to 
expect to go wrong.

The only reason for asking whether they will be used is to try to 
determine if it is worth defining the setup.  If it won't be used, then 
we have wasted our time.

Given that the deviations statement is a statement of non-compliance in 
and of itself, saying it MUST be used only as a last resort does not 
actually change anything.  The point of the statement (existing or new) 
is the "no substitute for implementing the standard correctly" text.

While we do look for bugs, and try to warn folks about particular 
problems with particular ways of failing to meet our standards, it does 
not really make sense to me to say "if you are going to violate the 
standard you MUST only do so as a last resort" inside an IETF RFC.

Yours,
Joel

Andy Bierman wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Andy Bierman <[email-masked]>
Date: Sat, 05 Sep 2009 11:43:44 -0700

> managed device does not comply with the model.

I would like to know how the IETF standards process
works wrt/ YANG modules.  I suspect other WGs who may
want to use YANG, and the IESG, who has to approve YANG,
are curious about the same thing.  That is, after all,
the official reason this work is being done.

If the standard YANG module defines a default, and the server
does not use that default, then the server is non-compliant
if the client does not work with that server.

The WG is proposing the opposite, by including the deviation
provided or altered default in the client MUST requirement,
which cannot be implemented anyway, according to the current spec.

> it is likely to try its best, can not reliably manage such a device. THe
> deviation statements do not change this.  What they do is increase the
> odds that the management operation will come sufficiently close. And
> they give a clean way for the vendor to tell the humans what to expect
> to go wrong.
> 
> Given that the deviations statement is a statement of non-compliance 

This is not what the WG is doing.
The proposal is that the draft says any deviation
(including a default) is proposed to be
a new part of the client conformance requirements in all those
sections I cited below.

in

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 05 Sep 2009 12:17:41 -0700

> I agree with both of you, but offer s/SHOULD/must/ as a possible

I think if all the various independent variables I cited
were either MUST or SHOULD, and the client default requirement
was SHOULD instead of MUST, then that would solve everything.

>From a standards perspective anyway.
Operationally, it is still a standards-track trainwreck,
but that is just my opinion.

However, a server that alters a standard YANG object
in any way is by definition non-compliant.
So the YANG spec needs to be clear that a client MAY
work with server-deviated-defaults, but not SHOULD.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Sat, 5 Sep 2009 10:55:10 -0700

Hi -

> Sent: Saturday, September 05, 2009 10:35 AM

I'm not sure where you get the idea that I'm arguing to make deviations
"part of the standard."  Since by definition a deviation breaks the contract,
I'm arguing quite the other direction:

Model + Deviation = Some_other_model
Model + Deviation != Model

Now, the point that might be causing difficulty is that people like to be
able to define a new model in terms of some existing model.  To say,
for example, "A is just like B except for C, D, and E."  The question
is the circumstances under which, both from the perspective of protocol
and of conformance, an A really *is* a B.  If we were talking objects here,
we could talk about inheritance or allomorphs.  What would this be
in YANG-speak?

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Sat, 5 Sep 2009 11:10:16 -0700

Hi -

> From: "Juergen Schoenwaelder" <[email-masked]>
> To: "Andy Bierman" <[email-masked]>
> Cc: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Saturday, September 05, 2009 10:42 AM
...
...

They serve different purposes, but to the same goal:  nailing down the
extent of variation permissible in implementations of "the same thing".

The STD 58 module update rules are further constrained by the lack
of an import-by-revision facility.

AGENT-CAPABILITIES, on the other hand, while it may have been
a nice idea, doesn't seem to have played out so well.  First, the
information was rarely available.  Secondly, and more importantly,
it didn't communicate the information about agent peculiarities that
management applications *really* needed to cope with various
implementations, such as: get-next order errors, index encoding quirks,
multi-varbind limits when using RowStatus, etc.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 07 Sep 2009 22:06:59 +0200 (CEST)

>     deviation-stmts will be used either.

As several others have pointed out, deviations document how a server
breaks the contract.  Not having deviations will not magically make
the server that needed a deviation in the first place follow the
contract. 

>    required by client can be inaccurate (9/3 capabilities related edits)

With meta-data, do you mean default or something else?

>    which can cause the meaning of a missing leaf to be ambiguous.

Phil replied to your email, but I never saw a follow-up?

>    purposes not the same as meta-data + deviations/

Isn't this the same as the first issue?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 08 Sep 2009 08:10:52 -0700

> the server that needed a deviation in the first place follow the

If the client MUST use the altered contract,
then no contract has been broken -- just
rewritten by the server.  The deviations can
contain semantic garbage, or no semantics at all.

This is a non-starter.  No client can implement
the contract using the syntax-only deviations-stmt
as a way to 'fix' the server problems.  This is pure myth.

> With meta-data, do you mean default or something else?

All meta-data that goes into the complicated algorithm
to deduce the value of a default leaf.

It's as simple as algebra:

   Creq(m) = Scap(m) + Sver(descendants(m)) + Sfeat(m) + Sdev(m)

If a RHS component is missing or inaccurate, then the
meta-data is incomplete, and possibly wrong.
The LHS answer is 100% dependent on RHS meta-data
provided by the server, and the server is dependent
on choices made by the data modeler.

> Phil replied to your email, but I never saw a follow-up?

SMI/SNMP standards developers figured out
how to let every manager retrieve a value
from every agent without complex algorithms
that produce ambiguous or wrong results.

Apparently, restricting YANG so it
does not produce garbage results in NETCONF
is not in the NETMOD charter.

> Isn't this the same as the first issue?

This is a different issue -- interoperability testing
for standards-track advancement.

Eventually, somebody might have to
deal the RFC 2026 advancement issues.
This assumes there will someday be a standard YANG module
ready for Draft Standard status.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 08 Sep 2009 22:06:06 +0200 (CEST)

> If the client MUST use the altered contract,

Deviations inform the client how a server breaks the contract.  A
client that implements the standard (contract) will not be able to
talk reliably with a server that broke the contract.  As we say in the
draft:

   Telling the application how a device fails to follow the standard
   is no substitute for implementing the standard correctly.

>    Creq(m) = Scap(m) + Sver(descendants(m)) + Sfeat(m) + Sdev(m)

Sfeat: Features do not change defaults. 

SDev: A client is not required to handle broken servers (i.e. servers
      which changes the contract, e.g. by changing the default).

Sver: Was this supposed to be the version?  If so, see below.

> is not in the NETMOD charter.

Phil asked if your opinion is:

  (a) Default statements cannot be added in new revisions of a module.

If so, Sver above goes away.

> for standards-track advancement.

Can you describe the issue you are seeing?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 08 Sep 2009 13:30:44 -0700

>    is no substitute for implementing the standard correctly.

Then change all the client MUST to MAY wrt/ deriving defaults.
The YANG spec can only deliver on MAY.

The client requirement can only be a MAY, because
the data modeler MAY choose no-revision or no-import-by-revision,
and the server MAY choose to omit module capability information.

>       which changes the contract, e.g. by changing the default).

Then change the text to say that.
The current proposal is that the deviation changes the
default (either adds or changes the default value).
This is a non-starter for a MUST requirement.
It is a MAY requirement at best.

> Sver: Was this supposed to be the version?  If so, see below.

the version (if available) of all nested imported modules
may impact the actual typedef or grouping used.

> If so, Sver above goes away.

That seems like a really harsh solution just
to keep the server from returning a default value.

I will never agree that there is a problem to solve
here.  This solution looks like a Rube Goldberg Contest entry
to me.  The server knows the default value it
is using (if any).  It also knows how to return a leaf value
to the client. Combining the 2 is trivial.

> Can you describe the issue you are seeing?

I already did.
Comparing apples to apples can be extremely complicated.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Sep 2009 14:36:54 -0400

>I've already posted emails to this list.

I believe every issue you've submitted has been discussed and
addressed.  Many are covered by more exact wording in the draft.
Others are simply cases where the WG disagress with you.

>of YANG mechanisms is selected by the data modeler.

This is a meaningless statement, since we can both agree with it
and both disagree about what it means.  It will help discussion if
you can use more precise wording for issues that you are trying to
raise.

The root issue is that default values are not part of the configuration
database.  The server is not required to return the default value for
any leaf not in the configuration database.  This is the current
wording in the YANG spec and has been discussed ad nauseum.  The
draft contains the WG concensus and is not likely to change.  Can
we record this as a closed issue and move on?  You've been reanimating
it for months, but the issue has been covered.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Sep 2009 15:23:42 -0400

>is not in the NETMOD charter.

You are making a serious claim, that YANG produces garbage results
in NETCONF.   Did you mean to claim that, or is this just the
continuing rant about default values not being in the configuration
database?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Sep 2009 13:05:13 -0700

> Others are simply cases where the WG disagress with you.

I think the co-Chairs should declare consensus -- not you.
I never saw any resolution fromhem.

> you can use more precise wording for issues that you are trying to

For a standards-track document it is not meaningless.
The WG does not understand the term MUST from RFC 2119.

> it for months, but the issue has been covered.

The issue is not closed because there is a MUST requirement
for the client that cannot be implemented.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Sep 2009 13:22:11 -0700

> continuing rant about default values not being in the configuration

Just insisting that the MUST requirement for the client-derived-default
works does not make it true. How does the client implement this MUST
requirement when conditions created by the server or data modeler
make this impossible?  Where is the normative text in the
draft that says how the client accomplishes this task in case
all those other conditions are false, and the meta-data is incomplete?

It is obvious the MUST requirement was reverse-engineered
from the desire to prevent the server from ever returning a default.
If that is what the WGs wants, then make the draft standards-worthy
by explaining how the client ALWAYS gets the derived default correct.

Or admit that the client-derived-default is not reliable
and just deal with it.  But pretending that 'no-instance-equals-default'
is reliable is simply not true.

There are also top-level mandatory nodes allowed in YANG
which will not work on all servers.  I do not agree that
standards-worthy means it will work on *some* (but not all)
compliant servers.  That is not useful to the application
developer because the YANG language makes it appear as if
all servers will support top-level mandatory nodes, instead
of using  if-feature or a NETCONF capability to indicate what
the server really implements.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Sep 2009 16:09:06 -0400

>if the stuff being managed is either flexible or diverse.

Amen.

>the perspective of configuration backup / restore and offline validation.

Should we make a list of model compliance, aka "what does it mean
to implement a model"?  Criteria like:
- syntatic compliance
  - every data node can be instantiated
- semantic compliance
  - every node performs the appropriate outcome
- constraint compliance
  - all constraints are enforced
    - data node entry constraints
    - edit-config constraints
    - validation constraints
- validation compliance
  - validate time actions (ABS nodes) are performed

I think much of this is already in the spec, but would pulling
it out into a new section help?  Or is this not what's not clear?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Sep 2009 16:51:55 -0400

>I think the co-Chairs should declare consensus -- not you.

Concensus was declared at the interim in DC long ago.  My opinion
is that it hasn't changed.

Would a new concensus call put the issue to bed (again)?
Should we continue to perform concensus calls until your vote wins?
Is no issue closed until the solution is Andy Approved? (tm)

>The WG does not understand the term MUST from RFC 2119.

Hmm....  So we need remedial 2119 schooling?

>for the client that cannot be implemented.

Details please:  what requirement cannot be implemented?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Sep 2009 16:57:01 -0400

Andy Bierman writes:
>works does not make it true.

And insisting that it's broken does not make it so.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Sep 2009 14:23:54 -0700

> is that it hasn't changed.

interesting -- that was before the current text
was written and the issues were known.

> Is no issue closed until the solution is Andy Approved? (tm)

I never got a resolution to the 'client MUST' issue.
Still waiting on that one.

> Hmm....  So we need remedial 2119 schooling?

yep -- the spec has an unimplementable MUST requirement.

> Details please:  what requirement cannot be implemented?

import-by-revision, no-revision-module, and missing
module capability meta-data cause problems for the
client wrt/ deriving the correct default or determining
the leaf does not exist.

The client MUST also be aware
of deviations that change defaults,
according to the spec.  It needs to be a MAY,
otherwise it changes the client requirements defined in
the YANG module itself.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 14 Sep 2009 14:29:27 -0700

Hi -

> From: "Phil Shafer" <[email-masked]>
> Sent: Monday, September 14, 2009 1:09 PM
...
...

Areas where things get fuzzy, at least as discussed on this list,
include:
  - handling of defaults specified in a model
  - deviations, especially if someone implements a deviation
    in order to be "compatible" with some widely deployed device
    with lots of deviations on which popular software depends.
    (I once worked on firmware for "Hayes-compatible" modems,
    so I have a pretty good idea of the horrors committed in
    the name of "compatibility.")
  - is the data retrieved by a get-config truly sufficient to bring
    any equivalent piece of equipment to an operationally equivalent
    state (nailing down what this means is not trivial)
  - what does it mean when two different models happen to be similar
    enough that configuration data read from one can be written
    to the other?  (a degenerate case of this is data type re-use)

Randy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Sep 2009 14:32:56 -0700

> And insisting that it's broken does not make it so.

It is broken -- a MUST requirement says *always*.
That is not possible the way the current spec is written.
Either change the MUST, or change the spec so that every default
leaf is always derivable by the client.  But if the MUST is changed,
then that is like saying the client does not need the
complete set of 'data+defaults -- it SHOULD or MAY get the
data it needs from the server.

If that is the level of interoperability the WG wants,
then make the spec say that, instead of pretending
the spec can deliver on MUST-level reliability.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Tue, 15 Sep 2009 12:43:55 +0200

----- Original Message -----
From: "Phil Shafer" <[email-masked]>
To: "Andy Bierman" <[email-masked]>
Cc: "NETMOD Working Group" <[email-masked]>
Sent: Monday, September 14, 2009 8:36 PM
Subject: Re: [netmod] capabilities related edits

> it for months, but the issue has been covered.

And it goes on being discussed because it is not resolved, there is no
consensus, it is not closed.

There are a few people who want the server not to return values that it
is using, there are a few people who want the server to return all the
values that it is using.  (It might help if there were a few more people).

I agree with Andy that the client must be able to get all the values in the
configuration.  The problem I have is being told that the client can retrieve
everything in the configuration and then find the definition of everything
excludes several items because they have been redefined as 'not configuration'.

My stab at this is

If the node in the data tree is defined in a yang module as configuration
and the server is using a value for that instance, then some variant of
get-config
(option=Gargantuan:-) must return that instance.  By all means have other
variants that return less, but the prime requirement is to be able to get
everything defined in a yang module as configuration.

Tom Petch

---

From: David Partain <[email-masked]>
Date: Tue, 15 Sep 2009 14:59:08 +0200

Hi,

We keep goin' round and round.  Just let me try to see if I'm getting any 
wiser...

Andy, do you agree that there is a need for mechanisms that will allow a 
server to choose NOT to send back defaults?  It appears that you think that 
the only way to ensure interoperability is to legislate that all data must 
always be returned, whether it's defined as 'default' in the YANG module or 
not.

Is that correct or not?

If you think all servers MUST always return all data, including defaults, then 
there is fundamental disagreement between members of the working group and 
someone's going to "lose".

You appear to assert that, as the spec is written now, unless the server 
returns all values, you cannot _really_ know what it's using, even though the 
YANG module defines it as a 'default'.  You say that 'default' is not 
reliable as a constant since "import-by-revision, no-revision-module, and 
missing module capability meta-data cause problems for the client wrt/ 
deriving the correct default or determining the leaf does not exist."

So... If these things that you think are broken (I know everyone doesn't 
agree) were fixed, would you agree that it's acceptable that the server MAY 
choose not to return values that are defaults?

Clearly, there are those who disagree with you about this, but I just want to 
ensure that I've got your point documented correctly.

Thanks.

David

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 06:02:13 -0700

> excludes several items because they have been redefined as 'not configuration'.

I've given up on YANG wrt/ standards value.
The market will dictate the minimum level of functionality,
even if the IETF can't.

But the use of MUST language for this client requirement
seems rather bizarre.  Those 4 paragraphs about 'if there is
a default, the client MUST use it' are only mentioned for defaults.
There is no text about "if the list has a key, then the client MUST use it."

The MUST terminology seems even more bizarre because there are so many ways for
the 'derived default' to be inconclusive or wrong.

IMO, there is no consensus that a deviation MUST be used by the client.
There is no consensus that a default within a grouping can be
re-written with a refine-stmt.

> everything defined in a yang module as configuration.

This seems to be an extra credit feature that not
all servers will provide.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 15 Sep 2009 15:02:57 +0200

> excludes several items because they have been redefined as 'not configuration'.

The problem with the text above is that the distinction between
configuration and operational state remains fuzzy and we seem to
differ on exactly this fundamental point.

The phrase "all the values it is using" translates to "operational
state" in my ears. And for me, the phrase "the client must be able to
get all the values in the configuration" translates into 'a value that
is operationally used but not part of the explicit configuration does
not have to be returned - because it is not configuration'.

The discussion around defaults is in my opinion a symptom of
fundamental disagreement what configuration really is. But this has
been stated before. What we really need is now a plan how to resolve
this disagreement.

/js

---

From: Phil Shafer <[email-masked]>
Date: Tue, 15 Sep 2009 09:26:01 -0400

>values that it is using.  (It might help if there were a few more people).

"it is using" is where we differ, I think.  "using" means operational
data, where "told" is configuration data.  Configuration data is
instructions to tell the device what to do.  If I leave out a step
of the instructions, the device will do something appropriate (where
this means (a) do nothing, (b) use a static default value, or (c)
use a dynamic default value).  But that missing step is not
configuration data.

When I configure an interface, I may not tell it the duplex or mtu
to use.  The duplex and mtu are not part of the configuration.

When I insert a interface card that matches the configured interface,
the device will find the appropriate config and set up the interface
according to those instructions.  Since I didn't provide a duplex or
mtu in the configuration, the device will do something appropriate
(a, b, or c).

I can now retrieve operational data to tell me the current values
for duplex and mtu for this interface.

But those values to not magically become part of the configuration
for that interface.  <duplex>half</duplex> isn't in the configuration
database.  It's not being redefined as 'not configuration', but it
is truly not configuration data.

If I tell you to drive to the market, the speed you are driving
isn't part of the instructions (configuration).  If I tell you to
drive 15 mph to the market, your current speed isn't part of the
instructions (configuration), but may be observed at any point in
time (as operational data).

This fuzzy line between "told" and "using" is something that imho
snmp got very wrong, allowing non-config data to masquerade as
configuration data.  But if you want the device to give you (b)
data as if it were real configuration, the with-defaults draft makes
this information available, without making it part of the configuration
database.

In any event, these are the same issues discussed at the interim
and in the end, the decision was to allow the device to avoid sending
default values.  This allows Andy to write code that populates the
configuration database with default values, allows devices that
don't track when default values are set explicitly by the user
(c-brand routers) and allows the configuration database to hold
exactly the instructions given by the user (which is how my code
works).

I haven't seen any new issues raised anywhere in this discussion.
I would be just as happy to say "the world MUST work my way" as
Andy would be to make the world work his way, but the WG decision
was to allow all three styles in a way that servers and clients
MUST live with.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 06:57:05 -0700

> always be returned, whether it's defined as 'default' in the YANG module or 

this is not my position.
There has never been any proposal by anybody that would require
every leaf to be returned every time, with no possibility
of filtering out defaults.  That is a strawman proposal.

> someone's going to "lose".

The current use of the term MUST is a problem wrt/
client using a derived default which MAY or SHOULD
be correct.

> deriving the correct default or determining the leaf does not exist."

It is not my opinion that the meta-data required by
the client to accurately deduce a default is optional.
It is a fact -- written in the spec.  The data modeler
and server are allowed to provide the client with incomplete
meta-data.

The RMONMIB WG could not agree which of the 9 groups
of RMON-1 should be mandatory, so they were all optional.
The RMON vendors were very good about spreading FUD on
their competitors lame excuse for an RMON implementation.
This market pressure, not the IETF, was the reason all
RMON probes implemented all of RMON-1.   I expect the
same market pressures will emerge for NETCONF, if and when
there is actually some common content to compare.

> choose not to return values that are defaults?

 1) 'client MUST use default' language is wrong, pointless, misleading
 2) client MUST use server deviated defaults is broken; the client
    only needs to code to the standard to be compliant with the standard
 3) refine-stmt cannot rewrite a default-stmt, as Martin claims is OK.
    This makes default-stmt inside a grouping immune to the change
    rules in sec 10, which is broken.

Note that none of my objections are related to a server MUST
return a leaf with a YANG default value.  I am not trying to change
that decision.   I am certainly not trying to force
all defaults to be returned all the time.

> ensure that I've got your point documented correctly.

As Joel pointed out, a application wants the consistent
but verbose data-set, even if the human operator usually
wants to ignore the default values.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 15 Sep 2009 09:57:53 -0400

>There is no text about "if the list has a key, then the client MUST use it."

IIRC this wording was added to clarify this point the last time we
circled this fire.  Are you now suggesting we remove this clarification?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 15 Sep 2009 16:37:48 +0200 (CEST)

> circled this fire.  Are you now suggesting we remove this clarification?

I can't find those 4 paragraphs.  I find 2 paragraphs, 7.13.3 and
7.14, and the text says says (7.13.3):

And the not-yet-published text says:

  If a leaf in the output tree has a default value, the NETCONF client
  MUST use this value in the same cases as described in
  ^leaf-default-value^.  In these cases, the client MUST operationally
  behave as if the leaf was present in the NETCONF RPC reply with the
  default value as its value.

Andy, what should we write instead of this?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 08:11:27 -0700

>From the email I sent on 9/5:

7.6.5: para 3:

7.13.2, para 3:

7.13.3, para 3:

7.14, para 4:

> Andy, what should we write instead of this?

I would just remove this noise about defaults.
The phrase 'is known to be the default value'
needs an asterisk next to it (* offer not available
in all areas).

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 11:58:04 -0700

> circled this fire.  Are you now suggesting we remove this clarification?

a) usage of 2119 language is normative, not a clarification
b) I posted all the MUST updates on the server side to show
   how absurd it would be to tighten up the DML usage that much.
   Apparently I was too subtle, because it was taken seriously.

IMO, here is what should be done to make the standard
reasonably likely to produce inter-operable implementations:

 1) Make revision-stmt mandatory to use
 2) Make advertisement of all module capabilities
    in the <hello> mandatory
 3) Make the sec. 10 rules wrt/ defaults apply to
    all defaults (even in groupings)
 4) Make import/include by revision SHOULD use (not specified now)
 5) Remove normative text wrt/ client MUST use the default;
    the text that the server MAY omit defaults is sufficient.
 6) Finish the with-defaults RFC ASAP; all servers SHOULD implement
    it to provide 'report-all' as a non-default retrieval mechanism
 7) Do not wait to define a new (possibly) non-backward compatible version
    of NETCONF (e.g., get-operational-state).  Stick to the decision
    to rely on description statements, special RPCs, and extensions
    to partition the database beyond the standard filters.  The NETCONF WG
    originally debated this issue a great deal and decided on
    1 'get-everything' operation (get).  This was a direct request
    from operators.

       Example: I just added set-log-level() as a special RPC
       because I don't want it saved in NV-store.  Is it perfect?
       Unified?  No -- but if the client doesn't know about the
       RPC <set-log-level>, why would it know about the config leaf <log-level>?
       This is a workaround that actually works.

 8) Pencils down.  No new features.  Bugfixes only from now on.
    Ship it already.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 15 Sep 2009 21:22:44 +0200

>     it to provide 'report-all' as a non-default retrieval mechanism

Sorry, but :with-defaults is at odds with explicit config and confuses
operational state with config data. Quoting -03:

   [NETCONF] does not define whether default data is part of the
   datastore/data model, or if it is meta-data that influences the
   behavior of the NETCONF server, but is not actually part of the
   datastore.  This document is intended to support either type of
   implementation, without deciding which approach is better.

I am not sure what meta-data refers to here. There are other places in
the document where things are rather unclear or where the text assumes
there are default configuration values while in reality there is
algorithmic determined operational state.

Take a look at the core definition:

   o  Default data: Data that is set or used by the NETCONF server
      whenever the NETCONF client does not provide a specific value for
      the relevant data item.  Default values SHOULD be specified in
      documents describing the data models supported by the NETCONF
      server.

Most of the many parameters determining system behaviour are
dynamically computed and not configuration values. We have discussed
several examples here on the mailing list. Treating such calculated
values as config data is at best an approximation of reality.

I believe explicit NETCONF support for retrieving operational state
plus proper support to separate config data from operational state
data from statistical data in YANG is the technically appropriate
solution.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 12:56:28 -0700

>    implementation, without deciding which approach is better.

It does not matter how the leaf instance /who/cares/what/it/is
is classified by the DML-of-the-month.  This has no
bearing on the <get> operation, which was designed
to get everything.  IMO, we could come up with clever
filters like 'get-operational-state' for many years,
and probably will.  These can always be added at any
time, and advertised with a capability -- exactly like 'with-defaults'.

> values as config data is at best an approximation of reality.

We are willing to accept approximations of reality
in machine-readable clauses all over the place.
We have no basis or framework to establish which
missing machine-readable clauses are more important than others.
We have no standard data -- no interoperability testing
of extensive 'apples to apples' content -- to decide what
standard content rules and mechanisms work best.  That is why
the Usage Guidelines is Informational, and the same logic
applies to new protocol features.

> data from statistical data in YANG is the technically appropriate

It could be done as an optional capability.
I want the <get> function to still get everything.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 15 Sep 2009 22:16:03 +0200

> time, and advertised with a capability -- exactly like 'with-defaults'.

The with-defaults ID says:

   Affected operations:
   o  <get>
   o  <get-config>
   o  <copy-config>

To me, this seems to imply that with-defaults works on config data and
in the explicit model this is kind of strange. Or do you propose to
restrict with-defaults to <get> only?
 
> applies to new protocol features.

I fail to see how all that related to my position. This jumping around
makes this discussion somewhat ineffective a bit frustrating if you
ask me.

> I want the <get> function to still get everything.

So you agree with me as long as <get> returns everything?

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 13:52:56 -0700

> restrict with-defaults to <get> only?

No.  I don't recall anyone ever suggesting that.

I don't understand your concerns wrt/ with-defaults.
A default is the YANG default-stmt value, and <get>
will access running config exactly the same as a <get-config>,
for source=running.

> So you agree with me as long as <get> returns everything?

I have not been following closely enough to
have an opinion on the new data classification work.
If it was an optional-to-implement RFC
(ala partial-lock or NETCONF-over-TLS), then vendors can
implement it if they need it, and it does not slow down
YANG or 4741bis or with-defaults.

I do not have any problems with the current <get> operation.
But (IMO) XPath is a million times better than subtree filtering,
and I can understand concerns over <get> if all you
had was subtree filtering.  It is also easy to add custom RPCs
like <get-schema> as well, so <get> is not overloaded.

Except for some 2119-cleanup work, the YANG standard is
precise enough, for the scope of its conceptual database model.
Refinements to this model over time should be expected.
I am sorry for getting us too focused on conformance language
in the DML RFC.  In the end, it is conformance to specific data
models that we measure for 2026 advancement, not the DML itself.
(Although I do not think the old checkbox form "Do you implement
the foo object?" is going to work anymore.)

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 15 Sep 2009 23:34:10 +0200

> will access running config exactly the same as a <get-config>,

To me, the default-stmt only covers rare corner cases - those where a
static constant exists. All other cases will either be dealt with in
description clauses are they are implementation specific. The argument
that with-defaults gets me the data I need to understand the behaviour
of a box is for me not at all convincing - and in particular if
default equates to YANG's default-stmt. To understand what a box
really does, I need a mechanism to retrieve its operational state.

The with-defaults capability does not provide the primitive needed to
analyze what a box is doing. This is why I am in favour of a primitive
that returns the operational state (and that allows me to relate it to
configuration state).

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 14:47:13 -0700

> that returns the operational state (and that allows me to relate it to

You seem to be describing a completely different problem
than the one addressed by with-defaults.  It is just supposed
to be a data reduction mechanism -- a way to get a subset
of the entire database.

I keep pointing out that my original draft was a simple
and direct way to ask for 'everything', since there
was no agreement in the NETCONF WG that the existing <get>
actually did that.  IMO there is clear consensus that
a way to get everything is still the point of with-defaults,
even if it is optional-to-implement.  That is all it
really needs to accomplish, especially since there is little
consensus on what 'trim' or 'explicit' modes really mean
on every server.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Sep 2009 00:06:15 +0200

> consensus on what 'trim' or 'explicit' modes really mean

We go into another circle because your interpretation of 'everything'
likely differs from my interpretation of 'everything' when we talk
about configuration.

For me, explicit seems pretty clear and useful. I fail to see why
someone wants to have trim. If you follow the explicit model, there 
is actually no difference between explicit and report-all since
operational state is not config.

I think we need help by the chairs to get us out of this loop.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2009 15:26:16 -0700

> likely differs from my interpretation of 'everything' when we talk

I do not put as much value as you do
in several specialized retrieval operations,
but RPCs are like Doritos -- just make more if you want more.

For the purposes of with-defaults, 'everything'
is defined as the same 'config+defaults' = 'everything'
in the YANG spec.

> I think we need help by the chairs to get us out of this loop.

What loop?

What exact changes to the with-defaults draft are you requesting?

I have posted precise requests wrt/ terminology clarification.
These remain to be resolved.  I would rather go back to
the simple boolean than expand the with-defaults problem
statement into an entire redesign of the NETCONF protocol
and database architecture.

Now that we are already late on delivering
final product to the IESG, it seems unwise to extend this
project by another year or two because there are some corner-cases
which do not fit this simple model.  It can be added on as a
new capability later.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2009 06:22:26 -0700

Juergen Schoenwaelder wrote:

> description clauses are they are implementation specific. 

The only reason we need with-defaults is to get the defaults.
I don't know how to put it any simpler than that.  It would be
really disappointing if the ability to retrieve the defaults
is not part of the standard, because there is no consensus
on all the cruft piled on top of the original functionality.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Fri, 18 Sep 2009 12:25:24 +0200

----- Original Message -----
From: "Juergen Schoenwaelder" <[email-masked]>
Sent: Tuesday, September 15, 2009 11:34 PM

> really does, I need a mechanism to retrieve its operational state.

Juergen

I think of defaults as being more widely used.

If I configure OSPF on a router, as I have done many times, then:

Globally,
OSPF set on
Router ID set (cannot default)
Area (default to 0.0.0.0)
Area type (default)
Database timers (default to 1800s, 3600s)
and per interface
OSPF set on
Active (default in most instances, sometimes set passive)
Interface type (default, derived from link type)
Area (default 0.0.0.0)
Hello, dead, delay, rexmit timers  (default, derive from link type)
Authentication set (for active interfaces)

so on a five interface router, I might set 12 values (two passive interfaces)
and use 39 defaults,.  There will be times when I set many of those
defaults to explicit values, timers being the least likely. (And yes, I may
configure
summarisation, ACLs etc but they tend to be downloaded from a central
source).

So defaults as I understand them really are useful and widespread, for me,
not at all rare.

Tom Petch

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 18 Sep 2009 14:17:28 +0200

> I think of defaults as being more widely used.

This is true, but our discussion is only about the defaults that can
reasonably be set in the data model. In your OSPF example, this would
most likely be the case of the OSPF area 0.0.0.0, but probably not of
things like timers, because this would lead the vendors in the desire
for differentiation to make deviations or even proprietary data models
for essentially the same thing, which is BAD.

IMO, small devices with a handful of config parameters could simply
forget about defaults and always send everything. On the other hand, big
boxes with zillions of parameters will have many more vendor-specific
defaults than those set in the generic data models and the clients will
anyway have to use other mechanisms to learn about the former.

So, it seems to me that the benefit of the 'default' statement in YANG
will not be all that big and certainly not worth the bandwidth spent on
it in this mailing list.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 18 Sep 2009 15:24:37 +0200

> So defaults as I understand them really are useful and widespread, for me,

Only a very small subset of these qualify as YANG data model defaults
and most of the parameters are operational state (some of them derived
algorithmically) you are most of the time happy with.

/js

---

From: "tom.petch" <[email-masked]>
Date: Fri, 18 Sep 2009 17:20:44 +0200

From: "Juergen Schoenwaelder" <[email-masked]>
Sent: Tuesday, September 15, 2009 3:02 PM

Juergen 

We could find out what we do agree on and what we disagree on.  
Things I thought clear turn out not to be and I am unclear how
much we disagree on - not you and me personally but the everyone
involved - and how much we agree on.  I suspect that at present
I am thinking that we disagree on more than we do.

And I think terminology is vital; Martin just referred to a 'missing leaf' 
err, missing from what?  No doubt obvious to some but increasing
the uncertainty for others (me:-). 

And Phil just suggested that 'data node ' is in the data tree
to which Lada and I objected; others have stayed silent.  My 
uncertainty just went up a notch. (I did check that the current
definition has been there since we started, nearly two years ago,
and yes it has, unchanged).  

For what we disagree on, then we define alternatives and, assuming that
we continue to disagree, let the 'marketplace' decide.  We provide 
alternatives - specify these options in YANG and you will never
be able to retrieve this value:-) - and then it is up to those who 
write the models, not us, as to which they choose.  We just have
to make sure that the consequences are spelt out more clearly
than when the Last Call started all those months ago.  SO much seems
to have changed in meaning since then.

Tom Petch

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 18 Sep 2009 10:25:05 -0700

Hi -

> From: "tom.petch" <[email-masked]>
> To: "Juergen Schoenwaelder" <[email-masked]>; "Andy Bierman" <[email-masked]>
> Sent: Friday, September 18, 2009 3:25 AM
...
...

But here's the real question: to what extent are these
defaults determined by the "standard" data model, and to what
extent are those defaults determined by vendor-specific "refinements"
of the "standard" model?

Randy

---

From: "tom.petch" <[email-masked]>
Date: Fri, 18 Sep 2009 19:01:30 +0200

----- Original Message -----
From: "Juergen Schoenwaelder" <[email-masked]>
Sent: Friday, September 18, 2009 3:24 PM

> algorithmically) you are most of the time happy with.

Add another one to the list of things we <do> disagree about:-(

Incidentally, most of the items I describe as default - eg interface timers -
have default values in the OSPF MIB Module [RFC4750] ospfIfTable.
My take is then that anyone writing the YANG equivalent would want
to specify a YANG equivalent  - 'use this value as part of the configuration
when no client explicitly sets a value'.

And more items from that table would be in this category were SMI 
as powerful as YANG.

Tom Petch

---

From: "tom.petch" <[email-masked]>
Date: Fri, 18 Sep 2009 19:01:51 +0200

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
Sent: Friday, September 18, 2009 2:17 PM

> for essentially the same thing, which is BAD.

Lada,

Well, as I said to Juergen, most of the items I describe as default - eg
interface timers - have default values in the OSPF MIB Module [RFC4750]
ospfIfTable.
My take is then that anyone writing the YANG equivalent would want
to specify a YANG equivalent  - use this value as part of the configuration
when no client explicitly sets a value.

And more items would be in this category were SMI as powerful as
YANG

Tom Petch

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 18 Sep 2009 20:55:56 +0200

> interface timers - have default values in the OSPF MIB Module [RFC4750]

But these values are not mandatory, right? In YANG, they are now
supposed to be mandatory, although we don't agree on what it exactly
means.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "tom.petch" <[email-masked]>
Date: Sat, 19 Sep 2009 18:14:37 +0200

----- Original Message -----
From: "Randy Presuhn" <[email-masked]>
To: "NETMOD Working Group" <[email-masked]>
Sent: Friday, September 18, 2009 7:25 PM
> extent are those defaults determined by vendor-specific "refinements"

Randy,

As you may see in my other answers, the authors of RFC4750 have
much the same ideas as me, even going further in places, suggesting
that the router ID could default, which I find a step too far.

Incidentally, I chose OSPF because it is a core and widespread
router function, and I see router configuration as a key part of any
effort such as this, and because it was one I could cite without
needing to look it up.  I was pleasantly surprised to find that
RFC4750 said much the same as me.

So very much the (SNMP) standard model.

Tom Petch

---

From: Phil Shafer <[email-masked]>
Date: Sat, 19 Sep 2009 14:35:05 -0400

>RFC4750 said much the same as me.

So the question is whether the default values for the four per-interface
timers (in your example) are part of the configuration if the user
does not specify them but wants the default value?  Should these
default values appear in the configuration if the user has decided
not to set them, intending to get the default values?

SNMP lacks a distinct configuration model, so fetching these variables
will give you the current operational "in use" value.  I may be
reading between the lines in rfc3535, but I think this is part of
the "clear distinction" between data sources they are asking for.

The JUNOS view is that if you didn't give an instruction, that
instruction isn't part of the configuration.  This certainly is
what our users want.

Thanks,
 Phil

---

From: "tom.petch" <[email-masked]>
Date: Mon, 21 Sep 2009 18:03:19 +0200

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
Sent: Friday, September 18, 2009 8:55 PM

> supposed to be mandatory, although we don't agree on what it exactly

Yes, they are mandatory; well, depends what you mean by mandatory:-)

The Hello and Dead interval timers are required by the OSPF protocol.
They must be set in the packets that an OSPF router sends to a
neighbour and the neighbour must use the same values ie be configured
with or be using the same values by default.

If the two routers do not have matching values, then an OSPF adjacency
will not form, no further exchange will take place, no OSPF, no IGP,
no routing (I've seen it happen:-(

So, defaults do really matter for me, and are in widespread use.

Tom Petch

---

From: "tom.petch" <[email-masked]>
Date: Mon, 21 Sep 2009 18:05:37 +0200

----- Original Message ----- 
From: "Phil Shafer" <[email-masked]>
Sent: Saturday, September 19, 2009 8:35 PM

Phil

My experience is that these timers are left to default by many,
perhaps most, people currently configuring routers by CLI. 

There is one set in common use in most cases, but there are
other sets of values for less common cases.

And, since the values have to match the neighbour router -
else no adjacency, no OSPF, no IGP, no routing - then it is
vital that the value in use is displayed in any display command.
Making assumptions that the value in use must be what the 
documentation says, because we live in a perfect world:-),
will make trouble shooting unnecessarily complex.

So I want a 'default' statement in YANG that means much the
same is in SMI, this will be used if not explicitly overridden by a
user, and I want to be able to read the value in use, without a
smart client or server making assumptions on my behalf,
assumptions which may be the source of the problem.

> the "clear distinction" between data sources they are asking for.

I see it differently.  I think that they are asking not to have to get the 
LSA database or BGP RIB when all they want is what they did
or might have specified.  It is the might haves, such as these two
timers, which do get and must be specified in places, but mostly
are left to default, where we are parting company.

As I said to Juergen, one way forward is to have this in black and
white in YANG statements and then let data modellers decide
which category, gotten in a get-config, gotten by some other means,
or not gettable, their data definitions come under.

Tom Petch

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 21 Sep 2009 13:49:27 -0700

Hi -

> To: "Phil Shafer" <[email-masked]>
> Cc: "Randy Presuhn" <[email-masked]>; "NETMOD Working Group" <[email-masked]>
> Sent: Monday, September 21, 2009 9:05 AM
...
> assumptions which may be the source of the problem.

That is *NOT* what DEFVAL means in SMI!
There is no assurance whatsoever that, if not supplied by
the manager in a set request, the value supplied by the
instrumentation will be the same as the DEFVAL specified
in the MIB.  This is a long-recognized shortcoming of the SMI.

RFC 2578 clause 7.9:
   The DEFVAL clause, which need not be present, defines an acceptable
   default value which may be used at the discretion of an agent when an
   object instance is created.  That is, the value is a "hint" to
   implementors.

Randy