Subject: [netmod] default and mandatory

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Jun 2008 16:31:00 -0700

Hi,

I would like to resolve the usage of default-stmt and mandatory-stmt,
within the YANG language, and the protocol operation behavior.
(This debate was never really resolved)

I can accept that default and mandatory MUST NOT be used together
in any fashion (e.g., derived default from the data type).
This means 'mandatory true;' can only be used if:
  1) there is no data model defined default value
  2) the agent will not supply any value for this leaf (or leaf-list)
  3) a <get> or <get-config> for this data will not contain any instances
     until one is provided by a manager

I think this is the behavior Phil wanted.
Mandatory means the manager must set it or it will not exist.
The config stays 'invalid' until the node is supplied.
No default value will be used by the agent under any condition.

I think the controversy is with agent-supplied defaults that
are not documented in the data model -- should the agent be required
to return a value for <get*> of the leaf in this case? (I say yes.)

Although data-model dependent, it may be valuable for an operator
to know that there is no 'foo' knob present, vs. knowing that the
knob is present and the agent supplied its own value, vs. knowing
the knob is present and some manager supplied the value.

XML attributes (ala operation, key, insert) could be defined
to add to <get*> replies for this purpose.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jun 2008 08:08:08 +0200 (CEST)

Hi,

Andy Bierman <[email-masked]> wrote:
> are not documented in the data model

Right.  We have three options here.  

  1. pretend that agent-supplied defaults don't exist

  2. realize that they do exit, but don't have any formal support for
     specifying them, and thus no formal support for a client to
     learn them.

  3. make it possible for an agent developer to specify them, and thus
     for a client to learn them and adapt to them.

The answer to the rest of your issues will follow from the outcome of
this one.

> -- should the agent be required

If we choose 2 above, yes; if we choose 3, no (then it would work like
data-model specific default, which can be controlled with
:with-defaults).

> to add to <get*> replies for this purpose.

One use-case to think about.  Suppose a manager sends a <get-config/>
in order to store a backup.  Later he sends this backup back in an
<edit-config/>.  If the get-config reply included defaults w/o any
annotations, the edit-config will explicitly set all those defaults,
and they are no longer defaults.  This is something I think we must
avoid.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jun 2008 10:35:28 +0200

Hi,

there are actually two forms of configuration data that YANG models
directly address:

A. Full configuration with all parameters that the agent uses in any
way.

B. Reduced configuration where some parameters may be missing because
they have default values in the full configuration.

In A, "optional" means that either the parameter is not essential (e.g.,
description) OR that it is in a subsystem that may not be operational
(that means inside a "presence" container that is missing).

In B, "optional" may mean the same things as above, but also that the
parameter in question has a default value.

I think it would be useful to clearly distinguish these three meanings
of mandatory/optional in YANG. The reduced form B is what traditional
CLIs present to the user, so I understand why Phil assumes it must exist
in the device, but in our implementantion it actually doesn't - the
reduction is only performed at the manager side.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jun 2008 11:05:43 +0200 (CEST)

Hi,

Ladislav Lhotka <[email-masked]> wrote:
> description)

For example:

  leaf description {
    mandatory false;
    type string;
  }

> OR that it is in a subsystem that may not be operational

For example:
  
  container ssh {
    presence "Enables ssh";
    
    leaf ip { 
      mandatory true;
      type inet:ip-address;
    }
  }

Are my examples what you had in mind?

You say that 'ip' is optional, b/c it is defined within a
presence-container.  In my view, it is still mandatory - but mandatory
always has a scope; the node MUST exist if its parent exist.

> of mandatory/optional in YANG.

First, note that we don't use the word "optional" to describe these.

I don't agree that there are different meanings of 'mandatory'.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jun 2008 11:24:05 +0200

>       type inet:ip-address;

This is a good example: IP adress is mandatory, since ssh server cannot
work without it. But a common practice applied by many devices is to set
it by default to an RFC 1918 address.

So in fact this is the case where it makes sense to have both mandatory
and default, if we interpret mandatory as "essential in the full
configuration".

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 05:41:50 -0700

Martin Bjorklund wrote:
>.....

This is not really possible.
If the leaf or typedef has a default, then the agent MUST use it.
If it doesn't, the agent is free to choose any default.
I do not want to change the meaning of default to be
the same as SMI.

This is not really a YANG issue, but rather a protocol issue
that should be solved with the 'with-defaults' flag.

The only argument against returning agent-supplied defaults
is that there might be lots of them.  IMO this is flawed logic.

1) It should be up to the operator, not the vendor, to decide
    when to return these agent-supplied defaults

2) It should be up to the operator, not the vendor, to decide
    if CPU and bandwidth would be 'wasted' or simply 'used'
    by retrieving these agent supplied defaults

3) It is well understood that silent changes to config defaults across
    versions and/or platforms is a source of network outages, which
    should be addressed by a robust standards-based configuration solution

> >....

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jun 2008 15:24:54 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> > Right.  We have three options here.  1. pretend that agent-supplied defaults don't exist

Do you mean that 3 is not possible?  I think it might be (think
positive!)  But is it something we want to do?

> If the leaf or typedef has a default, then the agent MUST use it.

Agreed.

> If it doesn't, the agent is free to choose any default.

Even when mandatory is true?  Even when mandatory if false?

These are the rules that we will have to spell out, if we go for 2) or
3) above.

> I do not want to change the meaning of default to be

Neither do I.

> is that there might be lots of them.  IMO this is flawed logic.

Agreed.

>     should be addressed by a robust standards-based configuration solution

Agreed.

So we agree that :with-defaults is the solution for controlling when
defaults are sent.  The issue here is how/if agent-supplied defaults
are specified and learned by the client.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 06:50:59 -0700

> positive!)  But is it something we want to do?

Why not just use <get> to retrieve the values at runtime?
It is a flawed hack to force operators to rely on external
documentation that may not even be correct on all platforms.
It is easier and more robust to simply ask the agent what
values are in effect at the moment.

> Even when mandatory is true?  Even when mandatory if false?

no -- if mandatory is true then the agent must not use
any default, documented or not.  The agent must wait
for the manager to provide a value, or it is an error.

If mandatory is false, and there is no default, documented
or otherwise, then the node is not needed for operation,
and the config will be accepted without the node.

> are specified and learned by the client.

I don't want to complicate the default-stmt with sort-of-default,
or sometimes-default statements.  Also, this issue could be viewed
as just one detail within the larger AGENT-CAPABILITIES problem.
I prefer to deal with the entire problem (but sometime in the future,
after YANG 1.0 is done.)

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 08:25:29 -0700

Hi,

 >>...

I think these rules make sense for default and mandatory:

  - if mandatory==true:
     * default-stmt MUST NOT appear in the same YANG construct
     * if the data type has a default defined, then the agent will ignore it
     * the agent will not supply a default value under any conditions
     * an rpc-error will be generated for <edit-config> to <running>
       or <commit> of <candidate> to <running>, if this node is missing
  - else:
     * if default-stmt is present in the leaf*, the agent will use it
     * else if the data type has a default defined, then the agent will use it
     * else the agent is free to supply any default value that is valid
       for the data type
     - if the agent supplies a value:
       - a node exists in the configuration with the value
       - the meta-variable 'set-by' is given the value 'agent'
     - else if the manager supplies a value:
       - a node exists in the configuration with the value
       - the meta-variable 'set-by' is given the value 'manager'
     - else:
       - a node does not exist in the configuration and the data model
         defines what happens if the node is missing

The 'set-by' meta-variable is maintained by the agent for each node.
If the 'with-defaults' flag (in retrieval requests) is set to 'true',
then nodes where set-by==agent are included in the response.
Otherwise these agent-supplied nodes are not returned in any
data retrieval operation, including <copy-config>.

A node is returned in <get*> operations (if with-defaults==false)
if the manager provides a value, even if the manager happens to
provide the default value for the leaf.

A standard read-only data model object called 'with-defaults-default' is needed,
to indicate which mode the agent will use if no 'with-defaults'
flag is present in a retrieval request.  IMO, RFC 4741 does not
recognize 'with-defaults==false', but real-world routers do
not work this way, so the default (for the default ;-) should
be false.

    leaf with-defaults-default {
       description
          "Indicates whether agent-supplied nodes will be returned
           in retrieval operations, if the 'with-defaults' attribute
           is not present.";
       type boolean;
       config false;
       default false;
    }

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jun 2008 18:53:24 +0200

> I think these rules make sense for default and mandatory:

[...]

So if there is a value with the meta-variable "set-by" with a value
'agent', how does this impact edit-config operations with say the
operation create? Will it fail or succeed?

It looks to me that we are facing a problem in the protocol which
ought to be solved there.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 10:22:21 -0700

> operation create? Will it fail or succeed?

It should fail.
That is why it is important that the entire configuration
be visible, if requested.

> ought to be solved there.

YANG is supposed to provide an XML mapping for NETCONF operations.
This seems to fall under that charter item, but you are probably
right.  I don't want one <get> operation for YANG data models
and another for non-YANG.  We already have that with the <edit-config>
operation, which is bad enough.

This does not have to be standardized at all, if there is
no consensus on the problem or the solution.

However, there are many aspects of CLI-based configuration management
that are not very inter-operable, and therefore are not appropriate
for IETF standards.  NETCONF + proprietary XML is not that much
better than SSH + proprietary CLI, and it appears the operators
know that.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jun 2008 20:14:23 +0200

> data retrieval operation, including <copy-config>.

But there still must be an option for the manager to ask the agent for
ALL values in its configuration database including data model defaults.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 12 Jun 2008 11:37:28 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Martin Bjorklund" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, June 12, 2008 6:50 AM
> Subject: Re: [netmod] default and mandatory
...
> or sometimes-default statements.

When the requirements design team hashed through this, our general
conclusion was that defaults were useful only if they specified an
interface contract.  Anything else (including the SMI-style sense) didn't
help provide a real configuration management solution.

>  Also, this issue could be viewed

A slightly different way of looking at this, which entirely eliminates the
need for AGENT-CAPABILITIES, is to derive a version or platform-
specific model from the "standard" model.  This way, vendor-specific
extensions and limitations, model-specific quirks, and all the stuff that
comes with multiple releases can be handled in the same way.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jun 2008 21:02:33 +0200 (CEST)

> >>>      for a client to learn them and adapt to them.

I think that you're saying that we should do 2, i.e. make sure the
document handles the case that agent-supplied defaults exist.  I would
prefer to explore 3, to see if that would work.

> documentation that may not even be correct on all platforms.

I haven't talked about using flawed external documentation.  There
might be other solutions to this problem.

> for the manager to provide a value, or it is an error.

Ladislav brought up a use-case where mandatory and agent-supplied
default would make sense.

Again, this depends on if we want to do 2 or 3.  If we do 2, then I
agree with you, b/c otherwise there would be no way (but
trial-and-error) for a manager to know if it has to provide values for
all mandatory leafs.   OTOH, if we do 3, then the manager would know
if a particular agent uses a default value or not.

Neither do I.

> I prefer to deal with the entire problem (but sometime in the future,

Yes maybe you are right.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 12:02:00 -0700

> help provide a real configuration management solution.

Fine, but that's not really the issue at hand.
The problem is that the CLI style of CM is to document
all the agent-supplied defaults off-line in an ad-hoc manner.
In YANG terms, these are the values the agent fills in when
there is no default-stmt to apply and mandatory==false.

In NETCONF terms, a manager has no way of knowing
(through the protocol) which knobs of this type
are not in use, and which are in use with an internal default,
or what that internal default value is on a particular agent.

These knobs could account for most of the CLI.
Given that vendors are even less likely to agree on standard
YANG defaults, when they can barely agree on SMIv2 DEFVAL suggested
defaults, a NETCONF manager is going to need to deal with
this problem.

IMO, this is an area where the 'native' interface is not
good enough for standards, and something better is needed.

> comes with multiple releases can be handled in the same way.

It seems like more work than just retrieving the values from
the agent, and it still requires a lot of offline documentation,
but it is machine-readable documentation at least.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Jun 2008 21:08:58 +0200 (CEST)

Hi,

> comes with multiple releases can be handled in the same way.

Yes, I have been thinking about using the standard module + an overlay
module which specifices implementation-specific defaults.  This
overlay module would be available through the schema discovery
mechanism.  Something like this:

  // std module
  module foo {
    ...
    container a {
       leaf b {
         type int32;
       }
    }
  }

  // implementation-specific overlay
  agent-module foo {
    container a {
      leaf b {
        default 42;
      }
    }
  }

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 12 Jun 2008 12:15:36 -0700

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, June 12, 2008 12:02 PM
...
...

Such things, while perhaps well-intentioned and convenient,
are *toxic* to real configuration management.

If the interface specifications are incomplete or inaccurate,
we shoudn't be surprised if configuration errors cause
significant operational problems.  If we follow a course that
guarantees that interface specifications are incomplete or
inaccurate, if we can't provide a way to determine *exactly*
what a device's configuration is, and, more importantly, how
to return that device to that *exact* configuration at some
later point in time, this exercise will have been a waste of time.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 12:33:58 -0700

> later point in time, this exercise will have been a waste of time.

+1

IMO, it is not a coincidence that all of the NMS projects
of this nature ever attempted (at my former employer),
failed because they could not deal with the massive data silo
maintenance.  Data-model driven tool automation is the only
viable answer, but it requires some changes from the 'CLI' way
of doing CM.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jun 2008 22:29:59 +0200

> later point in time, this exercise will have been a waste of time.

Setting the *exact* configuration at some later point in time on the
*same* device (same hardware plus same firmware) is a realistic goal.
However, once you allow changes in hardware or firmware, the problem
becomes really really hard and if we set the bar that high, we are
going to fail.

If you look at a plain Unix server, you likely have hundreds of knobs
and several interesting possible interactions between them. The only
way I personally survive is by trusting package and software
maintainers to set "sensible" defaults and most of the time this works
extremely well. So I am with Phil in the sense that most of the time,
I do not really want to see all the many defaults that exist. If I
have to dig deep to solve a specific problem, I am happy with finding
the relevant default _values_ and being able to change them. And no, I
do not use (=trust) documentation to tell me what the default is; I
like to see the setting on the device itself.

Perhaps we overestimate the value of the "default" statement in YANG
and we underestimate the value of a NETCONF feature (get-settings ;-)
to let me retrieve all knobs and their current settings when really
needed.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Jun 2008 13:48:39 -0700

> like to see the setting on the device itself.

We all share this position.
Most (but not all) of the time, leave the agent-set defaults out.

> to let me retrieve all knobs and their current settings when really

This would be OK.  Instead of changing the existing <get*> and
copy operations, create a new verb.
It doesn't solve the behavior of <get> being different
on different agents, but it provides a clean way to get every knob.

The <get*> operations need to be self-consistent on a given platform
(at least).  Phil and I came up with a good definition for property
for NETCONF, that goes all the way back to the XMLCONF design team:

   - The 'payload' output from a <copy-config> or <get-config> MUST be
     accepted by the agent without modification as input to an <edit-config>
     operation, when it has an empty <running> configuration.

Andy