Subject: [netmod] keyref CLRs

From: Andy Bierman <[email-masked]>
Date: Sun, 09 Nov 2008 11:06:57 -0800

Hi,

I suppose the leafref/keyref issues need to be cleared
up in general, but I just ran across a specific CLR
while testing some new code.

The ifStackEntry in IF-MIB.yang is indexed by ifStackHigherLayer
and ifStackLowerLayer.  These are non-config, because this
table is built by the agent, not the manager.

http://www.netconfcentral.com/modules/IF-MIB/2000-06-14#ifStackEntry.497

The ifInvStackEntry is indexed by keyrefs into the ifStackTable
(also called ifStackLowerLayer and ifStackHigherLayer),
but these entries are writable. (See ifInvStackStatus
for more details)

http://www.netconfcentral.com/modules/IF-INVERTED-STACK-MIB/2000-06-14#ifInvStackEntry.51

Either Juergen cannot do this translation in his smidump
program or the CLR about a config keyref pointing at
a non-config key needs to be fixed somehow.

I like the s/keyref/leafref idea with sub-clauses for
the properties that need to be controlled.  (Or another
option is to just have leafref with no constraints
and no extra sub-clauses.)

leaf foo {
  type leafref {
    path "...";

    // T: path target must be a config leaf
    require-config true/false [default: false];

    // T: path object must be a key leaf
    require-key true/false [default: false];

    // T: leafref leaf can only have values matching existing
    //    instances of the target leaf
    // F: leafref leaf can have any valid value for the data type
    //    of the target leaf
    require-instance true/false [default: false];
  }
}

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 10 Nov 2008 21:48:36 +0100 (CET)

Hi,

Andy Bierman <[email-masked]> wrote:
> the properties that need to be controlled.

So do I.

>     require-instance true/false [default: false];

I don't understand the "require-config" and "require-key" clauses.
This property will be given by the "path" argument - the leaf the path
points to is either config or not, and it is a key or not.

I think the "require-instance" clause is all that is needed.  A config
keyref can point to a non-config key, provided "require-instance" is
false.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 Nov 2008 14:15:07 -0800

> points to is either config or not, and it is a key or not.

So remove the CLRs about "keyref leaf must point at a key leaf" and
a "config keyref leaf must not point at a non-config key leaf"?

OK

> keyref can point to a non-config key, provided "require-instance" is

I don't think it has to be false.
Some of the SNMP examples have writable extension tables that
expect the read-only version of each row to be present.

How about no CLRs?

A leafref just needs to point at a leaf.
The 'config T/F' or 'key T/F' flags do not matter.
The 'require-instance' flag (or whatever it should be called)
does not have to be set, but if it is, the agent will
limit the leafref leaf to the existing values of the 'pointed-at' leaf.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 10 Nov 2008 23:55:52 +0100 (CET)

> a "config keyref leaf must not point at a non-config key leaf"?

No I meant that if the path doesn't point to a key, what does it mean
if it has "require-key true"?

> expect the read-only version of each row to be present.

So what happens if, at some time after commit, the read-only version
of the row is no longer present?  Is that an error?  The agent MUST
NOT remove any non-config key/leafs that config keyrefs points to??

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 Nov 2008 15:26:18 -0800

> NOT remove any non-config key/leafs that config keyrefs points to??

One way of looking at it is
that the schema nodes are always present and
there may or may not be any instances of the pointed-at leaf.

If the require-instance flag is true, then deleting
the instance of the pointed-at leaf set in the leafref
makes the leafref node invalid and a commit will fail.

There is no MUST NOT REMOVE rule, any more than a manager
messing up a config in other ways.  It just won't commit
(or edit-config on <running> will fail right away).

It require-instance is false, then the agent just needs
to know about the pointed-at object (to get its typedef).
The leafrefs will still validate if they contain
a value valid for the data type, but not a value
actually in use as an instance of the pointed-at leaf.

The problem is not 'instance T/F', but rather that
the <candidate> config and the <validate> operation
have no notion of mixing non-config from <running>
into the validation procedures.  That is a pure YANG-ism,
and needs special code to work.

I prefer a 'no CLR' approach.
The agent has to figure out what to do if
nodes in any XPath expression (must, when, path)
reference a non-config node.

If validation fails, then it fails.  We should avoid
rules that say "you MUST NOT saw off the branch you
are sitting on."

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 11 Nov 2008 08:57:32 +0100 (CET)

> (or edit-config on <running> will fail right away).

It's not the manager that messes things up if the agent removes the
read-only row.

The problem is that when you set the keyref which points to non-config
data, the non-config data might exist, and the config commits.  Then,
for some reason, the agent removes the non-config data.  At this
point, no manager is involved, and the running config is invalid.
(IMO, this in itself is the real issue - the invariant is that running
is always valid).  Now, some other manager might change some innocent
parameter and try to commit.  This will fail b/c of a missing
leafref.

> actually in use as an instance of the pointed-at leaf.

Yes.  I like this.

> and needs special code to work.

See above.  I don't think it's a YANG issue.  running is always
valid.

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 11 Nov 2008 15:42:01 +0100

Hello,
I would also like to cut down on little rules.

Someone had earlier the proposed:

---------------------
leafref foo {
	path ...
	strict true/false;
}

If strict=true, foo must point to configuration data that must also exist. (might or might not 
be key)

If strict=false then foo can point to config or non-config, it can point at existing or 
non-existing, the only requirements are that the path specified exists in the schema tree and 
that the value of foo conforms to the pointed to datatype.
---------------------

Both cases are simple, easy to understand. One gives strict control, one gives freedom. Some 
middle cases are not handled (e.g. pointer to strictly existing stuff which can be non-config), 
  but we don't have to handle every little corner case.

Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 11 Nov 2008 16:59:27 +0100

On Tue, Nov 11, 2008 at 03:42:01PM +0100, Balazs Lengyel wrote:

> If strict=true, foo must point to configuration data that must also 
> exist. (might or might not be key)
>
> If strict=false then foo can point to config or non-config, it can point 
> at existing or non-existing, the only requirements are that the path 
> specified exists in the schema tree and that the value of foo conforms to 
> the pointed to datatype.
> ---------------------
>
> Both cases are simple, easy to understand. One gives strict control, one 
> gives freedom. Some middle cases are not handled (e.g. pointer to 
> strictly existing stuff which can be non-config),  but we don't have to 
> handle every little corner case.

What does "must also exist" mean? Does this imply that any attempt to
remove ... fails? And if yes, which error can be expected if an
application tries to remove ...?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 11 Nov 2008 17:34:21 +0100

Hello,
Yes removal leads to an error, as the leafref constraint would be violated (just as for todays 
keyref).
Error code could be:
error-tag: missing-element (if we are allowed to mean "missing in the datastore")
error-app-tag: referenced-data-can-not-be-removed
or
error-tag: operation-failed
error-app-tag: referenced-data-can-not-be-removed

Balazs

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 Nov 2008 08:59:24 -0800

Balazs Lengyel wrote:
> Hello,
> Yes removal leads to an error, as the leafref constraint would be 
> violated (just as for todays keyref).
> Error code could be:

This is too complicated on the agent, and does not fit
with current practice.  If an ATM line card gets pulled,
and the agent is designed to remove the ATM data model
with the read-only table in it, then this CLR is going
to get in the way.

The SNMP examples I have looked at do not make sense
if the 'pointed-at' object goes away.  In every case,
the leafref object must go away too (or the notification
will not be generated anymore, etc.)  Instead of
preventing the r/o leafs from getting deleted,
the agent could issue a notification and delete
all the leafrefs that point at the deleted r/o leafs.

What if a manager sets the enableAtmStatistics knob to false?
Is that supposed to fail in the validation phase?  A centralized
agent is not going to know that setting this knob to
false will cause an agent callback to remove/disable other
r/o nodes from the object tree (after validatation).
There are lots of corner cases here.

The only things that can really be counted on:

    1) the agent MUST keep the <running> config valid
    2) a commit operation MUST fail if (1) cannot be met

There is a significant agent resource increase incurred
in order to replicate all of the <running> config,
not just the 'config true' nodes.  Are we sure there isn't
a better way to handle these corner cases?

Andy

>>> If strict=false then foo can point to config or non-config, it can 
>>> point at existing or non-existing, the only requirements are that the 
>>> path specified exists in the schema tree and that the value of foo 
>>> conforms to the pointed to datatype.
>>> ---------------------
>>>
>>> Both cases are simple, easy to understand. One gives strict control, 
>>> one gives freedom. Some middle cases are not handled (e.g. pointer to 
>>> strictly existing stuff which can be non-config),  but we don't have 
>>> to handle every little corner case.
>>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 11 Nov 2008 22:15:08 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> Balazs Lengyel wrote:
> > Hello,
> > Yes removal leads to an error, as the leafref constraint would be violated
> > (just as for todays keyref).
> > Error code could be:

Things gets too complicated if we allow config leafrefs with
"require-instance true" to point to non-config leafs.  If config
points to config with "require-instance true", then it is just a
normal validation rule to handle at commit time.

> a better way to handle these corner cases?

I think the following rule is simple to understand , easy to
implement, and does not break the invariant that running MUST always
be valid.

    A config leafref with require-instance true MUST NOT
    point to a non-config leaf.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 Nov 2008 14:22:51 -0800

OK

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Nov 2008 08:16:27 +0100 (CET)

> > Someone had earlier the proposed:

[...]

> application tries to remove ...?

This is how keyref works currently.  The reference is always
enforced.  The same effect would be achieved by setting
"require-instance"  (or "strict") to "true".

In the YANG spec there is no special error message defined.  In our
code (both client and server side) we use the generic
"operation-failed" with special error-info:

  <bad-keyref>
    <bad-element>...</bad-element>
    <missing-element>...</missing-element>
  </bad-keyref>

I'm all for putting detailed info in the errors - this means that the
manager application can give much better help to the user when things
go wrong.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Nov 2008 02:07:25 -0800

>>     point to a non-config leaf.

almost OK...

The NETCONF protocol does not actually support the concept
on non-config nodes in the <candidate>.  There is only
a <get-config> operation (and no <get> operation) for
this target database.

Pretending that non-config nodes exist for the purpose
of must, when, and keyref expressions, but you can never
actually see them -- is a hack.  Once the <candidate>
starts to differ significantly from the <running> config,
there is no way to tell what non-config nodes are really
supposed to be present anyway.  This is a fragile hack
that isn't going to work in the long run.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Nov 2008 11:23:39 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> actually see them -- is a hack.

I don't get this.  What do you mean by pretending?   If a config
points to non-config, no validation checks will be done.  As yu wrote
earlier, the only check will be the type check.

I don't understand what your conclusion is.  Are you suggesting that
we should keep keyref/leafref as is and NOT let config point to
non-config?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Nov 2008 03:38:57 -0800

> earlier, the only check will be the type check.

As the manager adds, deletes, and modifies nodes in
the <candidate>, there is no protocol support for
examining the non-config contents of the <candidate>.

I don't like solutions that assume the agent implements
exactly what the documentation says it will, therefore
why bother retrieving the data, since the manager
"knows" exactly what will happen.  This is fragile.
A manager needs to retrieve the actual config contents
from the agent, and that is not possible if the
<candidate> cannot report the existence of non-config nodes.

> we should keep keyref/leafref as is and NOT let config point to

I want an operationally robust solution.
I suspect there are more elegant and robust solutions
than what YANG has so far, but I don't have any
suggestions at this time.

The notification content use-case can be solved more
directly.  The oddball SNMP table constructions can
also be thought of as configuration that only the
agent (or the CLI) can set.  Perhaps the config T/F
flag needs more work.  The 'create-only' enum got
dropped for example.  IMO, the pointed-at SNMP stuff
is really 'out-of-band-created' and not 'config false'
at all.

Perhaps we have not specified the problem correctly.

Andy

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Nov 2008 10:00:57 -0800

> we should keep keyref/leafref as is and NOT let config point to

I'm starting to think the rule about a config leafref
cannot point at a non-config leaf is correct.

The classification of config as true/false is the part
that may not be correct.  Perhaps the original model used
in the inverse IF stack table (and other MIBs) is not valid.

It was a very common MIB practice to define
config objects read-only because (of course) the WG agrees
there will never be any standard configuration in the
future.  So even though these objects should be
writable configuration, they are defined as read-only,
and just used to report the configuration.

Perhaps an architectural decision (oh no!) is needed about
the very concept of 'report-only' configuration.
Just calling it state info is not good enough.
Is this category ignored?  Handled with existing rules?
Handled with special rules?  Do some SMIv2 'features' need
to be banned, and converted to the 'NETCONF' view of config,
or simply not translatable to NETMOD at all?

I like the concepts of config=true means a manager
can edit it and do all the other NETCONF operations,
and config=false means the manager cannot do any
of the 'write' operations to it.  There should only be
config=false nodes in the <running> database.

Andy