Subject: [netmod] leafref problems

From: Andy Bierman <[email-masked]>
Date: Thu, 13 Aug 2009 11:05:44 -0700

Hi,

>From yang-07, sec. 9.9.2

   The "path" XPath expression is conceptually evaluated in the
   following context, in addition to the definition in Section 6.4.1:

   o  The context node is the node in the data tree for which the "path"
      statement is defined.

   The accessible tree depends on the context node:

   o  If the leaf with the instance-identifier type represents
      configuration data, the tree is the data in one of the NETCONF
      datastores.  The XPath root node has all top-level configuration
      data nodes in all modules as children.

   o  Otherwise the tree is all state data on the device, and the
      <running/> datastore.  The XPath root node has all top-level data
      nodes in all modules as children.

1) this must be a bug, because it discusses the i-i type
in the section about the leafref builtin type.

2) This behavior is really complicated, especially when the leafref
is nested within a union typedef.  When checking a value against
its typedef, you need to know the protocol operation and which target
database (if any) is being referenced.  Maybe the agent just
picks a database at random if the target is not provided in
some operation?

3) There are no compelling use-cases for including leafref objects
in the database.  Their use in notifications and RPC operations
is different.  That allows custom RPCs and any event-type
to point at database objects for their transient messages

4) The issues I raised wrt mandatory/optional or just dealing
with objects that point at nonexistent nodes (not-supported
or if-feature=false, or when-stmt=false) make leafref almost
unusable in the database.   The implementation complexity issues
can be dismissed as subjective, but not the rest of them.

I think the leafref should not be allowed in config=true objects
at all.  They are not properly supported, and they are not really
needed either. Leafref is way more trouble than it is worth.
It hardly ever shows up in MIBs.  I know of 1 MIB -- the
RMON usrHistoryTable, which uses a leafref as part of its
self-contained config to monitor read-only data.

It isn't even very good for RPC/notification. A 'varbind' approach
would be much better and much easier to use.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 13 Aug 2009 12:40:40 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "NETMOD Working Group" <[email-masked]>
> Sent: Thursday, August 13, 2009 11:05 AM
> Subject: [netmod] leafref problems
...
> self-contained config to monitor read-only data.

Similar things abound in the disman MIB modules' configuration
data.  The patterns used in VACM are also *conceptually* similar,
though they're really built on syntactic punning.  There really are
cases where it's necessary for configuration data to refer to
other elements of configuration data.  If leafref in its current form
is problematic, a counter-proposal is in order.

> would be much better and much easier to use.

I'm inclined to agree.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 13 Aug 2009 13:01:42 -0700

> is problematic, a counter-proposal is in order.

these MIB modules are not widely used.
IMO, the leafref database object introduces a lot
of complexity for a very minor feature.

I don't have a counter-proposal to fix leafref,
other than to disallow the parts that don't work.

I think the normal process is for the WG to first agree or disagree
that any problem exists.  Somebody can go through all
the points I raised and explain why there are no concerns
for users of the YANG standard in any of them.

If somebody has a better solution then great.
I don't think the outcome of "nobody has
a good solution, so we will just ignore the problem"
works that well during the later review stages
of the IETF standards process.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 13 Aug 2009 22:13:18 +0200 (CEST)

> in the section about the leafref builtin type.

Yes, fixed in the upcoming -08.

> in the database.

Haven't we had this discussion several times before?  Are you now
suggesting that no leafref type whatsoever is needed, or are you
(still) concerned with leafref to non-key leafs?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 13 Aug 2009 13:17:06 -0700

> is problematic, a counter-proposal is in order.

this isn't really leafref we are talking about.
leafref has a hard-wired target object in the YANG file
that can never be changed.

The RMON use (and the varbind use) is really a
pair of objects -- an instance-identifier leaf
to identify the pointed-at-object, and an anyxml
node to represent the pointed-at-contents.

leafref does not provide anything like that.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 13 Aug 2009 13:33:04 -0700

> Yes, fixed in the upcoming -08.

this needs to be changed for for i-i too, because that is just
as bad, and confusing.  Sometimes an i-i points
to the <rpc>, sometimes to the <rpc-reply>,
sometimes to candidate, sometimes to running,
sometimes who knows where because the target
is specified in a description clause.

pg 132:

Does the <validate> operation check
the leafrefs against the candidate, and <commit>
checks them against the running config?
Or does the agent do whatever it wants?
Which "one of the NETCONF datastores" does an i-i represent?
This is very confusing.

IMO, i-i should have a static format:

  /nc:rpc/... --> in the <rpc> PDU
  /nc:rpc-reply/... --> in the <rpc-reply> PDU
  /ncn:notification/... --> in the <notification> PDU

  Everything else --> top-level node in the <running> database
  or conceptual state data available with <get>

The i-i data type is an absolute path expression,
so the parent context is irrelevant.

> (still) concerned with leafref to non-key leafs?

The recent issues I raised were discussed before and
resolved?  I do not remember that.  What was the resolution
to each issue?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 08:17:13 +0200 (CEST)

> > (still) concerned with leafref to non-key leafs?

You didn't reply to my clarifying question above, so I will assume
that you mean that you want to remove leafref completely.

> in the section about the leafref builtin type.

Fixed bug, as already mentioned.

> is nested within a union typedef.

That's why leafref is not allowed in union.

> 3) There are no compelling use-cases for including leafref objects

I strongly disagree.  Cross references are common in configuration
data.  For example, see the ipfix module.  For an SNMP example, see
how many MIBs have references to ifIndex.  Search the ML archives for
more examples.  Our customers use it (or rather the somewhat older
'keyref') all the time.

> unusable in the database.

IMO this falls under garbage in / garbage out.  You can get the same
problems with must expressions.  You can even get the same problem
with description clauses:

  leaf a {
     if-feature foo;
     type int32;
  }
  leaf b {
     type int32;
     mandatory true;
     description "This leaf MUST be a divisor of the leaf 'a'";
  }

> would be much better and much easier to use.

You agreed in
http://www.ietf.org/mail-archive/web/netmod/current/msg03104.html that
this should wait.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 03:20:16 -0700

> that you mean that you want to remove leafref completely.

no;
just leafref pointing at leafref and config=true leafref

> Fixed bug, as already mentioned.

i-i needs to be fixed as well.
The database MUST always be <running>, not up to the agent to decide.

> That's why leafref is not allowed in union.

good -- not easy to remember what types
are allowed where though.

> more examples.  Our customers use it (or rather the somewhat older

1) I am not convinced that the ipfix model is implementable
by mere mortals.  When somebody actually has it working in
running code, then I will be proved wrong.

2) INDEX by reference is trivial to support in SMIv2,
and could be as well in YANG.  YANG allows nested tabled
instead of flattened out tables like SMIv2, so table linkage
can be done multiple ways.

What is the use-case for leafref pointing at leafref?

>      description "This leaf MUST be a divisor of the leaf 'a'";

IMO, YANG has way too much GIGO!
YANG just ignores the broken protocol
behavior, and that is unacceptable.

You can define mandatory top-level objects that
cannot possibly be supported by any NETCONF agent.
You can have objects pointing at non-existent objects,
which also cannot possibly be implemented by any NETCONF agent.

YANG tools needs to check leafref:
  - mandatory pointing at optional
  - conditional pointing at different conditional
  - not conditional pointing at conditional
A YANG compiler MUST detect these fatal errors.
A YANG module is invalid and will not be usable
if any leafref object has these conditions.

If the agent contains any leafref objects whatsoever
which are no longer valid because the pointed-at object
is not available (e.g. due to deviations not-supported)
then what is supposed to happen to the leafref object(s)?
Is the agent supposed to remove all the leafrefs that
point at the not-supported leaf?  Just crash and burn?
How does YANG modify the NETCONF protocol (again)
so NETCONF works with YANG in this case?

> http://www.ietf.org/mail-archive/web/netmod/current/msg03104.html that

Yes -- eventually I want a varbind-based solution.
I don't care if it is added now or later.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 13:55:28 +0200 (CEST)

[resend]

> The database MUST always be <running>, not up to the agent to decide.

No, if the data store is candidate, then the i-i must point to
something in candidate, if you do a validate of candidate.

I propose the following change (and similar for the other case;
must/when/path):

OLD:

- If this leaf represents configuration data, the tree is the data in
  one of the NETCONF datastores.  The XPath root node has all
  top-level configuration data nodes in all modules as children.

NEW:

- If this leaf represents configuration data, the tree is the data in
  the NETCONF datastore where the leaf exists.  The XPath root node
  has all top-level configuration data nodes in all modules as
  children.

> What is the use-case for leafref pointing at leafref?

This is a cross reference to something which happens to have a cross
reference to something else.

If you want to discuss implementation strategies for leafrefs, I am
happy to do that (off-line).

> YANG tools needs to check leafref:

This is good - it is always easier to make progress when there is
a concrete proposal to discuss.

>   - mandatory pointing at optional

There is no reason this should be illegal - it might be bad design,
but the semantics are clear.  In order to create a valid
configuration, the mandatory leafref MUST be set.  And it MUST point
to something that exists, so the optional leaf (or more likeley, the
list entry) MUST also exist.

> if any leafref object has these conditions.

If by conditional you mean 'if-feature' I agree.  But it is not
trivial to describe this.  For example this should be valid:

  feature a { ... }
  feature b { if-feature a; ... }
  feature c { if-feature b; ... }

  list x {
    if-feature c;
    key y;
    ...
  }
  leaf z {
    if-feature a;
    type leafref { path /x/c; }
  }
    
Can you propose some text to describe this?  (Assuming this is what
you had in mind).

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 13:19:43 +0200 (CEST)

OLD:

NEW:

  feature a { ... }

  list x {
    if-feature c;
    key y;
    ...
  }
  leaf z {
    if-feature a;

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 06:23:37 -0700

>   has all top-level configuration data nodes in all modules as

rpc barf {
  input {
    leaf a {
      type leafref { path /foo/bar/baz; }
    }
  }
}

So which database is this leafref in?
Can the agent just pick one?

This proposal does not work.
The leafref MUST point at <running>.
The only valid config is <running>.
YANG specifies what goes in a valid database.
NETCONF specifies how the protocol operations work.

IMO, leafref and instance-identifier are both broken.

> happy to do that (off-line).

I think leafref and instance-identifier are really problematic
and need to be fixed or removed from YANG.

> list entry) MUST also exist.

bad design?  IMO broken YANG.
This is the mandatory-by-indirection part of leafref.
Anything a mandatory leafref points at is also mandatory.

> trivial to describe this.  For example this should be valid:

I mean any form of conditional object.
YANG has so many to choose from, and
a tool must get it right for all of them, not just 1.
This is close to impossible to implement,
but I could say that about several parts of YANG.

> Can you propose some text to describe this?  (Assuming this is what

yes - remove leafref -- it is broken,
complex, and easily misused.

I propose to let tools figure it out
like the rest of the complexity in YANG.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 15:32:32 +0200 (CEST)

> Can the agent just pick one?

Andy, please check the text.  See the last bullet in section 9.9.2.
With the change I propose for the first bullet, the new complete text
is:

   o  If the context node represents configuration data, the tree is the
      data in the NETCONF datastore where the context node exists.  The
      XPath root node has all top-level configuration data nodes in all
      modules as children.

> The leafref MUST point at <running>.

Yes, that how it works, since the last bullet apply in your rpc
example.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 07:10:49 -0700

> With the change I propose for the first bullet, the new complete text

I read the text.
I don't think it works.
How does the agent know from that PDU
which database it is in?

What would a leafref definition look like
that wasn't associated with <running> config?

  leaf huh {
     type leafref [path { /who/knows; }
  }

This definition is not specific to candidate or running or startup.
It is just a config leaf that exists in all 3 databases.
So the phrase 'the NETCONF datastore where the context node exists'
is totally wrong.  It assumes there is 1 datastore where the node
exists, and it assumes the agent will know which one.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 16:18:21 +0200 (CEST)

> How does the agent know from that PDU

Your example was this:

rpc barf {
  input {
    leaf a {

The context node is leaf "a" in the rpc.  Thus, the second bullet
applies, and the leafref points to <running/> and state.

> exists, and it assumes the agent will know which one.

No, if you validate the candidate, the context node is in the
candidate, and the node it refers to must also exist in the candidate.

/martin

---

From: "tom.petch" <[email-masked]>
Date: Fri, 14 Aug 2009 19:28:18 +0200

<inline>
Tom Petch

----- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "Martin Bjorklund" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, August 13, 2009 10:33 PM
Subject: Re: [netmod] leafref problems

> resolved?  I do not remember that.  What was the resolution
An alternative view.

1) Balazs flagged this bug soon after -07 appeared (and I updated my copy as a
result)

2) leafref in union is prohibited in s.9.12; I cannot recall any discussion on
this.

3) there was a lengthy discussion on leafref and its use cases commencing in May
2009, triggered by Joel saying, most circumspectly, that it was incomprehensible
(at least, that is what I thought).  Andy came up with five use cases, expanded
to six then contracted back to five.  I pointed out that it was Andy in a
similar thread in September who argued that leafref was needed, not just keyref.

As is all too often the case on this list, it is very hard, for me at least, to
understand what the consensus of this thread was (and I have re-read it at least
five times, including, coincidentally, this week and last), even on the
inclusion of require-instance; only when the I-D came out did I think that the
consensus had been for require-instance to be dropped from leafref. Balazs en
passant said it would be helpful if some clear statements made then were
included in the I-D, but, sadly, they were not.  Joel's characterisations I
found helpful, again not included.

So what was the consensus of that thread? I am unsure.  What does the I-D tell
me?  Again, in several places, I am unsure.

4) is a new one on me, although mandatory, in general, is another of those
topics which come up regularly, without a consensus that I can discern.  If, as
Joel/Balazs suggested, leafref constrains type and value, then a leafref to a
non-existent node means:
either there are no constraints, anything goes
or the constraint is total, the leafref is also non-existent

"All leafref nodes MUST reference
   existing leaf instances for the data to be valid"
suggests the latter.
Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 12:20:54 -0700

tom.petch wrote:
...
> 1) Balazs flagged this bug soon after -07 appeared (and I updated my copy as a

resolved

> 2) leafref in union is prohibited in s.9.12; I cannot recall any discussion on

resolved

> me?  Again, in several places, I am unsure.

I don't want to go over use-cases again.

All discussion of implementation details is kind
of irrelevant because it is too subjective.

But a manager needs to know what to expect from an agent
wrt/ all aspects of YANG, and leafref is no exception there.

> or the constraint is total, the leafref is also non-existent

I have about 5 warnings in yangdump related
to this.  I want them all to be fatal errors
instead.  The agent will use a module with warnings,
but not errors.

There are 2 new issues not resolved:
  1) YANG compile time:
     any possible scenario in which any path statement
     may not be valid in every possible compliant agent
     implementation MUST be considered a fatal error.
     The draft needs to make this clear.

  2) agent run time:
     - What to do when the agent has a leafref object
       with an invalid path statement?
     - What to do at the time the deviation 'not-supported'
       removes the pointed-at object?
    - What to do if a dynamic if-feature or when statement
      changes from true to false, with leafrefs pointing at the
      newly removed leaf?

>    existing leaf instances for the data to be valid"

So what happens to the pointing-leaf in the
running config when the pointed-at leaf goes away?
The running config cannot be invalid, but it is.

So the agent must iterate through a pruning process,
to make sure that each time it removes a dead leafref
that it is OK to do that, and no other leafrefs now
need to be pruned as well.

Or the agent can shutdown I guess,
but it cannot have an invalid running config.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Aug 2009 22:41:06 +0200 (CEST)

>      The draft needs to make this clear.

Please send me some suggestion for text to include!

>        with an invalid path statement?

Such a config is not valid.  It doesn't spontaneously happen - it
happens b/c an operator tries to modify the config.  This request must
be rejected, just as a request that would invalidate a must constraint.

>        removes the pointed-at object?

We had a proposal to state that the module must still be valid after
applying deviation statements.  But noone commented so it wasn't
included.  And I'm not convinced it is needed - deviations are
supposed to document what an implementation did, and it won't be
possible to implement this, as you noted.

>       changes from true to false, with leafrefs pointing at the

If a when statement changes to false, this is also an effect of an
operator modifying the config, which must be rejected.

> The running config cannot be invalid, but it is.

Again, the pointed-at leaf won't just "go away".  It goes away when a
client deletes it, and that will be an error.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 16:33:29 -0700

> Please send me some suggestion for text to include!

(Here's what yangdump checks anyway -- maybe there
is more to it than this...)

leafref node ==> leaf containing a leafref type
target leaf ==> leaf pointed at by a leafref path statement

The leafref node cannot be more conditional than
the target leaf.

The following restrictions apply:
  - if the leafref is mandatory, then the target leaf
    SHOULD be mandatory as well.  Managers need to make
    sure that an instance of the target leaf is present,
    or the leafref will not be valid.
  - if any if-feature statements apply to the leafref node,
    then they MUST also apply to the target leaf.
  - A when statement can apply to the leafref or target node
    in 3 ways (direct, inherited from all uses, inherited from
    augment).  The conceptual AND expression formed
    from these 3 XPath expressions MUST be identical
    in the leafref and target leafs.

> be rejected, just as a request that would invalidate a must constraint.

you are right.
this is no different than if the startup config
loaded into running is invalid.  The agent needs to deal
somehow with this situation already.

I would like there to be a standard way for an
agent to deal with a "good running config gone bad".

> operator modifying the config, which must be rejected.

maybe not the operator, but it does not matter.
the agent either has an internal or external trigger
to re-check the config, and the problem
will be dealt with at that time.

> client deletes it, and that will be an error.

I know the pointed-at leaf will go away.
So does the operator.   But the operator
may be astonished when the deleted-node ripple affect
deletes other nodes, not just the first one.
Side effects like this (and unlock(candidate) == discard-changes)
may cause the tech-support phone to ring a lot, and we don't
want that, do we?

Andy

---

From: WashamFan <[email-masked]>
Date: Sat, 15 Aug 2009 21:13:29 +0800

Hi,
 
>    existing leaf instances for the data to be valid"

+1
That is the exact way I understand leafref. A leafref pointing to
a non-existent node is invalid. And the removal of targeted-at
node either cause the leafref to be deleted or the removal request
is rejected at the beginning, IMO.

washam

---

From: Andy Bierman <[email-masked]>
Date: Sat, 15 Aug 2009 06:33:22 -0700

> is rejected at the beginning, IMO.

with 'deviate not-supported', there is no manager request.
what about that one?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 15 Aug 2009 09:12:40 -0700

WashamFan wrote:
> From: Andy Bierman <[email-masked]>
> Date: Saturday, August 15, 2009 9:34 pm
> To: WashamFan <[email-masked]>
> Cc: "tom.petch" <[email-masked]>, Martin Bjorklund <[email-masked]>, [email-masked]
> 
> 
>> WashamFan wrote:
>>> Hi,
>>>  

So some agents will magically delete extra nodes and others will
reject the request?  Can we pick one standard agent behavior here?

> IMO, The action to remove the targeted-at node would delete
> the leafref node either regardless of the manager or system
> trigger the action. Otherwise, the datastore would be invalid.
> 

so what is the text for the one standard agent behavior
for this situation?

Andy

---

From: WashamFan <[email-masked]>
Date: Sat, 15 Aug 2009 23:51:17 +0800

Hi,

----- Original Message -----
From: Andy Bierman <[email-masked]>
Date: Saturday, August 15, 2009 9:34 pm
Subject: Re: [netmod] leafref problems
To: WashamFan <[email-masked]>
Cc: "tom.petch" <[email-masked]>, Martin Bjorklund <[email-masked]>, [email-masked]

washam

---

From: WashamFan <[email-masked]>
Date: Sun, 16 Aug 2009 09:51:23 +0800

Hi,

> >>>> 4) is a new one on me, although mandatory, in general, is another 
> 
> >> of those
> >>>> Joel/Balazs suggested, leafref constrains type and value, then a 
> 
> >>>> leafref to a
> >>>> non-existent node means:

Then, I'd like pick the former.

> so what is the text for the one standard agent behavior

My interpretation, the referenced leaf defines the value
constraint for the corresponding referencing leaf. The value
constraint would change with the referenced leaf. When the
referenced leaf is gone (regardless of who trigger it), the 
constraint is total, which means, the referencing leaf's gone
with it.

So, IMO, there might be something like that:

With regard to any attempt (manager request, execution of
'not support' deviation,etc.) to remove a leaf which is referenced
by another leaf via leafref, the referencing leaf would be deleted 
automatically with the referenced leaf, if the attempt succeeds.

washam

---

From: "tom.petch" <[email-masked]>
Date: Sun, 16 Aug 2009 10:18:40 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, August 14, 2009 3:32 PM
Subject: Re: [netmod] leafref problems

> > >   has all top-level configuration data nodes in all modules as

'top-level' appears many times in the i-d and I would like to see the meaning
better nailed down .

It could refer to the substatements immediately below a module statement.

It could be extended to the substatements immediately below the submodule
statement of an included submodule.

Since the reference, as here, is frequently to top-level data nodes, it could be
read as any data node which has no other data node between it and the module
statement so that in

module m {grouping g { leaf x ...

then x is regarded as top-level

Or ...

I would like this clarified, here and in the I-D

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Sun, 16 Aug 2009 12:37:07 -0700

> Then, I'd like pick the former.

OK -- I'll go with that.
Whenever possible, pick the least astonishing choice,
and since when-stmt, case-stmt, and NP containers can
disappear as a side-effect, this should as well.

For the most common use-case (foreign key), this
is what you want.  You delete the main table entry,
and all the sub-ordinate entries get deleted as well.
If these other tables were nested within the main
table the way they should be, that is also what would happen.

> automatically with the referenced leaf, if the attempt succeeds.

a bit confusing, but Martin will fix it :-)

for deviate not-supported, there is no
request and maybe this will succeed.
If the syntax is valid, the agent is
declaring a module patch, and all leafrefs
pointing at this leaf are also not supported
and will be removed.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 10:39:28 +0200 (CEST)

>     or the leafref will not be valid.

According to this rule, would this model invalidate the SHOULD?

  list a {
    key x;
    leaf x { type int32; }
  }
  container b {
    presence "...";
    leaf y {
      type leafref { path /a/x; }
      mandatory true;
    }
  }

>     then they MUST also apply to the target leaf.

Is this text clear enough to just include in the draft?  Is it clear
what it means that an "if-feature statement apply"?

>     in the leafref and target leafs.

How do you check if the conceptual expressions are identitical?  If we
really want to include text along these lines, it will require some
effort to describe the logic in an interoperable way.

> agent to deal with a "good running config gone bad".

But running won't just go bad - in fact running cannot go bad.  So if
the server gets an edit-config with removes a leaf that is referenced
by a leafref, it will reply with a 'data-missing' error, with the
error-app-tag 'instance-required' (see section 13.6).

> deletes other nodes, not just the first one.

There is no "deleted-node ripple effect".  The delete on the first
node will fail.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Aug 2009 01:48:28 -0700

>       type leafref { path /a/x; }

why would it?
A key leaf is implicitly mandatory.

> effort to describe the logic in an interoperable way.

you really can't do more than string compare the XPath
expressions -- a totally stupid thing to do,
but that's already how deviations work.

> error-app-tag 'instance-required' (see section 13.6).

where is that specified in the draft?

I like Washam's plan better -- you delete the leaf,
and all its references go away as well.

> There is no "deleted-node ripple effect".  The delete on the first

what about a choice-stmt than contains a case
that contains a leaf that is pointed at by a leafref?
The agent is forced to remove that case when a new one
is entered.  This also leaves dangling references to
a deleted leaf.  You want that to work sometimes, but
not others?

We already have a deleted-node ripple effect.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 10:58:45 +0200 (CEST)

> A key leaf is implicitly mandatory.

B/c the list may have in 0 instances.  Why would that be different
from

  container a {
    leaf x { type int32; }
  }

I prefer to not add the SHOULD suggested above.

> where is that specified in the draft?

Section 8.3.3 says that constraints for running are checked at
edit-config time.  Section 9 says that a leafref is such a
constraint.  And section 13.6 specifies the error message to use.

> and all its references go away as well.

This has been discussed before, and it was rejected at the time.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 11:02:45 +0200 (CEST)

Hi,

"tom.petch" <[email-masked]> wrote:
> statement

Yes.

> so that in

No, the grouping statement does not instantiate any data nodes.  If
you also had:

  uses g;

then x would be a top-level data node.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Aug 2009 04:23:34 -0700

> This has been discussed before, and it was rejected at the time.

I do not remember this.
I remember you insisted an agent must
delete the old case in a choice when a new case
is set.  Are you saying now that is not what
the agent does?

  choice X {
     leaf a { type string; }
     leaf b { type string; }
  }

  leaf Y {
     type leafref { path /X/a/a; }
  }

Are you saying that if /b is set, so /a is deleted,
that this will fail if /Y exists, but work if /Y
does not exist?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 14:13:08 +0200 (CEST)

> > This has been discussed before, and it was rejected at the time.

See the mail thread "leafref issues", started by you 27 May 2009.  A
couple of emails objected against introducing automatic deletions of
references, and no one insisted that it should be added.

> is set.  Are you saying now that is not what

No.

Yes.

Just like this:

  leaf Y {
    type string;
    must ". = /X/a";
  }

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Aug 2009 05:53:36 -0700

>> that this will fail if /Y exists, but work if /Y

but it is not just like a must-stmt.
If the data model designer wants the behavior
you are are suggesting, then they will use the must-stmt.

Let's say an interface entry has 100 tables hanging
off of it, all dependent on the existence of that
specific interface entry.  It would be extremely cumbersome
to manually unwind all these data structures bottom up
because they were designed like SMIv2 tables (foreign
keys instead of nested tables).  If the operator deletes
an interface, then it is clear these tables with foreign
keys are no longer needed as well.

When an external config leafref points at a list entry,
the normal behavior is also to delete the reference.
Going back to an early example, it makes no sense at all
to keep the 'my-favorite-channel' leafref if the 'channel-list'
is deleted.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 15:07:51 +0200 (CEST)

>From a validation point of view, a leafref behaves like a leaf with a
must statement.  But I agree; a leafref is more than a must in that it
expclicitly defines a cross reference.

> keys are no longer needed as well.

But as you said earlier we expect people to not use the flat
structure, but rather make use of the hierarchy we have.

Going back to the example which you said is valid YANG:

  list a {
    key x;
    leaf x { type int32; }
  }
  container b {
    presence "...";
    leaf y {

Suppose /a[x=1] exists, and /b/y is 1.  Now if /a[x=1] is deleted,
/b/y should be auto-deleted, right?  Does this mean that /b is
auto-deleted as well?  This will indeed cause a delete ripple effect,
which I think would be very confusing and bad.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Mon, 17 Aug 2009 09:13:34 -0400

>which I think would be very confusing and bad.

Are you talking only about running?  In candidate, auto-deletion
of /b/y is not reasonable, since the user could immediate make/load
a new a[x=1].  The rules apply at validation time, so a workplace
like candidate can be invalid during the construction of a new
configuration.

Working backwards from there, running should be valid after each
edit-config, so an edit-config can deletes a[x=1] would make the
configuration invalid.  Is the best outcome to fail the operation
(if test-then-set) or to auto-delete to force the configuration to
be valid?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Aug 2009 15:40:07 +0200 (CEST)

> like candidate can be invalid during the construction of a new

This is another reason for keeping the current behavior (i.e. no
auto-deletion).  If we compare with 'choice', cases are auto-deleted,
but it works the same in the candidate and running.  I don't think we
should introduce another flavor of auto-delete for leafrefs.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 17 Aug 2009 15:41:58 +0200

> (if test-then-set) or to auto-delete to force the configuration to

My stomach says the operation should fail and we should ensure
referential integrity. If we do automatic deletions, then of course I
expect the same behaviour if I commit candidate to running at commit
time.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Aug 2009 06:49:07 -0700

> (if test-then-set) or to auto-delete to force the configuration to

It comes down to a simple unix analogy:

Do we want "rm -r" or "rm -rf"?  (both of course)

I am thinking about a similar code-point with a 'delete-all' value
for the nc:operation attribute (for list and leaf-list).
We could use the --force option in NETCONF by adding another
enum to the nc:operation.  (We could bundle all the new tweaks
into 1 :newstuff-1.0 capability, instead of a bunch of independent
tiny enhancements).

The default in unix is "rm" (reject with error),
so that seems to favor Martin's outcome.  Done.

I like your twist -- put this in the commit check,
so a dangling leafref reference in candidate is not an error.
(should already be coded that way).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 18 Aug 2009 15:55:29 +0200 (CEST)

> The leafref node cannot be more conditional than

Hmm.  It should be the other way around, right?

   list x {
     if-feature foo;
     key y;
     ...
   }
   leaf xref { // legal
     if-feature foo;
     if-feature bar;
     type leafref { path /x/y; }
   }
   leaf xref2 {  // illegal
     type leafref { path /x/y; }
   }

How about this:

  If the leaf that the leafref refers to is conditional based on one
  or more features (see ^if-feature^), then the leaf with the leafref
  type MUST also be conditional based on at least the same set of
  features.

/martin

---

From: "tom.petch" <[email-masked]>
Date: Tue, 18 Aug 2009 18:42:55 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 17, 2009 11:02 AM
> > Since the reference, as here, is frequently to top-level data nodes, it
could
> > be

Thanks Martin,

Logical but, taken together, that's a surprise.  So  with suitable imports and
prefixes

module n  {uses g ...

x is not top-level in module m but is top-level in module n.

I would like this spelt out in the I-D; I suggest amending
5.5 to become
    only top-level types and groupings
ie those appearing as substatements to a module or submodule statement
   can be used outside the module or

6.2.1 bullet 7 I am unclear about; does a
'module n { uses g
make leaf x top level in this case?

Add
A top-level configuration data node is one where there is no other data node
between it and a module or submodule statement.  Note that the 'grouping' and
'uses' statements do not themselves instantiate data nodes but that a 'uses'
statement that is not within a grouping instantiates the nodes that appear in
the  'grouping' statement that it references.

If that sounds complicated, well it is:-)

It is needed for the usual suspects, 'when' 'path' 'instance-identifier' 'must',
could appear in all four or perhaps better in 6.2.1 or 6.4.
Tom Petch

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Aug 2009 08:52:43 +0200 (CEST)

Yes.

>    can be used outside the module or

Ok, fixed.

> make leaf x top level in this case?

Yes.  I changed this a bit into:

- All leafs, leaf-lists, lists, containers, choices, rpcs,
  notifications, and anyxmls defined (directly or through a uses
  statement) within a parent node or at the top-level of the module or
  its submodules share the same identifier namespace.  

> could appear in all four or perhaps better in 6.2.1 or 6.4.

How about putting it in the Terminology section?  Also, I am not
convinced the "Note" part is necessary.  Section 7.11 on grouping
says:

   The grouping statement is not a data definition statement and, as
   such, does not define any nodes in the schema tree.

Note that we also have:

- data definition statement: A statement that defines new data
  nodes.  One of container, leaf, leaf-list, list, choice, case,
  augment, uses, and anyxml.

So I hope it is clear that 'uses' defines data nodes but groupings do
not.

I propose to add this to Terminology:

- top-level data node: A data node where there is no other data node
  between it and a module or submodule statement.

/martin

---

From: "tom.petch" <[email-masked]>
Date: Wed, 19 Aug 2009 15:47:14 +0200

---- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, August 19, 2009 8:52 AM

> > x is not top-level in module m but is top-level in module n.

Logical

> >    can be used outside the module or

Good

>   its submodules share the same identifier namespace.

Good

>   between it and a module or submodule statement.

Um.   We have schema tree and data tree and schema nodes go in the schema
tree and data nodes go in ... the schema tree:-(  I have tripped over this more
than once, so I am wary around data nodes.

But it is grouping that is the bigger problem.  It is not even a schema node,
yet its
position w.r.t schema nodes is significant, top-level or no, what parent it has.
I think of it as another type of node, but don't have a term for that type.

Thus there are
data nodes:
  container, leaf, leaf-list, list, anyxml
schema nodes:
  data nodes plus
  choice, case, rpc, input, output, notification
grouping:
  a reusable set of nodes
which aren't actually anything until a uses statement appears.

So while the clarification is there in 7.11 it is missing from terminology, and
until
I had grasped all the YANG statements, I had not noticed its absence.  Hence my
tendency to hammer the point that grouping may not be anything like you've seen
before (see prefixes in groupings:-).

Pro tem., I suggest adding your definition of top-level data node to Terminology
but also adding that sentence from 7.11 to grouping in Terminology.

And I will mull over it some more, how else grouping might be characterised.

Tom Petch

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Aug 2009 21:42:13 +0200 (CEST)

> but also adding that sentence from 7.11 to grouping in Terminology.

Just to make sure I get it right... when you say "that sentence", do
you mean:

/martin

---

From: "tom.petch" <[email-masked]>
Date: Fri, 21 Aug 2009 19:05:18 +0200

----- Original Message ----- 
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, August 20, 2009 9:42 PM
Subject: Re: Terminology2 was Re: [netmod] leafref problems

>   such, does not define any nodes in the schema tree.

Martin 

Yes, I meant add that statement to the definition of grouping in terminology.

Tom Petch