Subject: [netmod] leafref issues

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2009 08:07:46 -0700

Hi,

I am trying to keep up with all the leafref issues, but it
is difficult.  Maybe we can summarize them, eliminate
duplicate issues, etc.

------------------------------------------------------------------
use-case chart:

  1) a read-only leafref in a notification is like an alias

  2) a leafref acting as a key, and pointing at another key,
     is like an SQL foreign key

  3) a read-only leafref acting as a leaf, and pointing at a key,
     is almost like a RowPointer in SNMP

  4) a read-write leafref acting as a leaf, and pointing at any
     leaf, is like a 'MyServer' kind of config copy.  Some localized
     config is constrained to the global config values.

  5) an RPC input parameter, pointing at any database leaf,
     is just another form of constraint semantics.

  6) vip-slot example (not sure what to call this mode.)
     It is not a shadow, but a secondary knob which
     is constrained to the values of an external non-key leaf.
     (RMON users might think of the 'dataSource', which is a
     just a column in the main table, but an INDEX in other
     tables.)

-----------------------------------------------------------------

Issues:
   1) require-instance, needed or not?
   2) use-case 4, needed or not?
   3) use-case 6, needed or not?
   4) behavior when pointed-at leaf is deleted
     (or value in leafref no longer in use)?
   5) nested lists not supported (**)

------------------------------------------------------------------

** One issue not mentioned is that leafref is severely limited
in utility because all key value pairs except the innermost layer
need to be hardwired into the path expression.
For example, use-case (1) is not actually supported
by leafref for any nested lists, for this reason.

For notifications, the only thing that will really work
is a varbind list:

   list varbind {
      config false;

      leaf name {
         description
           "Identifies the leaf in the running database
            associated with this variable binding.";
         type instance-identifier;
         mandatory true;
      }

      leaf value {
         description
             "Contains the value of the leaf instance identified
              by the associated name field.  This field will
              not be present if the variable binding has no value,
              e.g., built-in type is 'empty'.";

         // could be union of all built-in types or just
         // the string representation of the canonical encoding
         type string;
      }
   }

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 27 May 2009 11:19:44 -0400

Pulling out one piece from Andy's list...

> by leafref for any nested lists, for this reason.

In an off-list conversation, I apparently got confused between "it 
doesn't mean that" and "it shouldn't mean that."

it seems to me that the current restriction in the path statement that 
only the last key value be unbound does not actually do what we want.
It allows one to know what leaf is being "referenced."
But a leafref is not actually a reference.  It is a value constraint.
As such, it would work just fine with multiple unspecified keys.  There 
would be no way from just the leafref to determine which leaf has the 
value that the leafref is using.  But that is not the purpose of the 
leafref.  (You can still have a leafref that points to a key, and where 
only the terminal key is unspecified.  But one would not be restricted 
to that.  Also, other xpath expressions would allow netconf applications 
to find the leaf with the given value.  YANG does not need to "find" the 
leaf itself.)

If we remove that key-specifying condition in the path statement, we 
pcik up significant value, and reduce the constraints on the structures 
with which leafref can be used as a value constraint (which seems to be 
what it is intended for.)

Joel

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2009 08:56:06 -0700

> Pulling out one piece from Andy's list...

Add one more: access control issues related to #4 and #6

> But a leafref is not actually a reference.  It is a value constraint.

No -- the agent/tool really does need to find the target leaf
in order to use the correct type statement for validation
and canonical output.

> to find the leaf with the given value.  YANG does not need to "find" the 

What does the last sentence mean?

> what it is intended for.)

IMO, the leafref is so complicated, that making even more variants
of it seems like the wrong direction.

In fact, at least 2 ACM implementations use an instance-identifier
as you described (optional key predicates), to specify the
data associated with an ACL.  But not a leafref -- an XPath string
that (almost) conforms to the instance-identifier data type.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 27 May 2009 12:05:27 -0400

Why does the agent need to find a specific leaf in order to find the type?
It seems that any leaf with the given path, but with different keys at 
different places, will have the same type.  (Different cases in a choice 
should result in different node names, not just different key values.)

Joel

Andy Bierman wrote:
>> As such, it would work just fine with multiple unspecified keys.  
>> There would be no way from just the leafref to determine which leaf 
>> has the value that the leafref is using.  But that is not the purpose 
>> of the leafref.  (You can still have a leafref that points to a key, 
>> and where only the terminal key is unspecified.  But one would not be 
>> restricted to that.  Also, other xpath expressions would allow netconf 
>> applications to find the leaf with the given value.  YANG does not 
>> need to "find" the leaf itself.)
>>
> 
>> pcik up significant value, and reduce the constraints on the 
>> structures with which leafref can be used as a value constraint (which 
>> seems to be what it is intended for.)
>>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 May 2009 18:26:15 +0200 (CEST)

> is difficult.  Maybe we can summarize them, eliminate

Why is the leaf read-only in this use-case?

IMO, the most important use-case is read-write leafref acting as key
or leaf, pointing to another key.  (this was all the original keyref
type did).

>      just a column in the main table, but an INDEX in other

People have reported that require-instance makes leafrefs more
complicated.  Personally, I don't see that, but we need to make
progress and solve this.  I prefer not to, but I can live with
removing require-instance and also add a CLR to prevent use-case 4.

>      (or value in leafref no longer in use)?

Is this an issue?  It works just like a false must constraint - the
config becomes invalid.

>    5) nested lists not supported (**)

No, any depth and number of keys are supported.  But in order to
identify a leaf, you need values for all keys on all levels.  One
leaf can only hold one value, so it means you need as many leafrefs as
there are keys.  An example from the draft:

  container default-address {
      leaf ifname {
          type leafref {
              path "../../interface/name";
          }
      }
      leaf address {
          type leafref {
              path "../../interface[name = current()/../ifname]"
                 + "/address/ip";
          }
      }
  }          

This syntax becomes clumsy when there are more than 2-3 keys, but it
is possible.  

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2009 10:08:20 -0700

> or leaf, pointing to another key.  (this was all the original keyref

this is use-case #2

>>      just a column in the main table, but an INDEX in other

module A {
   list base {
      key x;
      leaf x { type uint32; }
      leaf pointed-at {

   leaf a {
      type leafref {
      // is it OK to leave out the '[x=n]' key predicate?
      path "../base/pointed-at";
   }

   leaf b {
      type leafref {
      path "../a";
   }
}

Given this config:

   <base>
     <x>17</x>
     <pointed-at>/A:a</pointed-at>
   </base>
   <base>
     <x>42</x>
     <pointed-at>/A:b</pointed-at>
   </base>

It is clear that leaf 'a' is allowed to contain
the values /A:a or /A:b, and leaf 'b' is only allowed
to contain the current value of leaf 'a'.

So if the operator wants to configure an ACM rule
to restrict read access to '/base/pointed-at',
does that mean the operator needs to track down every single
leafref in the system that directly or indirectly
references '/base/pointed-at' and configure an ACM rule
for that leaf as well?

This sub-statement is also used in instance-identifier-specification.

Will require-instance go away completely?

> removing require-instance and also add a CLR to prevent use-case 4.

How do we distinguish #4 from #6?

> Is this an issue?  It works just like a false must constraint - the

so that means the constraint is enforced at commit-time
on candidate and edit-config-time on running.

> identify a leaf, you need values for all keys on all levels.

This is the problem for use-case #1.
It is too clumsy in YANG to specify what SMIv2 does so easily:

   - specify the database object to be included in the payload
   - the payload includes the instance-identifier of the selected
     instance of the specified object, and its current value

   One

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 May 2009 21:12:19 +0200 (CEST)

> > or leaf, pointing to another key.  (this was all the original keyref

Ok, so use-case #2 should be "acting as a key or leaf".

> references '/base/pointed-at' and configure an ACM rule

Well, there is no such thing as an ACM (yet).  But if this is a
leafref, rather than just type reuse and some description text, then
the ACM could actually handle this, so that a user that is not allowed
to read X is also not allowed to read a pointer to X (unless
explicitly allowed maybe).

Compare with SNMP - suppose you want to restrict read access to
ifIndex, then you will have to track down all places an interface is
referenced and configure VACM accordingly.

> Will require-instance go away completely?

If we remove it for leafref, I think it should be removed for
instance-identifier as well.  We should remove it if we see no value
in having references to not-yet configured data; this applies to both
types of "pointers" we have.

> How do we distinguish #4 from #6?

#4 is a config leafref that points to a non-key leaf.

#6 is a config leafref that points to a non-config leaf

> on candidate and edit-config-time on running.

Yes.

> This is the problem for use-case #1.

So do you still think that "nested lists are not supported"?

>      instance of the specified object, and its current value

I agree.  There are two reasons for this, I think.  One is XML vs. the
varbinds in SNMP, where the key in each key/value pair contains the
complete instance identifier.  The other reason is that all object
names are unique per module in SMIv2, which makes it easier to
reference an object.

It would be great to solve this problem in NETCONF/YANG.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2009 13:18:34 -0700

> #6 is a config leafref that points to a non-config leaf

I am strongly opposed to #6 because it requires
the agent to splice non-config parts of the running config
into the candidate database, to determine if the
config leafref is being set to a proper instance
of the the non-config leaf.

IMO, the candidate database MUST only contain config=true nodes.

I am in favor of disallowing both #4 and #6, and only
allowing use-case #2 for config=true leafref database nodes.

Removing require-instance (hardwire to true) is fine too.
The example I gave had a chicken-and-egg instance-identifier/leafref
problem.  The pointed-at leaf is not actually allowed to
contain the value /A:a or /A:b until either of those nodes exists,
but neither of those leafs is allowed to exist until pointed-at
exists first.

Setting require-instance to false on the pointed-at leaf
is the easiest way to solve this problem.  Or just remove
use cases #4 and #6 -- that also solves the problem.

> So do you still think that "nested lists are not supported"?

not easily supported, but this is a rare event,
so it probably doesn't matter

> names are unique per module in SMIv2, which makes it easier to

Yes -- in YANG, the schema-identifier would need to be
used instead:

(Think I posted this 2 or 3 times already.

   notification foo {

     varbind a {
         // 5 clauses, path mandatory, showing the defaults
         path "/interfaces/interface/ifMtu";
         status current;
         description "this is like an SMIv2 OBJECTS entry";
         reference "...";
         mandatory false;
     }

     varbind b {
         path "/interfaces/interface/ifInOctets";
         mandatory true;
     }

     leaf local-var {
         description
           "this is likse an SMIv2 a notify-only data definition.";
         type string;
     }
   }

   <notification>
      <foo xmlns:if="...">
         <eventTime>...</eventTime>
         <a>
            <name>
              /if:interfaces/if:interface[if:name='eth0']/if:ifMtu
            </name>
            <value>1500</value>
         </a>
         <b>
            <name>
              /if:interfaces/if:interface[if:name='eth0']/if:ifInOctets
            </name>
            <value>38765552</value>
         </b>
         <local-var>
             interface eth0 status string inserted here
         </local-var>
      </foo>
   </notification>

> It would be great to solve this problem in NETCONF/YANG.

yes

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 27 May 2009 16:35:46 -0400

Comments after Andy's text, with partial agreement.

> use cases #4 and #6 -- that also solves the problem.

Removing case 6 (config leaf with a leafref to non-config data) does 
seem like a good idea.  Otherwise, very strange inconsistencies would 
seem possible.

However, I do not see what the problem with case 4 is, and it seems to 
have value.  However, this is based on an assumption.  That assumption 
is that it is reasonable to multiple unbound keys in the path statement 
of a leafref.  The leafref is merely constrained to have a value pointed 
at by some key sequence.  In that model, a config leafref pointing to a 
non-key config leaf still has multiple values, so it can be set (i.e. is 
meaningful config information.)  If a leafref pointing to a non-key leaf 
can not have any unbound keys in the path, then there is no point in 
such, since although it claims to be config data, it can not 
meaningfully be set (its value is constrained to a single value at any 
given time.)

Yours,
Joel

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 May 2009 23:36:43 +0200 (CEST)

> of the the non-config leaf.

No, because in when a config point to a non-config, require instance
must be false.

> IMO, the candidate database MUST only contain config=true nodes.

Absolutely.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 May 2009 15:00:52 -0700

> No, because in when a config point to a non-config, require instance

but what if require-instance goes away,
and is hard-wired to true?

Will it be hard-wired to false for any leafref
pointing at a non-config leaf?

This is all a bit confusing.
It is not clear what is allowed, or what sub-statements
are being kept, or going away, or changing again.

I still do not like this because the XPath
target is associated with a specific database,
e.g., the <source> parameter in the <get-config> operation.

An XPath expression that is evaluated against the candidate database
should be limited to nodes actually in the candidate database.
Although 'unknown-element' is a just a no-match condition,
it is extremely confusing to treat references to non-config
nodes in some special way, within an XPath expression.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Thu, 28 May 2009 09:43:03 +0200

Something simple for a change:-)

I like the coverage of XPath in -05 but think it needs a little tweaking.

XPath appears in 'must' (7.5.3), 'augment' (7.15), 'when' (7.19.5), 'path'
(9.9.2), 'instance-identifier' (9.13) and 6.4 but is not treated consistently.

XPath needs a context - node, position, size, variable bindings, function
library and namespace declarations - and, since this is not a document, a
specification of the XPath root node.

6.4 says something about context node and namespaces

'must' and 'when' cover context node, function library, namespace declarations,
XPath root node.

'augment', 'path', 'instance identifier' cover none of this.

Why the difference?  Can some of this be in 6.4, eg on function library, if it
is common to all use cases of XPath?  Possibly likewise with namespaces (I
struggle to decide which namespaces are current for XPath within the assemblage
of module/submodule/grouping/import/include/uses/augments)

And 'augment', 'path', 'instance-identifier' prohibit axes, which leaves me
stumped.   'self', 'parent', 'child', 'descendant' are all axes and without
them, I struggle to create a meaningful XPath expression.

I also struggle with 'sibling'; this is not an axis but 'following-sibling' and
'preceding-sibling' are, but we do not have a document order so presumably can
never use either form of sibling axis; if so, this would fit with 6.4.

Tom Petch

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 28 May 2009 11:05:55 +0200 (CEST)

> pointing at a non-config leaf?

One reason we added require-instance was to be able to handle #6.  So
if we remove require-instance, I think a config leafref MUST
refer to a config leaf, and a non-config leafref can refer to
anything.

> are being kept, or going away, or changing again.

Nothing has been decided yet, we're still trying to understand the
consequences of making various changes.  

If we remove require-instance, I believe we get these combinations:

  a) config leafref can refer to config leaf or key
     use-case: #2 and #4
 
  b) non-config leafref can refer to any config or non-config leaf or
     key 
     use-case: #1, #3, and #5

It seems we have a couple of people that want to remove
require-instance (and thus use-case #6), and a couple that can accept
this decision.  Any objections to removing require-instance for
leafrefs?

If we remove require-instance for leafrefs, the question is if we
should also remove it for instance-identifiers.  I earlier said yes,
but I have changed my mind.  One example is if you have an access
control model based on instance-identifiers.  Then it would be very
difficult to specify rules only for existing instances.

The other issue is if we should add a CLR to prevent usecase #4.  I am
not sure if anyone is actually proposing this, although a couple of
people have said they can accept it.  Joel explcitily said he wants to
keep it.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 28 May 2009 11:56:55 +0200 (CEST)

Hi Tom,

"tom.petch" <[email-masked]> wrote:
> Why the difference?

Ladislav pointed this out as well.  Specifically, he wanted this spec
for 'path'.

> Can some of this be in 6.4, eg on function library, if it
> is common to all use cases of XPath?

The function library is common to all use cases which allow
functions (must and when).  Note that augment, path and
instance-identifier all have a very restricted form of XPath where
functions and variables cannot appear.

> Possibly likewise with namespaces (I

The intention is to allow the abbreviated syntax which has the 'child'
axis as default, but prohibit explicit axis specification.
E.g. /foo/bar is legal, but /foo/child::bar is not.  Do we need to
clarify this in the text?

As an alternative, these expressions could be defined w/o mentioning
XPath at all, e.g. augment is a '/' separated string of node
identifiers.   But I'm not sure this would be better / simpler.

> never use either form of sibling axis; if so, this would fit with 6.4.

Andy wrote about this in the guidelines docuement.  Do you think we
need to include anything from his text in this document, or can be
leave it there?

/martin

---

From: "tom.petch" <[email-masked]>
Date: Thu, 28 May 2009 18:42:25 +0200

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Thursday, May 28, 2009 11:56 AM
> Ladislav pointed this out as well.  Specifically, he wanted this spec

Ah, I missed that; I would like it for all three, or else a note as
to why this is not an issue.  I think that namespaces is the aspect I cannot
readily work out (import/include/uses/augments ...etc)

> functions and variables cannot appear.

Well, to be picky, path allows the current() function.

> clarify this in the text?

I think so; I would stay with XPath as a base since it is used elsewhere but
I think that the text on axes should be explicit - self, parent, child,
descendant? - rather than implicit by saying that any axis that can be specified
by abbreviated syntax is valid (except attribute because there are none) or what
we have now
which for me is a contradiction; axes specified via the abbreviated syntax are
still axes to me.

> need to include anything from his text in this document, or can be

I would include it in this document

Tom Petch

---

From: "tom.petch" <[email-masked]>
Date: Thu, 28 May 2009 18:51:03 +0200

Andy

I like the taxonomy but struggle with two potential attributes of each case:
- whether there is a second instance created, which has an independent life
cycle as opposed to creating a second path to a single instance.

In 1), I had assumed one instance multiple paths but Martin says not.

- and whether the leafref has write access; in 2), I assume that the foreign
key is read only ie as a data modeller, if I have designed and specified a
key and its use, I don't want someone else coming along and changing it.
I don't doubt that you can implement multiple write but would not like to see it
on offer.

I would like to be clearer on this before saying how many of the use cases I
would like to bin:-)

Tom Petch

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 28 May 2009 15:54:11 -0400

I believe the intent and agreement is that there are two separate 
instances.  The leaf of type leafref has a value.  The leaf (or leaves) 
pointed to by the paths have their own value(s).  The path selected 
leaf(s) define the value type of the leaf which has a type of leafref. 
The values of the targeted leaf(s) define the permissible values, just 
as a range restriction does for other types.
This is not two paths pointing to the same place.  It is two separate 
things, with a constraint relationship.  For non-config data, 
particularly for non-config data where the path points to a single leaf, 
this will look like it is two paths to the same operational data.  But 
it is still really two separate leaves.

Joel

tom.petch wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 May 2009 15:07:58 -0700

> this decision.  Any objections to removing require-instance for

I'm still unclear on all the implications of removing
require-instance.  We should think this through carefully
and remember why it was added in the first place.

I finally understand use-case #6, wrt/ use-cases
for pointing at state data with config data.

A good example is the "favorite channels" feature
built into most set-top CATV boxes.

You don't get to pick the set of valid channels.
That is done by the service provider.  So the
set of pointed-at values is read-only to you.
The set of 10 favorite channels is read-write to you,
but each member of the set must be one of your
enabled channels.

It only makes sense if require-instance=true though.

BTW, there is no reason a leaf-list could not work here:

   leaf-list channel {
      type uint32 { range 1..9999; }
      config false;
   }

   leaf-list favorite-channel {
      type leafref { path /channel; }
      max-elements 10;
   }

So why is leaf-list disallowed again?
All instances are required to be unique.
It's like pointing at a key leaf, but the
rest of the list is missing ;-)

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 28 May 2009 18:21:30 -0400

a) I have no problem with allowing leafrefs to point to leaf-lists.  I 
actually thought it was allowed since it made so much sense to me.

b) With regard to the example of config data pointing at state data, (in 
this case, favorite channel), the quesiton is what should happen if the 
favorite channel is no longer available.
b1) If that favorite will get deleted, then a leafref may well make 
sense, but the operational handling requirement is hard to describe.
b2) If the entry will remain, even though the channel no longer 
"exists", then it is not a leafref.  It is a number, which corresponds 
to a channel iff there is one.  While we could try to capture that level 
of semantic in the type system, it seems quite sensible to just say no.

Just my personal reaction to a clear example.
Joel

> rest of the list is missing ;-)
...

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 May 2009 15:54:47 -0700

> of semantic in the type system, it seems quite sensible to just say no.

this is a great summary.
I agree that this issue is very important to clarify
in the draft.  Saying that the config becomes invalid
doesn't really work because it's the running config
where this matters, and an invalid running config is
not allowed.

So in the candidate config, the constraint is not being
enforced.  If the <commit> operation is eventually given,
then it will fail.

I think solution (b1) makes the most sense, when
dealing with running config.

So, when is leafref:require-instance=true enforced
in the running config?

 From an implementation POV, it is undesirable to check
if any possible leafrefs changed from valid to invalid,
every time some non-config node is modified or deleted.
But that is what solution b1 for use-case #6 requires.
That is one reason I prefer to avoid #6 completely.

Requiring that all usage like #6 (config --> non-config)
has to set require-instance=false seems less that useful,
especially if solution b2 is used instead of b1.
Might as well just declare a common typedef (ChannelNumber)
and use it in both objects.  There is no instance coupling,
just a common type.

I am in favor of removing require-instance for leafref
but leaving it in place for instance-identifier.
That means that use-case #6 is illegal because
require-instance is going to be hard-wired to true.

Andy

---

From: Per Hedeland <[email-masked]>
Date: Fri, 29 May 2009 07:10:46 +0200

> It only makes sense if require-instance=true though.

And if the service provider changes the number for one of your favorite
channels, you can't watch TV at all until you update the favorite list,
because the box refuses to run with an invalid configuration.:-) I agree
that it's a good example, but I think it makes sense with
require-instance=false - think of it as a SHOULD rather than a MUST,
perhaps.

--Per Hedeland

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 29 May 2009 08:55:47 +0200 (CEST)

> > actually thought it was allowed since it made so much sense to me.

I agree.  At first we just had keyref, but when we generalized into
leafref, we should have thought about this.  Ok, we just need to make
sure this is captured in the text (withouth making the text even more
complex...)

> > b) With regard to the example of config data pointing at state data, (in this
> > case, favorite channel), the quesiton is what should happen if the favorite
> > channel is no longer available.
> > b1) If that favorite will get deleted, then a leafref may well make sense,
> > but the operational handling requirement is hard to describe.
> > b2) If the entry will remain, even though the channel no longer "exists",
> > then it is not a leafref.  It is a number, which corresponds to a channel iff
> > there is one.  While we could try to capture that level of semantic in the
> > type system, it seems quite sensible to just say no.
> > 
> 
> this is a great summary.

But this is the very reason for specifying this with
require-instance=false!  If you do that, your system continues to work
when your service provider deletes a channel to which you pointed.

When looking at the datamodel, it is very clear that the favorite list
is a pointing into the channels list.  So having
require-instance=false makes this a much clearer datamodel, IMO.

The alternative woould be:

   leaf-list channel {

   leaf-list favorite-channel {
      type uint32 { range 1..9999; }
      max-elements 10;
      description
         "A number in this leaf-lists refers to an entry in the
         '/channel' list, if it exists.  If it doesn't exist in the
         channel list, then the favorite channel is unvailable.";
   }
 
So, w/o require-instance=false, people will have to catch this
semantics in the description clause.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 00:50:09 -0700

> require-instance=false - think of it as a SHOULD rather than a MUST,

No -- If the agent uses the delete-dead-leafref behavior,
then the invalid config is deleted immediately,

IMO, the leafref datatype should be used for situations
that require a real coupling of instances, not merely using
the same data type.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 29 May 2009 10:30:04 +0200 (CEST)

> that require a real coupling of instances, not merely using

So, should it be removed also from notifications?

In your channels example, these *is* a coupling.  The channels and
favorite lists do not just by accident use the same datatype.

/martin

---

From: Per Hedeland <[email-masked]>
Date: Fri, 29 May 2009 10:49:58 +0200

> then the invalid config is deleted immediately,

Is such a behavior even being considered? To me it seems like a bizarre
idea that the type of a leaf could imply automatic deletion from the
configuration. If a config leafref's require-instance=true is not
satisfied, it means that the configuration is invalid, just as if the
value of an integer is out of range - you wouldn't automatically modify
such an integer value to fall within the closest range...

--Per

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 29 May 2009 14:21:34 +0200

> readily work out (import/include/uses/augments ...etc)

The context is quite clear for "augment", but the document is the
corresponding schema tree, unlike the other uses of XPath.

I was thinking about 'path' under 'leafref' and IMO the accessible tree
should be the main data tree. Are there any use cases for, say, a
leafref in RPC output referring to a leaf inside the same RPC output
tree?

I am not sure about instance identifier, for me it's quite an exotic
data type.
 
> Well, to be picky, path allows the current() function.

I don't think current() is allowed in those three cases. On the other
hand, I am wondering about other XPath constructs that seem to be legal
e.g. in the 'path' argument, for instance "foo//bar". I think this
should be banned, too, such a 'leafref' would be really mindboggling.

Lada
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 29 May 2009 09:30:33 -0400

It does not look like a leafref to the available channels to me.
If you want to view it as a leafref, it is a leafref to the list of all 
offered channels from the operator, including all currently unpaid for 
premiums, etc.  And if someone has a "favorite" on that list, you might 
as well delete it, because it is gone.

I think we get a much simpler conceptual model, that covers the cases 
correctly, if we just use leafref for the cases when the values are 
coupled (which includes notifications.)

Leafref with require-instance=false is just a type alias.  It is not a 
value restriction.  Displaying the leaf as "pointing-to" some range of 
other leafs is actually misleading, since its value space is not 
limited.  It is as if we included a "typical" option in a range 
statement to say that the range restriction was not mandatory, only 
advisory (maybe the GUI should highlight it if it is outside the 
"typical" range".)

Yours,
Joel

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 07:04:55 -0700

> coupled (which includes notifications.)

Seems like require-instance should always be true for notifications.
You are reporting on a database node (or about-to-be-deleted node),
so it makes sense to report only real instances.

> advisory (maybe the GUI should highlight it if it is outside the 

I agree with this concern about require-instance=false.
At the interim, I also raised the concern about forcing
the YANG reader (our highest priority) to read XPath expressions
and track down the correct version of the module containing the pointed-at
leaf, just to see its data type.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 07:28:50 -0700

> So, should it be removed also from notifications?

Notifications in NETCONF/YANG are rather limited and
unless the WG(s) do something to fix them,
it isn't really worth worrying about.

A 'useful' notification will need to include
all the keys from the relevant entries, as
well as some data selected from these entries.

Since leafref just returns the value of the pointed-at node
with no indication of the actual instance selected,
it is not very useful for notifications.

> favorite lists do not just by accident use the same datatype.

But you could say this about any 2 leafs that happen
to use the same derived data type.

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 29 May 2009 18:04:41 +0200

Hello,
One reason why leafref is so complex is that it is overloaded. 6 different use-cases are IMHO 
too many for a single construct.

I would strongly vote for keeping require-instance on instance-identifier. I see the use of it 
and it is an easily understandable concept.

I much liked Andy's idea about varbinds, because its a clean and simple solution for use-case 1.

I am strongly against automatically deleting leafrefs. What if it is a mandatory leaf or part 
of a leaf-list with min-elements > 0 or part of a mandatory choice, or if a must checks it's 
existence, or if a when references it?

I also liked Martin's proposal: require-instance hardwired to true if leafref referres to a 
config leaf, otherwise hardwired to false.

About documentation:
I was very much missing the following from the text:
- The leafref and the referred to leaf/leaf-list have separate values.
- The aim of the leafref is to document the semantic connection between the leafref and the 
referred to leaf AND to constrain the allowed values of the leafref to a set defined by the 
referred to leafs/leaf-lists.

Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320                  email: [email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 10:41:08 -0700

> readily work out (import/include/uses/augments ...etc)

Remember that the target strings used in augment,
refine, and deviation statements are all schema-nodeid strings
and they must include choice and case nodes in the string.
The identify a node in the object tree.

The path-stmt in a leafref identifies one or more conceptual
nodes in the value tree, so there are no choice or case nodes
and there can be predicates for list keys.

The value of an instance-identifier is an absolute path
expression representing one conceptual node in the value tree.
The prefixes are relative, so comparing instance-identifiers
is an implementation detail, i.e., not strcmp().

I find it much easier to conceptualize YANG by thinking
of how it would be implemented.  Imagine a tree of
all available objects (rpc, notification, data-def-stmt).
Then imagine another 'mirror' tree, with zero or more instances
of each object node.  It isn't really necessary to copy
all the YANG defaults into the value tree, so some
implementations don't.  (That's why we need with-defaults.)

The <candidate> value tree doesn't have any config=false nodes in it,
but the object tree has them, so implementing require-instance=false
only requires some special code -- just a major XPath implementation
requirement to process the internal object tree, not some
XML instance document.  It would be really inefficient to
process XPath by creating an XML document, so it is more
likely the value tree will be processed directly anyway.

Then there is the issue of deleting invalid config leafrefs
when the non-config pointed-at leaf changes value.
This is a problem for implementors.  I was thinking it was OK
to leave the invalid config leafref in <running>, but what
about <startup>?  Either direct via <copy-config> or
indirect via mirroring, the invalid leafref will end up
in the <startup> config, causing a problem later.

Removing the leafref may cause more problems than leaving it there,
due to must, when, mandatory, min-elements statements.

IMO, config leafrefs are not very important to have,
except as foreign keys.  But the problem doesn't
go away for foreign keys, so might as well support #4.

SMIv2 can have simple INDEX clause because all the leaf names
must be unique.  YANG would need to specify the schema path
as follows:

    container interfaces {
       list interface {
          key name;
          leaf name { ... }
          leaf ifIndex {
            // what to do about read-only ifIndex?
            // should that be supported? IMO, no
          }
          ...
       }
    }

    list oldIfEntry {
       key "/interfaces/interface/ifIndex";
       // no leaf-stmt for ifIndex
       leaf foo {}
    }

    list extOldIfEntry {
       key "/interfaces/interface/ifIndex localIndex";
       // no leaf-stmt for ifIndex
       leaf localIndex {}
       leaf foo {}
    }

Note how a local key is also a valid path expression,
consistent with the foreign keys?

IMO, this would eliminate the need for config leafrefs,
and it is more direct.  The SMIv2 'copy-fooIndex"
approach is too cut-and-paste.  If you want a foreign
key, then it should be specified in the key-stmt.
Mixing foreign and local keys from the same namespace
would require some name collision checking, but
the YANG compiler does that already.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 29 May 2009 19:48:15 +0200 (CEST)

> it is not very useful for notifications.

We changed keyref to leafref mainly for supporting leafref in
notifications.  If you think we don't need to worry about
notifications, can we change leafref back to keyref?

> to use the same derived data type.

?

Are you suggesting that two leafs in this example are just as
coupled as favorite and channel in your example?  If so, I am
confused.

  typedef AdminString {
     type string { ... }
  }

  leaf sys-name {
      type AdminString;
  }
  leaf entity-name {
      type AdminString;
  }

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 10:51:24 -0700

> tom.petch wrote:
.....
> must be unique.  YANG would need to specify the schema path

oops -- forgot that ifIndex needs a unique-stmt:

            unique ifIndex;

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 29 May 2009 13:55:53 -0400

While there is some semantic information in a config leaf whose type is 
a leafref with require-instance=false, it is a very small amount of 
information.  It will allow a few uses, but usually won't allow what an 
NMW would like (for example, it will not allow the nms to provide a list 
of valid values to the user.)
It seems a complication with little actual use.  Real semantics is 
always in the description clause anyway.

Joel

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 11:05:03 -0700

> notifications, can we change leafref back to keyref?

I really don't like the cut-and-paste problems that
show up because of leafref/keyref:

    list A {
       key x;
       leaf x { type string; }
       ...
    }

    list B {
       key x;
       leaf x { type leafref { path /A/x; }}
       ...
    }

    list C {
       key x;
       leaf x { type leafref { path /B/x; }}
       ...
    }

This copy-of-x approach allows copy-of-copy-of-x
as well.  This is either a bug or a feature,
depending on your POV.

I prefer to put the path-stmt directly into
the key-stmt, eliminating the need for any
duplication of the foreign leaf used as a key.

    list A {
       key x;
       leaf x { type string; }
       ...
    }

    list B {
       // valid
       key "/A/x";
       ...
    }

    list C {
       // invalid because /B/x is a foreign key
       key "/B/x";
       ...
    }

We don't really need 2 or 3 different ways to declare
a foreign leaf as a list key.

> coupled as favorite and channel in your example?  If so, I am

They are sort of the same because the set of
channel numbers is static and valid.
The actual stations enabled to produce
content instead of a blank screen is
a dynamic read-only list.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 11:10:22 -0700

.....
> a dynamic read-only list.

I got this use-case wrong.
This is not read-only (config-false) data.
The enabled-station list is simply config data
that access control is allowing the user to read,
but preventing the user from changing.

IMO, there is no use-case for config leafrefs pointing at
non-config leafs.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 29 May 2009 22:33:37 +0200 (CEST)

> as well.  This is either a bug or a feature,

This is only a problem with leafref, it was not in the original
keyref.

> duplication of the foreign leaf used as a key.

But this would solve only the foreign key problem.  It does not solve
the leaf-to-key reference problem.  (Compare with channelIfIndex from
RMON-MIB, which is a reference to ifIndex, but this info is only
available in the description clause)

> a foreign leaf as a list key.

I think we have one way to declare a foreign leaf as a key - a leaf of
type leafref.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 May 2009 13:45:02 -0700

> always in the description clause anyway.

You are being fooled by the name 'leafref',
when all that is being supported is really 'typeref'.
The leaf or leaf-list node containing the leafref type
statement can override any of the sub-statements
(e.g., units, default, description, if-feature).

I would think 'leafref' meant that there was a coupling
between the pointer and pointed-at (as 'ref' suggests).

I would also think the thing being referenced was
the entire leaf, not just the type-stmt from that leaf.

I would also not guess that there were at least 6 different
use-cases allegedly supported, given the existing text.
Some of them are clearly under-specified, if in fact
they remain in the draft.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Sat, 30 May 2009 21:23:35 +0200

---- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
To: "tom.petch" <[email-masked]>
Cc: "Martin Bjorklund" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Friday, May 29, 2009 2:21 PM
> should be banned, too, such a 'leafref' would be really mindboggling.

Well, reading the ABNF, current() is explicitly mentioned for path:-)
But functions is the least concern for me.

Biggest is namespaces.  We (have  rough consensus to) deviate from
XPath 1.0 by making names without prefixes part of the current module
and that is called out for 'must and 'when'.  Hence I think it needs calling
out for the others; omitting this must surely confuse.  Perhaps a candidate
for 6.4.

Second biggest is root.  Again, this was discussed (January) and we do
(have rough consensus to) not deviate from XPath 1.0, that is the XPath
(virtual)
root node sits above the root element nodes of the 'document' and 'must'
and 'when' clearly call out what that means.  Again, I think that this needs
calling
out for the others, particularly for 'augment'; else we may not get
interoperability for absolute paths.

And yes, I can see a leafref in output or notification referring to a leaf node
therein, but I cannot think of a case where the same thing could not be achieved
by a second reference to the data tree, so a short cut for the data modeller
rather than a must have.

'instance-identifier' yes, I have not got my mind around it yet.

Tom Petch