Subject: [Netconf] 4741bis (007) - return from XPath filters

From: Martin Bjorklund <[email-masked]>
Date: Mon, 09 Nov 2009 14:52:39 +0100 (CET)

Hi,

http://trac.tools.ietf.org/wg/netconf/trac/wiki/Issues_4741bis#issue-007

This issue was discussed at the NETCONF session today.

The question is what the resulting XML from an XPath filter looks
like.  For example, with this data model:

   list interface {
     key name;
     ...
     leaf mtu { ... }
   }

What should the expression: select="/interface/mtu" return?

1.  (no keys)

    <nc:data>
      <if:interface>
        <if:mtu>1500</if:mtu>
      </if:interface>
      <if:interface>
        <if:mtu>1500</if:mtu>
      </if:interface>
    </nc:data>

2.  (keys)

    <nc:data>
      <if:interface>
        <if:name>eth0<if:name>
        <if:mtu>1500</if:mtu>
      </if:interface>
      <if:interface>
        <if:name>eth2</if:name>
        <if:mtu>1500</if:mtu>
      </if:interface>
    </nc:data>

At least two implementations (mine and Andy's) return the keys, as in
alternative 2.

I suggest we add this to 8.9.1:

    The response message contains the subtrees selected by the filter
    expression.  For each such subtree, the path from the data model
    root node down to the subtree, including any elements or
    attributes necessary to uniquely identify the subtree, are
    included in the response message.

This is slightly different from how subtree filtering works.  Compare
these filters, with a data model where 'user' is a list with key
'name':

    <filter xmlns:t="http://example.com/schema/1.2/config"
            type="xpath"
            select="/t:top/t:users/t:user/t:company-info"/>

and

    <filter type="subtree">
      <top xmlns="http://example.com/schema/1.2/config">
        <users>
          <user>
            <company-info/>
          </user>
        </users>
      </top>
    </filter>

which look very similar.   The first one returns:

    <top xmlns="http://example.com/schema/1.2/config">
      <users>
        <user>
          <name>fred</name>
          <company-info>
            <id>2</id>
          </company-info>
        </user>
      </users>
    </top>

but the second one doesn't return any keys (which is not very
useful...)

    <top xmlns="http://example.com/schema/1.2/config">
      <users>
        <user>
          <company-info>
            <id>2</id>
          </company-info>
        </user>
      </users>
    </top>

Andy commented that his code returns the keys also in the subtree
filter case.  I'm not convinced the current text on subtree filtering
allows this, but maybe we should clarify that key nodes MAY be added
to the response message in subtree filtering.

Since the text on subtree filtering says: 

   The server does not need to utilize any data-model-specific
   semantics during processing,

we cannot say that the keys MUST be added, so a client that needs the
keys will still have to ask for them.

/martin

---

From: David Reid <[email-masked]>
Date: Mon, 09 Nov 2009 10:11:29 -0500

>     included in the response message.

I think the keys should be included, so I agree with the proposed text above.

> 
> This is slightly different from how subtree filtering works. 
> 

Our implementation also returns the keys, which I think is the only useful
approach, but maybe not correct according the the current spec. I support
the proposed clarification saying key nodes MAY be added.

-David Reid

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Nov 2009 00:34:23 +0900

>     included in the response message.

Hmm, how do you recognize such elements or attributes? NETCONF has no
notion of list keys, so there may be multiple ways how to satisfy this
requirement. For instance, in a user database both uid and username will
be unique although only one of them will be used as the list key in
YANG.

It's true that the XPath capablity is underspecified but this will be
hard to fix because specification of database queries belongs to the
"Content" layer.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 09 Nov 2009 16:41:51 +0100 (CET)

> be unique although only one of them will be used as the list key in

Sure, that will be data model dependent.  Is that a problem?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Nov 2009 08:30:57 +0900

> Sure, that will be data model dependent.  Is that a problem?

Let's say we have a YANG data model that has "uid" as the only key.
Still, the return value for the XPath query may use either "uid" or
"username", both satisfying the above requirement. Even "fullname" could
do, if this item happens to be unique in a particular instance, right?

You really want to say "list keys" here.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 10 Nov 2009 00:35:14 +0100 (CET)

> do, if this item happens to be unique in a particular instance, right?

Or maybe there is just one user with length 202 cm, so that would be
unique?

What sort of text do we need to make the intention clear?

> You really want to say "list keys" here.

No, b/c that is YANG specific.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Nov 2009 10:42:49 +0900

> Or maybe there is just one user with length 202 cm, so that would be

Yes, or if there is only one subtree, you don't need anything to
identify it.

> What sort of text do we need to make the intention clear?

I think the cleanest solution would be to just define the "schema" for
RPC requests utilizing the XPath capability and leave the spec of XPath
query sematics to YANG (or other DM languages, for that matter).

YANG already fills some gaps in the semantics of NETCONF operations
(e.g., edit-config for lists in Sec. 7.8.6), so it wouldn't be a really
new situation.

> No, b/c that is YANG specific.

Indeed, because it is specific to the Content layer, so NETCONF spec
shouldn't address it at all.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 10 Nov 2009 08:48:12 +0100 (CET)

> query sematics to YANG (or other DM languages, for that matter).

I think we can and should do better.  Regardless of DM language, we
want the XPath filter to return XML "as for subtree filters", right?

How about this:

    The response message contains the subtrees selected by the filter
    expression.  For each such subtree, the path from the data model
    root node down to the subtree, including any elements or attributes
    necessary, as defined by the data model, to uniquely identify the
    subtree, are included in the response message.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Nov 2009 23:21:50 +0900

> want the XPath filter to return XML "as for subtree filters", right?

Is the subtree filtering mechanism supposed to fill-in all "list keys"
in the ancestor hierarchy? Where is this said?

>     subtree, are included in the response message.

It would be so easy to express this in YANG terms. :-) I don't know: If
you want to determine whether a given reply satisfies this requirement,
you have to look-up the DML spec to find out what these identifying
elements/attributes are. You cannot tell this just by looking to the
XML.

I think it is pretty much the same problem as deciding about the
duplicate leaf in edit-config, which we classified as not being NETCONF
protocol issue.

Anyway, what is it really supposed to mean? If I have

module foo {
    namespace "http://example.org/foo";
    prefix foo;
    container top {
        list outer {
            key outer-key;
            leaf outer-key { ... }
            list inner {
                key inner-key;
                leaf inner-key { ... }
                leaf bar { ... }
            }
        }
    }
}

and then use XPath selector "/foo:top/foo:outer/foo:inner/foo:bar", will
the returned output contain both outer-keys and inner-keys?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 10 Nov 2009 15:30:58 +0100 (CET)

> in the ancestor hierarchy? Where is this said?

No, see my first post in this thread.  Is might be useful to at least
allow the subtree filtering mechanism to fill in keys; Andy and David
R. reported that their implementations do this.

> the returned output contain both outer-keys and inner-keys?

Yes.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2009 08:16:22 +0900

> R. reported that their implementations do this.

The original RFC 4741 behaviour should remain compliant. In principle,
there needn't be any key-like distinguishing elements/attributes. For
example, the datastore content may be the following valid XML:

<control xmlns="http://example.com/dual-toaster">
  <temperature>75</temperature>
  <temperature>75</temperature>
</control>

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Nov 2009 16:06:37 -0800

>   <temperature>75</temperature>

I don't see why it is a problem if the server fills in the
missing keys.  If there are no keys, like in your example,
then there will be no added leafs.  But what if the 'toaster'
is a list:

  <filter>
     <toaster>
        <temperature/>
     </toaster>
  </filter>

  <data>
     <toaster>
        <id>1</id>
        <temperature>75</temperature>
     </toaster>
     <toaster>
        <id>2</id>
        <temperature>234</temperature>
     </toaster>
  </data>

This way, the client will use its YANG definition,
see that /toaster is a list, and expect a key leaf
to be the first element returned after that.

Consider the utility if no list key is returned:

  <filter>
     <toaster>
        <onfire/>
     </toaster>
  </filter>

  <data>
     <toaster>
        <onfire/>
     </toaster>
  </data>

Which toaster is on fire?

IMO, not only should the server return keys. it should
also normalize the output so no duplicates are returned:

  <filter>
     <toaster>
        <onfire/>
     </toaster>
     <toaster>
        <temperature/>
     </toaster>
  </filter>

  <data>
     <toaster>
        <id>2</id>
        <onfire/>
        <temperature>234</temperature>
     </toaster>
  </data>

> Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2009 13:35:26 +0900

> then there will be no added leafs.  But what if the 'toaster'

I don't mind if the server fills in the keys but I'd object if the
server was no more allowed to return multiple identical trees, like in
this example.

But the fact that the replies to get/get-config queries with both
subtree and XPath filters may depend on the data modeling framework is a
new situation compared to the old RFC 4741.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C