Subject: [netmod] XPath warnings

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Jul 2009 06:48:13 -0700

Hi,

I am conflicted regarding Phil's proposal
about 'dead nodes in an XPath expression are errors'.
When coding this (yangdump has 4 or 5 warnings
just on this 1 topic), I thought "gee this is stupid,
but that's what XPath says to do, and YANG says I MUST
support valid XPath".

But these need to be warnings, not errors.

First, when analyzing the object tree to generate
warnings about what will happen at run-time, any expression
with positional axes, or certain function calls or wildcard combos,
can fool the compiler.

  preceding-sibling::*[3][some-child='fred']

Is there a 'some-child' node in the data tree?  Maybe.
(Do you have any idea how hard it would be for the
XPath 'parse_predicate' code to get this right every time?
And what about all the preceding siblings that don't
have a 'some-child' leaf?  They are intended as 'no-match',
not an error at all.)

There is also the 'some minus child' arithmetic expression
that will trip up the compiler in this example.

Then there is the problem of intended usage within YANG.

  must "NOT_YET";  // obviously a bug

  when "NOT_YET";   // not a bug

Checking for expressions that always yield a constant result
is tricky (true or false, doesn't matter, both are bugs).
The easy ones are not likely to ever show up (must "3 > 4";)

Summary:  pay attention to warnings when you are debugging,
or you deserve what you get.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 27 Jul 2009 16:16:37 +0200

Hi,

I agree with Andy. In addition, from the viewpoint of XPath, (extended)
names of nonexistent nodes are nothing illegal, they just select the
empty set of nodes with all consequences on more complicated
expressions, e.g. "count(foo:nonexistent)" returns zero.

Lada

Andy Bierman píše v Po 27. 07. 2009 v 06:48 -0700:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 27 Jul 2009 12:30:03 -0400

>But these need to be warnings, not errors.

Not true.  We can say "XPath expressions should not refer to
nodes that are not defined in the module".  This makes "../foo"
an error if "foo" is not defined.  More complex expression may
be tricky, but the tool can know that when it finds them, they
are an error.  Otherwise "../Interface" (instead of "interface")
or "../../time-frame" instead of "TimeFrame" will not be a
compile error, since it's perfectly valid.

Many typos will be published in RFCs if they aren't detected.

>  when "NOT_YET";   // not a bug

must 1 ; when 0;

>or you deserve what you get.

Um, I'm a -Werror addict, since I despise errors, but I think
that's rare.  The vast majority of programmers happily ignore
warnings, allowing them to pile up until the warnings caused
by new code changes are lost in the noise of the old warnings.

Garbage in, garbage out.  So why allow garbage?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Jul 2009 10:30:06 -0700

> Many typos will be published in RFCs if they aren't detected.

This assumes that the WG members will not be
able to review the YANG module and find the
problems.  If that is the case, then XPath
is too hard to use and it needs to be tossed out
of YANG completely.

But I think it will be OK, because YANG authors
who don't know XPath will avoid must/when statements completely.
They will write description statements instead, just like in SMIv2.

>>  when "NOT_YET";   // not a bug

both expressions evaluate to the same thing -- undefined element,
but in a 'when' expression, this might be the intent.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 27 Jul 2009 13:36:31 -0400

Andy Bierman writes:
>problems.

Are you assuming reviewers will catch every mistake?

>but in a 'when' expression, this might be the intent.

1 and 0 are numbers.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Jul 2009 11:15:59 -0700

> Are you assuming reviewers will catch every mistake?

No better or worse than any other IETF document

>> but in a 'when' expression, this might be the intent.

yes, but not sure what this has to do with the
issue I raised about requiring the XPath parser
to treat when and must expressions differently.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Jul 2009 16:24:47 -0700

> Are you assuming reviewers will catch every mistake?

I am not assuming a compiler can always tell that
certain valid XPath is OK sometimes, and other times not.

I think must and when statement usage will be problematic,
but could also enable a lot of automatic code generation as well.
Don't ignore compiler warnings, and use the power saw carefully.

I think yangdump and pyang will support an "IETF review" mode,
and garbage XPath will certainly be elevated to an error in this mode.

But I do not want to put any rules in YANG about what is
and is not garbage XPath.  YANG compilers MUST report invalid
XPath usage  (e.g., "x and and y"). They SHOULD report warnings
for expressions which may produce constant (or empty) results at run-time.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 28 Jul 2009 03:55:28 -0400

>and garbage XPath will certainly be elevated to an error in this mode.

I don't really understand this stance.  You are saying that generic
YANG should be fine with references to undefined elements, but that
ietf mode will make this an error.  If the goal is to make it an
error, why not simply declare from the start that references to
undefined elements are illegal.

Consider:
- the cost of allowing such simple, completely preventable errors
- the benefit of detecting errors as early as possible
- every YANG module author will not be a YANG expert
- rfc reviewers don't catch errors (consider 4741 as proof ;^)

I don't understand why this is a hangup.  You are already saying
that tools need to support detecting this (for your ietf mode),
so there's no win in calling it legal YANG.  It will just be
a source of errors for our fledgling langauge.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 28 Jul 2009 10:39:02 +0200

Hi,

the upcoming XSD Schema 1.1 introduces asserts which serve pretty much
the same purpose as 'must' in YANG. They use XPath 2.0 but have the
following limitations:

- no access outside the subtree where the assert appears
- they are not schema-aware (which is the topic of this thread)

I don't know the reasons behind these limitations but at least they
indicate something.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: David Partain <[email-masked]>
Date: Tue, 28 Jul 2009 16:51:29 +0200

Hi all,

This issue has generated a fair bit of discussion here at the IETF.  
Essentially, the discussion has demonstrated clearly that there are pretty 
set stances in the working group.

I, as technical contributor, agree with Phil that this garbage should be 
thrown back in the face of the person who handed it to the compiler.

However, I also see in the mail and in discussions that have been happening in 
Stockholm that there are several others who think the YANG specification 
should not start down this path.  If it's valid XPath, it's a YANG warning 
and not an error.  Rather, it will be up to YANG compilers to deal with 
garbage in XPath expressions if they choose to do so.

That means that, while nonsense, the following would be considered "valid" 
(but stupid) YANG:

module a {
    namespace “http://example.com/a”;
    prefix a;
    container foo {
        leaf a {
            type int32;
            must “. < ../c”;
        }
        leaf b { … }
    }
}

We will try to put this issue to bed tomorrow in the working group meeting.

Cheers,

David

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2009 17:02:42 -0700

> We will try to put this issue to bed tomorrow in the working group meeting.

I changed my mind on this one.

The issue is the usage of a complete QName within an absolute
or relative path expression, without any wildcards or references
to relative position of instances in the data tree.

When a complete QName is specified (a specific local-name in
a specific namespace), it MUST refer to an existing node in
the schema tree.  However, if any wildcards are present which
make the exact location in the schema tree unknown or ambiguous,
then the agent may not be able to detect these errors.

So while I agree with the sentiment, I do not know if
a CLR can be written which is going to be future-proof,
such that the CLR will make sense 5 or 10 years from now.

If you can write the CLR properly,
then this should be an error in YANG.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2009 10:48:10 +0200 (CEST)

> then the agent may not be able to detect these errors.

In the following examples, the nodes "x" are undefined, but all others
are defined.  Would an ipmlementation have to detect:
 
   ../x 
   current()/../x
   ../a[./x]
   ../a[./b/x]
   ../a[./b[./x]]
   ../a[current()/x < 0]

The problem is that it requires not only parsing the expression, but
also some semi-interpretation of the expression.

> then this should be an error in YANG.

I think this is more or less bound to fail, and it would be confusing
if some of these references are illegal, but some are not, depending
on how good our CLR is.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 29 Jul 2009 07:42:23 -0400

>also some semi-interpretation of the expression.

I'm suggesting two bits: (a) that we say that this isn't valid yang,
and (b) that we detect simple brokenness and not compile it.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2009 14:11:53 +0200 (CEST)

> I'm suggesting two bits: (a) that we say that this isn't valid yang,

So you're saying that all above examples are invalid?

> and (b) that we detect simple brokenness and not compile it.

Define "simple brokenness".

/martin