Subject: Re: [netmod] Order of evaluation for when?

From: Balazs Lengyel <[email-masked]>
Date: Tue, 13 Oct 2015 14:25:28 +0200

Hello Martin,
I agree that A1 is what follows the spirit of YANG, but then IMHO you 
should change/correct 8.2.1 in YANG because that implies A2 and error.
regards Balazs

On 2015-10-13 13:30, Martin Bjorklund wrote:
> Balazs Lengyel <[email-masked]> wrote:
>> Hello Martin,
>> If I had a model
>>
>> leaf a {type boolean;}
>> leaf b {
>>      when a
>>      type int8;
>> }
>>
>> if "a" is false and then you get an edit config with: set b=5 ; set a=true
>> What should be the result?
>> A1)  this should succeed as at commit time the "when" will be true.
> yes.
>
>> A2) this should fail as at parsing (as described in 8.2.1 of the YANG
>> RFC) the when is false - yet. This would also mean that you need 2
>> edit-configs to set b=5.
>> regards Balazs
>
> /martin

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
Senior Specialist
ECN: 831 7320
Mobile: +36-70-330-7909              email: [email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Oct 2015 21:25:18 +0200 (CEST)

> should change/correct 8.2.1 in YANG because that implies A2 and error.

Ok, I agree.  I suggest we remove from 8.2.1:

   o  If data for a node tagged with "when" is present, and the "when"
      condition evaluates to "false", the server MUST reply with an
      "unknown-element" error-tag in the rpc-error.

and add to 8.2.2:

  o  Modification requests for nodes tagged with "when", and the "when"
     condition evaluates to "false".  In this case the server MUST reply
     with an "unknown-element" error-tag in the rpc-error.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Oct 2015 12:36:37 -0700

>      with an "unknown-element" error-tag in the rpc-error.

This seems like a BIG protocol change to <edit-config> behavior.
IMO this not an error at all.  In our server the false-when data is just
pruned, and no error is ever sent for a pruned when=false data node.

It may be a client programming error for the client to provide
false when nodes or not.  What if the client is reusing some
code that sends 5 parameters, it it's OK if 1 of them gets
pruned sometimes because of a false when (e.g, product
feature-specific knob and that feature is not installed)

BTW, this is a really good example of what not to do, if one
wants to make the YANG specification protocol independent.

> /martin

Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Oct 2015 21:48:13 +0200 (CEST)

> pruned, and no error is ever sent for a pruned when=false data node.

So you are not following the current RFC 6020 spec?

I don't think this is a BIG protocol change, since the spec already
says that requests for nodes w/ false when expressions MUST send
error.  The change is to say that this behavior is true regardless of
evaluation order.

> feature-specific knob and that feature is not installed)

Well, it might be simpler to send if-featured nodes that the specific
server doesn't support - this is also an error in 6020.

> wants to make the YANG specification protocol independent.

That statement is true for the entire section 8.2, it is not
specifically true for this change.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Oct 2015 12:49:12 -0700

> So you are not following the current RFC 6020 spec?

Yes we are following it.
The schema for the edit-config RPC operation contains
an 'anyxml' for <config> node.  It does not contain any
when-stmts for the data nodes that get passed in the <config> node.
The correct behavior is to just enforce the error on the when-stmts
that appear in the rpc-stmt.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Oct 2015 22:26:42 +0200 (CEST)

> > So you are not following the current RFC 6020 spec?

Ok.

> that appear in the rpc-stmt.

I don't understand what you are trying to say.

Here's an example:

  augment /if:interfaces/if:interface {
    when "if:type = 'ianaift:ethernetCsmacd'";

    container ethernet {
      leaf duplex {
        type enumeration {
          enum "half";
          enum "full";
        }
      }
    }
  }

Suppose the db is empty.

What if the edit-confif contains
 
  <interfaces>
    <interface>
      <name>eth0</name>
      <eth:duplex>full</eth:duplex>
      <type>ianaift:ethernetCsmacd</type>
    </interface>
  </interfaces>

will that work or not?  I.e., will the eth0 interface be created with
duplex full?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Oct 2015 14:06:46 -0700

> I don't understand what you are trying to say.

I know about the text that says a false-when in an RPC is an error.
Show me the when-stmts  "interface" in the "edit-config" rpc-stmt.

> will that work or not?  I.e., will the eth0 interface be created with

Yes -- because these are data nodes and the rules for when-stmt
on data nodes are different than for rpc-stmt.  Then the when-stmt
is a test on whether the node should exist in the candidate or running
datastore.

Our server applies all the edits first, when checks all the when-stmts
that might have changed value.  Nodes that have already existed in the
datastore may get pruned, not just the new nodes.

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 15 Oct 2015 10:00:40 +0200

See below, Balazs

On 2015-10-14 23:06, Andy Bierman
wrote:

On Wed, Oct 14, 2015 at 1:26 PM,
Martin Bjorklund <[email-masked]> wrote:

Andy
Bierman <[email-masked]>
wrote:

> On Wed, Oct 14, 2015 at 12:48 PM, Martin Bjorklund
<[email-masked]>
wrote:

> > > On Wed, Oct 14, 2015 at 12:25 PM, Martin
Bjorklund <[email-masked]>

> > wrote:

> > > > > should change/correct 8.2.1 in
YANG because that implies A2 and

> > error.

> > > >      condition evaluates to "false".
In this case the server MUST

> > reply

Ok.

I know about the text that says a false-when in an RPC
is an error.

Show me the when-stmts  "interface" in the
"edit-config" rpc-stmt.

Here's an example:

container ethernet {

leaf duplex {

type enumeration {

enum "half";

enum "full";

}

}

}

}

Suppose the db is empty.

<interfaces>

<interface>

<name>eth0</name>

</interface>

</interfaces>

will that work or not?  I.e., will the eth0 interface be
created with

duplex full?

Yes -- because these are data nodes and the rules for
when-stmt

on data nodes are different than for rpc-stmt.  Then
the when-stmt

is a test on whether the node should exist in the
candidate or running

datastore.

Our server applies all the edits first, when checks all
the when-stmts

that might have changed value.  Nodes that have already
existed in the

/martin

Andy

> Andy

And this contradicts the current rfc6020bis-07#section-8.2.1 which
states that already during parsing you must check

If data for a node tagged with "when" is present, and the "when"
condition evaluates to "false", the server MUST reply with an
"unknown-element" error-tag in the rpc-error.

So already during parsing <eth:duplex>full</eth:duplex> you MUST send back an error;
before processing <type>ianaift:ethernetCsmacd</type>.
(I also assume this is independent from the document order of the edit-config request.)
So this MUST be corrected in the draft!
regards Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 15 Oct 2015 11:04:07 +0200

> datastore may get pruned, not just the new nodes.
Hello Andy,

We definitely need a new section/paragraph in RFC6020 about the scenario when the when statement is changed in a transaction and something under the when statement is also changed in the same transaction!

---------------------------------------------------------------
So if you have the model

leaf a {type boolean;}
leaf b {
      when a
      type int8;
}

We have 3 cases:
1) if "a" is false and then you get an edit config with: set b=5 ; set
a=true: result OK
2) if "a" is false and then you get an edit config with: set b=5: result unknown-element
3) if "a" is true and then you get an edit config with: set b=5 ; set
a=false: result OK, b gets pruned (contradicts 8.2.1)

Are these the results dictated by YANG and supported by your implementation?

regards Balazs

---

From: Andy Bierman <[email-masked]>
Date: Thu, 15 Oct 2015 04:46:56 -0700

Hi,

You are incorrect.

Within the PAYLOAD (as this section describes), there is no when-stmt
for data nodes within the datastore.  Look at the YANG for edit-config.
There are no when-stmts for "interface" in "edit-config".

So explain which constraint in the payload is being violated?

Andy

On Thu, Oct 15, 2015 at 1:00 AM, Balazs Lengyel <[email-masked]

> On Wed, Oct 14, 2015 at 1:26 PM, Martin Bjorklund < <[email-masked]>
> [email-masked]> wrote:
>
>> > > > > Balazs Lengyel < <[email-masked]>
>> [email-masked]> wrote:
>> > > > > > Hello Martin,
> And this contradicts the current rfc6020bis-07#section-8.2.1 which states
> that already during parsing you must check
>

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 15 Oct 2015 13:53:34 +0200 (CEST)

> There are no when-stmts for "interface" in "edit-config".

Andy, there is some confusion here.  The section talks about:

  For configuration data, there are three windows when constraints
  MUST be enforced:

  - during parsing of RPC payloads
  - during processing of NETCONF operations
  - during validation

So the entire section talks about constraints *on configuration data*.

If you interpret this section to talk about the nodes in the
definition of edit-config, nothing in the section makes any sense at
all.   For example:

  If all keys of a list entry are not present, the server MUST reply
  with a "missing-element" error-tag in the rpc-error.

you might say that there are no lists in the definition of
edit-config, so this bullet doesn't apply.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 15 Oct 2015 08:02:04 -0700

> So the entire section talks about constraints *on configuration data*.

http://www.netconfcentral.org/modules/ietf-netconf/2011-06-01#edit-config.421

Here is the YANG for edit-config?
Please point out the when-stmts in this rpc-stmt
specific to the "interface" node.
I just see an "anyxml" that has no when-stmts at all.

So enforcing the when constraint on the RPC PAYLOAD
clearly has nothing to do with "interface" -- just the parameters
specified in the rpc-stmt.

> edit-config, so this bullet doesn't apply.

edit-config is the next section  -- 8.2.2

Andy

> > >> > > > On Wed, Oct 14, 2015 at 12:25 PM, Martin Bjorklund <
> [email-masked]>
> > >> > > wrote:
> > >> > > >
> > >> > > > > > should change/correct 8.2.1 in YANG because that implies A2
> and
> > >> > > error.
> > >> > > > >
> > >> > > > >    o  If data for a node tagged with "when" is present, and
> the
> > >> "when"
> > >> > > > >       condition evaluates to "false", the server MUST reply
> with
> > >> an
> > >> > > > >   o  Modification requests for nodes tagged with "when", and
> the
> > >> "when"
> > >> > > > >      condition evaluates to "false".  In this case the server
> MUST
> > >> > > reply
> > >> > > > IMO this not an error at all.  In our server the false-when
> data is
> > >> just
> states

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 15 Oct 2015 17:33:10 +0200

Hello,

I had the same interpretation as Martin. The when could be on the
config database model. Not just on the when defined in a definition
of an rpc or action.

regards Balazs

On 2015-10-15 17:02, Andy Bierman
wrote:

On Thu, Oct 15, 2015 at 4:53 AM,
Martin Bjorklund <[email-masked]> wrote:

Andy

> There are no when-stmts for "interface" in
"edit-config".

Andy, there is some confusion here.  The section talks
about:

For configuration data, there are three windows when
constraints

MUST be enforced:

- during validation

So the entire section talks about constraints *on
configuration data*.

clearly has nothing to do with "interface" -- just the
parameters

If
you interpret this section to talk about the nodes in the

definition of edit-config, nothing in the section makes
any sense at

all.   For example:

If all keys of a list entry are not present, the server
MUST reply

/martin

Andy

wrote:

> >> > On Wed, Oct 14, 2015 at 12:48 PM,
Martin Bjorklund <[email-masked]>

> >> > > > On Wed, Oct 14, 2015 at 12:25
PM, Martin Bjorklund <[email-masked]>

> >> > > > > > I agree that A1 is
what follows the spirit of YANG, but then

> >> IMHO you

> >> > > > > > should
change/correct 8.2.1 in YANG because that implies A2 and

> >> > > > >    o  If data for a node
tagged with "when" is present, and the

> >> > > > >       condition
evaluates to "false", the server MUST reply with

> >> > > > >   o  Modification
requests for nodes tagged with "when", and the

> >> > > > >      condition evaluates
to "false".  In this case the server MUST

> >> > > > >      with an

> >> > > > This seems like a BIG
protocol change to <edit-config> behavior.

> >> > > > IMO this not an error at
all.  In our server the false-when data is

"edit-config" rpc-stmt.

> >> > > > It may be a client
programming error for the client to provide

> > So already during parsing
<eth:duplex>full</eth:duplex> you MUST send
back an error;

> > (I also assume this is independent from the
document order of the edit-config request.)

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 15 Oct 2015 18:03:57 +0200 (CEST)

> specified in the rpc-stmt.

Ok, you're right.  8.2.1 should be kept as it is.  (we may need to
rephrase the intro text in 8.2) But I think Balazs is also right.
Suppose you have:

  leaf a {
    when "../b = 42";
    type int32;
  }
  leaf b {
    type int32;
  }

and the db contains b=10.

Suppose I send an edit-config with a=2.  What is the result?

  1)  you get an error back
  2)  you get ok; the request to set a to 2 is silently dropped
  3)  something else

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 15 Oct 2015 14:05:49 -0700

Hi,

Show me the text that says an anyxml passes the constraints of
the hidden data models through to the RPC processing.
The error for false-when only applies to parameters specified
in the RPC.

The processing of the rpc-stmt does not have any when-stmts that
need to be checked.

The config data is validated as part of a datastore, not as part
of the RPC payload.

Andy

On Thu, Oct 15, 2015 at 8:33 AM, Balazs Lengyel <[email-masked]

> Hello,
> I had the same interpretation as Martin. The when could be on the config
> database model. Not just on the when defined in a definition of an rpc or
> action.
> regards Balazs
>
> On 2015-10-15 17:02, Andy Bierman wrote:
>
>
>
> On Thu, Oct 15, 2015 at 4:53 AM, Martin Bjorklund < <[email-masked]>
> [email-masked]> wrote:
>
>> > On Thu, Oct 15, 2015 at 1:00 AM, Balazs Lengyel <
>> <[email-masked]>[email-masked]
>> > > wrote:
>> >
>> > > See below, Balazs
>> > >
>> > >> > On Wed, Oct 14, 2015 at 12:48 PM, Martin Bjorklund <[email-masked]
>> >
>> > >> wrote:
>> > >> >
>> > >> > > Andy Bierman < <[email-masked]>[email-masked]> wrote:
>> > >> > > > > Balazs Lengyel < < <[email-masked]>
>> [email-masked]>
>> > >> > > > > > I agree that A1 is what follows the spirit of YANG, but
>> then
>> > >> IMHO you
>> > >> > > > > > should change/correct 8.2.1 in YANG because that implies
>> A2 and
>> > >> > > error.
>> > >> > > > >
>> > >> > > > >      condition evaluates to "false".  In this case the
>> server MUST
>> > >> > > reply
>> > > Mobile: +36-70-330-7909              email:
>> <[email-masked]>[email-masked]
>> > >
>> > >
>>
>
>
> --

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 16 Oct 2015 09:43:04 +0200

> On 15 Oct 2015, at 18:03, Martin Bjorklund <[email-masked]> wrote:
> 

IMO YANG spec shouldn't deal with this kind of things in the first place. It should be sufficient to define two validation levels, e.g. syntactic (schema) and semantic validity. Stating what a protocol does if either of these validity types is violated would then be up to the protocol spec, and different protocols may use different approaches.

That said, I like #1 best.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 16 Oct 2015 12:38:41 +0200

Hello Andy, Martin,

If that is what is meant by 8.2.1 then I have a few comments

- Clarify that this is for RPCs defined by the Yang RPC statement
and not for all RPCs. After all edit-config is also carried in an
RPC.

- Add actions as they should be validated the same way.

- I don't understand the last 2 bullets in 8.2.1:

"If the attributes "before" and "after" appears in any element"

What would before and after mean in a YANG defined RPC or ACTION. I
thought they are used defined only for edit-config. Remove last
bullet.

- There are other constraint not mentioned here (min/max-elements,
must). Some are mentioned in the section some are not. Confusing.

Also this does not cover the questions about how to evaluate when.

regards Balazs

On 2015-10-15 23:05, Andy Bierman
wrote:

Hi,

Show me the text that says an anyxml passes the constraints
of

The error for false-when only applies to parameters
specified

in the RPC.

The processing of the rpc-stmt does not have any when-stmts
that

need to be checked.

The config data is validated as part of a datastore, not as
part

of the RPC payload.

Andy

On Thu, Oct 15, 2015 at 8:33 AM, Balazs
Lengyel <[email-masked]>
wrote:

Hello,

I had the same interpretation as Martin. The when could be
on the config database model. Not just on the when defined
in a definition of an rpc or action.

regards Balazs

On Thu, Oct 15, 2015 at
4:53 AM, Martin Bjorklund <[email-masked]>
wrote:

Andy

> There are no when-stmts for "interface" in
"edit-config".

Andy, there is some confusion here.  The section
talks about:

For configuration data, there are three
windows when constraints

MUST be enforced:

- during validation

So the entire section talks about constraints
*on configuration data*.

Please point out the when-stmts in this
rpc-stmt

I just see an "anyxml" that has no when-stmts
at all.

So enforcing the when constraint on the RPC
PAYLOAD

clearly has nothing to do with "interface" --
just the parameters

If

you interpret this section to talk about the
nodes in the

definition of edit-config, nothing in the
section makes any sense at

all.   For example:

If all keys of a list entry are not present,
the server MUST reply

with a "missing-element" error-tag in the
rpc-error.

you might say that there are no lists in the
definition of

/martin

Andy

wrote:

> > On Wed, Oct 14, 2015 at 1:26 PM,
Martin Bjorklund < <[email-masked]>

wrote:

> >> > On Wed, Oct 14, 2015 at 12:48

> >> > > > On Wed, Oct 14,
2015 at 12:25 PM, Martin Bjorklund <[email-masked]>

wrote:

> >> > > > > > Hello
Martin,

> >> > > > > > I agree
that A1 is what follows the spirit of YANG, but
then

> >> > > > > > should
change/correct 8.2.1 in YANG because that
implies A2 and

> >> > > > >    o  If data
for a node tagged with "when" is present, and
the

condition evaluates to "false", the server MUST
reply with

> >> > > > > and add to
8.2.2:

> >> > > > >   o
Modification requests for nodes tagged with
"when", and the

> >> > > > >      condition
evaluates to "false".  In this case the server
MUST

> >> > > > This seems like a
BIG protocol change to <edit-config>
behavior.

> >> > > > IMO this not an
error at all.  In our server the false-when data
is

> >> > > > pruned, and no
error is ever sent for a pruned when=false data
node.

<name>eth0</name>

> >> > > I don't think this is a
BIG protocol change, since the spec already

> >> > > says that requests for
nodes w/ false when expressions MUST send

> >> > > error.  The change is to
say that this behavior is true regardless of

> >> > > Well, it might be
simpler to send if-featured nodes that the
specific

> >> > > > BTW, this is a
really good example of what not to do, if one

> > So already during parsing
<eth:duplex>full</eth:duplex> you
MUST send back an error;

> > (I also assume this is independent
from the document order of the edit-config
request.)

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 16 Oct 2015 13:05:28 +0200

> On 16 Oct 2015, at 12:27, Balazs Lengyel <[email-masked]> wrote:
> 
> IMHO YANG should define the behavoir, and I would want it to be the same on Netconf/Restconf/CLI etc.
> I agree that " 1) you get an error back" would be the best: because it is the easiest to understand for the operator, with the fewest corner cases.
> Also we must define if in the same transaction we set b=42 and a=2. which of the 3 options are taken?
> 
> 
> We should not leave such complex cases undefined, or alternatively state that they are implementation dependant.

I might be difficult to say what is a complex case, unless we abandon XPath in "when" statements and use something else - and considerably simpler.

Here is another interesting corner case - does it qualify as being complex?

leaf-list bar {
   type uint8;
   when "count(../*) > 1";
}
leaf foo {
   type uint8;
}

Assuming no instances exist, is this edit allowed? (This was essentially your question.)

<bar>1</bar>
<foo>2</foo>

But what about this?

<bar>1</bar>
<bar>2</bar>

Lada

> regards Balazs
> 
> On 2015-10-16 09:43, Ladislav Lhotka wrote:
>>> Andy Bierman 
>>> <[email-masked]>
> [email-masked]
> 
>

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Oct 2015 13:11:59 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> > I agree that " 1) you get an error back" would be the best: because it
> > is the easiest to understand for the operator, with the fewest corner
> > cases.
> > We should not leave such complex cases undefined, or alternatively
> > state that they are implementation dependant.
> 
> I might be difficult to say what is a complex case, unless we abandon
> XPath in "when" statements and use something else - and considerably
> simpler.
> 

Yes.

> <bar>2</bar>

No.

(Note the new text that specifies that 'bar' is tentatively replaced
with a dummy node during when processing)

> >> IMO YANG spec shouldn't deal with this kind of things in the first
> >> place. It should be sufficient to define two validation levels,
> >> e.g. syntactic (schema) and semantic validity. Stating what a protocol
> >> does if either of these validity types is violated would then be up to
> >> the protocol spec, and different protocols may use different
> >> approaches.
> >>

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 16 Oct 2015 13:24:04 +0200

> On 16 Oct 2015, at 13:11, Martin Bjorklund <[email-masked]> wrote:
> 

Yes, I know, but it is *extremely* weird. I can guarantee you that XPath experts would be upset when seeing this.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Oct 2015 14:01:43 +0200 (CEST)

Hi,

> If that is what is meant by 8.2.1 then I have a few comments

Sorry for the confusion on this topic.  I have now done some digging
in the archives and I think that section 8.2 is really intended to
apply only to *configuration data* (which it actually says upfront).
The text in 8.2.1 is not intended to be for generic rpcs; it is
intended for rpcs that modify configuration datastores (edit-config).

The whole idea is that there are really three points in time where
"checks" are done - (1) simple type checks and structural checks when
the edit-config is parsed (2) check that the requested _operations_
are valid and (3) check that the _resulting datastore_ is valid.

This said, note that section 8.1 applies to *all* valid data trees,
including rpc/action input/output.

So, in summary, 8.1 defines the generic rules for valid data trees,
and 8.2 defines the NETCONF-specific behavior for edit-config
processing.

In order to resolve Balazs' original issue, we need to agree what
should happen in these cases *for NETCONF*.

Suppose we have:

 leaf a {
   when "../b = 42";
   type int32;
 }
 leaf b {
   type int32;
 }

Scenario A
----------
The DB contains b=10

The server gets an edit-config with:

   <a>2</a>

What is the result?

 1)  an error is returned
 2)  ok; the request to set a to 2 is silently dropped

Scenario B
----------
The DB contains b=10.

The server gets an edit-config with:

   <a>2</a>
   <b>42</b>

What is the result?

 1)  an error is returned
 2)  ok, db now has b=42; the request to set a to 2 is silently
     dropped
 3)  ok, db now has a=2 and b=42

Scenario C  (same as 2, but different order in edit-config)
----------
The DB contains b=10.

The server gets an edit-config with:

   <b>42</b>
   <a>2</a>

What is the result?

 1)  an error is returned
 2)  ok, db now has b=42; the request to set a to 2 is silently
     dropped
 3)  ok, db now has a=2 and b=42

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 16 Oct 2015 14:52:07 +0200

Hello,
AFAIK the document order in edit-config was not seen as important till 
today, with the single exception :
- user-ordered list/leaf-list
Making it significant now would be a new concept. I don't want that. It 
makes it more difficult to make a correct edit-config.
So Scenario B(2) and C should have the same result.
regards Balazs

---

From: Andy Bierman <[email-masked]>
Date: Fri, 16 Oct 2015 08:03:32 -0700

Hi,

I find all this fretting over when-stmt corner-cases to be a waste of time.
I certainly have no intention of spending 100s of hours coding for
corner-cases
that have no operational value whatsoever.  When-stmt has always been full
of problems that exist on paper but not in real servers.

There is no need for the YANG spec to point out that the constraints
only apply to YANG. That is self-evident.

Why doesn't when-stmt apply to a plain rpc?

   rpc plot-point {
     input {
        leaf 3d {
          type boolean;
           default false;
        }
        leaf X { type uint32; }
        leaf Y { type uint32; }
        leaf Z {
          when "../3d";
          type uint32;
       }
     }
   }

Are you saying that the when-stmt for Z is not allowed?
Or that it gets ignored and not enforced?
IMO this section was rather clear -- it applies to when-stmt
in the RPC input.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 16 Oct 2015 17:37:46 +0200

> On 16 Oct 2015, at 17:03, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 
> I certainly have no intention of spending 100s of hours coding for corner-cases

This may well be because YANG module authors so far have been aware of the pitfalls of "when". Now that YANG is mushrooming all around, it may change very soon.

Lada

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 16 Oct 2015 17:42:24 +0200

Hello,

I don't have a problem with the when in rpc/action as it is a simple
case, it is meaningful understandable. I have problems with the
undespecified behavior of when in  edit-config and with the MANY!
complicated "corner" cases. I would love to just prohibit 90% of
them.

regards Balazs

On 2015-10-16 17:03, Andy Bierman
wrote:

Hi,

I find all this fretting over when-stmt corner-cases to be
a waste of time.

I certainly have no intention of spending 100s of hours
coding for corner-cases

that have no operational value whatsoever.  When-stmt has
always been full

There is no need for the YANG spec to point out that the
constraints

rpc plot-point {

input {

leaf 3d {

type boolean;

default false;

}

leaf X { type uint32; }

leaf Y { type uint32; }

leaf Z {

when "../3d";

type uint32;

}

}

}

IMO this section was rather clear -- it applies to
when-stmt

in the RPC input.

Andy

On Fri, Oct 16, 2015 at 5:01 AM, Martin
Bjorklund <[email-masked]>
wrote:

Hi,

Balazs Lengyel <[email-masked]>

> If that is what is meant by 8.2.1 then I have a few
comments

Sorry for the confusion on this topic.  I have now done some
digging

in the archives and I think that section 8.2 is really
intended to

apply only to *configuration data* (which it actually says
upfront).

The text in 8.2.1 is not intended to be for generic rpcs; it
is

intended for rpcs that modify configuration datastores
(edit-config).

The whole idea is that there are really three points in time
where

"checks" are done - (1) simple type checks and structural
checks when

the edit-config is parsed (2) check that the requested
_operations_

are valid and (3) check that the _resulting datastore_ is
valid.

This said, note that section 8.1 applies to *all* valid data
trees,

So, in summary, 8.1 defines the generic rules for valid data
trees,

and 8.2 defines the NETCONF-specific behavior for
edit-config

processing.

In order to resolve Balazs' original issue, we need to agree
what

Suppose we have:

leaf a {

when "../b = 42";

type int32;

}

leaf b {

type int32;

}

Scenario A

----------

The DB contains b=10

<a>2</a>

What is the result?

1)  an error is returned

Scenario B

----------

The DB contains b=10.

<a>2</a>

<b>42</b>

What is the result?

1)  an error is returned

2)  ok, db now has b=42; the request to set a to 2 is
silently

dropped

----------

The DB contains b=10.

<b>42</b>

<a>2</a>

What is the result?

1)  an error is returned

2)  ok, db now has b=42; the request to set a to 2 is
silently

dropped

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 16 Oct 2015 17:55:01 +0200

Hello,
We already have embedded choice  and when in choice in the OSPF module 
https://tools.ietf.org/html/draft-ietf-ospf-yang-02.
So unless we do something fast the nasty complications will be happenning.
regards Balazs

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Oct 2015 21:49:35 +0200 (CEST)

No.

No.

> IMO this section was rather clear -- it applies to when-stmt

No - and that's the problem which we need to fix.  It might be that we
all agree on the expected server behavior ,and if that's the case, we
just need to fix the text to align with what we meant.  In order to
resolve this, it would be very helpful if you could provide your

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 16 Oct 2015 12:55:50 -0700

> opinion on the questions in the three scenarios below:

We implement (2)

> > > Scenario B

We implement (3)

> > > Scenario C  (same as 2, but different order in edit-config)
We implement (3)

Order of edits never matters.
We apply all the edits in a non-destructive manner and then
run all the when-stmt tests that are needed.

The order of when-stmt evaluation can change the answer but we
have not seen any YANG modules where that really happens.

Andy

---

From: Xiang Li <[email-masked]>
Date: Fri, 16 Oct 2015 17:49:56 -0500

> opinion on the questions in the three scenarios below:

I would expect 1).

>>> Scenario B

I would expect 3)
>>> Scenario C  (same as 2, but different order in edit-config)
I would expect 3)

In SNMP, when a set operation contains multiple variable bindings, the 
rule is:

"Each variable assignment specified by the SetRequest-PDU should be 
effected as if
simultaneously set with respect to all other assignments specified in
the same message."

I would expect the same rule applies for edit-config. That is , the 
order is not
significant.

-Xiang Li

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 18 Oct 2015 11:52:00 +0200

>   2)  you get ok; the request to set a to 2 is silently dropped

Isn't the simplest to always make the changes that were requested in
the rpc/action (e.g. edit-config) and then to validate the result and
if it fails to validate to return an error? No magic addition or
removal of nodes while trying to guess what the client wanted to
achieve. I am likely missing details since I never implemented this
but having a processing logic that is simple to understand would be
good and I think it is fair to expect that clients send edits that
do not require the server to guess what should happen.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 18 Oct 2015 14:21:22 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> achieve.

Note that "when" is like a generalized "choice".  If nodes for one
case is present, and you want another case to be set, you just have to
create the new nodes.  The server deletes the nodes from the old case
for you.  In the same way, the server deletes nodes from "when"
statements that become false.

The "must" statement behaves the way you describe above.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 18 Oct 2015 15:00:25 +0200

> The "must" statement behaves the way you describe above.

Not sure this automatic deletion of nodes was a smart idea since this
leads to the confusion of what the server does with certain edits. I
am not sure the savings this brings (you do not have to delete the old
stuff) are worth the trouble. Are clients really relying on this
automatic deletion of nodes?

/js

PS: These are really specific protocol aspects that ideally would be
    in some other document (but its difficult to refactor things in
    the IETF).

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 18 Oct 2015 16:43:07 +0200

> On 18 Oct 2015, at 11:52, Juergen Schoenwaelder <[email-masked]> wrote:
> 

That would be the type of behaviour I'd prefer. The auto-deletion feature also goes against the principle of least embarrassement - a trivial error can inadvertently erase substantial parts of a data tree.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 18 Oct 2015 08:22:25 -0700

On Sun, Oct 18, 2015 at 7:43 AM, Ladislav Lhotka <[email-masked]> wrote:

>
> > On 18 Oct 2015, at 11:52, Juergen Schoenwaelder <
> [email-masked]> wrote:
> >
> That would be the type of behaviour I'd prefer. The auto-deletion feature
> also goes against the principle of least embarrassement - a trivial error
> can inadvertently erase substantial parts of a data tree.
>
>
This goes against the Postel Principle,
You can make your code as fragile as possible if you want.
I have always written servers that try to figure out what the client is
doing.

It seems obvious to me that when-stmt is applied after edits are applied,
just like a choice-stmt.

The server should not be guessing that valid edits from the client
are really programming errors.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 19 Oct 2015 21:04:34 +0200

> are really programming errors.

To me, auto-deletion feels fragile.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 19 Oct 2015 12:22:03 -0700

> To me, auto-deletion feels fragile.
Why?  Returning an error seems fragile to me.
What if the client is using a template that creates some data structure.
The request says "add foo when X is true, add bar when Y is true).
Pruning the false when-stmt is what the client should expect.
Otherwise the template is very fragile and the client has to make a
different
template for every possible permutation of the data structure with
and without false-when nodes.

/js
Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 20 Oct 2015 09:40:12 +0200

> On 19 Oct 2015, at 21:04, Juergen Schoenwaelder <[email-masked]> wrote:
> 

+1

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 20 Oct 2015 09:53:54 +0200

> On 19 Oct 2015, at 21:22, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> On Mon, Oct 19, 2015 at 12:04 PM, Juergen Schoenwaelder <[email-masked]> wrote:
> Otherwise the template is very fragile and the client has to make a different

I see it otherwise. If a client wants to delete existing configuration, he should do it explicitly, perhaps in the same transaction. With the complexity of "when" evalution, it is way too easy to mess up things. Especially when concurrent clients modify the configuration at the same time, the only correct server behaviour IMO is to return a validity error: "I don't accept your edit because another client did some edits in the mean time that are incompatible with your stuff."

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 20 Oct 2015 10:09:42 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> > On Mon, Oct 19, 2015 at 12:04 PM, Juergen Schoenwaelder
> > <[email-masked]> wrote:
> I see it otherwise. If a client wants to delete existing
> configuration, he should do it explicitly, perhaps in the same
> transaction. With the complexity of "when" evalution, it is way too
> easy to mess up things. Especially when concurrent clients modify the
> configuration at the same time, the only correct server behaviour IMO
> is to return a validity error: "I don't accept your edit because
> another client did some edits in the mean time that are incompatible
> with your stuff."

I agree with Andy.  This is one of these cases where the feature might
be complicated in theoretical corner-cases, but extremely useful in
practice.  People that use YANG try to capture real-world semantics.
I have never seen anyone trying to refer to the conditional nodes in a
when expression - simply b/c it doesn't make any sense.

And anyway, changing this would be backwards incompatible, so I don't
know how constructive this conversion is right now.

At this point, it would be useful to focus on the text in 7.21.5 to
make sure it captures the current semantics.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 20 Oct 2015 10:18:15 +0200

> On 20 Oct 2015, at 10:09, Martin Bjorklund <[email-masked]> wrote:
> 

Note that this also applies to cases of a plain choice. If user A adds case C1 and user B (who is not aware of A's changes) then adds case C2, I don't want the server to automatically nuke A's changes. It's the same as in source code management systems - if there is a conflict, it is left to human users to resolve it.

> know how constructive this conversion is right now.

This is not true, if we stick to the YANG 1.1 constraints that are in the charter - no 1.0 module becomes invalid.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Oct 2015 02:05:09 -0700

+1

The false-when deletion has been there from the start,
and works just like the remove-old-case-stmt if the
client selects a new case from a choice-stmt.

The 'shared candidate  without a lock' argument does not apply at all.
That is fragile with or without when-stmts.

The argument "the client might not know what it is doing"
is also really weak.  How does the server know that?

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 21 Oct 2015 13:45:58 +0200

I would love to get rid of the autodelete feature. It really complicates 
things.
regards Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 21 Oct 2015 13:55:52 +0200

Fully agree autodelete is fragile. Kill it.

One more reason why it is fragle is that the autodeletion might change 
another "when's" value thereby creating defaults or deleting something 
that can change some other when statement ad infinitum. Detecting such 
possible loops is very difficult.
regards Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 21 Oct 2015 14:16:58 +0200

Hello Martin,
I would want to codify this. My earlier proposal was:

- when MUST NOT be dependent on a data node controlled by a when or 
choice statement

Notice the strong MUST NOT statement. This would simplify life greatly.
regards Balazs

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Oct 2015 05:33:47 -0700

Hi,

IMO we do not need lots of rules for when-stmt.
They are harder to enforce than just implementing the auto-deletion.

Note that auto-deletion also applies to nodes already in candidate or
running.
It is just a derivative case to have a newly-created node deleted right
away.
If you add node /foo it may cause node /bar and node /baz to get deleted.

I strongly object to treating a false when-stmt in a datastore validation
as an error.  This is not how YANG 1.0 works, and this is not
backward-compatible.

Andy

On Wed, Oct 21, 2015 at 5:16 AM, Balazs Lengyel <[email-masked]

> - when MUST NOT be dependent on a data node controlled by a when or choice
> statement
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 21 Oct 2015 14:46:59 +0200

> On 21 Oct 2015, at 14:33, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 
> Note that auto-deletion also applies to nodes already in candidate or running.
> It is just a derivative case to have a newly-created node deleted right away.

I think it has nothing to do with YANG (1.0 or whatever), and RFC 6020 correctly describes this auto-deletion behaviour for "choice" in sec. 7.9.6 NETCONF <edit-config> Operations. It is indeed protocol business - YANG spec should just define what's valid and what isn't.

IMO RESTCONF spec doesn't require auto-deletion.

Lada

> 
> 
> Andy
> 
> 
> On Wed, Oct 21, 2015 at 5:16 AM, Balazs Lengyel <[email-masked]> wrote:
> Hello Martin,
> - when MUST NOT be dependent on a data node controlled by a when or choice statement
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Oct 2015 06:07:53 -0700

On Wed, Oct 21, 2015 at 5:46 AM, Ladislav Lhotka <[email-masked]> wrote:

> I think it has nothing to do with YANG (1.0 or whatever), and RFC 6020
> correctly describes this auto-deletion behaviour for "choice" in sec. 7.9.6
> NETCONF <edit-config> Operations. It is indeed protocol business - YANG
> spec should just define what's valid and what isn't.
>

Our server uses the same validation engine for both protocols.
RESTCONF does not change the behavior of YANG in any way.
I don't see how YANG validation procedures would not apply to RESTCONF.

YANG says that the node semantics apply IFF the when-stmt evaluates to true.
It is up to the implementation to enforce that.  It applies to
server-created
nodes or nodes created via some protocol.

Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 21 Oct 2015 15:22:21 +0200

> On 21 Oct 2015, at 15:07, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

The validation procedure does apply (the notion of a valid data tree has to be the same) but auto-deletion doesn't because it is specified in "NETCONF <edit-config> ..." sections (7.9.6 and 8.3.2), and RESTCONF doesn't use <edit-config>.

> It is up to the implementation to enforce that.  It applies to server-created

Yes, but it can be enforced either by auto-deleting offending nodes, or by refusing to accept changes that lead to an invalid configuration.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Oct 2015 06:31:17 -0700

On Wed, Oct 21, 2015 at 6:22 AM, Ladislav Lhotka <[email-masked]> wrote:

> The validation procedure does apply (the notion of a valid data tree has
> to be the same) but auto-deletion doesn't because it is specified in
> "NETCONF <edit-config> ..." sections (7.9.6 and 8.3.2), and RESTCONF
> doesn't use <edit-config>.
>
> >
> Yes, but it can be enforced either by auto-deleting offending nodes, or by
> refusing to accept changes that lead to an invalid configuration.
>
>

But the "when-stmt" never causes an error for application within
a datastore.

The text in sec. 8 does not apply because the when-stmt is not
on any object in the RPC being processed.

Only this text applies:

   The "when" statement makes its parent data definition statement
   conditional.  The node defined by the parent data definition
   statement is only valid when the condition specified by the "when"
   statement is satisfied.

The NETCONF specific text needs to change.
Simply putting , "For example, NETCONF ..." might be enough.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Oct 2015 20:15:43 +0200 (CEST)

[...]

> Simply putting , "For example, NETCONF ..." might be enough.

Can you be more specific - exactly where do you suggest this change?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Oct 2015 13:21:21 -0700

> Can you be more specific - exactly where do you suggest this change?
Not really -- there are way too many places the draft refers to
NETCONF operations, where it could just be referring to a configuration
datastore.

The draft does not even cover proprietary access RPCs.
If somebody defined <my-edit-config> the same datastore validation
procedures should apply to the conceptual configuration datastore.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 22 Oct 2015 09:21:25 +0200

> On 21 Oct 2015, at 22:21, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

Ideally, YANG spec should contain only language definition and statements about validity of instance data. Protocol-related stuff should go into protocol-specific "adaptation" documents, and encodings should also be separate.

Lada

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 22 Oct 2015 12:45:56 +0200

Hello,

I STRONGLY agree with Andy, Interfaces MUST work the same way.
Autodeletion MUST work or NOT work for all interfaces (Netconf,
Restconf, CLI, GUI, etc.) the same way. IMO it is not a protocol
issue. It is part of the YANG definition.

The whole idea behind model driven OAM is that we have one model
that works (mostly) the same way on all interfaces. The more
differences we have the less usable the product, the more difficult
to implement.

regards Balazs

On 2015-10-21 15:07, Andy Bierman
wrote:

On Wed, Oct 21, 2015 at 5:46 AM,

> backward-compatible.

I think it has nothing to do with YANG (1.0 or whatever),
and RFC 6020 correctly describes this auto-deletion
behaviour for "choice" in sec. 7.9.6 NETCONF
<edit-config> Operations. It is indeed protocol
business - YANG spec should just define what's valid and
what isn't.

Our server uses the same validation engine for both
protocols.

RESTCONF does not change the behavior of YANG in any
way.

I don't see how YANG validation procedures would not
apply to RESTCONF.

YANG says that the node semantics apply IFF the
when-stmt evaluates to true.

It is up to the implementation to enforce that.  It
applies to server-created

Lada

Andy

--

PGP Key ID: E74E8C0C

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 22 Oct 2015 13:08:07 +0200

Hello,
I would propose:
Change
"If the XPath expression references any node that also has associated 
"when" statements, these "when" expressions MUST be evaluated first."
To
"If the expression in a when statement is dependent on a data node 
controlled by another when or choice statement, the order of evaluation 
of this when and the other when/choice statement is implementation 
dependent."

Result:
- As this makes such cases non-interoperable it effectively says: Don't 
use them as the results are unpredictable.
- We should explicitly state this in 6087bis
- This takes away the most nasty side effects of autodelete (and 
auto-usage of default values, which can be just as big problem.)
- This would make implementation easier
- It is YANG 1.0 compatible, as the same yang module (YAM) is still 
accepted.
- It is YANG compatible as the to-be deleted sentence was only inserted 
by YANG 1.1. It is not present in YANG 1.0.
- As Martin stated, this is anyway an unseen, crazy corner-case

regards Balazs

P.S.1. My preference would be to say that such case are not just 
implementation specific but rather forbidden, but that might be 
incompatible, although I would consider them as error, so don't care 
about compatibility.

P.S.2.  I still don't like autodelete.

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 22 Oct 2015 13:15:22 +0200

> On 22 Oct 2015, at 12:45, Balazs Lengyel <[email-masked]> wrote:
> 
> Hello,
> I STRONGLY agree with Andy, Interfaces MUST work the same way. Autodeletion MUST work or NOT work for all interfaces (Netconf, Restconf, CLI, GUI, etc.) the same way. IMO it is not a protocol issue. It is part of the YANG definition.

This however limits the use of YANG to NETCONF and closely related protocols, which is IMO short-sighted. People have already started using YANG models with other protocols:

https://mailarchive.ietf.org/arch/msg/netmod/h_0jZbqfdcWFA8_hXx0gGzoi4X0

Putting too much protocol details into data modelling language definition actually undermines the value of the standard - users of other protocols will simply ignore those MUSTs and MUST NOTs they cannot or don't want to implement.

Lada

>  
> 
> The whole idea behind model driven OAM is that we have one model that works (mostly) the same way on all interfaces. The more differences we have the less usable the product, the more difficult to implement.
> regards Balazs
> 
> On 2015-10-21 15:07, Andy Bierman wrote:
>> 
>>

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 22 Oct 2015 13:26:31 +0200

Hello Lada,
The issue is what is "too much protocol details" ?
I agree that there are many things that are not part of the YANG 
language/metamodel itself. On the other hand if a simple create leaf 
operation on different interfaces can result in different datastores and 
different operation of the network node, then IMHO the different 
interfaces use different models, NOT the same.

I consider autodelete a basic property of the YANG model. A mechanism 
that results in deleting data nodes should work (or not) the same way on 
all interfaces.
regards Balazs

P.S. I still hate autodelete

---

From: Andy Bierman <[email-masked]>
Date: Thu, 22 Oct 2015 10:51:28 -0700

On Wed, Oct 21, 2015 at 4:45 AM, Balazs Lengyel <[email-masked]

> I would love to get rid of the autodelete feature. It really complicates

So how would when-stmt work?
It would be an error if a false when-stmt ever occurred?

  leaf X { type int32; }
  leaf Y { when "../X=3"; type int32; }

Let's say the client creates node in candidate:

  create /X value=3

works OK

  create /Y value=42

works OK

   merge /X value=5

Is this an error because it would cause /Y to be deleted?
Or does it work like choice-stmt, and the new /X is accepted
and the old /Y is deleted?

Why would this scenario be treated differently then if the auto-delete
happens
on a node provided in the <config> payload?

regards Balazs

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 22 Oct 2015 20:48:35 +0200 (CEST)

Balazs Lengyel <[email-masked]> wrote:
> operation on different interfaces can result in different datastores
> and different operation of the network node, then IMHO the different
> that results in deleting data nodes should work (or not) the same way
> on all interfaces.

I agree.  This is what makes "when" different than "must".
auto-deletion in choice/when should be described as a property of the
data model for the datastore.  Parts of the text from Section 8.2.2
should be made more generic and moved, probably to a new section
8.1.1.   I will have a look at this.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 22 Oct 2015 15:19:17 -0700

Hi,

I have to report that at least 1 customer agrees with you about
auto-deletion.
The comment was "how are we supposed to tell there is a bug in the client
if we do not get back an error instead of silent deletion?"

A: use "must" instead of "when" and you will get an error right away.

In hindsight, perhaps NETCONF should have the equivalent of the --force
option
in many Unix commands.  e.g., only do the auto-deletion if --force is
present
otherwise return an error that auto-deletion would have occurred.

Andy

On Thu, Oct 22, 2015 at 3:45 AM, Balazs Lengyel <[email-masked]

> Autodeletion MUST work or NOT work for all interfaces (Netconf, Restconf,
> CLI, GUI, etc.) the same way. IMO it is not a protocol issue. It is part of
> the YANG definition.
>
> The whole idea behind model driven OAM is that we have one model that
> works (mostly) the same way on all interfaces. The more differences we have
> the less usable the product, the more difficult to implement.
> regards Balazs
>
> On Wed, Oct 21, 2015 at 5:46 AM, Ladislav Lhotka < <[email-masked]>
> [email-masked]> wrote:
>
>>
>> > On 21 Oct 2015, at 14:33, Andy Bierman < <[email-masked]>
>> [email-masked]> wrote:
>> >
>> > Hi,
>> >
>> > On Wed, Oct 21, 2015 at 5:16 AM, Balazs Lengyel <
>> <[email-masked]>[email-masked]> wrote:
>> > Hello Martin,

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 23 Oct 2015 10:35:48 +0200

> I agree.  This is what makes "when" different than "must".

Right, that's why I tend to avoid "when", except in augments.

> 8.1.1.   I will have a look at this.

I think that defining a general datastore API as a part of YANG spec is
not useful. Protocols that potentially might use YANG (gRPC, Cap'n
Proto) won't be changed, so should we tell them not to use YANG?

IMO YANG spec should tell what's valid and what isn't, and stop there.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 23 Oct 2015 10:45:28 +0200

> Is this an error because it would cause /Y to be deleted?

No, it is an error because the resulting data tree is invalid.

> on a node provided in the <config> payload?

I don't advocate auto-deleting anything in <config> payload either.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 23 Oct 2015 10:58:29 +0200 (CEST)

> Proto) won't be changed, so should we tell them not to use YANG?

It is not a datastore API.  It would specify what happens within a
datastore as "when" expressions become false.

Note that the concept of datastores, and specifically config vs state
is a core feature of YANG.  By default, if you just specify some
nodes, they are config true.  If you want to use YANG define some kind
of other datastructure, you need to put the nodes in some extension.
For example, we use:

  tailf:structure foo {
    leaf a { ... }
    container b { ... }
  }

to define a structure that doesn't have any real semantics associated.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 23 Oct 2015 10:55:50 +0200

> if we do not get back an error instead of silent deletion?"

This is not only a matter of bugs in client code but also of human errors
and typos: an operator might not realize that making a configuration
change might trigger auto-deletion in remote parts of the data tree.

I think that sane models should use "when" in a very restricted and
localised way, as most modules so far do, e.g. making a node conditional
with respect to the value of a sibling leaf. 

> otherwise return an error that auto-deletion would have occurred.

I was thinking about the same. It is conceivable that different apps and
users might want different behavior for the same node - but it would be
impossible if one alternative is hard-wired in YANG spec.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 23 Oct 2015 11:29:54 +0200

> On 23 Oct 2015, at 10:58, Martin Bjorklund <[email-masked]> wrote:
> 

Well, I assume you want *all* "NETCONF <edit-config> ..." sections to hold for all protocols using YANG - key handling, list insertions etc. And these sections together represent quite a significant part of a datastore API that in fact reflect limitations in <edit-config> design.

> is a core feature of YANG.  By default, if you just specify some

Yes, "config true/false" is rather NETCONF-specific. We are already seeing protocols (I2RS) that might want something else.

> to define a structure that doesn't have any real semantics associated.

It could also be done the other way around, e.g. remove the "config" and "key" statements from the YANG core and define them as extensions for NETCONF-like protocols. I know this idea is not going to become popular but then IMO YANG is bound to remain a niche technology, unfortunately.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 23 Oct 2015 07:02:45 -0700

> datastore as "when" expressions become false.

Agreed -- it should be possible to specify what happens inside a
configuration datastore without needing to specify that <edit-config>
or <copy-config> are the only ways to alter a datastore.

Since a datastore is just a conceptual collection of top-level YANG data
nodes,
there is no problem defining this concept in YANG.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 23 Oct 2015 16:19:19 +0200

> On 23 Oct 2015, at 16:02, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> Since a datastore is just a conceptual collection of top-level YANG data nodes,

I agree it can be done but the question is whether it needs to be done - given that there is no consensus about what the behaviour should be.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 23 Oct 2015 22:36:17 +0200

[...]

> IMO YANG spec should tell what's valid and what isn't, and stop there.

As technical contributor, I tend to agree. The purpose of validation
should be to return a boolean - datastore is valid or invalid. I find
the idea scary that validation causes changes of a datastore in an
attempt to make an invalid the datastore valid. And it is even more
scary if the attempt to make the datastore valid requires to remember
what the last edit was that triggered the validation procedure in
order to decide how to try to make the datastore valid (if we consider
different protocols with different primitives to trigger edits on a
datastore).

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 23 Oct 2015 14:42:26 -0700

> different protocols with different primitives to trigger edits on a

There is a very strong use-case in "augment when" and also
just "when <correct-subclass>".

Does that mean that implementing "when" statement should not
be 10X harder than anything else in YANG?  Who knows.

I suspect there does not exist anything close to a perfect
implemention of YANG when evaluation.  If customers really wanted
to use the corner-case examples you guys come up with, then
there would be a problem.  So far it is mostly the 2 use cases I described.

The current auto-deletion is actually consistent behavior
because it applies to nodes in the PDU and nodes already
in the datastore.  Auto-deletion avoids forcing the client to
make multiple edits, possibly leaving the datastore in
a vulnerable state in between edits.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 24 Oct 2015 00:09:32 +0200

On Fri, Oct 23, 2015 at 02:42:26PM -0700, Andy Bierman wrote:

> Auto-deletion avoids forcing the client to make multiple edits,
> possibly leaving the datastore in a vulnerable state in between
> edits.

A single edit can say 'delete this, add that' and then you validate
the result. This is simple to understand and matches the behaviour of
other systems people are familiar with. I do not buy your 'vulnerable
state in between argument'. If the client prefers to send multiple
edits, it better uses locks anyway.

Right now, we seem to accept 'add that' and when validation of the
result fails, the server is expected to try 'delete that' in an
attempt to restore happiness. And this try 'delete that' can have
ripple effects. A validate operation that changes was is being
validated is scary. Its like pyang modifying foo.yang while parsing it
in an attempt to finish without parsing errors...

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 23 Oct 2015 15:40:25 -0700

Hi,

I buy Martin's argument that "when" stmt is like "choice".
The draft says exactly what will happen if the constraint
is not satisfied.  Why don't we require the client to delete
the old case and create the new case all at once?

The "when" auto-deletion is no less scary then "case" auto-deletion.
It has the exact same ripple effects.  For that matter, if a client
incorrectly deletes a trunk interface, scary things will happen as well.
So why don't we take out the "delete" command?
A buggy client may get *any* of the commands wrong, so
better take out all editing commands to protect everyone..

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 24 Oct 2015 15:07:41 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> should be to return a boolean - datastore is valid or invalid.

Right.  This is what "must" does.  "when" is different.  If a node's
"when" expression becomes false, that node is deleted, and its other
constraints are no longer used (e.g. must, mandatory etc).  These are
two different use cases, two different tools available to the data
model designer.

If we put "when" to the side for a moment, do you also think that
there should be no auto-deletion of cases in a choice?

If this discussion had started from implementation/deployment
experience that said that "when" could not be implemented or that it
made it difficult to write NMS system or something else, things would
be different.  But now we have a feature that has been in use for 5+
years, and there are several implementations of it out there, and now
we say that it should be removed?  Or worse, keep the syntax but
radically change the semantics.

This said, I am all for guidelines and that we check for bad uses of
"when" when we do reviews of data models.

/martin

> I find

---

From: Andy Bierman <[email-masked]>
Date: Sat, 24 Oct 2015 09:41:52 -0700

> radically change the semantics.

The one special case that has been identified is data provided
as part of an edit that is silently deleted and <ok/> returned.
This could be considered different than providing an edit that
deletes existing data nodes.

> "when" when we do reviews of data models.
We will do that, but it does not address the runtime error (or not)
of auto-deletion of data provided in the edit. Should this be
an error, a warning, or return OK?

Auto-deletion of existing data nodes is fine.
The client needs to be aware of the YANG module
or watch netconf-config-change notifications.  Setting /X=3 or /X=5
requires understanding of the semantics for object X.  The when-stmt is just
part of that requirement.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 25 Oct 2015 14:58:48 +0100 (CET)

> deletes existing data nodes.

Right.  I think an error should be returned to the client in this
case.  This is nicer to the client.  Otherwise, the client might set
something, get <ok/> back and the might be surprised when the changes
did not actually happen.

(FWIW, this is how we interpreted the spec and thus have implemented)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sun, 25 Oct 2015 08:45:01 -0700

> (FWIW, this is how we interpreted the spec and thus have implemented)

What text says return an error here?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 25 Oct 2015 16:55:17 +0100 (CET)

> What text says return an error here?

I think we already had this discussion.  I thought that 8.3.1 was
supposed to cover this, but I can see how it doesn't.  So I agree
there is no such clear text in 6020, even though that was the
intention.  I am open to an discussion about the best behavior in this
case.  It is trivial to change the code to not return the error.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sun, 25 Oct 2015 11:17:56 -0700

On Sun, Oct 25, 2015 at 8:55 AM, Martin Bjorklund <[email-masked]> wrote:

> > > > > > > > auto-deletion in choice/when should be described as a
> property
> > > of the

I don't have enough data to know if this is more of a bug than a feature.
I agree that it is a slippery slope to say the hidden when-stmts behind
the <config> anyxml node are covered by 8.3.1.

The text for if-feature looks similar to when-stmt, but they are
not treated the same. If an edit contains a node where the if-feature
is false, an 'unknown-element' error is always returned.
But YANG features are stable, and for "when", the results are
possibly changing because of the edit in progress.

We have 'delete' to be fussy and 'remove' to delete if it exists,
and the client can decide which one to use. I would prefer
a protocol solution that let the client decide if payload auto-delete
is OK or not.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Oct 2015 10:42:07 +0100

> there should be no auto-deletion of cases in a choice?

If I were to start from scratch, my answer would be most likely yes.

> radically change the semantics.

Do independent implementations really behave the same? I am not sure,
the discussion around this makes me believe that it is somewhat likely
that they do not all do the same.

Reading section 7.19.5 of RFC 6020 again, I do not find the auto-deletion
described. I do see auto-deletion defined in section 7.9.6 for
NETCONF's edit-config operation. (The text does not say anything about
what happens if an attempt is made to create multiple cases, I guess
it is implementation dependent which choice will become effective but
one might also consider this an error - but clearly this is not
defined). I note that RFC 6020bis has lifted this auto-deletion up
from being a NETCONF edit-config property to a property that applies
to all protocols. (It still does not define what happens if an edit
creates multiple case branches.)

A proposal was made to declare "when" auto-deletion to be part of the
YANG data store validation process, that is, it applies to every
protocol interface. This means that a datastore not only needs to
maintain data that is being validated, it also needs to remember the
last edit (at least) in order to _guess_ what should be deleted during
validation.  It is the guessing part and the idea that the server
somehow tries to infer what the intention of the client was that makes
me (as a technical contributor) feel somewhat uncomfortable.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 11:44:05 +0100

> just "when <correct-subclass>".

Essentially, these are cases that can be expressed with a regular
grammar like this:

X -> a A | b B

where terminals "a" and "b" are the types of two classes and
nonterminals "A" and "B" represent their contents.

Incidentally, this is also a type of choice that can be easily expressed
in RELAX NG:

  (element type { "a" } & A) | (element type { "b" } & B)

Unfortunately, YANG's choice cannot be used in this fashion because the
same node (here it is "type") cannot be used in multiple cases.

> implemention of YANG when evaluation.  If customers really wanted

One (rather restrictive) way would be to allow when expressions only
with functions derived-from() and derived-from-or-self(). So classes and
sublasses would be modelled exclusively via identities.

> there would be a problem.  So far it is mostly the 2 use cases I

Yes, but then it would be much better to have corresponding syntactic
restrictions. The fact that any XPath expression is allowed is not only
scary, it could be exploited as a security hole.

> a vulnerable state in between edits.

Auto-deletion just makes the problems of "when" more serious because it
can have catastrophic consequences.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 11:47:04 +0100

+1

And auto-deleting a case of a choice can have the same ripple effects
if, for example, some "when" expression depends on the auto-deleted case. 

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 11:53:52 +0100

> The "when" auto-deletion is no less scary then "case" auto-deletion.

Correct.

> So why don't we take out the "delete" command?

In a delete command, the client has to explicitly identify the node to
be deleted. With auto-deletion this is not the case.

> better take out all editing commands to protect everyone..

It's vulnerable not only to bugs in client or server code but also to
human errors - an operator might not fully understand the contrived logic
of the data model.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 12:31:52 +0100

> On 26 Oct 2015, at 10:42, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Same here.

> that they do not all do the same.

For one, libnetconf [1] doesn't do auto-deletion because it uses DSDL schemas for validation, and DSDL deliberately restricts the ways in which the validated XML infoset can be manipulated.

[1] https://github.com/cesnet/libnetconf

> to all protocols. (It still does not define what happens if an edit

Not yet, I believe, and I would be opposed to such a change.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 26 Oct 2015 12:55:02 +0100 (CET)

> that they do not all do the same.

Apart from the error reporting issue we have discussion, I think at
least yuma and tail-f behave the same for the use cases that people
actually use.  I'm sure we don't handle all these corner cases the
same way, but this should be fixed by adjusting the code to match the
new text in 6020bis.

> described.

It is described in 8.3.2.

> I do see auto-deletion defined in section 7.9.6 for
> validation.

No guessing is involved.  If a node's when expression evaluates to
false, it is deleted.

/martin

---

From: Jonathan Hansford <[email-masked]>
Date: Mon, 26 Oct 2015 12:24:35 +0000

A theme that appears to have been running through the discussion of languages like YANG since (at least) 2007 (see draft-schoenw-sming-lessons-01, section 3.1) is that “preference should be give to solutions that simplify the task of readers.  Reduction of the efforts required by writers is of secondary importance and the reduction of the efforts required by tool developers is of least important preference.” However, it sounds as though “when” auto-deletion complicates the task for both readers (e.g. network operators) and tool developers.

Jonathan

From: Ladislav Lhotka
Sent: 26 October 2015 10:54
To: Andy Bierman;Juergen Schoenwaelder;Andy Bierman;Martin Bjorklund;Balazs Lengyel;[email-masked]
Subject: Re: [netmod] Order of evaluation for when?

Correct.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 13:48:36 +0100

> On 26 Oct 2015, at 12:55, Martin Bjorklund <[email-masked]> wrote:
> 

Are there any use cases where a client wants to send an edit that makes a "when" expression false, in order to remove the instance of the parent node, i.e. when it is not a mistake?

In the two use cases that Andy described it is probably not the case - e.g., changing the type of an interface in ietf-interfaces is IMO always an error, and changing the type of routing protocol in ietf-routing isn't possible at all because "type" is part of the list key.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Oct 2015 08:10:16 -0700

> same way, but this should be fixed by adjusting the code to match the

Some of the new text that has been discussed in not implementable.
For example, there are no tools that take an arbitrary set of XPath
expressions
against the same XML instance document, and determine which order
(if any) is the one true correct order that must be used.  Requirements
like this in YANG 1.1 are just not implementable.

The "when" statement is the sharpest knife in the drawer.
If it is too complicated to get right, then it will be
used incorrectly.  Same as the "pattern" statement.
We get those wrong all the time and fix them in Errata.
That doesn't seem to be a problem.  I don't hear calls to
remove pattern-stmt because he syntax is really hard.

We are not going to spend any time coding for
clever corner-cases.  We do spend time coding for use-cases.
I agree the interoperability for use-cases is quite acceptable.
All this fuss over corner-cases is not that interesting.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 26 Oct 2015 16:57:35 +0100

> On 26 Oct 2015, at 16:10, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> For example, there are no tools that take an arbitrary set of XPath expressions

And even if it is implementable, it would be terribly expensive.

> remove pattern-stmt because he syntax is really hard.

The difference is that efficient algorithms are available for regex matching, and there are no dark corners.
So a pattern may not do exactly what it was intended for but it should never cause the server or client software to fail or accidentally erase some data.

> All this fuss over corner-cases is not that interesting.

I think it is only matter of time when such corner cases show up in the wild. People like to use sharp knives.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Oct 2015 09:14:32 -0700

On Mon, Oct 26, 2015 at 8:57 AM, Ladislav Lhotka <[email-masked]> wrote:

> So a pattern may not do exactly what it was intended for but it should
> never cause the server or client software to fail or accidentally erase
> some data.
>
>

If the regex test fails then the leaf or leaf-list will be rejected with
an invalid-value error.  If an invalid pattern passes then it will be
accepted as a valid string for that typedef.

This is bad enough but it can in turn cause ripple effects (e.g. leafref)
that are just as bad as deleting data that the when-stmt says is going
to be deleted.

> wild. People like to use sharp knives.
And we fix then one at a time when they come up.
As the number of inter-connected modules grows the possibility
for when-stmt race conditions in real modules will also grow.
Hopefully YANG Doctors will be paying attention and identify problems
in IETF modules before they are published.

Lada

Andy