Subject: [netmod] mandatory choice with non-presence container case

From: Michal Vaško <[email-masked]>
Date: Mon, 24 Jun 2019 18:39:18 +0200

Hi,
I have encountered a situation that I think is not covered by RFC 7950. My specific use-case was as follows.

model:

container TOP {
  leaf L {
    type empty;
  }
  choice A {
    mandatory true;
    container C;
  }
}

data:

<TOP>
  <L/>
  <C/>
</TOP>

Parsing was successful, but printing these data back to XML produced:

<TOP>
  <L/>
</TOP>

and parsing this correctly failed with missing mandatory choice. According to section 7.5.7 [1], I think the C container could be omitted but the whole situation does not seem correct. Thank you for any input.

Regards,
Michal

[1] https://tools.ietf.org/html/rfc7950#section-7.5.7

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Mon, 24 Jun 2019 16:52:34 +0000

Hi Michal,

My thoughts:

According to 7.5.1:

   In the first style, the container has no meaning of its own, existing
   only to contain child nodes.  In particular, the presence of the
   container node with no child nodes is semantically equivalent to the
   absence of the container node.  YANG calls this style a "non-presence
   container".  This is the default style.

Hence your request (because the NP container does not have any children) is equivalent to:

 <TOP>
   <L/>
 </TOP>

which fails the "mandatory" check.

Thanks,
Rob

> -----Original Message-----
> From: netmod <[email-masked]> On Behalf Of Michal Vaško
> Sent: 24 June 2019 17:39
> To: netmod <[email-masked]>
> Subject: [netmod] mandatory choice with non-presence container case
> 
> Hi,
> and parsing this correctly failed with missing mandatory choice. According
> to section 7.5.7 [1], I think the C container could be omitted but the
> whole situation does not seem correct. Thank you for any input.
> 
> Regards,
> Michal
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Michal Vaško <[email-masked]>
Date: Mon, 24 Jun 2019 19:01:29 +0200

Hi Rob,
I think there is a problem in the RFC because using only allowed steps I got invalid data from initially valid data. That cannot be correct.

Regards,
Michal

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Jun 2019 10:11:39 -0700

On Mon, Jun 24, 2019 at 10:01 AM Michal Vaško <[email-masked]> wrote:

> Hi Rob,
> I think there is a problem in the RFC because using only allowed steps I
> got invalid data from initially valid data. That cannot be correct.
>
>
No.  See sec. 7.5.7

   If a non-presence container does not have any child nodes, the
   container may or may not be present in the XML encoding.

Just because your retrieval does not contain the NP-container, that does
not mean the
NP-container was not present in the server for the mandatory-stmt
validation.

Regards,
> Michal
Andy

> > > I have encountered a situation that I think is not covered by RFC
> 7950. My
> > > specific use-case was as follows.
> > >
> > > model:
> > >
> > > container TOP {
> > >   leaf L {
> > >     type empty;
> > >   }
> > >   choice A {
> > >     mandatory true;
> > >     container C;
> > >   }
> > > }
> > >
> > > data:
> > >
> > > <TOP>
> > >   <L/>
> > >   <C/>
> > > </TOP>
> > >

---

From: Michal Vaško <[email-masked]>
Date: Mon, 24 Jun 2019 19:15:18 +0200

Hi Andy,

> NP-container was not present in the server for the mandatory-stmt

I agree, but these valid data were correctly printed into invalid data. I do not think printing is allowed to change the validity of data.

Michal

---

From: Kent Watsen <[email-masked]>
Date: Tue, 25 Jun 2019 01:20:31 +0000

Hi Michal,

> I agree, but these valid data were correctly printed into invalid data. I do not think printing is allowed to change the validity of data.

The NP-container text is unclear.  I'm unsure if you saw the thread in NETCONF, but I filed this issue over the weekend.  Please add to it, if only to include a link to this thread:

	https://github.com/netmod-wg/yang-next/issues/88.

Thanks,
Kent

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Jun 2019 09:39:01 +0200

Kent Watsen <[email-masked]> writes:

> Hi Michal, 
>  
>> I agree, but these valid data were correctly printed into 
>> invalid data. I do not think printing is allowed to change the 
>> validity of data. 
>  
> The NP-container text is unclear.

This issue IMO has more to do with the "mandatory" statement under 
"choice". Similar problems can be caused by data nodes that depend 
on "when". For example:

choice sel {
    madatory true;
    leaf foo {
        when "...";
        ...
    }
    leaf bar { ... }
    leaf baz { ... }

If "foo" exists in instance data but its when condition becomes 
false, the server is expected to remove the "foo" instance, but 
what next? The data becomes invalid, but the server can hardly 
include "bar" or "baz" on its own.

Lada

> I'm unsure if you saw the thread in NETCONF, but I filed this 
> issue over the weekend.  Please add to it, if only to include a 
> link to this thread: 
> 
> Thanks, Kent 
> _______________________________________________ netmod mailing 
> list [email-masked] 

-- 
Ladislav Lhotka Head, CZ.NIC Labs PGP Key ID: 0xB8F92B08A9F76C67

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Jun 2019 09:58:12 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> >> I agree, but these valid data were correctly printed into invalid
> >> data. I do not think printing is allowed to change the validity of
> >> data.
> "choice". Similar problems can be caused by data nodes that depend on
> "when". For example:
> 
> choice sel {
>    madatory true;
>    leaf foo {
>        when "...";
>        ...
>    }
>    leaf bar { ... }
>    leaf baz { ... }
> 
> If "foo" exists in instance data but its when condition becomes false,
> the server is expected to remove the "foo" instance, but what next?
> The data becomes invalid, but the server can hardly include "bar" or
> "baz" on its own.

Right, so that change (that would make the when condition false) will
be rejected by the server, since the resulting config would be
invalid.

/martin

> 
> Lada
> 
> > I'm unsure if you saw the thread in NETCONF, but I filed this issue
> > over the weekend.  Please add to it, if only to include a link to this
> > thread: https://github.com/netmod-wg/yang-next/issues/88. Thanks, Kent
> > _______________________________________________ netmod mailing list
> > [email-masked] https://www.ietf.org/mailman/listinfo/netmod
> 
> --

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 25 Jun 2019 09:08:24 +0000

Hi Michal,

It is not the printing of the data that makes it valid/invalid.

I don't think that your input data was ever valid, because "container C" doesn't satisfy the mandatory statement because it isn't a real data node in the tree - it is instantiated when required and may be deleted when it is no longer required.

I.e. your model has been designed such that it can never be satisfied.

If your model was instead:

container TOP {
  leaf L {
    type empty;
  }
  choice A {
    mandatory true;
    container C {
      leaf L2 {
        type empty;
      }
    }
  }
}

Then this data is valid:

<TOP>
  <L/>
  <C>
   <L2/>
  </C>
</TOP>

But this data is not:

<TOP>
  <L/>
</TOP>

Nor is this, which is directly equivalent to the one above, because the <C/> container doesn't really exist if it doesn't have a child node present.

<TOP>
  <L/>
  <C/>
</TOP>

Thanks,
Rob

> From: Michal Vaško <[email-masked]>
> Sent: 24 June 2019 18:15
> To: Andy Bierman <[email-masked]>
> Cc: Rob Wilton (rwilton) <[email-masked]>; netmod <[email-masked]>
> Subject: Re: [netmod] ?= mandatory choice with non-presence container cas
> 
> Hi Andy,
> 
> > > I think there is a problem in the RFC because using only allowed
> > > steps I got invalid data from initially valid data. That cannot be
> correct.
> > >
> > >
> > No.  See sec. 7.5.7
> >
> > Just because your retrieval does not contain the NP-container, that
> > does not mean the NP-container was not present in the server for the
> > mandatory-stmt validation.
> 
> I agree, but these valid data were correctly printed into invalid data. I
> do not think printing is allowed to change the validity of data.
> 
> Michal
> 
> >
> > Regards,
> > > Michal
> > >
> > >
> > Andy
> >
> >
> > > > Hence your request (because the NP container does not have any
> > > > children)
> > > is equivalent to:
> > > >
> > > >  <TOP>
> > > >    <L/>
> > > >  </TOP>
> > > >
> > > > > to section 7.5.7 [1], I think the C container could be omitted
> > > > > but the whole situation does not seem correct. Thank you for any
> input.
> > > > >
> > > > > Regards,
> > > > > Michal
> > > > >

---

From: Michal Vaško <[email-masked]>
Date: Tue, 25 Jun 2019 13:19:37 +0200

Hi Rob,
actually, I have used model with the container TOP just for simplification, I have encountered the issue while implementing ietf-ssh-server model from its current draft. I have created the container "users" [1] without any "user" list instances. Now, you may argue that this is still not a valid use-case because there are no users but I only tried to satisfy the condition. There are some users on the system but they are generated into the configuration on-demand when operational data is requested.

Regards,
Michal

[1] https://tools.ietf.org/html/draft-ietf-netconf-ssh-client-server-14#page-22

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Jun 2019 13:31:51 +0200

> be rejected by the server, since the resulting config would be

Yes, this is actually the only sane behaviour that should be applied each time a
when constraint becomes invalid - rather then auto-delete the conditional nodes.

Regarding NP-containers, I think their character implies that they can never be
empty in the schema. This should probably be stated in the YANG spec.

Lada

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 25 Jun 2019 13:59:02 +0200 (CEST)

Michal Vaško <[email-masked]> wrote:
> Hi Rob,
> actually, I have used model with the container TOP just for
> simplification, I have encountered the issue while implementing
> ietf-ssh-server model from its current draft. I have created the
> container "users" [1] without any "user" list instances. Now, you may
> argue that this is still not a valid use-case because there are no
> users but I only tried to satisfy the condition.

Yes, I think that this list (user) should have a "min-elements 1".  I
think that matches the intent.

/martin

> There are some users
> on the system but they are generated into the configuration on-demand
> when operational data is requested.
> 
> Regards,
> Michal
> 
> [1]
> https://tools.ietf.org/html/draft-ietf-netconf-ssh-client-server-14#page-22
> 
> > I don't think that your input data was ever valid, because "container
> > C" doesn't satisfy the mandatory statement because it isn't a real
> > data node in the tree - it is instantiated when required and may be
> > deleted when it is no longer required.
> > 
> > Nor is this, which is directly equivalent to the one above, because
> > the <C/> container doesn't really exist if it doesn't have a child
> > node present.
> > 
> > <TOP>
> >   <L/>
> >   <C/>
> > </TOP>
> > 
> > Thanks,
> > Rob
> > 
> > 
> > > data. I

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Jun 2019 15:14:15 +0200

On Tue, 2019-06-25 at 13:59 +0200, Martin Bjorklund wrote:
> think that matches the i ntent.

Interestingly, the "users" container actually becomes a P-container: its
presence indicates that the corresponding case is selected. It might make sense
for an admin to select this case even before any users are configured.

This example also exposes the drawback of the XML representation - it cannot
distinguish between an empty list and nothing. In JSON, the problems of this
thread could potentially be circumvented by configuring

"users" : {
    "user" : [
    ]
}

Lada

> > > > > > >    container node with no child nodes is semantically equivalent
> > > > > > > to
> > > > the

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 25 Jun 2019 13:49:10 +0000

> From: netmod <[email-masked]> On Behalf Of Ladislav Lhotka
> Sent: 25 June 2019 14:14
> To: [email-masked]
> Subject: Re: [netmod] ?= ?==?utf-8?q? mandatory choice with non-presence
> container cas
> 
> > > container "users" [1] without any "user" list instances. Now, you
> > > may argue that this is still not a valid use-case because there are
> > > no users but I only tried to satisfy the condition.
> >
> presence indicates that the corresponding case is selected.

I don't think that this makes it a P-container.

 It might make
> sense for an admin to select this case even before any users are
> configured.

Sure, the "users" container could have been marked as having presence in the YANG model.

> 
> This example also exposes the drawback of the XML representation - it
> cannot distinguish between an empty list and nothing. In JSON, the
> problems of this thread could potentially be circumvented by configuring
> 
> "users" : {
>     "user" : [
>     ]
> }

I don't think that an empty list "exists" in a configuration datastore, i.e. I don't think that it should impart any meaning, in that regard is seems somewhat like an NP-container.

Thanks,
Rob

> 
> Lada
> 
> >
> > /martin
> >
> >
> >
> > > There are some users
> > > on the system but they are generated into the configuration
> > > on-demand when operational data is requested.
> > >
> > > Regards,
> > > Michal
> > >
> > > [1]
> > > > I don't think that your input data was ever valid, because
> > > > "container C" doesn't satisfy the mandatory statement because it
> > > > isn't a real data node in the tree - it is instantiated when
> > > > required and may be deleted when it is no longer required.
> > > >
> > > > Nor is this, which is directly equivalent to the one above,
> > > > because the <C/> container doesn't really exist if it doesn't have
> > > > a child node present.
> > > >
> > > > <TOP>
> > > >   <L/>
> > > >   <C/>
> > > > </TOP>
> > > >
> > > > Thanks,
> > > > Rob
> > > >
> > > >
> > > > > On Monday, June 24, 2019 19:11 CEST, Andy Bierman
> > > > > <[email-masked]>
> > > > > wrote:
> > > > >
> > > > > > On Mon, Jun 24, 2019 at 10:01 AM Michal Vaško
> > > > > > <[email-masked]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Rob,
> > > > > > > I think there is a problem in the RFC because using only
> > > > > > > allowed steps I got invalid data from initially valid data.
> > > > > > > That cannot be
> > > > > correct.
> > > > > > No.  See sec. 7.5.7
> > > > > >
> > > > > > Just because your retrieval does not contain the NP-container,
> > > > > > that does not mean the NP-container was not present in the
> > > > > > server for the mandatory-stmt validation.
> > > > >
> > > > > > > On Monday, June 24, 2019 18:52 CEST, "Rob Wilton (rwilton)"
> > > > > > > < [email-masked]> wrote:
> > > > > > >
> > > > > > > > Hi Michal,
> > > > > > > >
> > > > > > > > My thoughts:
> > > > > > > >
> > > > > > > > According to 7.5.1:
> > > > > > > >
> > > > > > > >    In the first style, the container has no meaning of its
> > > > > > > > own,
> > > > > existing
> > > > > > > >    container node with no child nodes is semantically
> > > > > > > > equivalent to
> > > > > the
> > > > > > > >    absence of the container node.  YANG calls this style a
> > > > > > > > "non-
> > > > > presence
> > > > > > > > > Parsing was successful, but printing these data back to
> > > > > > > > > XML
> > > > > produced:
> > > > > > > > > <TOP>
> > > > > > > > >   <L/>
> > > > > > > > > </TOP>
> > > > > > > > >
> > > > > > > > > and parsing this correctly failed with missing mandatory
> choice.
> > > > > > > According
> > > > > > > > > to section 7.5.7 [1], I think the C container could be
> > > > > > > > > omitted but the whole situation does not seem correct.
> > > > > > > > > Thank you for any
> > > > > input.
> > > > > > > > > Regards,
> > > > > > > > > Michal
> > > > > > > > >

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 26 Jun 2019 09:01:00 +0200

> I don't think that this makes it a P-container.

Not formally, according to the current rules, but effectively "the container
itself carries some meaning" (sec. 7.5.1).

> I don't think that an empty list "exists" in a configuration datastore, i.e. I

I am not sure about this, it possibly depends on an implementation.

Lada

> don't think that it should impart any meaning, in that regard is seems
> somewhat like an NP-container.

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 26 Jun 2019 09:15:20 +0000

> From: Ladislav Lhotka <[email-masked]>
> Sent: 26 June 2019 08:01
> To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
> > > Subject: Re: [netmod] ?= ?==?utf-8?q? mandatory choice with
> > > non-presence container cas
> > >
> > > > > container "users" [1] without any "user" list instances. Now,
> > > > > you may argue that this is still not a valid use-case because
> > > > > there are no users but I only tried to satisfy the condition.
> > > >
> > > > Yes, I think that this list (user) should have a "min-elements 1".
> > > > I think that matches the i ntent.
> > >
> > > Interestingly, the "users" container actually becomes a P-container:
> > > its presence indicates that the corresponding case is selected.
> >
> Not formally, according to the current rules, but effectively "the
> container itself carries some meaning" (sec. 7.5.1).

In a data tree, I think that the only meaning that an NP-container conveys is the existence of a descendent leaf, leaf-list, p-container, anydata, or anyxml data node.

> > > This example also exposes the drawback of the XML representation -
> > > it cannot distinguish between an empty list and nothing. In JSON,
> > > the problems of this thread could potentially be circumvented by
> > > configuring
> > >
> > > "users" : {
> > >     "user" : [
> > >     ]
> > > }
> >

Perhaps, but I argue that this is probably the only robust interpretation, particularly given that XML cannot represent a list that exists with no entries.

Thanks,
Rob

> > > > > https://tools.ietf.org/html/draft-ietf-netconf-ssh-client-server
> > > > > -14#
> > > > > page-22
> > > > >
> > > > > > "container C" doesn't satisfy the mandatory statement because
> > > > > > it isn't a real data node in the tree - it is instantiated
> > > > > > when required and may be deleted when it is no longer required.
> > > > > >
> > > > > > because the <C/> container doesn't really exist if it doesn't
> > > > > > have a child node present.
> > > > > >
> > > > > > <TOP>
> > > > > >   <L/>
> > > > > >   <C/>
> > > > > > </TOP>
> > > > > >
> > > > > > Thanks,
> > > > > > Rob
> > > > > >
> > > > > >
> > > > > > > >    If a non-presence container does not have any child
> > > > > > > > nodes,
> > > the
> > > > > > > > Just because your retrieval does not contain the
> > > > > > > > NP-container, that does not mean the NP-container was not
> > > > > > > > present in the server for the mandatory-stmt validation.
> > > > > > >
> > > > > > > invalid data. I do not think printing is allowed to change
> > > > > > > the validity of data.
> > > > > > >
> > > > > > > Michal
> > > > > > >
> > > > > > > > Regards,
> > > > > > > > > Michal
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Andy
> > > > > > > >
> > > > > > > >
> > > > > > > > > >    In the first style, the container has no meaning of
> > > > > > > > > > its own,
> > > > > > > existing
> > > > > > > > > > > Parsing was successful, but printing these data back
> > > > > > > > > > > to XML
> > > > > > > produced:
> > > > > > > > > > > <TOP>
> > > > > > > > > > >   <L/>
> > > > > > > > > > > </TOP>
> > > > > > > > > > >
> > > > > > > > > > > to section 7.5.7 [1], I think the C container could
> > > > > > > > > > > be omitted but the whole situation does not seem
> correct.
> > > > > > > > > > > Thank you for any
> > > > > > > input.
> > > > > > > > > > > Regards,
> > > > > > > > > > > Michal
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > > > > > > > > > > https://tools.ietf.org/html/rfc7950#section-7.5.7
> > > > > > > > > > >