Subject: Re: [netmod] yang-instance-file include-defaults leaf

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Mon, 23 Aug 2021 10:27:20 +0000

Hi Balazs, Andy, Netmod,

Sorry for the delayed response.  I would still like to strength the description of the defaults.  E.g., RFC 6243 uses MUSTs rather than SHOULDs.

Hence, I have generated some proposed alternative descriptions, that are somewhat stricter, but also more generically focussed only on the default values.

With these definitions, I think that we could define the “include-defaults” default value to be “explicit”, since if the leaf if not included, then I think that this effectively what the meaning would be anyway.

In particular, I would propose changing the descriptions as follows:

       leaf includes-defaults {
         type enumeration {
           enum report-all {
             value 1;
             description
               "All data nodes SHOULD be included independent of
                 any default values.";
           }
           enum trim {
             value 2;
             description
               "Data nodes that have a default defined and where
                 the actual value is the default value SHOULD
                 NOT be included.";
           }
           enum explicit {
             value 3;
             description
               "Data nodes that have a default defined and where
                 the actual value is the default value SHOULD NOT be
                 included. However, if the actual value was set by
                 a NETCONF client or other management application
                 by the way of an explicit management operation the
                 data node SHOULD be included.";
           }
         }

Proposed:

       leaf includes-defaults {
         type enumeration {
           enum report-all {
             value 1;
             description
               "The instance data set includes all data nodes,
                including those that contain the schema default.”;
           }
           enum trim {
             value 2;
             description
               "The instance data set excludes all data nodes
                that contain the schema default.";
           }
           enum explicit {
             value 3;
             description
               "The instance data set may include some data nodes
                that match the schema default and may exclude some
                data nodes that match the schema default.”;
           }
         }
         description
           "This leaf provides an indication of how default data
            is presented within an instance data set, modelled on
            RFC 6243.

            Interpretation of the use of defaults depends on the
            context of what the instance data set represents.

            E.g., if the instance data set represents configuration,
            Then include-defaults aligns to the meaning of the
            default-handling basic modes in RFC 6243.  If the
            instance data set represents operational data from the
            operational state datastore [RFC 8342], then
            include-defaults aligns to the definition of that
            datastore in RFC 8342.”;

Would text along these lines work?

Thanks,
Rob

From: Balázs Lengyel <[email-masked]>
Sent: 28 July 2021 23:08
To: Rob Wilton (rwilton) <[email-masked]>; Andy Bierman <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hello Rob,
Removing the “default trim;” will address Andy’s comment.

Your in-use-values is very specific to one of the use-cases: reading/documenting operational values. It is not useful for the other use-cases. I think the “documenting operational datastore” use-case could be handled by indicating the includes-defaults=report-all. Case (i) would contain the value case (ii) will not.
Regards Balazs

From: Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>>
Sent: 2021. július 27., kedd 17:38
To: Andy Bierman <[email-masked]<mailto:[email-masked]>>; Balázs Lengyel <[email-masked]<mailto:[email-masked]>>
Cc: NetMod WG <[email-masked]<mailto:[email-masked]>>
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hi Andy, Balazs,

So, the reason that I want a flag to indicate whether default values are in use is because of this definition of operational in RFC 8342:

   Requests to retrieve nodes from <operational> always return the value
   in use if the node exists, regardless of any default value specified
   in the YANG module.  If no value is returned for a given node, then
   this implies that the node is not used by the device.

It was written this way because otherwise a consumer of operational data cannot differentiate between:

(i)                  This value is not present because it matches the default value specified in the YANG module, and

(ii)                This value is not present because the server has failed to return it for some reason (e.g., perhaps the daemon that would have provided this value is down or not available, or perhaps it is a bug, or perhaps it is not implemented and is a missing deviation).

So, I think that in some cases, the absence of a data node does not necessarily mean that the default value is in effect, and I wanted the instance-data document to be able to contain and correctly report this data.

I think that this behaviour could be captured by a single leaf.  Another way of articulating this would be:

leaf in-use-values {
  type boolean;
  default false;
  description
    “Only if set to true, the absence of a value in the
     instance data for a given data node implies that the
    node is not used rather than implicitly taking the
     default value specified by any corresponding
    ‘default’ statement specified in the YANG schema.”;
}

With this, I’m not sure whether we need the “includes-default” leaf currently specified in the draft, but if we do, then I would think that leaf should be entirely optional, i.e., without the default “trim”.

Regards,
Rob

From: Andy Bierman <[email-masked]<mailto:[email-masked]>>
Sent: 10 July 2021 17:41
To: Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>>
Cc: NetMod WG <[email-masked]<mailto:[email-masked]>>; Balázs Lengyel <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] yang-instance-file include-defaults leaf

On Fri, Jul 9, 2021 at 5:23 AM Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>> wrote:
Andy,

Yes, when I suggested this, I was thinking that a boolean flag might be sufficient.  My point being that automatically filtering out default values isn’t always the right thing to do.

The solution is simple.
Get rid of the inappropriate "default trim" statement.

If the leaf is present then it identifies the basic-mode that was used to include defaults.
If not then the information is either not known, not applicable, or defaults were not added.

The "default" statement is a bug because there is no default basic-mode.
All of the basic-modes are in use in deployments and no camp has ever
been able to convince the others that theirs is right.

Andy

E.g., something along these lines:

leaf exclude-defaults {
  type boolean;
  default true;
  description
    “Can be used to reduce the size of the content data file.

      When unset or set to true, data nodes that have a default defined and
      where the actual value is the default value are excluded from the content
      data.

      When set to false, data nodes with default value are not filtered, and
      may appear in the content data.”
}

Would this satisfy your concern?

Regards,
Rob

From: netmod <[email-masked]<mailto:[email-masked]>> On Behalf Of Andy Bierman
Sent: 08 July 2021 18:16
To: NetMod WG <[email-masked]<mailto:[email-masked]>>
Subject: [netmod] yang-instance-file include-defaults leaf

Hi,

The module has this object:

    leaf includes-defaults {

       type enumeration {

         enum report-all {

           value 1;

           description

             "All data nodes SHOULD be included independent of

               any default values.";

         }

         enum trim {

           value 2;

           description

             "Data nodes that have a default defined and where

               the actual value is the default value SHOULD

               NOT be included.";

         }

         enum explicit {

           value 3;

           description

             "Data nodes that have a default defined and where

               the actual value is the default value SHOULD NOT be

               included. However, if the actual value was set by

               a NETCONF client or other management application

               by the way of an explicit management operation the

               data node SHOULD be included.";

         }

       }

       default trim;

The draft is extremely server-centric, like most IETF standards, but this
leaf is too server-centric to ignore.

Consider the possibility that the source of the file is NOT a NETCONF server.
This data may not be known so the default of "trim" may not be correct.

IMO this leaf is noise because any tool that knows the schema will also
know the YANG defaults.  The solution is incomplete anyway because
the presence of a leaf that has a YANG default is not enough.
The  "report-all-tagged" mode must be used to identify defaults.
IMO this leaf should be removed, but at least add an enum called "unknown".

Andy

---

From: Balázs Lengyel <[email-masked]>
Date: Mon, 23 Aug 2021 12:17:24 +0000

Hello Rob,

I think this won’t fly. 

In sections 1.2 and 2 we state:

“Instance data files MAY contain partial data sets.”
Which is important for many use-cases.  This means you cannot say that a default value will or must be included, as they might be omitted because they are not part of the partial data set.
In a way it is difficult to separate between leaves that are missing because
-        They are not part of the partial data-set
-        They are omitted because they have the default value and one of the trim or explicit options is used
If this becomes important the report-all options shall be used.
 
This is the reason I used the SHOULD word.
Regards Balazs

From: Rob Wilton (rwilton) <[email-masked]> 
Sent: 2021. augusztus 23., hétfő 12:27
To: Balázs Lengyel <[email-masked]>; Andy Bierman <[email-masked]>; NetMod WG <[email-masked]>
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hi Balazs, Andy, Netmod,

 

       leaf includes-defaults {

         type enumeration {

           enum report-all {

             value 1;

             description

                 any default values.";

           }

           enum trim {

             value 2;

             description

                 NOT be included.";

           }

           enum explicit {

             value 3;

             description

           }

         }

Proposed:

       leaf includes-defaults {

         type enumeration {

           enum report-all {

             value 1;

             description

           }

           enum trim {

             value 2;

             description

           }

           enum explicit {

             value 3;

             description

           }

         }

         description

            RFC 6243.

            datastore in RFC 8342.”;

Thanks,

Rob

From: Balázs Lengyel <[email-masked] <mailto:[email-masked]> > 
Sent: 28 July 2021 23:08
To: Rob Wilton (rwilton) <[email-masked] <mailto:[email-masked]> >; Andy Bierman <[email-masked] <mailto:[email-masked]> >
Cc: NetMod WG <[email-masked] <mailto:[email-masked]> >
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hello Rob,

Regards Balazs

From: Rob Wilton (rwilton) <[email-masked] <mailto:[email-masked]> > 
Sent: 2021. július 27., kedd 17:38
To: Andy Bierman <[email-masked] <mailto:[email-masked]> >; Balázs Lengyel <[email-masked] <mailto:[email-masked]> >
Cc: NetMod WG <[email-masked] <mailto:[email-masked]> >
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hi Andy, Balazs,

leaf in-use-values {

  type boolean;

  default false;

  description

}

Regards,
Rob

From: Andy Bierman <[email-masked] <mailto:[email-masked]> > 
Sent: 10 July 2021 17:41
To: Rob Wilton (rwilton) <[email-masked] <mailto:[email-masked]> >
Cc: NetMod WG <[email-masked] <mailto:[email-masked]> >; Balázs Lengyel <[email-masked] <mailto:[email-masked]> >
Subject: Re: [netmod] yang-instance-file include-defaults leaf

On Fri, Jul 9, 2021 at 5:23 AM Rob Wilton (rwilton) <[email-masked] <mailto:[email-masked]> > wrote:

Andy,

 

The solution is simple.

 

Andy

leaf exclude-defaults {

  type boolean;

  default true;

  description

      data.

}

Regards,
Rob

From: netmod <[email-masked] <mailto:[email-masked]> > On Behalf Of Andy Bierman
Sent: 08 July 2021 18:16
To: NetMod WG <[email-masked] <mailto:[email-masked]> >
Subject: [netmod] yang-instance-file include-defaults leaf

Hi,

    leaf includes-defaults {
       type enumeration {
         enum report-all {
           value 1;
           description

 

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 23 Aug 2021 09:57:49 -0700

On Mon, Aug 23, 2021 at 5:17 AM Balázs Lengyel <[email-masked]>
wrote:

> *“**Instance data files MAY contain partial data sets.”*
>

I thought we already agreed there cannot be a default or there is no way to
represent "no defaults added".

Note that "report-all" is not useful if basic-mode=explicit, since a leaf
reporting the YANG default
could be set by the client.  Only report-all-tagged will clearly identify
defaults in this case.

Also note that if basic-mode=report-all then there will be no defaults ever
reported.
This mode means the server does not consider any node to be a default and
always returns
every node (if with-defaults used or not).

Andy

>
>
> *From:* Rob Wilton (rwilton) <[email-masked]>
> *Sent:* 2021. augusztus 23., hétfő 12:27
> *To:* Balázs Lengyel <[email-masked]>; Andy Bierman <
> [email-masked]>; NetMod WG <[email-masked]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Balazs, Andy, Netmod,
>
>
>
> Sorry for the delayed response.  I would still like to strength the
> description of the defaults.  E.g., RFC 6243 uses MUSTs rather than SHOULDs.
>
>
>
> Hence, I have generated some proposed alternative descriptions, that are
> somewhat stricter, but also more generically focussed only on the default
> values.
>
>
>
> With these definitions, I think that we could define the
> “include-defaults” default value to be “explicit”, since if the leaf if not
> included, then I think that this effectively what the meaning would be
> anyway.
>
>
>
>
>
> *From:* Balázs Lengyel <[email-masked]>
> *Sent:* 28 July 2021 23:08
> *To:* Rob Wilton (rwilton) <[email-masked]>; Andy Bierman <
> [email-masked]>
> *Cc:* NetMod WG <[email-masked]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hello Rob,
>
> Your *in-use-values* is very specific to one of the use-cases:
> reading/documenting operational values. It is not useful for the other
> use-cases. I think the “documenting operational datastore” use-case could
> be handled by indicating the *includes-defaults=report-all*. Case (i)
> would contain the value case (ii) will not.
>
> Regards Balazs
>
>
>
> *From:* Rob Wilton (rwilton) <[email-masked]>
> *Sent:* 2021. július 27., kedd 17:38
> *To:* Andy Bierman <[email-masked]>; Balázs Lengyel <
> [email-masked]>
> *Cc:* NetMod WG <[email-masked]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Andy, Balazs,
>
>
>
> So, the reason that I want a flag to indicate whether default values are
> in use is because of this definition of operational in RFC 8342:
>
>
>
> (i)               This value is not present because it matches the
> default value specified in the YANG module, and
>
> (ii)              This value is not present because the server has failed
> to return it for some reason (e.g., perhaps the daemon that would have
> provided this value is down or not available, or perhaps it is a bug, or
> perhaps it is not implemented and is a missing deviation).
>
>
>
> So, I think that in some cases, the absence of a data node does not
> necessarily mean that the default value is in effect, and I wanted the
> instance-data document to be able to contain and correctly report this data.
>
>
>
> With this, I’m not sure whether we need the “includes-default” leaf
> currently specified in the draft, but if we do, then I would think that
> leaf should be entirely optional, i.e., without the default “trim”.
>
>
>
> Regards,
> Rob
>
>
>
>
>
> *From:* Andy Bierman <[email-masked]>
> *Sent:* 10 July 2021 17:41
> *To:* Rob Wilton (rwilton) <[email-masked]>
> *Cc:* NetMod WG <[email-masked]>; Balázs Lengyel <
> [email-masked]>
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Fri, Jul 9, 2021 at 5:23 AM Rob Wilton (rwilton) <[email-masked]>
> wrote:
>
> Andy,
>
>
>
> Yes, when I suggested this, I was thinking that a boolean flag might be
> sufficient.  My point being that automatically filtering out default values
> isn’t always the right thing to do.
>
>
>
>
>
>
>
> The solution is simple.
>
> *From:* netmod <[email-masked]> *On Behalf Of *Andy Bierman
> *Sent:* 08 July 2021 18:16
> *To:* NetMod WG <[email-masked]>
> *Subject:* [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi,
>
>
>

---

From: Balázs Lengyel <[email-masked]>
Date: Tue, 24 Aug 2021 08:41:44 +0000

Hello Andy,

In the -17 I removed the default value for includes-defaults as you proposed.

I am not sure I understand the rest of the comments as instance-file-format does not use the concept of “basic-mode”. It has a single leaf to indicate what is the situation with defaults in the specific instance-data-set.

As this is not a live server request/reply situation we do not want to specify a basic and additional modes, we just want to specify the handling used for this specific instance data set.

Regards Balazs

From: Andy Bierman <[email-masked]> 
Sent: 2021. augusztus 23., hétfő 18:58
To: Balázs Lengyel <[email-masked]>
Cc: Rob Wilton (rwilton) <[email-masked]>; NetMod WG <[email-masked]>
Subject: Re: [netmod] yang-instance-file include-defaults leaf

On Mon, Aug 23, 2021 at 5:17 AM Balázs Lengyel <[email-masked] <mailto:[email-masked]> > wrote:

Hello Rob,

I think this won’t fly. 

 

Note that "report-all" is not useful if basic-mode=explicit, since a leaf reporting the YANG default

could be set by the client.  Only report-all-tagged will clearly identify defaults in this case.

Also note that if basic-mode=report-all then there will be no defaults ever reported.

This mode means the server does not consider any node to be a default and always returns

Andy

From: Rob Wilton (rwilton) <[email-masked] <mailto:[email-masked]> > 
Sent: 2021. augusztus 23., hétfő 12:27
To: Balázs Lengyel <[email-masked] <mailto:[email-masked]> >; Andy Bierman <[email-masked] <mailto:[email-masked]> >; NetMod WG <[email-masked] <mailto:[email-masked]> >
Subject: RE: [netmod] yang-instance-file include-defaults leaf

Hi Balazs, Andy, Netmod,

 

       leaf includes-defaults {

         type enumeration {

           enum report-all {

             value 1;

             description

                 any default values.";

           }

           enum trim {

             value 2;

             description

                 NOT be included.";

           }

           enum explicit {

             value 3;

             description

           }

         }

Proposed:

       leaf includes-defaults {

         type enumeration {

           enum report-all {

             value 1;

             description

           }

           enum trim {

             value 2;

             description

           }

           enum explicit {

             value 3;

             description

           }

         }

         description

            RFC 6243.

            datastore in RFC 8342.”;

Thanks,

Rob

Hello Rob,

Regards Balazs

Hi Andy, Balazs,

leaf in-use-values {

  type boolean;

  default false;

  description

}

Regards,
Rob

 

Andy,

 

The solution is simple.

 

Andy

leaf exclude-defaults {

  type boolean;

  default true;

  description

      data.

}

Regards,
Rob

Hi,

    leaf includes-defaults {
       type enumeration {
         enum report-all {
           value 1;
           description

 

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 24 Aug 2021 04:34:46 -0700

On Tue, Aug 24, 2021 at 1:41 AM Balázs Lengyel <[email-masked]>
wrote:

> I am not sure I understand the rest of the comments as
> instance-file-format does not use the concept of “basic-mode”. It has a
> single leaf to indicate what is the situation with defaults in the specific
> instance-data-set.
>
> As this is not a live server request/reply situation we do not want to
> specify a basic and additional modes, we just want to specify the handling
> used for this specific instance data set.
>

The draft as written does not actually provide the same utility as
<with-defaults>.
(Without the "default" attribute the "explicit" mode is not actually
supported.)

The "with-defaults" mechanism works exactly the same no matter what
the XML representation is used for.  The mode used to write the data will
correspond to the basic-mode with the same name.

Andy

> *Sent:* 2021. augusztus 23., hétfő 18:58
> *To:* Balázs Lengyel <[email-masked]>
> *Cc:* Rob Wilton (rwilton) <[email-masked]>; NetMod WG <[email-masked]
> >
> On Mon, Aug 23, 2021 at 5:17 AM Balázs Lengyel <
> [email-masked]> wrote:
>
> Hello Rob,
>
> I think this won’t fly.
>
> *“Instance data files MAY contain partial data sets.”*
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 25 Aug 2021 09:29:16 -0700

Hi,

Here is the latest text. It is inconsistent with RFC 6243, section 3.
IMO the subsections should be cited instead of the copy-and-change approach.

   leaf includes-defaults {
         type enumeration {
           enum report-all {
             value 1;
             description

AB: should follow 6243, 3.1

           }
           enum trim {
             value 2;
             description

AB: should follow 6243, section 3.2

           }
           enum explicit {
             value 3;
             description

AB: should follow 6243, section 3.3

           }
         }
         description
           "As instance-data-sets MAY contain incomplete data sets,
             thus any data node MAY be absent.

             Providing the instance-data-set intends to contain a
             full data set, this leaf specifies whether the data set
             includes data nodes that have a default defined and
             where the actual value is the same as the default value.

             Data nodes that have no default values should always
             be included.
             Data nodes that have a default value, but the actual
             value is not equal to the schema defined default
             should always be included.";

AB: The last paragraph should be removed or changed. Why are these
nodes special?

Nodes that are actually present and do not contain the YANG default

are not relevant to this object.

         reference
           "RFC 6243 <https://datatracker.ietf.org/doc/html/rfc6243>:
With-defaults Capability for NETCONF";
       }

The best way to indicate a representation of a YANG default in the data set
is to include
the "default" attribute in each default node.
https://datatracker.ietf.org/doc/html/rfc6243#section-6
This actually works for explicit mode and leaf-lists (unlike the current
solution).

Andy