Subject: Re: [netmod] [Netconf] LC of NDMA NETCONF/RESTCONF drafts

From: Mahesh Jethanandani <[email-masked]>
Date: Thu, 1 Feb 2018 09:22:07 -0800

This closes the LC for the two NDMA drafts for NETCONF and RESTCONF.

As part of the LC, there were a couple of comments/questions raised. In particular

- Vladmir reported an error, which Martin said is fixed in his local copy.
- Robert suggested that “/yang-library/checksum” should be a reference. Juergen supported that comment, so I am assuming that that will be incorporated into the draft.
- Andy had questions around datastore set to “conventional”, about origin filter limited to 1 source and the behavior of with-defaults. I see some discussion around it with the authors agreeing that some of them need some text clarifying the position. Can the authors please post the suggested text/additions for the WG to review.
- Anything else??

Once an updated draft has been posted, I will do a writeup on the drafts and send it to IESG. 

Thanks.

> On Jan 31, 2018, at 10:16 AM, Juergen Schoenwaelder <[email-masked]> wrote:
> 
> On Wed, Jan 31, 2018 at 04:53:48PM +0000, Eric Voit (evoit) wrote:
>> 
>> I do have one additional thought below on draft-ietf-netmod-revised-datastores section 5.3 default handling process.  See in-line...
>> 
> 
> Well, this document is with the RFC editor now. I do not think it needs
> clarification. It already has text in 5.3 such as:
> 
>   Requests to retrieve nodes from <operational> always return the value
>   in use if the node exists, regardless of any default value specified
>   in the YANG module.  If no value is returned for a given node, then
>   this implies that the node is not used by the device.
> 
> /js
> 
>> From: Robert Wilton -X, January 31, 2018 6:31 AM
>> 
>> 
>> Hi Andy,
>> 
>> On 31/01/2018 09:22, Andy Bierman wrote:
>> 
>> 
>> On Wed, Jan 31, 2018 at 12:11 AM, Juergen Schoenwaelder <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>> On Tue, Jan 30, 2018 at 12:35:33PM -0800, Andy Bierman wrote:
>>> Hi,
>>> 
>>> I have some questions about these drafts.
>>> 
>>> 1) what if datastore set to "conventional"?
>>>    There are many places where a datastore-ref type is used.
>>>    However, "conventional" is valid for base "datastore", even though
>>>    it is ambiguous as a datastore selector.
>> 
>> We can add explicit text that an identity that does not resolve to a
>> datastore implemented by the server results in an invalid value error.
>> 
>> 
>> OK
>> 
>> 
>>> 2) origin filter is limited to 1 source
>>>   This filtering seems rather limited.  A client must retrieve
>>> <with-origin> and check
>>>    all the values in use, then make repeated requests for each source as a
>>> different
>>>    <origin-filter> leaf
>> 
>> If the client does <with-origin>, then it has all origin information
>> and it can filter locally. That said, we could make origin-filter a
>> leaf-list which is logically ORed so that one can retrieve
>> origin-filter=or:system or origin-filter=or:learned in one request.
>> 
>> 
>> OK
>> 
>>> 3) with-defaults broken
>>>    The operational datastore does not support with-defaults.
>>>     Instead, the client must use origin-filter=or:default or with-origin
>>>     and check all the origin attributes.  Since a client needs to use
>>>     with-defaults for other datastores, this special handling of
>>> <operational>
>>>     seems unhelpful.
>> 
>> I think the with-defaults semantics for conventional configuration
>> datastores are much more complicated than necessary for the
>> operational state datastore. Note that that the operational state
>> datastore reports in-use values not really defaults:
>> 
>>  <leaf or:origin='default'>foo</leaf>
>> 
>> This reports that the value 'foo' is in use and that it originates
>> from a default value. Note that this could also be
>> 
>>  <leaf or:origin='intended'>foo</leaf>
>> 
>> in case the intended configuration datastore configured the value
>> 'foo' (despite this value matching the default). The with-defaults
>> solution is pretty complex because it tries to handle how different
>> systems deal with configuration defaults. The idea is to not carry
>> this complexity over to in-use values in the operational state
>> datastore.
>> 
>> 
>> Before NMDA, the client could decide if it wanted to retrieve default nodes or not.
>> This client-choice has been removed from NMDA, which is a problem.
>> We tried to reach a sensible compromise on the data returned from operational (defined in section 5.3 of the NMDA architecture):
>> - it should return explicit values for everything that is affecting the actual running state of the device (regardless of whether the operational value matches a schema default value).
>> - it does not need to, and should not, return operational values for stuff that isn't actually in use, i.e. don't return needless and unwanted data.
>> 
>> In particular, if no value is returned from a particular data node in <operational> then, barring mgmt protocol errors, a client can assume that any functionality associated with that data node is off (i.e. not in use).
>> 
>> Some examples to illustrate the behavior:
>> 
>> (i) If a protocol, e.g. OSPF,  is not enabled/running then <operational> does not need to return any data for it.  It would be reasonable to return a flag to indicate that OSPF is not enabled/running.
>> 
>> (ii) If you have some funky widget on an interface that defaults to being off and isn't being used then <operational> don't need to return any data for it.
>> 
>> (iii) But, if you have some funky widget on an interface that defaults to being on, then the server should return data for it. If it is actually enabled, then it would indicate that it is on and return any associated values with its operational state, or if it is disabled then it should explicitly report that it is off.
>> 
>> (iv) I would regard that all applied configuration is "in use" by the system, even if it matches the default value, and hence it should be reported.
>> 
>> 
>> This behavior for <operational> is obviously slightly different from the existing with-default handling that is supported for configuration datastores.  As I recall, there were a couple of reasons that we decided to have a different behavior for <operational>:
>> (a) to have consistent semantics for all servers, rather than different servers supporting different with-defaults behaviors (which makes life harder for clients because they must cope with all variants).
>> (b) to remove any potential ambiguity if data isn't returned.  I.e. with the existing with-defaults semantics it is not clear to me that servers will always return an explicit value to indicate that a particular widget is off if the schema defines that the default it that is enabled.  If the server doesn't support a given widget at all, it is quite plausible that it will just return no data for it.  In theory features/deviations should handle this, but those don't work so well if different linecards have different capabilities.  Hence being explicit about stuff that is in use seems more robust.
>> 
>> <eric> These are good examples.  It would be great if section 5.3 could be tweaked to make clearer the relationship between running datastore defaults and other operational datastore defaults for the same tree.
>> 
>> For example, let’s say I create a configured subscription, and the default transport protocol is NETCONF.  NETCONF will be used for that subscription even though the node might not be populated.  In this case, the object would not appear in the running datastore, but MUST* appear in the operational datastore with the default origin (as it is in-use).
>> 
>> This to me is the desired behavior as it doesn’t incorrectly add information to the running datastore, but shows what is in-use within operational.   I suspect other such relationships for other operational tree defaults could be asserted, perhaps based on the origin.
>> 
>> (* Maybe ‘MUST eventually’, as obviously there is a temporal relationship between the two datastores.)
>> 
>> Eric
>> 
>> Thanks,
>> Rob
>> 
>> 
>> 
>> 
>> 
>> 
>> /js
>> 
>> Andy
>> 
>> --
>> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
>> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
>> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> 
>> netmod mailing list
>> 
>> [email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>
>> 
>> https://www.ietf.org/mailman/listinfo/netmod <https://www.ietf.org/mailman/listinfo/netmod>
>> 
> 
>> _______________________________________________
>> netmod mailing list
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/netmod <https://www.ietf.org/mailman/listinfo/netmod>
> 
> 
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/ <https://www.jacobs-university.de/>>
> 
> _______________________________________________
> netmod mailing list
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/netmod <https://www.ietf.org/mailman/listinfo/netmod>
Mahesh Jethanandani
[email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Feb 2018 18:43:12 +0100 (CET)

Hi,

> - Vladmir reported an error, which Martin said is fixed in his local copy.

Fixed.

> - Robert suggested that “/yang-library/checksum” should be a
>   reference. Juergen supported that comment, so I am assuming that
>   that will be incorporated into the draft.

Yes, fixed.

> - Andy had questions around datastore set to “conventional”

Fixed.

>   , about origin filter limited to 1 source

Fixed.

>   and the behavior of with-defaults.

There were no additional changes to the document from the discussion
about this.

>   I see some discussion around it with the authors
>   agreeing that some of them need some text clarifying the
>   position. Can the authors please post the suggested text/additions
>   for the WG to review. 
> - Anything else??
> 

The issues above are now addressed, in
draft-ietf-netconf-nmda-netconf-03.

There were no additional comments on
draft-ietf-netconf-nmda-restconf-02, so I believe this version is
ready.

/martin

---

From: Mahesh Jethanandani <[email-masked]>
Date: Mon, 5 Feb 2018 13:33:13 -0800

For folks that provided comments as part of LC, please verify that your comments have been adequately addressed by -03 version of the draft. 

Thanks 

Mahesh Jethanandani 
[email-masked]

> On Feb 5, 2018, at 9:43 AM, Martin Bjorklund <[email-masked]> wrote:
> 
> Hi,
>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 6 Feb 2018 16:18:34 +0000

Yes, my comment on the YANG library checksum reference has been 
adequately addressed by the -03 version of the draft.

Thanks,
Rob

On 05/02/2018 21:33, Mahesh Jethanandani wrote:
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Feb 2018 18:33:37 -0800

On Mon, Feb 5, 2018 at 1:33 PM, Mahesh Jethanandani <[email-masked]
> wrote:

> For folks that provided comments as part of LC, please verify that your
> comments have been adequately addressed by -03 version of the draft.
>
>

Most comments have been addressed.

    The "with-defaults" parameter does not apply when interacting with an

        operational datastore.

There is no explanation of why the with-defaults parameter does not apply
to <operational>.
This is confusing. The solution that has been a standard for years still
applies to
all datastores, except a completely different mechanism (origin-filter) is
used instead
for 1 datastore.

If the server code can identify a default so it can be tagged
origin=or:default, then it can
also support with-defaults.

I prefer the sentence above be changed, so that a server MAY implement
with-defaults
for <operational>.  If the client sends <with-defaults> it should be OK to
honor it instead
of returning an error.

Andy

> >>> On Jan 31, 2018, at 10:16 AM, Juergen Schoenwaelder <
> [email-masked]> wrote:
> >>>
> >>>> I do have one additional thought below on draft-ietf-netmod-revised-datastores
> section 5.3 default handling process.  See in-line...
> >>>>
> >>>
> >>>> On Wed, Jan 31, 2018 at 12:11 AM, Juergen Schoenwaelder <
> [email-masked] <mailto:j.schoenwaelder@
> jacobs-university.de><mailto:[email-masked] <mailto:
> [email-masked]>>> wrote:
> >>>> We tried to reach a sensible compromise on the data returned from
> operational (defined in section 5.3 of the NMDA architecture):
> >>>> - it should return explicit values for everything that is affecting
> the actual running state of the device (regardless of whether the
> operational value matches a schema default value).
> >>>> - it does not need to, and should not, return operational values for
> stuff that isn't actually in use, i.e. don't return needless and unwanted
> data.
> >>>>
> >>>> In particular, if no value is returned from a particular data node in
> <operational> then, barring mgmt protocol errors, a client can assume that
> any functionality associated with that data node is off (i.e. not in use).
> >>>>
> >>>> (i) If a protocol, e.g. OSPF,  is not enabled/running then
> <operational> does not need to return any data for it.  It would be
> reasonable to return a flag to indicate that OSPF is not enabled/running.
> >>>>
> >>>> (ii) If you have some funky widget on an interface that defaults to
> being off and isn't being used then <operational> don't need to return any
> data for it.
> >>>>
> >>>> (iii) But, if you have some funky widget on an interface that
> defaults to being on, then the server should return data for it. If it is
> actually enabled, then it would indicate that it is on and return any
> associated values with its operational state, or if it is disabled then it
> should explicitly report that it is off.
> >>>>
> >>>> (iv) I would regard that all applied configuration is "in use" by the
> system, even if it matches the default value, and hence it should be
> reported.
> >>>>
> >>>>
> >>>> This behavior for <operational> is obviously slightly different from
> the existing with-default handling that is supported for configuration
> datastores.  As I recall, there were a couple of reasons that we decided to
> have a different behavior for <operational>:
> >>>> (a) to have consistent semantics for all servers, rather than
> different servers supporting different with-defaults behaviors (which makes
> life harder for clients because they must cope with all variants).
> >>>> (b) to remove any potential ambiguity if data isn't returned.  I.e.
> with the existing with-defaults semantics it is not clear to me that
> servers will always return an explicit value to indicate that a particular
> widget is off if the schema defines that the default it that is enabled.
> If the server doesn't support a given widget at all, it is quite plausible
> that it will just return no data for it.  In theory features/deviations
> should handle this, but those don't work so well if different linecards
> have different capabilities.  Hence being explicit about stuff that is in
> use seems more robust.
> >>>>
> >>>> <eric> These are good examples.  It would be great if section 5.3
> could be tweaked to make clearer the relationship between running datastore
> defaults and other operational datastore defaults for the same tree.
> >>>>
> >>>> For example, let’s say I create a configured subscription, and the
> default transport protocol is NETCONF.  NETCONF will be used for that
> subscription even though the node might not be populated.  In this case,
> the object would not appear in the running datastore, but MUST* appear in
> the operational datastore with the default origin (as it is in-use).
> >>>>
> >>>> This to me is the desired behavior as it doesn’t incorrectly add
> information to the running datastore, but shows what is in-use within
> operational.   I suspect other such relationships for other operational
> tree defaults could be asserted, perhaps based on the origin.
> >>>>
> >>>> (* Maybe ‘MUST eventually’, as obviously there is a temporal
> relationship between the two datastores.)
> >>>>
> >>>> Eric
> >>>>
> >>>> Thanks,
> >>>> Rob
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> /js
> >>>>
> >>>> Andy
> >>>>
> >>>> --
> >>>> https://www.ietf.org/mailman/listinfo/netmod <
> https://www.ietf.org/mailman/listinfo/netmod>
> >>>>
> >>>
> >>>> https://www.ietf.org/mailman/listinfo/netmod <
> https://www.ietf.org/mailman/listinfo/netmod>
> >>>
> >>>
> >>> --
> >>> https://www.ietf.org/mailman/listinfo/netmod <
> https://www.ietf.org/mailman/listinfo/netmod>
> >> Mahesh Jethanandani
> >> [email-masked]
> >>
>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 7 Feb 2018 11:14:58 +0000

Hi Andy,

On 07/02/2018 02:33, Andy Bierman wrote:
>
>
> On Mon, Feb 5, 2018 at 1:33 PM, Mahesh Jethanandani 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     For folks that provided comments as part of LC, please verify that
>     your comments have been adequately addressed by -03 version of the
>     draft.
>
>
>
> There is no explanation of why the with-defaults parameter does not 
> apply to <operational>.
> This is confusing. The solution that has been a standard for years 
> still applies to
> all datastores, except a completely different mechanism 
> (origin-filter) is used instead
> for 1 datastore.
>
> for <operational>.  If the client sends <with-defaults> it should be 
> OK to honor it instead
> of returning an error.
I have two concerns with changing this to a MAY:

(1) The most useful "with-defaults" behavior differs for <operational> 
vs the configuration datastores, but with-defaults only allows a single 
standard behavior to be specified.

E.g. for configuration datastores the most appropriate semantics (if the 
client doesn't explicitly ask for something else) is "explicit". i.e. 
you give back exactly what was put in.

But, for operational, the most appropriate semantics (if the client 
doesn't explicitly ask for something else) is something like 
"report-all", i.e. the device reports the precise current state 
including any defaults.  However, we felt that this would return too 
much unnecessary data, hence why the datastore architecture defines 
"in-use" data, allowing the server to prune out any data that is clearly 
irrelevant.

(2) <operational> is a new datastore.  I personally don't want each 
server choosing how the data is returned, which requires that clients 
must handle all variants.  It would be better for the draft to specify 
the standard semantics to use unless a client has explicitly requested 
something else.

I'm not opposed to a "with-defaults-bis", or a new draft covering 
"with-defaults" for <operational>", but I think that:
(i) We shouldn't delay the NMDA protocol drafts for this, this can be 
done as separate draft adding extra optional functionality.
(ii) The semantics for retrieving data from operational (or 
notifications) should be as defined by "in-use" in the NMDA 
architecture, unless a client has explicitly specified, or configured, a 
different behavior.
(iii) Probably the only existing option defined in "with-defaults" that 
makes sense for <operational> is a variant of "trim" that is specified 
to return what is defined as returning the "in-use" values, but also 
excluding any values that match a default value specified in the schema.

Thanks,
Rob

>
>
> Andy
>
>
>     Thanks
>
>     Mahesh Jethanandani
>     [email-masked] <mailto:[email-masked]>
>
>     > On Feb 5, 2018, at 9:43 AM, Martin Bjorklund <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > Hi,
>     >
>     > Mahesh Jethanandani <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >>> On Jan 31, 2018, at 10:16 AM, Juergen Schoenwaelder
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >>>
>     >>>> I do have one additional thought below on
>     draft-ietf-netmod-revised-datastores section 5.3 default handling
>     process.  See in-line...
>     >>>>
>     >>>
>     >>>> On Wed, Jan 31, 2018 at 12:11 AM, Juergen Schoenwaelder
>     <[email-masked]
>     <mailto:[email-masked]>
>     <mailto:[email-masked]
>     <mailto:[email-masked]>><mailto:[email-masked]
>     <mailto:[email-masked]>
>     <mailto:[email-masked]
>     <mailto:[email-masked]>>>> wrote:
>     >>>> We tried to reach a sensible compromise on the data returned
>     from operational (defined in section 5.3 of the NMDA architecture):
>     >>>> - it should return explicit values for everything that is
>     affecting the actual running state of the device (regardless of
>     whether the operational value matches a schema default value).
>     >>>> - it does not need to, and should not, return operational
>     values for stuff that isn't actually in use, i.e. don't return
>     needless and unwanted data.
>     >>>>
>     >>>> In particular, if no value is returned from a particular data
>     node in <operational> then, barring mgmt protocol errors, a client
>     can assume that any functionality associated with that data node
>     is off (i.e. not in use).
>     >>>>
>     <operational> does not need to return any data for it.  It would
>     be reasonable to return a flag to indicate that OSPF is not
>     enabled/running.
>     >>>>
>     >>>> (ii) If you have some funky widget on an interface that
>     defaults to being off and isn't being used then <operational>
>     don't need to return any data for it.
>     >>>>
>     defaults to being on, then the server should return data for it.
>     If it is actually enabled, then it would indicate that it is on
>     and return any associated values with its operational state, or if
>     it is disabled then it should explicitly report that it is off.
>     >>>>
>     >>>> (iv) I would regard that all applied configuration is "in
>     use" by the system, even if it matches the default value, and
>     hence it should be reported.
>     >>>>
>     >>>>
>     >>>> This behavior for <operational> is obviously slightly
>     different from the existing with-default handling that is
>     supported for configuration datastores.  As I recall, there were a
>     couple of reasons that we decided to have a different behavior for
>     <operational>:
>     different servers supporting different with-defaults behaviors
>     (which makes life harder for clients because they must cope with
>     all variants).
>     >>>> (b) to remove any potential ambiguity if data isn't
>     returned.  I.e. with the existing with-defaults semantics it is
>     not clear to me that servers will always return an explicit value
>     to indicate that a particular widget is off if the schema defines
>     that the default it that is enabled.  If the server doesn't
>     support a given widget at all, it is quite plausible that it will
>     just return no data for it.  In theory features/deviations should
>     handle this, but those don't work so well if different linecards
>     have different capabilities.  Hence being explicit about stuff
>     that is in use seems more robust.
>     >>>>
>     >>>> <eric> These are good examples.  It would be great if section
>     5.3 could be tweaked to make clearer the relationship between
>     running datastore defaults and other operational datastore
>     defaults for the same tree.
>     >>>>
>     >>>> For example, let’s say I create a configured subscription,
>     and the default transport protocol is NETCONF.  NETCONF will be
>     used for that subscription even though the node might not be
>     populated. In this case, the object would not appear in the
>     running datastore, but MUST* appear in the operational datastore
>     with the default origin (as it is in-use).
>     >>>>
>     >>>> This to me is the desired behavior as it doesn’t incorrectly
>     add information to the running datastore, but shows what is in-use
>     within operational.  I suspect other such relationships for other
>     operational tree defaults could be asserted, perhaps based on the
>     origin.
>     >>>>
>     >>>> Fax:   +49 421 200 3103       
>      <https://www.jacobs-university.de/
>     <https://www.jacobs-university.de/>>
>     >>>>
>     >>>>
>     >>>>
>     >>>>
>     >>>>
>     <mailto:[email-masked]
>     <mailto:[email-masked]>><mailto:[email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>>
>     >>>>
>     >>>> https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     <https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>>
>     >>>>
>     >>>
>     <mailto:[email-masked] <mailto:[email-masked]>>
>     >>>> https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     <https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>>
>     >>>
>     >>>
>     >>> --
>     >>> Fax:   +49 421 200 3103       
>      <https://www.jacobs-university.de/
>     <https://www.jacobs-university.de/>
>     <https://www.jacobs-university.de/
>     <https://www.jacobs-university.de/>>>
>     >>>
>     <mailto:[email-masked] <mailto:[email-masked]>>
>     >>> https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     <https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>>
>     >> Mahesh Jethanandani
>     >> [email-masked] <mailto:[email-masked]>
>     >>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netconf>
>
>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 7 Feb 2018 06:23:22 -0800

On Wed, Feb 7, 2018 at 3:14 AM, Robert Wilton <[email-masked]> wrote:

> On Mon, Feb 5, 2018 at 1:33 PM, Mahesh Jethanandani <
> [email-masked]> wrote:
>
> (1) The most useful "with-defaults" behavior differs for <operational> vs
> the configuration datastores, but with-defaults only allows a single
> client doesn't explicitly ask for something else) is "explicit". i.e. you
> give back exactly what was put in.
>
> doesn't explicitly ask for something else) is something like "report-all",
> i.e. the device reports the precise current state including any defaults.
> However, we felt that this would return too much unnecessary data, hence
> why the datastore architecture defines "in-use" data, allowing the server
> to prune out any data that is clearly irrelevant.
>
> (2) <operational> is a new datastore.  I personally don't want each server
> choosing how the data is returned, which requires that clients must handle
> all variants.  It would be better for the draft to specify the standard
> semantics to use unless a client has explicitly requested something else.
>
> (i) We shouldn't delay the NMDA protocol drafts for this, this can be done
> as separate draft adding extra optional functionality.
> (ii) The semantics for retrieving data from operational (or notifications)
> should be as defined by "in-use" in the NMDA architecture, unless a client
> has explicitly specified, or configured, a different behavior.
> makes sense for <operational> is a variant of "trim" that is specified to
> return what is defined as returning the "in-use" values, but also excluding
> any values that match a default value specified in the schema.
>
>

I think your approach violates the Postel Principle.
"Be liberal in what you accept" is about robustness.
Rejecting parameters for no good reason is about fragility.

I never said change the behavior for <operational> if no <with-defaults> is
present.
If the parameter is provided it is trivial for the server to honor it.
The most useful value (report-all) is the default, not leave out all
defaults, like <get-config>.

Andy

>> [email-masked] <mailto:j.schoenwaelder@jacobs
>> -university.de><mailto:[email-masked] <mailto:
>> >>>> In particular, if no value is returned from a particular data node
>> in <operational> then, barring mgmt protocol errors, a client can assume
>> that any functionality associated with that data node is off (i.e. not in
>> use).
>> >>>>
>> >>>> (iv) I would regard that all applied configuration is "in use" by
>> the system, even if it matches the default value, and hence it should be
>> reported.
>> >>>>
>> >>>>
> Netconf mailing [email-masked]://www.ietf.org/mailman/listinfo/netconf
>
>
>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 7 Feb 2018 16:11:03 +0000

On 07/02/2018 14:23, Andy Bierman wrote:
>
>
> On Wed, Feb 7, 2018 at 3:14 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi Andy,
>
>
>>         For folks that provided comments as part of LC, please verify
>>         that your comments have been adequately addressed by -03
>>         version of the draft.
>>
>>
>>
>>     There is no explanation of why the with-defaults parameter does
>>     not apply to <operational>.
>>     This is confusing. The solution that has been a standard for
>>     years still applies to
>>     I prefer the sentence above be changed, so that a server MAY
>>     implement with-defaults
>>     for <operational>.  If the client sends <with-defaults> it should
>>     be OK to honor it instead
>>     of returning an error.
>     (1) The most useful "with-defaults" behavior differs for
>     <operational> vs the configuration datastores, but with-defaults
>     only allows a single standard behavior to be specified.
>
>     E.g. for configuration datastores the most appropriate semantics
>     (if the client doesn't explicitly ask for something else) is
>     "explicit". i.e. you give back exactly what was put in.
>
>     But, for operational, the most appropriate semantics (if the
>     client doesn't explicitly ask for something else) is something
>     like "report-all", i.e. the device reports the precise current
>     state including any defaults.  However, we felt that this would
>     return too much unnecessary data, hence why the datastore
>     architecture defines "in-use" data, allowing the server to prune
>     out any data that is clearly irrelevant.
>
>     (2) <operational> is a new datastore.  I personally don't want
>     each server choosing how the data is returned, which requires that
>     clients must handle all variants.  It would be better for the
>     draft to specify the standard semantics to use unless a client has
>     explicitly requested something else.
>
>     (i) We shouldn't delay the NMDA protocol drafts for this, this can
>     be done as separate draft adding extra optional functionality.
>     architecture, unless a client has explicitly specified, or
>     configured, a different behavior.
>     (iii) Probably the only existing option defined in "with-defaults"
>     that makes sense for <operational> is a variant of "trim" that is
>     specified to return what is defined as returning the "in-use"
>     values, but also excluding any values that match a default value
>     specified in the schema.
>
>
>
> I never said change the behavior for <operational> if no 
> <with-defaults> is present.
OK, so one option is to allow the "with-defaults" parameter for the 
<operational> datastore, but specify in both the NETCONF NMDA and 
RESTCONF NMDA drafts how "with-defaults" semantics apply to any 
operational datastore:

1) Regardless of what is reported in the "basic-mode" capability 
parameter, the default behavior for <operational> is "report-all", with 
semantics as described below:
2) "report-all" for operational datastores is defined as returning all 
"in use" values (i.e. as specified in section 5.3 of the NMDA 
architecture, so default values that are not "in use" are not returned).
3) "trim" for operational datastores is defined as returning all 
"in-use" values (... as 5.3) except that values that match the value 
specified in the schema "default" statement are omitted.  Note, clients 
cannot distinguish between a value that is omitted because it is not in 
use, vs a value that is omitted because it matches the schema default.
4) "explicit" for operational datastores is defined as returning the 
same as "report-all", defined above.
5) "report-all-tagged" for operational datastores is defined as 
returning the same as "report-all", except that all values that match 
the values specified in the schema "default" statement are tagged, as 
described in with-defaults (RFC 5243).

Also note - there is no change in semantics or behavior to how 
"with-defaults" works for conventional datastores.

Thoughts?

Thanks,
Rob

>>         > On Feb 5, 2018, at 9:43 AM, Martin Bjorklund
>>         <[email-masked] <mailto:[email-masked]>> wrote:
>>         >
>>         > Hi,
>>         >
>>         >> Once an updated draft has been posted, I will do a writeup
>>         on the
>>         >> drafts and send it to IESG.
>>         >
>>         draft-ietf-netmod-revised-datastores section 5.3 default
>>         handling process.  See in-line...
>>         >>>>
>>         >>>
>>         >>>> We tried to reach a sensible compromise on the data
>>         returned from operational (defined in section 5.3 of the NMDA
>>         architecture):
>>         >>>> - it should return explicit values for everything that
>>         is affecting the actual running state of the device
>>         (regardless of whether the operational value matches a schema
>>         default value).
>>         >>>> - it does not need to, and should not, return
>>         operational values for stuff that isn't actually in use, i.e.
>>         don't return needless and unwanted data.
>>         >>>>
>>         >>>> In particular, if no value is returned from a particular
>>         data node in <operational> then, barring mgmt protocol
>>         errors, a client can assume that any functionality associated
>>         with that data node is off (i.e. not in use).
>>         >>>>
>>         >>>> (i) If a protocol, e.g. OSPF,  is not enabled/running
>>         then <operational> does not need to return any data for it. 
>>         It would be reasonable to return a flag to indicate that OSPF
>>         is not enabled/running.
>>         >>>>
>>         >>>> (iii) But, if you have some funky widget on an interface
>>         that defaults to being on, then the server should return data
>>         for it. If it is actually enabled, then it would indicate
>>         that it is on and return any associated values with its
>>         operational state, or if it is disabled then it should
>>         explicitly report that it is off.
>>         >>>>
>>         >>>> (iv) I would regard that all applied configuration is
>>         "in use" by the system, even if it matches the default value,
>>         and hence it should be reported.
>>         >>>>
>>         >>>>
>>         supported for configuration datastores.  As I recall, there
>>         were a couple of reasons that we decided to have a different
>>         behavior for <operational>:
>>         >>>> (a) to have consistent semantics for all servers, rather
>>         than different servers supporting different with-defaults
>>         behaviors (which makes life harder for clients because they
>>         must cope with all variants).
>>         returned.  I.e. with the existing with-defaults semantics it
>>         is not clear to me that servers will always return an
>>         explicit value to indicate that a particular widget is off if
>>         the schema defines that the default it that is enabled.  If
>>         the server doesn't support a given widget at all, it is quite
>>         plausible that it will just return no data for it.  In theory
>>         features/deviations should handle this, but those don't work
>>         so well if different linecards have different capabilities. 
>>         Hence being explicit about stuff that is in use seems more
>>         robust.
>>         >>>>
>>         >>>> <eric> These are good examples.  It would be great if
>>         section 5.3 could be tweaked to make clearer the relationship
>>         between running datastore defaults and other operational
>>         datastore defaults for the same tree.
>>         >>>>
>>         >>>> For example, let’s say I create a configured
>>         subscription, and the default transport protocol is NETCONF.
>>         NETCONF will be used for that subscription even though the
>>         node might not be populated. In this case, the object would
>>         not appear in the running datastore, but MUST* appear in the
>>         operational datastore with the default origin (as it is in-use).
>>         >>>>
>>         >>>> This to me is the desired behavior as it doesn’t
>>         incorrectly add information to the running datastore, but
>>         shows what is in-use within operational.   I suspect other
>>         such relationships for other operational tree defaults could
>>         be asserted, perhaps based on the origin.
>>         >>>>
>>         >>>> (* Maybe ‘MUST eventually’, as obviously there is a
>>         temporal relationship between the two datastores.)
>>         >>>>
>>         >>>> Eric
>>         >>>>
>>         >>>> Thanks,
>>         >>>> Rob
>>         >>>>
>>         >>>>
>>         >>>>
>>         >>>>
>>         >>>>
>>         >>>>
>>         >>>> /js
>>         >>>>
>>         >>>> Andy
>>         >>>>
>>         >>>> --

---

From: Andy Bierman <[email-masked]>
Date: Wed, 7 Feb 2018 10:14:29 -0800

On Wed, Feb 7, 2018 at 8:11 AM, Robert Wilton <[email-masked]> wrote:

>> all datastores, except a completely different mechanism (origin-filter)
>> is used instead
>> for 1 datastore.
>>
>> for <operational>.  If the client sends <with-defaults> it should be OK
>> to honor it instead
>> of returning an error.
>>
>> server choosing how the data is returned, which requires that clients must
>> handle all variants.  It would be better for the draft to specify the
>> standard semantics to use unless a client has explicitly requested
>> something else.
>>
>> notifications) should be as defined by "in-use" in the NMDA architecture,
>> unless a client has explicitly specified, or configured, a different
>> behavior.
> I never said change the behavior for <operational> if no <with-defaults>
> is present.
> <operational> datastore, but specify in both the NETCONF NMDA and RESTCONF
> NMDA drafts how "with-defaults" semantics apply to any operational
> datastore:
>
> 2) "report-all" for operational datastores is defined as returning all "in
> use" values (i.e. as specified in section 5.3 of the NMDA architecture, so
> default values that are not "in use" are not returned).
> 3) "trim" for operational datastores is defined as returning all "in-use"
> values (... as 5.3) except that values that match the value specified in
> the schema "default" statement are omitted.  Note, clients cannot
> distinguish between a value that is omitted because it is not in use, vs a
> value that is omitted because it matches the schema default.
> 4) "explicit" for operational datastores is defined as returning the same
> as "report-all", defined above.
> 5) "report-all-tagged" for operational datastores is defined as returning
> the same as "report-all", except that all values that match the values
> specified in the schema "default" statement are tagged, as described in
> with-defaults (RFC 5243).
>
This looks good.

Showing the work...

1) there is no YANG default for <with-defaults> parameter.
    The behavior if this parameter is missing is determined by the operation

2) The <get-data> operation returns all values in use.
    The only way to suppress defaults is to use <origin-filter>
    (e.g., request all origins except 'default')

3) The <with-defaults> parameter for <operational> is mostly a NO-OP.
    It does not add or remove any nodes that would be returned.
    The "report-all-tagged" value does add the "default> attribute, which
is useful
    for clients that process these attributes already

4) The values that are tagged default are the same ones that the server tags
     as origin=default.

5) It still seems to up to the server "basic-mode" as to what is considered
     "set by default". This messy aspect of NETCONF is minimized in
<get-data> because
     the server usually returns all values in use.

Thanks,

Andy

>>> >>>> - it does not need to, and should not, return operational values
>>> for stuff that isn't actually in use, i.e. don't return needless and
>>> unwanted data.
>>> >>>>
>>> >>>> This behavior for <operational> is obviously slightly different
>>> from the existing with-default handling that is supported for configuration
>>> <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g>
>>> <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 07 Feb 2018 19:28:03 +0100 (CET)

>     The behavior if this parameter is missing is determined by the operation

Right.  So in this case (get-data of operational), it would return all
default values in use.

>     (e.g., request all origins except 'default')

Or use with-defaults = trim.

/martin

---

From: Mahesh Jethanandani <[email-masked]>
Date: Wed, 7 Feb 2018 13:58:22 -0800

It appears that there is some consensus around the issue of “with-defaults” for operational datastore. Andy, would you agree with what has been suggested?

It also appears that text needs to be added to explain the behavior w.r.t. operational datastore. If everyone agrees, authors, can you propose the text that needs to be updated.

Thanks.

> On Feb 7, 2018, at 10:28 AM, Martin Bjorklund <[email-masked]> wrote:
> 
> Andy Bierman <[email-masked] <mailto:[email-masked]>> wrote:
>>>>> <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g>>
>>>>> <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g <https://maps.google.com/?q=Campus+Ring+1+%7C+28759+Bremen+%7C+Germany&entry=gmail&source=g>>
> https://www.ietf.org/mailman/listinfo/netconf <https://www.ietf.org/mailman/listinfo/netconf>
Mahesh Jethanandani
[email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Wed, 7 Feb 2018 14:04:13 -0800

On Wed, Feb 7, 2018 at 1:58 PM, Mahesh Jethanandani <[email-masked]

> It appears that there is some consensus around the issue of
> “with-defaults” for operational datastore. Andy, would you agree with what
> has been suggested?
>
>
Yes.

> It also appears that text needs to be added to explain the behavior w.r.t.
> operational datastore. If everyone agrees, authors, can you propose the
> text that needs to be updated.
>
> Thanks.
>
>
Andy

> - Vladmir reported an error, which Martin said is fixed in his local
>
> copy.
>
>
> Fixed.
>
> Well, this document is with the RFC editor now. I do not think it
>
> needs
>
> Requests to retrieve nodes from <operational> always return the
>
> value
>
>  However, "conventional" is valid for base "datastore", even
>
> though
>
> datastore implemented by the server results in an invalid value
>
> error.
>
>
>
> OK
>
>
>  all the values in use, then make repeated requests for each
>
> source as a
>
> different
>  <origin-filter> leaf
>
>
>   Instead, the client must use origin-filter=or:default or
>
> with-origin
>
>   and check all the origin attributes.  Since a client needs to
>
> use
>
> Before NMDA, the client could decide if it wanted to retrieve
>
> default nodes or not.
>
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen |
> Germany
> [email-masked] <mailto:[email-masked]><mailto:[email-masked]
>
> <mailto:[email-masked]>>
>
>
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen |
> Germany
> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/ <
>
> https://www.jacobs-university.de/>>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 7 Feb 2018 15:03:49 -0800

On Wed, Feb 7, 2018 at 10:28 AM, Martin Bjorklund <[email-masked]> wrote:

> your
> apply
> still
> <operational> vs
> "report-all",
> must
> architecture,
> that
> RESTCONF
> "in
> "in-use"
> returning

OK

> Or use with-defaults = trim.

Yes -- because the definition in RFC 6243 is worded to exclude nodes.

It should be clear in some draft how basic-mode applies to origin=default
within <operational>.

Applying sec 2 of 6243...

config=true:

If basic-mode=report-all then origin=default will never be present

If basic-mode=trim then origin=default is only possible if the value-in-use
is the YANG default

If basic-mode=explicit then origin=default is only possible if the
configured value was not
explicitly set by a client.  Sec 2.3.1 is not clear if the YANG default
value is relevant or not.
It could be that if the configured value not explicitly set, then any
value-in-use (not just the
YANG default) could be tagged origin=default.

config=false:

report-all: default ignored, no nodes treated as default
trim: node removed if value=YANG default
explicit: all config=false nodes are set by the server, so no nodes treated
as default

This draft makes with-defaults mandatory-to-implement.
It is a SHOULD implement now.  (I approve!).

The with-defaults capability MUST be advertised by the NMDA server,
including
the basic-mode parameter. The also-supported parameter MAY be included.

Is it possible for report-all-tagged to apply to nodes that are learned
(i.e., not origin=default)?

Andy

> from
> affecting
> node
> defaults to
> (which makes
> 5.3

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 8 Feb 2018 08:36:17 +0100

> It should be clear in some draft how basic-mode applies to origin=default

Frankly, carrying the different basic modes over to <operational>
sounds like a mistake. Complexity for no real value.
 
> explicit: all config=false nodes are set by the server, so no nodes treated

Who needs all this to manage a network?
 
> (i.e., not origin=default)?

So here is an alternate proposal: The NMDA documents are silent about
with-defaults and if someone wants to use with-defaults with
datastores then an update of RFC 6243 needs to be written. This way,
implementations can choose to not do any of the with-defaults magic.

What we may consider, though, is to have a way to negate origin-filter
so that we can exclude specific origins - right now to emulate this
one has to (a) know all possible origins and then (b) list all origins
except the one not wanted.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 Feb 2018 09:11:58 -0800

> sounds like a mistake. Complexity for no real value.

API consistency is never a mistake.
Special case treatment of 1 datastore sounds like a mistake.

> except the one not wanted.
Then remove the text that says an error is sent if with-defaults attempted
on <operational>.
None of this new text needs to go into NMDA. It can be a vendor-specific
mystery what gets set as origin=default.  Implementors can read RFC 6243
and figure it out on their own.

Sometimes default leafs might take up 50% or more of the retrieval response.
Insisting that all clients always want this extra data seems overly
simplistic.
Assuming all networks are fast enough to ignore a 2X increase in data size
for no benefit
is not a good idea either.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 Feb 2018 09:55:47 -0800

> ....

I sometimes get comments from people about NETCONF defaults, like
"You think this is a standard? Why does a vendor get to decide what
is a default leaf?"

CoMI has taken a different approach.
Every server MUST implement "trim" mode and nothing else.
NETCONF should do the same (but won't)

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 8 Feb 2018 13:01:35 -0500

>sounds like a mistake. Complexity for no real value.

+1

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 08 Feb 2018 19:40:31 +0100 (CET)

> If basic-mode=report-all then origin=default will never be present

Note that origin=default can be used for more than just static YANG
"default" statement defaults.  But then RFC 6243 talks about "schema
defaults", and presumably this includes dynamic defaults, even if they
are defined in prose in a description statement.

So I think your statement above is correct, since the default value is
present even in <running> and <intended>.

> If basic-mode=trim then origin=default is only possible if the value-in-use

Yes, for the same reason as above.

> explicitly set by a client.

Yes.

> Sec 2.3.1 is not clear if the YANG default

Yes I think so, as per above.

> config=false:

Note that the origin annotation is not present on config false nodes.

Yes.

Yes.

Yes.

> It is a SHOULD implement now.  (I approve!).

But is this what the WG wants?  (side note; it would be good with a
"if-module-implemented" statement, similar to if-feature.  as it is
today, if we want with-defaults to be optional we'd have to define an
additional feature, which really is unnecessary)

> (i.e., not origin=default)?

No, I think that if report-all-tagged is requested, you'd get both the
"default" attribute and origin=default together.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 08 Feb 2018 19:44:50 +0100 (CET)

Hi,

> except the one not wanted.

  leaf-list include-origin-filter { ... }
  leaf-list exclude-origin-filter { ... }

Or we have a single leaf origin-filter which is a boolean expression
with origin values.  Just like if-feature.

  <origin-filter>(not default) and (not learned)</origin-filter>

but maybe this is overkill...

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 8 Feb 2018 19:44:53 +0100

> and figure it out on their own.

As I said, I am absolutely fine with the option of being silent about
with-defaults and if needed someone can spin an update of RFC 6243.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 Feb 2018 10:55:53 -0800

> with-defaults and if needed someone can spin an update of RFC 6243.

This is not needed.

   If the "with-defaults" parameter
   is present in a request to such a datastore, then the server MUST

      return an error, as specified in "ietf-netconf-nmda" (see Section 4
<https://tools.ietf.org/html/draft-ietf-netconf-nmda-netconf-03#section-4>).

         The 'with-defaults' parameter does not apply to operational
         datastores. If the 'with-defaults' parameter is present in a
         request to an operational datastore, then the server MUST
         return an <rpc-error> element with an <error-tag> value of
         'invalid-value'.";

There are 2 places in the draft that say MUST send an error.
You need to add detailed text explaining what "harm to the Internet"
is caused if this parameter is accepted. You cannot use MUST
unless interoperability is harmed by allowing a server to
omit response data containing the YANG default value.
Please explain all the problems solved by this MUST constraint.

Please explain in detail in the draft WHY

         The 'with-defaults' parameter does not apply to operational
         datastores.

This assertion is false.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 8 Feb 2018 20:00:08 +0100

>          The 'with-defaults' parameter does not apply to operational

I don't get your message. Trying again: I am fine to remove all text
that talks about with-default and if needed someone can spin an update
of RFC 6243 to use with-default.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 08 Feb 2018 20:27:32 +0100 (CET)

> NETCONF should do the same (but won't)

Since we're now defining a new datastore, <operational>, we have the
opportunity to define one single behavour that all servers MUST
implement.  That's why the draft says that defaults are injected as
data flows into <operational>, and that all values that are in use
(including defaults), are present in <operational> and always
returned.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 Feb 2018 14:34:19 -0800

> of RFC 6243 to use with-default.

It is now clear that the current "trim" and "report-all-tagged" enums apply
to <operational>.
These retrieval options apply to config=false nodes for <get>.
It is not clear why <get-data> is removing this functionality for
config=false nodes.

Andy