Subject: [netmod] WG adoption poll draft-nmdsdt-netmod-revised-datastores-00

From: Lou Berger <[email-masked]>
Date: Fri, 2 Dec 2016 16:26:35 -0500

All,

This is start of a two week poll on making
draft-nmdsdt-netmod-revised-datastores-00 a NetMod working group
document.  This document is unusual in that WG last call will be jointly
held in both the NetConf and NetMod WGs, while adoption and day-to-day processing
will take place in NetMod.

Please send email to the list indicating "yes/support" or "no/do not
support".  If indicating no, please state your reservations with the
document.  If yes, please also feel free to provide comments you'd like
to see addressed once the document is a WG document.

The poll ends December 16.

Thank you,
NetConf and NetMod WG Chairs

---

From: Andy Bierman <[email-masked]>
Date: Fri, 2 Dec 2016 13:44:34 -0800

Hi,

Which specific NETMOD WG charter item authorizes this work?

I have concerns about impact of this work on all YANG-based protocols.
I have asked several times "how do you decide which servers need to
implement the intended and applied datastores?" and never got an answer.

I think an Applicability Statement is needed for this work because some
systems
converge so fast that the difference between intended and applied
(for real edits, forget cards not plugged in) will be insignificant.

Andy

On Fri, Dec 2, 2016 at 1:26 PM, Lou Berger <[email-masked]> wrote:

> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Fri, 2 Dec 2016 21:46:25 +0000

Yes/Support

On 12/2/16, 4:26 PM, "netmod on behalf of Lou Berger"
<[email-masked] on behalf of [email-masked]> wrote:

>netmod mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/netmod

---

From: Lou Berger <[email-masked]>
Date: Fri, 2 Dec 2016 17:04:21 -0500

Andy,

There are different approaches to detail in charters, I'm personally
comfortable with this work being covered by our/netmod current
charter(under supporting ongoing deployment of YANG).  This said, the
chairs expect there to be discussions with our AD on respective charter
updates.  Obviously the WG will have an opportunity to provide input on
any update. 

Do you have any specific  changes you'd like to propose?

Thanks,

Lou

On 12/2/2016 4:44 PM, Andy Bierman wrote:
> I think an Applicability Statement is needed for this work because
> some systems
> On Fri, Dec 2, 2016 at 1:26 PM, Lou Berger <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     All,
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netconf>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 2 Dec 2016 14:15:46 -0800

> Do you have any specific  changes you'd like to propose?
I assume you mean this bullet in the charter:

The NETMOD WG may also develop any additional data models written in YANG
that the WG considers core building blocks and that do not fall under the
charters of other active IETF working groups. The NETMOD WG may simply add
such milestones as it sees fit, with the advice and consent of the
responsible AD.

I do not have charter changes to propose.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 3 Dec 2016 10:01:25 +0100

> implement the intended and applied datastores?" and never got an answer.

I do not think people proposed to make the implementation of intended
and applied datastores mandatory to implement. What this proposal is
trying to achieve is to provide an architectural framework that allows
to provide access to additional datastores without having to write
data models in certain ways. Keeping the specific datastore semantics
out of data models (and thereby simplifying data models) is what I see
as the main value of this work.

> (for real edits, forget cards not plugged in) will be insignificant.

The current I-D says in section 6.1:

   o  Support for <intended>, <applied>, and <operational-state> should
      be optional to implement.

/js

PS: Do we have to run the discussion via cross-posts on two lists or
    can the chairs agree to just use one list, assuming that those who
    care can manage to follow this one list?

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Dec 2016 10:18:32 +0100

> On 2 Dec 2016, at 22:26, Lou Berger <[email-masked]> wrote:
> 
> All,
> 

There seems to be no impact on YANG syntax or semantics - the one mentioned in sec. 6.3 is in fact protocol stuff that doesn't belong to the definition of YANG the language. Therefore, this work has nothing to do with data modelling and in fact belongs to the NETCONF WG. This would also solve the indicated WG schizophrenia that should IMO be avoided in any case.

A useful thing to do in the NETMOD WG would be to remove all NETCONF-specific text from the YANG spec because whenever YANG is used outside the NETCONF context (I2RS, CORE), that text is mostly ignored anyway.

Lada 

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Dec 2016 10:38:58 +0100

On Mon, Dec 05, 2016 at 10:18:32AM +0100, Ladislav Lhotka wrote:
> There seems to be no impact on YANG syntax or semantics - the one mentioned in sec. 6.3 is in fact protocol stuff that doesn't belong to the definition of YANG the language. Therefore, this work has nothing to do with data modelling and in fact belongs to the NETCONF WG. This would also solve the indicated WG sch
izophrenia that should IMO be avoided in any case.

I disagree that the datastore model is a protocol specific aspect. I
consider datastores an architectural component binding data models and
protocols together. In fact, the 'traditional' datastore model
together with the semantics of the <get/> operation caused us to write
data models in a very specific way. Since the number of protocols
transporting YANG defined data is growing, it is crucial that
architectural aspects are more clearly spelled out as such. (And the
only architectural document we have so far was done in NETMOD. But at
the end, I find the discussion which WG is responsible somewhat
pointless, it is the same set of active technical contributors
anyway.)

> A useful thing to do in the NETMOD WG would be to remove all
> NETCONF-specific text from the YANG spec because whenever YANG is
> used outside the NETCONF context (I2RS, CORE), that text is mostly
> ignored anyway.

If there is an opportunity to update all core documents together, we
may clean up some of this; so far, we never had such an opportunity.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Dec 2016 11:36:11 +0100

> On 5 Dec 2016, at 10:38, Juergen Schoenwaelder <[email-masked]> wrote:
> 

I would agree with this if datastores were a general concept in YANG, but the revised-datastores draft explicitly introduces the "intended" and "applied" datastores that may be irrelevant to other protocols using YANG, and even needn't be used in all NETCONF implementations. I wouldn't call this "an architectural component" of YANG.

If you are saying that it will have nontrivial impact on YANG, I would like to see it explained in sec. 6.3. Without this information I am quite reluctant to agree with the adoption.

> data models in a very specific way. Since the number of protocols

Yes, to work around a flaw in the NETCONF protocol.

> architectural aspects are more clearly spelled out as such. (And the

See above - architectural aspects need to be relevant to all protocols.

> pointless, it is the same set of active technical contributors

Mehmet rightly argued in Seoul that this work should be done in NETMOD WG because it will certainly have implications for the NETCONF protocol. It is thus understandable that NETCONF chairs want to exercise some control.

> may clean up some of this; so far, we never had such an opportunity.

I don't know what you mean by all core documents. In my view, it would be sufficient to split RFC 7950 into two documents - the other one could be something like "Adapting NETCONF for use with YANG".

If we don't do this, it is difficult to propose YANG for use with other protocols (as we did for I2RS) because we have to say: "You know, some parts of the YANG spec appear mandatory but they are irrelevant for you, so just ignore them". This was actually the case already for RESTCONF.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Dec 2016 11:40:14 +0100

> On 5 Dec 2016, at 11:36, Ladislav Lhotka <[email-masked]> wrote:
> 
>> 
> Mehmet rightly argued in Seoul that this work should be done in NETMOD WG because it will certainly have implications for the NETCONF

Sorry, of course I meant NETCONF WG.

Lada

>  protocol. It is thus understandable that NETCONF chairs want to exercise some control.
> 
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Dec 2016 13:02:10 +0100

> I would agree with this if datastores were a general concept in YANG, but the revised-datastores draft explicitly introduces the "intended" and "applied" datastores that may be irrelevant to other…

An architectural component of this new management framework (that does
not have a name). The fact that not all protocols may expose all
datastores is IMHO not a reason that the datastore model is not an
architectural framework.

> If you are saying that it will have nontrivial impact on YANG, I would like to see it explained in sec. 6.3. Without this information I am quite reluctant to agree with the adoption.

An operational state datastore has implications how one writes data
models. It may not directly affect YANG itself but surely the usage of
YANG.

> See above - architectural aspects need to be relevant to all protocols.

Yes, but relevant to all protocols does not mean every protocol needs
to expose say all datastores. But every protocol should be clear about
how what it exposes relates to the architectural framework.

/js

---

From: Lou Berger <[email-masked]>
Date: Mon, 5 Dec 2016 07:40:31 -0500

On December 3, 2016 4:02:02 AM Juergen Schoenwaelder
<[email-masked]> wrote:

> ...

I agree that this adoption call is not the usual.

The reason for this is basically due to the "which WG" issue that Lada
is raising.  The agreement between the chairs and the AD is that this
work would be run in NetMod and that any resulting netconf/restconf
protocol work would be done in netconf (i.e., the group that owns the
protocol).

To ensure all are in sync on this activity, the agreement between the
chairs is to do the adoption discussion in both groups, and then once
the normal process has been run in NetMod to do a joint WG last call.
We know this makes for some duplicate messages, but it ensures the WGs
are in sync.

Lou

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Dec 2016 09:18:59 -0800

On Mon, Dec 5, 2016 at 4:02 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > I would agree with this if datastores were a general concept in YANG,
> but the revised-datastores draft explicitly introduces the "intended" and
> "applied" datastores that may be irrelevant to other protocols using YANG,
> and even needn't be used in all NETCONF implementations. I wouldn't call
> this "an architectural component" of YANG.
> >
>
> > If you are saying that it will have nontrivial impact on YANG, I would
> like to see it explained in sec. 6.3. Without this information I am quite
> reluctant to agree with the adoption.
>

There is a "current solution" consisting of hard-wired object semantics
(e.g., ifAdminStatus and ifOperStatus).  This solution does not require
special protocols or datastores, but it is being replaced by a generic
solution.

If the "generic" solution requires special procedures which differ on each
protocol,
then it might end up be worse than the hard-wired solution that works on
every protocol.
So I agree with Juergen that this is primarily an architectural issue.

/js

Andy

---

From: MehmetErsue <[email-masked]>
Date: Fri, 09 Dec 2016 12:25:54 +0000

Hi All,

I think the adoption of the DT draft is fine. We agreed in IETF 97 to adopt
and finalize the DT draft in NETMOD WG, which I still support.

I believe the bigger issue is to agree on a plan concerning the related
work and the re-organization of existing standards. As a matter of fact
this plan will lead to updating the charter of two WGs and the work we are
going to start.

I see the DT document as a datastore solution proposal. There are different
gaps and issues which still need to be addressed and the solution proposal
needs yet to be discussed and finalized. The document is providing
information on the history, explaining the need for a generic solution as
well as is discussing different scenarios. As such I believe the datastore
solution proposal from the DT should be published with the intended status
Informational RFC.

Based on the finalized and agreed datastore solution we should do different
updates to existing documents in NETCONF and NETMOD WGs. With this action
we can also fix well-known issues.

Concerning the NETCONF WG I would see it as valuable if we develop:
- a RFC6241bis draft removing the current datasore concept specification,
and getting rid of well-known issues e.g. with the <get> operation,
- a new protocol- and language-independent standard document (RFC XYZ)
defining the generic datastore concept and framework and describing its use
(based on and following the DT solution draft),
- adding potential extensions to RFC6241bis (following the DT draft and
with a normative reference to RFC XYZ),
- adding potential extensions to a RESTCONF-bis RFC (following the DT draft
and with a normative reference to RFC XYZ),

Concerning the NETMOD WG I would see it as valuable if we develop:
- RFC7950bis deleting protocol-dependent details and specifying the
datastore usage with YANG on a generic level (with a normative reference to
RFC XYZ),
- adding potential extensions to RFC7950bis, e.g. concerning the proposed
<notification> element,
- possibly an RFC 6244bis to describe architectural aspects. However
RFC6244 is Informational and a RFC6244bis would be still Informational. I'm
not sure whether this is really necessary. The DT proposal does already
describe such a solution and can be seen as an update to RFC 6244.
- RFC6087bis giving guidelines on how to use YANG with the new datastore
concept.

Referring to Lada's proposal concerning the spin off document from RFC7950
("Adapting NETCONF for use with YANG"),
I think this can be provided in the corresponding protocol RFCs, i.e.
for NETCONF a section on "Using NETCONF with YANG" in RFC6241bis and for
RESTCONF "Using RESTCONF with YANG" in RESTCONF-bis RFC.

Hope this helps as a starting point on the way to a good plan.

PS: As Benoit suggested some time ago we might also consider to rename
NETCONF WG as it is not only on NETCONF protocol anymore.

Regards,
Mehmet

On Mon, Dec 5, 2016 at 6:19 PM Andy Bierman <[email-masked]> wrote:

> Phone: +49 421 200 3587 <0421%202003587>         Campus Ring 1 | 28759
> Bremen | Germany
> Fax:   +49 421 200 3103 <0421%202003103>         <
> http://www.jacobs-university.de/>
>
Cheers,
Mehmet

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Dec 2016 13:48:35 +0100

Hi Mehmet,

I think I could just sign your text at the bottom.

Lada

> On 9 Dec 2016, at 13:25, MehmetErsue <[email-masked]> wrote:
> 
> Hi All,
> 
> I think the adoption of the DT draft is fine. We agreed in IETF 97 to adopt and finalize the DT draft in NETMOD WG, which I still support.
> 
> I believe the bigger issue is to agree on a plan concerning the related work and the re-organization of existing standards. As a matter of fact this plan will lead to updating the charter of two WGs and the work we are going to start.
>  
> I see the DT document as a datastore solution proposal. There are different gaps and issues which still need to be addressed and the solution proposal needs yet to be discussed and finalized. The document is providing information on the history, explaining the need for a generic solution as well as is discussing different scenarios. As such I believe the datastore solution proposal from the DT should be published with the intended status Informational RFC. 
> 
> Based on the finalized and agreed datastore solution we should do different updates to existing documents in NETCONF and NETMOD WGs. With this action we can also fix well-known issues.
> 
> - a RFC6241bis draft removing the current datasore concept specification, and getting rid of well-known issues e.g. with the <get> operation,
> - a new protocol- and language-independent standard document (RFC XYZ) defining the generic datastore concept and framework and describing its use (based on and following the DT solution draft), 
> - adding potential extensions to RFC6241bis (following the DT draft and with a normative reference to RFC XYZ),
> - adding potential extensions to a RESTCONF-bis RFC (following the DT draft and with a normative reference to RFC XYZ),
> 
> - RFC7950bis deleting protocol-dependent details and specifying the datastore usage with YANG on a generic level (with a normative reference to RFC XYZ),
> - adding potential extensions to RFC7950bis, e.g. concerning the proposed <notification> element,
> - possibly an RFC 6244bis to describe architectural aspects. However RFC6244 is Informational and a RFC6244bis would be still Informational. I'm not sure whether this is really necessary. The DT proposal does already describe such a solution and can be seen as an update to RFC 6244.
> - RFC6087bis giving guidelines on how to use YANG with the new datastore concept.
> 
> Referring to Lada's proposal concerning the spin off document from RFC7950 ("Adapting NETCONF for use with YANG"), 
> for NETCONF a section on "Using NETCONF with YANG" in RFC6241bis and for RESTCONF "Using RESTCONF with YANG" in RESTCONF-bis RFC.
> 
> PS: As Benoit suggested some time ago we might also consider to rename NETCONF WG as it is not only on NETCONF protocol anymore. 
> 
> Regards,
> Mehmet 
> 
> On Mon, Dec 5, 2016 at 4:02 AM, Juergen Schoenwaelder <[email-masked]> wrote:
> special protocols or datastores, but it is being replaced by a generic solution.
> 
> If the "generic" solution requires special procedures which differ on each protocol,
> then it might end up be worse than the hard-wired solution that works on every protocol.

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Tue, 13 Dec 2016 13:37:32 +0000

I support adoption, and like Mehmet's thinking as well.

Also worth focusing on is transport protocol independent yang filtering.  So along with how to frame get operations against one of the datastores, how do you know which subtrees/nodes should be included/excluded.

Eric

> From: Netconf, December 9, 2016 7:49 AM
> 
> Hi Mehmet,
> 
> > I think the adoption of the DT draft is fine. We agreed in IETF 97 to adopt and
> finalize the DT draft in NETMOD WG, which I still support.
> >
> > I believe the bigger issue is to agree on a plan concerning the related work
> and the re-organization of existing standards. As a matter of fact this plan will
> lead to updating the charter of two WGs and the work we are going to start.
> >
> needs yet to be discussed and finalized. The document is providing information
> on the history, explaining the need for a generic solution as well as is discussing
> different scenarios. As such I believe the datastore solution proposal from the
> DT should be published with the intended status Informational RFC.
> >
> updates to existing documents in NETCONF and NETMOD WGs. With this
> action we can also fix well-known issues.
> >
> > - a RFC6241bis draft removing the current datasore concept
> > specification, and getting rid of well-known issues e.g. with the
> > <get> operation,
> > defining the generic datastore concept and framework and describing
> > its use (based on and following the DT solution draft),
> > - adding potential extensions to RFC6241bis (following the DT draft
> > - adding potential extensions to a RESTCONF-bis RFC (following the DT
> > draft and with a normative reference to RFC XYZ),
> >
> > datastore usage with YANG on a generic level (with a normative
> > reference to RFC XYZ),
> > - possibly an RFC 6244bis to describe architectural aspects. However RFC6244
> is Informational and a RFC6244bis would be still Informational. I'm not sure
> whether this is really necessary. The DT proposal does already describe such a
> solution and can be seen as an update to RFC 6244.
> > Referring to Lada's proposal concerning the spin off document from
> > RFC7950 ("Adapting NETCONF for use with YANG"), I think this can be
> provided in the corresponding protocol RFCs, i.e.
> > for NETCONF a section on "Using NETCONF with YANG" in RFC6241bis and
> for RESTCONF "Using RESTCONF with YANG" in RESTCONF-bis RFC.
> >
> > On Mon, Dec 5, 2016 at 4:02 AM, Juergen Schoenwaelder
> > > > I disagree that the datastore model is a protocol specific aspect.
> > > > I consider datastores an architectural component binding data
> > > > models and protocols together. In fact, the 'traditional'
> > > > datastore model
> > >
> > > I would agree with this if datastores were a general concept in YANG, but
> the revised-datastores draft explicitly introduces the "intended" and "applied"
> datastores that may be irrelevant to other protocols using YANG, and even
> needn't be used in all NETCONF implementations. I wouldn't call this "an
> architectural component" of YANG.
> > >
> >
> > > If you are saying that it will have nontrivial impact on YANG, I would like to
> see it explained in sec. 6.3. Without this information I am quite reluctant to
> agree with the adoption.
> >
> > There is a "current solution" consisting of hard-wired object
> > semantics (e.g., ifAdminStatus and ifOperStatus).  This solution does
> > not require special protocols or datastores, but it is being replaced by a
> generic solution.
> >
> > If the "generic" solution requires special procedures which differ on
> > each protocol, then it might end up be worse than the hard-wired solution
> that works on every protocol.

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Dec 2016 09:37:57 -0800

On Tue, Dec 13, 2016 at 5:37 AM, Eric Voit (evoit) <[email-masked]> wrote:

> Also worth focusing on is transport protocol independent yang filtering.
> So along with how to frame get operations against one of the datastores,
> how do you know which subtrees/nodes should be included/excluded.
>
>

This architectural change needs to be implemented in various protocols.
I am not sure a 6241bis is the best approach because it is not clear which
servers really need to implement the revised datastores.  Since RD is
purely optional
to implement, it should not obsolete 6241 in any way.  It should be possible
to add new operations and/or new parameters to existing operations without
needing to redefine what is already there.

The new protocol features need to explain how to include/exclude subtrees.
IMO we should only support YANG defined data.  This allows the solutions
to be generalized and reusable across protocols (e.g., using YANG
extensions).

Eric

Andy

> information
> RFC6244

---

From: "Mehmet Ersue" <[email-masked]>
Date: Tue, 13 Dec 2016 22:41:12 +0100

Hi Andy,

> servers really need to implement the revised datastores.  

Mehmet

From: Andy Bierman [mailto:[email-masked]] 
Sent: Dienstag, 13. Dezember 2016 18:38
To: Eric Voit (evoit) <[email-masked]>
Cc: Ladislav Lhotka <[email-masked]>; MehmetErsue <[email-masked]>; NetMod WG Chairs <[email-masked]>; NetConf WG Chairs <[email-masked]>; NetMod WG <[email-masked]>; Netconf <[email-masked]>
Subject: Re: [netmod] [Netconf] WG adoption poll draft-nmdsdt-netmod-revised-datastores-00

On Tue, Dec 13, 2016 at 5:37 AM, Eric Voit (evoit) <[email-masked] <mailto:[email-masked]> > wrote:

 

servers really need to implement the revised datastores.  Since RD is purely optional

to be generalized and reusable across protocols (e.g., using YANG extensions).

Eric

Andy

> > On 9 Dec 2016, at 13:25, MehmetErsue <[email-masked] <mailto:[email-masked]> > wrote:
> >
> > Hi All,
> >
> > On Mon, Dec 5, 2016 at 6:19 PM Andy Bierman <[email-masked] <mailto:[email-masked]> >
> wrote:
> <[email-masked] <mailto:[email-masked]> > wrote:
> > [email-masked] <mailto:[email-masked]> 

_______________________________________________
netmod mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Dec 2016 13:51:50 -0800

On Tue, Dec 13, 2016 at 1:41 PM, Mehmet Ersue <[email-masked]> wrote:

> specification, and getting rid of well-known issues e.g. with the <get>
> operation,
>
>
I do not agree with the text you wrote.
I do not want to remove candidate, running, and startup from RFC 6241.
IMO the new datastores can be defined in a new document that does not
redefine the existing datastores.

I also do not want to get rid of <get>,  It works as intended.
It is not a problem on small devices.  It is not a problem on large devices
if
sufficient filtering is used.  It does not differentiate between intended
and applied config
or understand different types of config=false nodes.  Use a new operation to
add these features.

Andy

>
>
> *From:* Andy Bierman [mailto:[email-masked]]
> *Sent:* Dienstag, 13. Dezember 2016 18:38
> *To:* Eric Voit (evoit) <[email-masked]>
> *Cc:* Ladislav Lhotka <[email-masked]>; MehmetErsue <[email-masked]>;
> NetMod WG Chairs <[email-masked]>; NetConf WG Chairs <
> [email-masked]>; NetMod WG <[email-masked]>; Netconf <
> [email-masked]>
> *Subject:* Re: [netmod] [Netconf] WG adoption poll
> draft-nmdsdt-netmod-revised-datastores-00
>
>
>
>
>
>
>

---

From: "Mehmet Ersue" <[email-masked]>
Date: Tue, 13 Dec 2016 23:47:05 +0100

Technically spoken actually both options work for me: 

Keeping current concept in 6241 or having everything in one document.

I prefer the latter.

We actually don’t want to remove <get>, only solve issues.

Any change to <get> is for sure subject to WG decision.

Mehmet

From: Andy Bierman [mailto:[email-masked]] 
Sent: Tuesday, December 13, 2016 10:52 PM
To: Mehmet Ersue <[email-masked]>
Cc: Eric Voit (evoit) <[email-masked]>; Ladislav Lhotka <[email-masked]>; NetMod WG Chairs <[email-masked]>; NetConf WG Chairs <[email-masked]>; NetMod WG <[email-masked]>; Netconf <[email-masked]>
Subject: Re: [netmod] [Netconf] WG adoption poll draft-nmdsdt-netmod-revised-datastores-00

On Tue, Dec 13, 2016 at 1:41 PM, Mehmet Ersue <[email-masked] <mailto:[email-masked]> > wrote:

Hi Andy,

IMO the new datastores can be defined in a new document that does not redefine the existing datastores.

It is not a problem on small devices.  It is not a problem on large devices if

sufficient filtering is used.  It does not differentiate between intended and applied config

add these features.

 

Mehmet

Andy

From: Andy Bierman [mailto:[email-masked] <mailto:[email-masked]> ] 
Sent: Dienstag, 13. Dezember 2016 18:38
To: Eric Voit (evoit) <[email-masked] <mailto:[email-masked]> >
Cc: Ladislav Lhotka <[email-masked] <mailto:[email-masked]> >; MehmetErsue <[email-masked] <mailto:[email-masked]> >; NetMod WG Chairs <[email-masked] <mailto:[email-masked]> >; NetConf WG Chairs <[email-masked] <mailto:[email-masked]> >; NetMod WG <[email-masked] <mailto:[email-masked]> >; Netconf <[email-masked] <mailto:[email-masked]> >
Subject: Re: [netmod] [Netconf] WG adoption poll draft-nmdsdt-netmod-revised-datastores-00

 

 

Eric

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Dec 2016 12:07:54 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> It is not a problem on small devices.

Andy, the problem with <get> has nothing to do with the size of the
device.  The problem is that <get> returns two things (running config
+ operational state) in one merged output document.  This forces
people to split data models so that config and state are mutually
exclusive (/interfaces and /interfaces-state).  This draft proposes a
fix for this, which makes <get> less useful.

/martin

> It is not a problem on large devices
> if

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Dec 2016 06:09:11 -0800

On Wed, Dec 14, 2016 at 3:07 AM, Martin Bjorklund <[email-masked]> wrote:

> its use
This "problem" exists for some new overloaded use of <get> that did not
exist before.
The <get> operation only mixes config=true and config=false. It never had
anything
to do with intended vs. applied.  IMO your proposed solution is not very
backward compatible
with simple systems that do not have delays between intended and applied.

Andy

> > > > > I think the adoption of the DT draft is fine. We agreed in IETF 97
> to
> > > adopt and
> > > > and the re-organization of existing standards. As a matter of fact
> this
> > > plan will
> > > > on the history, explaining the need for a generic solution as well
> as is
> > > discussing
> > > > different scenarios. As such I believe the datastore solution
> proposal
> > > from the
> > > > is Informational and a RFC6244bis would be still Informational. I'm
> not
> > > sure
> > > > whether this is really necessary. The DT proposal does already
> describe
> > > such a
> > > > > not require special protocols or datastores, but it is being
> replaced
> > > by a
> > > > generic solution.
> > > > >

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 14 Dec 2016 15:25:26 +0100

> redefine the existing datastores.

As for candidate, it is optional and we all know that it is quite
problematic if concurrent access of multiple clients is
possible. Therefore, it would IMO be a good riddance.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Dec 2016 14:42:09 +0000

On 14/12/2016 14:09, Andy Bierman wrote:
>
>
> On Wed, Dec 14, 2016 at 3:07 AM, Martin Bjorklund <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Andy Bierman <[email-masked] <mailto:[email-masked]>> wrote:
>     > On Tue, Dec 13, 2016 at 1:41 PM, Mehmet Ersue <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > > Hi Andy,
>     > >
>     > >
>     > >
>     > > > I am not sure a 6241bis is the best approach because it is
>     not clear
>     > > which
>     > >
> This "problem" exists for some new overloaded use of <get> that did 
> not exist before.
> The <get> operation only mixes config=true and config=false. It never 
> had anything
> to do with intended vs. applied.  IMO your proposed solution is not 
> very backward compatible

I've been informed (repeatedly ;-) that <running> has always only ever 
meant intended configuration.  I.e. that it is reasonable behaviour for 
a system to accept config into <running> and then fail to apply that 
configuration for some reason (daemon is unavailable, linecards is down, 
resources have been exceeded, programming error, etc).  If this 
interpretation is correct then the NETCONF <get> operation is poorly 
defined because it is mixing "intended configuration of the system" 
along with "actual running state".

What NETCONF <get> should really be providing is "actual configuration 
in use by the system" along with "actual running state".  Of course this 
is effectively what the new operational state datastore is defined as 
containing.

If I understand it correctly, I think that Andy's point is that for lots 
of systems "intended configuration of the system" and "actual 
configuration of the system" are effectively one and the same thing.  If 
this is the case then it should be easy for clients/servers to migrate 
from an existing <get> request to a <get-data> request that just returns 
the data from the operational state datastore.  It sounds like the 
actual data returned in both cases should be the same?

Thanks,
Rob

>     > > *From:* Andy Bierman [mailto:[email-masked]
>     <mailto:[email-masked]>]
>     > > *To:* Eric Voit (evoit) <[email-masked] <mailto:[email-masked]>>
>     > > *Cc:* Ladislav Lhotka <[email-masked] <mailto:[email-masked]>>;
>     MehmetErsue <[email-masked] <mailto:[email-masked]>>;
>     > > NetMod WG Chairs <[email-masked]
>     <mailto:[email-masked]>>; NetConf WG Chairs <
>     > > [email-masked] <mailto:[email-masked]>>;
>     NetMod WG <[email-masked] <mailto:[email-masked]>>; Netconf <
>     > > [email-masked] <mailto:[email-masked]>>
>     > > On Tue, Dec 13, 2016 at 5:37 AM, Eric Voit (evoit)
>     <[email-masked] <mailto:[email-masked]>>
>     > > wrote:
>     > >
>     Since RD is
>     > > to implement, it should not obsolete 6241 in any way.  It
>     should be
>     > > possible
>     > >
>     > > > > On 9 Dec 2016, at 13:25, MehmetErsue <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > > > >
>     > > > > Hi All,
>     > > > >
>     > > > > I think the adoption of the DT draft is fine. We agreed in
>     IETF 97 to
>     > > adopt and
>     > > > > I believe the bigger issue is to agree on a plan
>     concerning the
>     > > related work
>     > > > and the re-organization of existing standards. As a matter
>     of fact this
>     > > plan will
>     > > > lead to updating the charter of two WGs and the work we are
>     going to
>     > > start.
>     > > > >
>     > > > > I see the DT document as a datastore solution proposal.
>     There are
>     > > different
>     > > > gaps and issues which still need to be addressed and the
>     solution
>     > > proposal
>     > > > on the history, explaining the need for a generic solution
>     as well as is
>     > > discussing
>     > > > different scenarios. As such I believe the datastore
>     solution proposal
>     > > from the
>     > > > > Based on the finalized and agreed datastore solution we
>     should do
>     > > different
>     > > > > - adding potential extensions to RFC7950bis, e.g.
>     concerning the
>     > > > > proposed <notification> element,
>     > > > is Informational and a RFC6244bis would be still
>     Informational. I'm not
>     > > sure
>     > > > whether this is really necessary. The DT proposal does
>     already describe
>     > > such a
>     > > > > On Mon, Dec 5, 2016 at 6:19 PM Andy Bierman
>     <[email-masked] <mailto:[email-masked]>>
>     > > > wrote:
>     > > > <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > > > > > I would agree with this if datastores were a general
>     concept in
>     > > YANG, but
>     > > > the revised-datastores draft explicitly introduces the
>     "intended" and
>     > > "applied"
>     > > > > > If you are saying that it will have nontrivial impact on
>     YANG, I
>     > > would like to
>     > > > see it explained in sec. 6.3. Without this information I am
>     quite
>     > > reluctant to
>     > > > agree with the adoption.
>     > > > >
>     > > > > not require special protocols or datastores, but it is
>     being replaced
>     > > by a
>     > > > generic solution.
>     > > > >
>     > > > > each protocol, then it might end up be worse than the
>     hard-wired
>     > > solution
>     > > > > Fax:   +49 421 200 3103       
>      <http://www.jacobs-university.de/ <http://www.jacobs-university.de/>>
>     > > > >
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     > > > > --
>     > > > > Cheers,
>     > > > > Mehmet
>     > > >
>     > > > --
>     <https://www.ietf.org/mailman/listinfo/netmod>
>     > >
>     > >
>     > >
>
>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Dec 2016 07:41:53 -0800

On Wed, Dec 14, 2016 at 6:42 AM, Robert Wilton <[email-masked]> wrote:

>> <get>
> meant intended configuration.  I.e. that it is reasonable behaviour for a
> system to accept config into <running> and then fail to apply that
> defined because it is mixing "intended configuration of the system" along
> with "actual running state".
>
> What NETCONF <get> should really be providing is "actual configuration in
> use by the system" along with "actual running state".  Of course this is
> effectively what the new operational state datastore is defined as
> containing.
>
> of systems "intended configuration of the system" and "actual configuration
> of the system" are effectively one and the same thing.  If this is the case
> then it should be easy for clients/servers to migrate from an existing
> <get> request to a <get-data> request that just returns the data from the
> operational state datastore.  It sounds like the actual data returned in
> both cases should be the same?
>
>

I do not understand the new solution because it has not been well-defined.

In the old solution, I have 2 leafs /foo and /foo-state.
I can retrieve both of these leafs at once with <get>.
I can decide if /foo and /foo-state are the values I expect.

But if I overload /foo such that it represents different semantics in
different datastores, now a retrieval operation can only get 1 at a time.
If I set /foo, then get-state(/foo), how do I know the config value for /foo
has not changed in the meantime?  If the server returns 2 /foo leafs in the
same
message body, then it is no longer YANG schema-valid (only 1 /foo node is
allowed)

The  <get> operation does not overload objects with different semantics.
Only a new server that eliminates /foo-state is overloading the semantics
of /foo.
The ability to retrieve /foo-state is lost for a client that only knows
<get> (and not <get-state).
This does not mean <get> is broken.  It just means the ability to access
/foo-state is removed.

Thanks,
Andy

>> > > *Cc:* Ladislav Lhotka <[email-masked]>; MehmetErsue <[email-masked]
>> >;
>> > > > > I think the adoption of the DT draft is fine. We agreed in IETF
>> 97 to
>> > > adopt and
>> > > > the revised-datastores draft explicitly introduces the "intended"
>> and
>> > > "applied"
>> > > > > Fax:   +49 421 200 3103         <http://www.jacobs-university.de/
>> >
>> > > > >
> netmod mailing [email-masked]://www.ietf.org/mailman/listinfo/netmod
>
>
>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Dec 2016 16:25:13 +0000

Hi Andy,

Thanks for you comments, please see inline.

On 14/12/2016 15:41, Andy Bierman wrote:
>
>
> On Wed, Dec 14, 2016 at 6:42 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>
>>         > On Tue, Dec 13, 2016 at 1:41 PM, Mehmet Ersue
>>         <[email-masked] <mailto:[email-masked]>> wrote:
>>         >
>>         > > Hi Andy,
>>         > >
>>         > >
>>         > >
>>         > > > I am not sure a 6241bis is the best approach because it
>>         is not clear
>>         > > which
>>         > >
>>     This "problem" exists for some new overloaded use of <get> that
>>     did not exist before.
>>     The <get> operation only mixes config=true and config=false. It
>>     never had anything
>>     to do with intended vs. applied.  IMO your proposed solution is
>>     not very backward compatible
>     I've been informed (repeatedly ;-) that <running> has always only
>     ever meant intended configuration.  I.e. that it is reasonable
>     behaviour for a system to accept config into <running> and then
>     fail to apply that configuration for some reason (daemon is
>     unavailable, linecards is down, resources have been exceeded,
>     programming error, etc).  If this interpretation is correct then
>     the NETCONF <get> operation is poorly defined because it is mixing
>     "intended configuration of the system" along with "actual running
>     state".
>
>     What NETCONF <get> should really be providing is "actual
>     configuration in use by the system" along with "actual running
>     state".  Of course this is effectively what the new operational
>     state datastore is defined as containing.
>
>     If I understand it correctly, I think that Andy's point is that
>     for lots of systems "intended configuration of the system" and
>     "actual configuration of the system" are effectively one and the
>     same thing.  If this is the case then it should be easy for
>     clients/servers to migrate from an existing <get> request to a
>     <get-data> request that just returns the data from the operational
>     state datastore.  It sounds like the actual data returned in both
>     cases should be the same?
>
>
>
Yes, this is true. But to be a programmatically easily consumable 
solution it requires that the structure under /foo is also exactly 
replicated under /foo-state.  This probably means that most config needs 
to be put into groupings, and IMO, I think that it makes the models hard 
to read, and also harder to maintain.

> different datastores, now a retrieval operation can only get 1 at a time.
Yes, this is true.  Although it should be possible to define RPC 
operations that fetch more than one datastore in the same message if 
required, although I'm not convinced that is truly needed.

> has not changed in the meantime?
You don't, unless you also do a get on the running config datastore as well.

But I also don't see why this is a problem.  Either the client knows 
what the config should be, in which case it probably doesn't need to 
explicitly ask the device, but can infer it from the operational state.  
Alternatively, if another client may also be modifying the configuration 
at the same time then the client needs to be designed to expect and 
handle this.

Even with the existing <get> request, I don't think that any non trivial 
device can guarantee that all the data returned in that get request is 
atomically consistent with itself.

>   If the server returns 2 /foo leafs in the same
> message body, then it is no longer YANG schema-valid (only 1 /foo node 
> is allowed)
The message body would need to return two separate data trees, one for 
each datastore that is being returned.

> Only a new server that eliminates /foo-state is overloading the 
> semantics of /foo.
The new server doesn't overload the same object with different 
semantics, it is really just saying that the value of an object depends 
on what datastore it is being read from, which is true in NETCONF even 
today.  I.e. the value for a given config leaf could already differ 
between candidate, running, startup.  What is being proposed doesn't 
seem to fundamentally change the meaning of the config leaves in a new 
and different way, it is just defining new datastores with associated 
more tightly specified semantics.

> The ability to retrieve /foo-state is lost for a client that only 
> knows <get> (and not <get-state).
> This does not mean <get> is broken.  It just means the ability to 
> access /foo-state is removed.
I think that <get> is broken in the sense that it is combining together 
two sets of data that cannot always be combined in a meaningful way.  
For me the bigger problem is that it is only under rare conditions that 
the problem scenarios arise, meaning that clients may not be coded to be 
robust under those scenarios devaluing the YANG automation proposition.

Thanks,
Rob

---

From: "Jonathan Hansford" <[email-masked]>
Date: Mon, 19 Dec 2016 08:09:36 -0000

> -----Original Message-----
> From: netmod [mailto:[email-masked]] On Behalf Of Ladislav
> Lhotka
> Sent: 14 December 2016 14:25
> To: Andy Bierman <[email-masked]>; Mehmet Ersue
> <[email-masked]>
> Cc: NetMod WG Chairs <[email-masked]>; NetConf WG Chairs
> <[email-masked]>; NetMod WG <[email-masked]>; Netconf
> <[email-masked]>
> As for candidate, it is optional and we all know that it is quite
problematic if
> concurrent access of multiple clients is possible. Therefore, it would IMO
be a
> good riddance.

For someone who is yet to see NETCONF and YANG used in anger, can you
explain why judicious use of candidate and lock is problematic with
concurrent access and why, as a consequence, it should be got rid of? One of
the features of NETCONF that attracted us to it is its support for
transactioning, a feature which it would appear came from previous
experience with JUNOS. Is it current guidance that transactioning should not
be used?

Jonathan

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 19 Dec 2016 11:37:57 +0100

> On 19 Dec 2016, at 09:09, Jonathan Hansford <[email-masked]> wrote:
> 

I am not proposing to ban candidate or something but IMO it needn't be part of the base NETCONF (or whatever protocol) spec.

A typical problem of candidate combined with NACM is that user A edits item X and B edits Y in candidate. If B doesn't have write access to X and A to Y, then none of them is able to make a commit.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Dec 2016 10:48:34 +0000

Hi Jonathan,

These are just my thoughts ...

I'm not advocating getting rid of candidate/locking, but if I was 
designing a new network management solution, I would not plan on using 
candidate/locking.

Candidate is great when you have got a human operator typing in some 
config on the CLI and want to stop anyone (or anything) else changing 
the config at the same time under their feet.

But once you only have machines programming the config, then they should 
be able to easily construct a single message for each edit of the config 
(which can be applied/failed in its entirety). Candidate and locking 
don't seem to help here.

Further, if you want to update multiple devices at the same time then 
you end up in the realm of distributed transactions which get very 
complicated and are hard to get right in a fully robust fashion.

It is at this point, that I would try hard to build a network management 
architecture that doesn't rely on transactions or locking, and instead 
only relies on individual updates being consistently applied in a 
serialized fashion.

Rob

On 19/12/2016 08:09, Jonathan Hansford wrote:
> .

---

From: Jan Lindblad <[email-masked]>
Date: Mon, 19 Dec 2016 12:31:07 +0100

> But once you only have machines programming the config, then they should be able to easily construct a single message for each edit of the config (which can be applied/failed in its entirety). Candidate and locking don't seem to help here.
> 
> Further, if you want to update multiple devices at the same time then you end up in the realm of distributed transactions which get very complicated and are hard to get right in a fully robust fashion.

Harder still without the candidate. Any case where transaction phases or timing is involved would be harder to get right without the candidate and confirmed commit. Being far from perfect, it's the best standardized configuration protocol out there.

/jan

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 19 Dec 2016 13:48:53 +0100

> I am not proposing to ban candidate or something but IMO it needn't be part of the base NETCONF (or whatever protocol) spec.

Lets recall that candidate is a _capability_. Nobody is required to
implement it.

> A typical problem of candidate combined with NACM is that user A edits item X and B edits Y in candidate. If B doesn't have write access to X and A to Y, then none of them is able to make a commit.

The problem is caused by allowing users with inconsistent access
rights to both use candidate. So you get what you asked for. But I
assume you can still do <discard-changes> and recover.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 19 Dec 2016 13:56:17 +0100

> On 19 Dec 2016, at 12:31, Jan Lindblad <[email-masked]> wrote:
> 

Per-user candidate datastores are much easier to deal with, and this is what we do in our RESTCONF implementation:

https://github.com/CZ-NIC/jetconf

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 19 Dec 2016 14:14:49 +0100

> On 19 Dec 2016, at 13:48, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Yes, this is what I wrote. My point is that this capability could/should be moved out of the NETCONF spec to a separate document. The latter can then be modified without affecting the former. 

> rights to both use candidate. So you get what you asked for. But I

Well, as soon as you let clients manage their private data (user accounts, routing instances etc.) you get these "inconsistent" access rights automatically.

> assume you can still do <discard-changes> and recover.

..., or find somebody with superuser privileges to perform the commit, but it is clumsy either way. Automated solutions are likely to break.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 19 Dec 2016 14:28:44 +0100

> Yes, this is what I wrote. My point is that this capability could/should be moved out of the NETCONF spec to a separate document. The latter can then be modified without affecting the former. 

I believe it does not matter where a capability is defined. You can
even declare the capability dead by writing a document declaring it
dead; this does not require to revise the RFC the capability is
defined in. Since the candidate datastore has been in NETCONF since
day one and it is out there and apparently used, I do not see a strong
reason to move the definition. Those who do not like it simply do not
implement it.

/js

---

From: Lou Berger <[email-masked]>
Date: Mon, 19 Dec 2016 08:33:47 -0500

All,

This poll is closed and this document is adopted in the NETMOD WG.

There have been many good topics raised during the call adoption call
that we will work through. Kent and I are already working on a draft of
the updated charter.

Authors,

    Please resubmit your draft with the new name
draft-nmdsdt-netmod-revised-datastores-00 and with that and the date
being the only change.

Please consider how you want to track and resolve the topics/issues
raised during the adoption call.  WG trac and github issue tracker are
available to you.  Please also set a topic-specific Subject line in any
list followup. 

-- Draft specific discussion can now be limited to just the netmod list.

Thank you!
Lou

---

From: "Mehmet Ersue" <[email-masked]>
Date: Mon, 19 Dec 2016 15:25:22 +0100

Dear Lou,

as I stated earlier and some people agreed, I believe this document should be published as Informational RFC.

The reason I gave is below:
> I see the DT document as a datastore solution proposal. There are different gaps and issues which still need to be addressed and the solution proposal needs yet to be 
> discussed and finalized. The document is providing information on the history, explaining the need for a generic solution as well as is discussing different scenarios. 
> As such I believe the datastore solution proposal from the DT should be published with the intended status Informational RFC.

I would like to recommend to change the intended status of the draft accordingly.

Thanks,
Mehmet

-----Original Message-----
From: Lou Berger [mailto:[email-masked]] 
Sent: Monday, December 19, 2016 2:34 PM
To: NetMod WG <[email-masked]>; NetConf WG <[email-masked]>
Cc: NetMod WG Chairs <[email-masked]>; NetConf WG Chairs <[email-masked]>
Subject: Re: WG adoption poll draft-nmdsdt-netmod-revised-datastores-00

All,

There have been many good topics raised during the call adoption call that we will work through. Kent and I are already working on a draft of the updated charter.

Authors,

    Please resubmit your draft with the new name
draft-nmdsdt-netmod-revised-datastores-00 and with that and the date being the only change.

Please consider how you want to track and resolve the topics/issues raised during the adoption call.  WG trac and github issue tracker are available to you.  Please also set a topic-specific Subject line in any list followup. 

Thank you!
Lou

On 12/2/2016 4:26 PM, Lou Berger wrote:
> document.  This document is unusual in that WG last call will be 
> jointly held in both the NetConf and NetMod WGs, while adoption and 
> day-to-day processing will take place in NetMod.
>
> document.  If yes, please also feel free to provide comments you'd 
> like to see addressed once the document is a WG document.
>