Subject: [Netconf] RESTCONF content=all on conflicts?

From: Robert Varga <[email-masked]>
Date: Tue, 21 Jun 2016 10:08:13 +0200

Hello,

Reading through section 4.8.1 and  D.3.1, it seems that in case of
content=all there is no guidance what the server should do if it
encounters a leaf which has different values in config and oper.

Would this constitute an error, or should one of them take precedence?

Thanks,
Robert

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 10:43:43 +0200 (CEST)

> encounters a leaf which has different values in config and oper.

Note that the 'data' resource contains "the combined configuration and
state data".  This has the same problem as NETCONF <get/>.  If we
introduce separate datastores for intended, applied, operational-state
as proposed in draft-schoenw-netmod-revised-datastores (which I think
your question sort of implies), we will need to update this model both
in NETCONF and RESTCONF.  draft-schoenw-netmod-revised-datastores
suggests how this can be done.

The WG should be aware of this though.  If we introduce the separare
datastores (which I think we should do), NETCONF <get/> won't work
properly, and plain RESTCONF GET w/o additional query parameters would
also not work properly, just as you indicate.  Maybe we can solve this
by saying that one of them takes precedence.

/martin

---

From: Robert Varga <[email-masked]>
Date: Tue, 21 Jun 2016 11:13:06 +0200

On 06/21/2016 10:43 AM, Martin Bjorklund wrote:
> If we

Right, sorry about silent implication, this comes from ODL
implementation, which has an in-between architecture with separate
config and oper stores, but without running/intended/applied separation.

Thanks for the pointer.

Bye,
Robert

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 11:20:15 +0200 (CEST)

Robert Varga <[email-masked]> wrote:
> config and oper stores

So does the schema for the oper store contain also all config true
nodes?  If it doesn't there's no risk for a conflict, right?

How does ODL handle this case for NETCONF <get/>?

/martin

---

From: Robert Wilton <[email-masked]>
Date: Tue, 21 Jun 2016 10:20:51 +0100

Hi Robert,

The existing definition of the datastores as defined in NETCONF (RFC 
6241) doesn't allow a config=true leaf to have a different value in 
config and oper.  There is only one leaf with one value.

As Martin has indicated, Juergen (with Martin input) has written up a 
revised datastore proposal as one way of splitting the configuration 
into the intended and applied values and defining an operational state 
datastore.

I've also written up an alternative architecture for splitting up the 
datastores (based on the opstate discussions), as
draft-wilton-netmod-refined-datastores.  In the model that I propose, a 
NETCONF get-config request would return the intended configuration sent 
by the client, and a NETCONF get request would return the operational 
state (which includes the applied configuration).   For servers that 
don't support a split between intended vs applied configuration then 
they can be regarded as having the same values, and hence the behaviour 
is identical to the NETCONF behaviour today.

My proposed refinement in GET semantics for clients may not be 
acceptable, in which case the existing GET operation would probably need 
to be superseded by a new GET-0PER operation.

Thanks,
Rob

On 21/06/2016 09:08, Robert Varga wrote:
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 21 Jun 2016 11:46:28 +0200

> Would this constitute an error, or should one of them take precedence?

The notion of "all" may prove problematic depending on how we move
forward with multiple datastore. However, I think the most pressing
issue to address before RESTCONF goes to the RFC editor is to move the
default of the content query parameter back to 'config' instead of
'all'. Having the content parameter default to 'all' makes it hard to
possibly deprecate 'all' in the future.

Note that the current RESTCONF I-D is also inconsistent:

  4.8.1.  The "content" Query Parameter

  If this query parameter is not present, the default value is "all".
  This query parameter MUST be supported by the server.

  D.3.1.  "content" Parameter

  To retrieve only the configuration child resources, the "content"
  parameter is set to "config" or omitted since this is the default
  value.  Note that the "ETag" and "Last-Modified" headers are only
  returned if the content parameter value is "config".

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 12:21:17 +0200 (CEST)

> possibly deprecate 'all' in the future.

I don't think this change would help.  Suppose we introduce
intended/applied/operational-state as proposed in your draft.  Config
true nodes would then exist in all three.  So which one would be
returned w/o any 'content' param (or 'content=config')?

IMO the problem is that 'data' contains *combined* config and state,
which doesn't really work if the same schema nodes can exist in both
config and state.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Tue, 21 Jun 2016 11:50:38 +0100

> which doesn't really work if the same schema nodes can exist in both
This is one of the reasons why I prefer having only 2 datastores (i.e. 
intended + operational state (inc applied config)) rather than the 3 
described above.

The content=all operation can then return the contents of the 
operational state datastore which happens to include the applied 
configuration, and is also the same as what would be returned from a 
RESTCONF device today.

Full clean access to the separate datastores will require datastore 
awareness in the REST API, but I can imagine that for a lot of simple 
devices a simple combined view is useful.

Thanks,
Rob

> .

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 12:59:31 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> This is one of the reasons why I prefer having only 2 datastores
> (i.e. intended + operational state (inc applied config)) rather than
> the 3 described above.
> 
> operational state datastore

This would be fine, but this is NOT how the content parameter is
defined today.   Hmmm, maybe it is not clear what the "content"
parameter actually does.  It says:

    | config    | Return only configuration descendant data nodes     |
    | nonconfig | Return only non-configuration descendant data nodes |
    | all       | Return all descendant data nodes                    |

Whatever this means, it seems to imply that:

  GET(content=config) + GET(content=nonconfig) = GET(content=all)

It is just a simple filter to select one subset of all data.

This means that the "content" parameter does not affect the source of
the data; the source is still the "combination of config ans state".

> which happens to include the applied

This would be ok.

/martin

---

From: Robert Varga <[email-masked]>
Date: Tue, 21 Jun 2016 13:28:39 +0200

> nodes?  If it doesn't there's no risk for a conflict, right?

Yes, it does, but the config -> oper propagation is done asynchronously
by individual plugins, hence there is a window when the two are not
consistent.

> How does ODL handle this case for NETCONF <get/>?

That is a good question, I had to look this one up -- it takes just the
oper state (and assumes config was already mirrored). That feels like
something that should be changed.

Bye,
Robert

---

From: t.petch <[email-masked]>
Date: Tue, 21 Jun 2016 12:55:44 +0100

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, June 21, 2016 11:59 AM

> the data; the source is still the "combination of config ans state".

Martin

I think that it is sort of clear.  I did comment earlier on the non-use
of the term 'state' in this I-D and it has been changed in most places,
but not here.  I would prefer the choice to be 'config' 'state' 'all' to
be in line with NETCONF.

There is a further ambiguity in this thread that  NETCONF discarded the
term operational state and uses state for both data learnt by other
means, such as a hot-plugged card or a routing protocol, and statistics.
When someone uses the term operational state, are they including
statistics?  According to NETCONF yes, but I wonder if they understand
that.  An unfiltered 'get' 'all' could be big.

Tom Petch

<snip>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:05:24 -0700

> that.  An unfiltered 'get' 'all' could be big.

The config parameter simply provides a filter based on the YANG config-stmt.
It is not a datastore filter.  RESTCONF does not expose datastores.

You keep seeing proof that NETCONF is in chaos because some people
want to refine the datastore model wo support new features.  It doesn't
look that way to me.

> Tom Petch

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 21 Jun 2016 14:19:14 +0200

> intended + operational state (inc applied config)) rather than the 3

I fail to see why the # of datastores matters. If the 'content'
parameter really just filters by the YANG config true|false property,
then using config true data nodes to also report state in a different
datastore won't work, regardless what the number is.

> The content=all operation can then return the contents of the operational
> state datastore which happens to include the applied configuration, and is
> also the same as what would be returned from a RESTCONF device today.

But this is not how the parameter is defined.

/js

---

From: Robert Wilton <[email-masked]>
Date: Tue, 21 Jun 2016 13:19:16 +0100

> the data; the source is still the "combination of config ans state".
OK, so perhaps refine/redefine that GET requests on "/restconf/data" are 
always against the operational state (inc applied config) datastore.  
I.e. with the content=config filter you just get applied-config.

POST/POST/etc requests would write to what I have labelled as the 
persistent config ds.  Semantically, this doesn't seem to be that 
different to writing to the candidate ds under the covers if necessary 
that RESTCONF does today.

I appreciate that this is not how this is defined at the moment, but 
this would still seem to be backwards compatible refinement.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Tue, 21 Jun 2016 13:44:15 +0100

On 21/06/2016 13:19, Juergen Schoenwaelder wrote:
> I fail to see why the # of datastores matters.
Because if you regard that for an existing server intended=applied, and 
ignore system created config, then for the definition that I propose:

running config + config=false nodes == operational state ds.

As such, I think that it should be possible to find a viable upgrade 
path from an existing NETCONF/RESTCONF server to the new datastore 
definitions.

I believe that with the definition of the operational-state ds that you 
have proposed, this is not true, i.e.
running config + config=false nodes != operational state ds.

I might be missing something, but it looks like upgrading to 
NETCONF/RESTCONF to the definition that you propose is significantly 
more work because the operational state ds doesn't necessarily reflect 
the actual running configuration at all (or certainly not in a way that 
a client can even programmatically rely on).

>   If the 'content'
Yes, thanks.  But RESTCONF GET requests could be refined to always be 
against the operational-state datastore in which case the filtering 
would work exactly as specified.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 14:46:59 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> OK, so perhaps refine/redefine that GET requests on "/restconf/data"
> are always against the operational state (inc applied config)
> datastore.  I.e. with the content=config filter you just get
> applied-config.

So how would you read what you call "persistent config db" below?

/martin

---

From: Robert Wilton <[email-masked]>
Date: Tue, 21 Jun 2016 13:55:04 +0100

> So how would you read what you call "persistent config db" below?
Via a GET operation on new datastore specific RESTCONF path.
E.g. "/restconf/ds/persistent/"

I would also allow, and recommend, POST/PUT requests on 
"/restconf/ds/persistent/".  POST/PUT requests on "/restconf/data" would 
be handled by the server as if the request had been made on 
"/restconf/ds/persistent/".

Similarly, a datastore specific path would be defined for the opstate 
datastore (e.g. /restconf/ds/opstate/):
- POST/PUT requests to this datastore are not allowed (it is read-only)
- GET requests to "/restconf/data" would be handled by the server as if 
the request had been made on "/restconf/ds/opstate/" instead.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Jun 2016 14:59:38 +0200 (CEST)

> E.g. "/restconf/ds/persistent/"

So this is clearly not backwards compatible either.  I don't think the
wilton solution is better in this regard than the schoenwaelder
solution.

/martin

> "/restconf/ds/persistent/".  POST/PUT requests on "/restconf/data"
> would be handled by the server as if the request had been made on
> - the request had been made on "/restconf/ds/opstate/" instead.
> 
> Thanks,
> Rob
> 
> >
> >

---

From: [email-masked] (Dale R. Worley)
Date: Tue, 21 Jun 2016 11:09:55 -0400

> something that should be changed.

Speaking based only on reading 6020bis and restconf, it seems to me that
the "device" has a "running" data tree, some items of which are
"configuration" (settable by Netconf et al.) and some of which are
"state" (set by the device).  There's also a concept of a "new" data
tree of configuration, which can be created in steps by protocol
operations by modifying an implicitly-created copy of the running
configuration data.  When the "new" tree is committed, it is checked for
consistency (the infamous "constraints"!) and written into the running
data tree.

I can see the value of this model, because it solves a lot of messy
problems arising from consistency requirements.

Within this model, Restconf never really sees this split between the
"running" and the "new" configuration, because after each Restconf
operation a commit is done.  (Which I take to imply that Restconf
operations edit a fresh copy of the running data tree.)

I can see the value of this model, because it means that Restconf
doesn't incorporate the complexity of managing transactions.

Within this, the meaning of the "content" parameter is simple, because
Restconf can't address a "new" data tree.

Dale

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 Jun 2016 13:15:39 +0200

> running config + config=false nodes == operational state ds.

I meant to say it is not relevant for this thread. I am not sure we
should hijack this thread to discuss different datastore proposals.

> As such, I think that it should be possible to find a viable upgrade path
> from an existing NETCONF/RESTCONF server to the new datastore definitions.
> 
> I believe that with the definition of the operational-state ds that you have
> proposed, this is not true, i.e.
> NETCONF/RESTCONF to the definition that you propose is significantly more
> work because the operational state ds doesn't necessarily reflect the actual
> running configuration at all (or certainly not in a way that a client can
> even programmatically rely on).

I think "my" proposed definition of an operational state datastore
pretty much matches what is called so far 'operational state' in the
NETCONF/YANG world.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 Jun 2016 13:19:12 +0200

On Tue, Jun 21, 2016 at 11:09:55AM -0400, Dale R. Worley wrote:
> "state" (set by the device).

This is not 100% correct. See the definitions of configuration data
and configuration datastore imported from RFC 6241.

/js

---

From: Robert Wilton <[email-masked]>
Date: Wed, 22 Jun 2016 17:13:00 +0100

> pretty much matches what is called so far 'operational state' in the
Sorry, I don't understand this.

draft-ietf-netmod-opstate-reqs defines Operational State as:

    Operational State:  Operational State is the current state of the
        system as known to the various components of the system (e.g.,
        control plane daemons, operating system kernels, line cards).
        The operational state includes both applied configuration and
        derived state.

The definition in draft-schoenw excludes the applied configuration as 
part of the operational state datastore which I would read as being 
inconsistent with the definition above.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 23 Jun 2016 22:49:19 +0200

On Wed, Jun 22, 2016 at 05:13:00PM +0100, Robert Wilton wrote:
> The definition in draft-schoenw excludes the applied configuration as part
> of the operational state datastore which I would read as being inconsistent
> with the definition above.

The <operational-state> datastore in the I-D represents _all_
operational state, irrespective of its origin. I think partitioning
the operational state is not useful. Again, this discussion does not
really belong into this thread.

/js