Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

From: Robert Wilton <[email-masked]>
Date: Thu, 1 Oct 2015 10:50:09 +0100

On 01/10/2015 00:55, Mahesh Jethanandani wrote:
> One comment.
>
>> On Sep 30, 2015, at 8:02 AM, Robert Wilton <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi Kent,
>>
>> Just some quick comments inline ...
>>
>> On 30/09/2015 15:31, Kent Watsen wrote:
>>> [As a contributor]
>>>
>>>
>>> I find that the term "system" is a bit ambiguous in this context.  
>>> It is talking about the NMS, the server, or both together?
>>>
>>> [KENT] I believe that we're talking about the NETCONF/RESTCONF/<foo> 
>>> server, specifically in how it processes update requests.
>>>
>>>
>>> Anyway, I've tried to define them relative to the configuration 
>>> operations being performed.
>>>
>>> [KENT] Perhaps these could be collapsed if we use the terms 
>>> "a/synchronous server" ?
>>
>> Personally, I think that defining the operations is arguably more 
>> useful because it is feasible that you could have a server that 
>> supports both sync and async operations and allows the client to 
>> choose which semantics should be used for a particular request.
>>
>>
>>>
>>>
>>> Synchronous configuration operation - A configuration request that 
>>> the server applies synchronously with respect to the client 
>>> request.  Before the server replies back to the client indicating 
>>> the success or failure of the operation it MUST semantically 
>>> validate the contents of the request and update the intended 
>>> configuration of the target datastore.  If the request is to the 
>>> running datastore then the configuration change MUST also be applied 
>>> in the system before the server replies back to the client.  The 
>>> reply to the client indicates whether there are any semantic errors 
>>> or system errors from applying the configuration change.
>>>
>>>
>>> [KENT]  This generally matches my understanding, but here are some 
>>> thoughts to improve it:
>>>
>>> 1. I think the language "semantically validate" would exclude an 
>>> ephemeral datastore.  We could put a qualifier on it, but I think it 
>>> can be removed entirely as each datastore already has its own 
>>> semantics around how it processes update requests.
>>
>> My aim here is potentially tied to the definition of intended config, 
>> were I am suggesting that the system has to at least have been 
>> checked that the request is well formed and any YANG constraints in 
>> the data model have been met, before it is accepted as intended 
>> config, otherwise it would be rejected.
>>
>>
>>>
>>> 2. I like how you call out the running datastore, but it is somewhat 
>>> NETCONF-specific.  How about something like "If the request impacts 
>>> the intended configuration (see term), then..."
>>
>> Yes your text is better and I agree that we should avoid making the 
>> description NETCONF specific at all.
>>
>>>
>>> 3. "applied in the system" seems too open ended, how about this: 
>>>  "...MUST also be propagated to and processed by the operational 
>>> components of the server before..." ???
>>
>> So my aim here was to tie it back to the definition of "applied 
>> configuration", but this could probably be expressed in a more direct 
>> way, e.g. perhaps ".... MUST update the applied configuration in the 
>> system before the server replies …"
>
> If I understand this correctly, we are still talking about “applied 
> configuration” as configuration that has been written to 
> datastore/hardware/line card etc. It does not imply that anything has 
> come out of it, including whether the configuration is usable. That 
> operating configuration (and I just introduced another term) will be 
> reflected by derived state.
>
> Is that true?
Yes.

Rather than operating configuration I would perhaps say the "observable 
system behaviour is reflected by derived state".

E.g. if you were to change the MTU of an interface to a different value 
(that was say incompatible with a peer device) then the "applied 
configuration" for the MTU leaf would only tell you whether or not that 
MTU was actively being used by the system.   It wouldn't tell you that 
an ISIS session on that interface had gone down due to the MTU 
incompatibility.  That could only be observed by the changing of the 
derived state representing the status of the ISIS session (or an alarm).

Thanks,
Rob

---

From: Kent Watsen <[email-masked]>
Date: Tue, 6 Oct 2015 00:52:13 +0000

Rob,

Can you take all the comments from this thread to update the proposed definitions for "synchronous server" and "asynchronous server" terms?

Thanks,
Kent

From: Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Thursday, October 1, 2015 at 5:50 AM
To: Mahesh Jethanandani <[email-masked]<mailto:[email-masked]>>
Cc: Kent Watsen <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

On Sep 30, 2015, at 8:02 AM, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

Hi Kent,

I find that the term "system" is a bit ambiguous in this context.  It is talking about the NMS, the server, or both together?

[KENT] I believe that we're talking about the NETCONF/RESTCONF/<foo> server, specifically in how it processes update requests.

Anyway, I've tried to define them relative to the configuration operations being performed.

[KENT] Perhaps these could be collapsed if we use the terms "a/synchronous server" ?

Personally, I think that defining the operations is arguably more useful because it is feasible that you could have a server that supports both sync and async operations and allows the client to choose which semantics should be used for a particular request.

Synchronous configuration operation - A configuration request that the server applies synchronously with respect to the client request.  Before the server replies back to the client indicating the success or failure of the operation it MUST semantically validate the contents of the request and update the intended configuration of the target datastore.  If the request is to the running datastore then the configuration change MUST also be applied in the system before the server replies back to the client.  The reply to the client indicates whether there are any semantic errors or system errors from applying the configuration change.

[KENT]  This generally matches my understanding, but here are some thoughts to improve it:

1. I think the language "semantically validate" would exclude an ephemeral datastore.  We could put a qualifier on it, but I think it can be removed entirely as each datastore already has its own semantics around how it processes update requests.

My aim here is potentially tied to the definition of intended config, were I am suggesting that the system has to at least have been checked that the request is well formed and any YANG constraints in the data model have been met, before it is accepted as intended config, otherwise it would be rejected.

2. I like how you call out the running datastore, but it is somewhat NETCONF-specific.  How about something like "If the request impacts the intended configuration (see term), then..."

Yes your text is better and I agree that we should avoid making the description NETCONF specific at all.

3. "applied in the system" seems too open ended, how about this:  "...MUST also be propagated to and processed by the operational components of the server before..." ???

So my aim here was to tie it back to the definition of "applied configuration", but this could probably be expressed in a more direct way, e.g. perhaps ".... MUST update the applied configuration in the system before the server replies …"

If I understand this correctly, we are still talking about “applied configuration” as configuration that has been written to datastore/hardware/line card etc. It does not imply that anything has come out of it, including whether the configuration is usable. That operating configuration (and I just introduced another term) will be reflected by derived state.

Is that true?
Yes.

Rather than operating configuration I would perhaps say the "observable system behaviour is reflected by derived state".

E.g. if you were to change the MTU of an interface to a different value (that was say incompatible with a peer device) then the "applied configuration" for the MTU leaf would only tell you whether or not that MTU was actively being used by the system.   It wouldn't tell you that an ISIS session on that interface had gone down due to the MTU incompatibility.  That could only be observed by the changing of the derived state representing the status of the ISIS session (or an alarm).

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Tue, 6 Oct 2015 14:59:29 +0100

Hi Kent,

Feeding in the various input, I think that this is the best refinement 
that I've come up with:

Synchronous configuration operation - A configuration request to update 
the running configuration of a server that is applied synchronously with 
respect to the client request.  The server SHOULD ensure that the 
request is valid, and MUST fully effect the configuration change to all 
impacted components in the server, updating both the server's intended 
and applied configuration (see terms), before replying to the client.  
The reply to the client indicates whether there are any errors in the 
request or errors from applying the configuration change.

Asynchronous configuration operation - A configuration request to update 
the running configuration of a server that is applied asynchronously 
with respect to the client request.  The server SHOULD ensure that the 
request is valid, and MUST update the server's intended configuration 
(see term) before replying to the client indicating whether the request 
will be processed.  The server's applied configuration state (see term) 
is updated after the configuration change has been fully effected to all 
impacted components in the server.  The reply to the client only 
indicates whether there are any errors in the original request.

Synchronous system -  Client/server interactions that process all 
configuration requests as synchronous configuration operations (see term).

Asynchronous system - Client/server interactions that process all 
configuration requests as asynchronous configuration operations (see term).

Any further comments?

Based on the feedback from Andy and others, it seems that the prevailing 
view is that existing NETCONF and RESTCONF should be regarded as being 
asynchronous in their behaviour in that the config nodes in the running 
datastore only represent the intended configuration and not the applied 
configuration.

Thanks,
Rob

On 06/10/2015 01:52, Kent Watsen wrote:
>
> Rob,
>
> Can you take all the comments from this thread to update the proposed 
> definitions for "synchronous server" and "asynchronous server" terms?
>
> Thanks,
> Kent
>
> From: Robert Wilton <[email-masked] <mailto:[email-masked]>>
> Cc: Kent Watsen <[email-masked] <mailto:[email-masked]>>, 
> "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous 
> vs asynchronous (esp. wrt intended and applied)
>
>
>
>>>> [KENT] I believe that we're talking about the 
>>>> NETCONF/RESTCONF/<foo> server, specifically in how it processes 
>>>> update requests.
>>>>
>>>>
>>>> running datastore then the configuration change MUST also be 
>>>> applied in the system before the server replies back to the 
>>>> client.  The reply to the client indicates whether there are any 
>>>> semantic errors or system errors from applying the configuration 
>>>> change.
>>>>
>>>>
>>>> ephemeral datastore.  We could put a qualifier on it, but I think 
>>>> it can be removed entirely as each datastore already has its own 
>>> My aim here is potentially tied to the definition of intended 
>>> config, were I am suggesting that the system has to at least have 
>>> been checked that the request is well formed and any YANG 
>>> constraints in the data model have been met, before it is accepted 
>>> as intended config, otherwise it would be rejected.
>>>
>>>
>>>>
>>>> 2. I like how you call out the running datastore, but it is 
>>>> somewhat NETCONF-specific.  How about something like "If the 
>>>> request impacts the intended configuration (see term), then..."
>>>
>>> configuration", but this could probably be expressed in a more 
>>> direct way, e.g. perhaps ".... MUST update the applied configuration 
>>> in the system before the server replies …"
>>
> Rather than operating configuration I would perhaps say the 
> "observable system behaviour is reflected by derived state".
>
> E.g. if you were to change the MTU of an interface to a different 
> value (that was say incompatible with a peer device) then the "applied 
> configuration" for the MTU leaf would only tell you whether or not 
> that MTU was actively being used by the system.   It wouldn't tell you 
> that an ISIS session on that interface had gone down due to the MTU

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Oct 2015 18:16:36 +0200

> request or errors from applying the configuration change.

What does "SHOULD ensure that the request is valid" mean? RFC 6020
says:

   When datastore processing is complete, the final contents MUST obey
   all validation constraints.  This validation processing is performed
   at differing times according to the datastore.  If the datastore is
   <running/> or <startup/>, these constraints MUST be enforced at the
   end of the <edit-config> or <copy-config> operation.  If the
   datastore is <candidate/>, the constraint enforcement is delayed
   until a <commit> or <validate> operation.

Are we talking about datastore validation or validation of a request?
If the former, are we watering down a MUST to a SHOULD?

> configuration requests as synchronous configuration operations (see term).

I think "synchronous system" is misleading, perhaps "synchronous
configuration server". The definitions talk about how a server
processes requests, they are silent about clients:

  Synchronous configuration server - a configuration server that
  processes all configuration operations as synchronous configuration
  operations.

> configuration requests as asynchronous configuration operations (see term).

  Asynchronous configuration server - a configuration server that
  processes all configuration operations as asynchronous configuration
  operations.

And I would not be surprised if we also need this sooner or later:

  Mixed configuration server - a configuration server that processes
  some configuration operations as synchronous configuration
  operations and some configuration operations as asynchronous
  configuration operations.

> datastore only represent the intended configuration and not the applied 

Depends on the definition of applied configuration - where is the latest
version of it?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Oct 2015 10:01:53 -0700

>   configuration operations.
As I already commented, this is how servers works today.
The data model details, server design, sub-system design, ASIC design, etc.
can all factor into whether or not the config is applied
when <ok/> is returned for the edit or commit.

It is also unwise to assume that the server can determine the applied
state for every possible leaf.

> Depends on the definition of applied configuration - where is the latest

Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 6 Oct 2015 21:42:37 +0100

Hi Juergen,

> If the former, are we watering down a MUST to a SHOULD?
Really it is datastore validation, particularly for an async request 
where the intended config nodes are updated before replying. I'm not 
intentionally trying to water down a MUST to a SHOULD, but Kent had 
concerns that my previous description using "semantically validate" 
would exclude an ephemeral datastore, so I was trying to water down the 
description and also describe the behaviour in a way that isn't specific 
to either RESTCONF/NETCONF or datastores.

Perhaps, the start of sentence could simply be changed to:

The server MUST fully effect the configuration change to all
impacted components in the server ...

And equivalently for the asynchronous case below:

The server MUST update the server's intended configuration ...

> processes requests, they are silent about clients:
I agree.  Some of the requirement text would probably need to be tweaked 
to accommodate this.

>    configuration operations.
Yes, I would expect that clients may want to define the expected 
behaviour, potentially on a per request basis.

> Depends on the definition of applied configuration - where is the latest
The last proposed text for intended/applied is as below:

       intended configuration - this data represents the configuration
       state that the network operator intends the system to be in, and 
that
       has been accepted by the system as valid configuration.  This 
data is
       colloquially referred to as the 'configuration' of the system.

      applied configuration - this data represents the configuration
       state that the network element is actually in, i.e., the
       configuration state which is currently being being used by system
       components (e.g., control plane daemons, operating system
       kernels, line cards).

 From Thursday's interim meeting, Rob Shakir clarified that the desired 
intention is that applied configuration should mean that the 
configuration is fully applied everywhere.  I don't know if that means 
that the definition of applied configuration should be strengthened, or 
if it is sufficient?

Thanks,
Rob

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Oct 2015 14:25:32 -0700

On Tue, Oct 6, 2015 at 1:42 PM, Robert Wilton <[email-masked]> wrote:

> Really it is datastore validation, particularly for an async request where
> the intended config nodes are updated before replying. I'm not
> concerns that my previous description using "semantically validate" would
> exclude an ephemeral datastore, so I was trying to water down the
> description and also describe the behaviour in a way that isn't specific to
> either RESTCONF/NETCONF or datastores.
>
>

I have been thinking about I2RS validation, and it there might be good
reasons
to say SHOULD instead of MUST (wrt/ ephemeral datastore anyway).

The server instrumentation can do a better job than the server at
deciding how to prune bad config.  It may be much faster to have the server
do syntax validation but not YANG constraint validation.  The
instrumentation
examines the ephemeral datastore to apply what it can. The rest gets
ignored.
This may be too sloppy for standards, but I2RS is supposed to be fast.

Andy

Perhaps, the start of sentence could simply be changed to:
>       state that the network operator intends the system to be in, and that
>       has been accepted by the system as valid configuration.  This data is
> intention is that applied configuration should mean that the configuration
> is fully applied everywhere.  I don't know if that means that the
> definition of applied configuration should be strengthened, or if it is
> sufficient?
>
> Thanks,
> Rob
>
>
>> /js
>>
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Oct 2015 10:48:14 +0200

> The server MUST update the server's intended configuration ...

Works for me. Sometimes less is more.

> behaviour, potentially on a per request basis.

I doubt that servers will offer clients to choose; I am more with Andy
that in real systems, depending on the data model, certain parts of a
data model may be synchronous while others may be asynchronous.

>       components (e.g., control plane daemons, operating system

Well, sometimes the config goes to a control plane daemon and then to
the OS kernel and finally to the line cards. This definition leaves
some room what an applied configuration is (which is IMHO needed if
you want to have something implementable) and hence a NC server can
either be considered synchronous or asynchronous.

> that the definition of applied configuration should be strengthened, or 

As said before, an OS kernel usually does not track where resource
parameters were coming from. (An interface has a set of IP addresses
and the kernel usually does not know which addresses were coming from
a configuration daemon, a dhcp daemon, a tunneling daemon, a command
line utility, ...) The same may be true for line card. So in order to
implement a very tight definition, one has to either 'guess' which
'subset' of operational state can be related 'somehow' to the intended
config and then report the result of the guess work as applied config
or one would have to to change daemons/kernels/linecards to have a
tracking number or something like that.

Anyway, as long as a regular NC/RC server does not have to pay a price
for this applied config idea, I have no real problem with this since I
am sure the market will sort this out.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Oct 2015 10:51:52 +0200

On Tue, Oct 06, 2015 at 02:25:32PM -0700, Andy Bierman wrote:
> reasons to say SHOULD instead of MUST (wrt/ ephemeral datastore anyway).

Ephemeral config is a different dimension that is yet to be worked
out. That said, RFC 6020 is quite clear what is expected from a
configuration datastore in terms of validation and we should not
change that.

> deciding how to prune bad config.  It may be much faster to have the
> server do syntax validation but not YANG constraint validation.  The
> instrumentation examines the ephemeral datastore to apply what it
> can. The rest gets ignored.  This may be too sloppy for standards,
> but I2RS is supposed to be fast.

Again, this is an ephemeral state issue. Do we bundle ephemeral state
into the open config discussion? I guess we need to do this but then
we also have a more complex set of players in this discussion.

/js

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Oct 2015 13:59:52 +0100

Hi,

> data model may be synchronous while others may be asynchronous.
I think that the question regarding synchronous vs asynchronous is 
really about what a client can usefully do with that information.

If a client knows that a request is synchronous then it doesn't have to 
poll or check whether the configuration has been applied.  Unless it 
gets told in the reply that some configuration failed, it should be able 
to assume that all the configuration has been successfully applied and 
it can proceed to the next step (if any) in provisioning a service.

If a client knows that a request is asynchronous then it has to poll (or 
rely on a notification) to determine when the configuration has been 
applied.  If, when it polls, parts of the config are applied and parts 
of it are not then it might be able to continue with some provisioning 
tasks but have to wait before it can continue with other parts of the 
provisioning.

If a server is a hybrid (where I define a hybrid server as one that 
replies somewhere between when a sync or async server would), then I 
would expect that a client would end up treating this the same as an 
asynchronous server.  I.e. it would expect to query to find out which 
parts of the configuration are applied and which are not.

But I agree that existing NC servers are potentially neither exactly 
synchronous or asynchronous in their behaviour and hence my trying to 
labelling them as such may be a mistake.  Hence, perhaps it would be 
better to classify servers as one of sync, async, hybrid?

> either be considered synchronous or asynchronous.
Yes, I understand your concern about having something implementable, but 
equally I think that the operators want a definition that is strong 
enough that they can usefully use it.  I also think that it is better to 
have a reasonably tight definition so that the implementers know what 
they should be aiming for, even if it isn't 100% achievable in all 
scenarios.

If we provide a definition of a hybrid server (i.e. somewhere between 
sync and async) then we could still have a tight definition of applied 
config, but aren't forcing any change in semantics to existing NC/RC 
servers since we just define their behaviour as being somewhere between 
an async and sync response.

> tracking number or something like that.

Inferring the applied config state from the operation state is one way 
of doing this.

An alternative way is for the NC/RC server to maintain separate internal 
state for intended vs applied cfg nodes.  Then the NC/RC server would 
update the applied cfg node when the internal IPC to program that 
configuration had completed for all affected subsystems.

> am sure the market will sort this out.
Agreed.

Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Oct 2015 21:13:50 +0200

> configuration had completed for all affected subsystems.

But the reality is that the NC/RC won't know. I invoke a system call
to tell the kernel what I want changed. The kernel sometimes says OK
before all components have been finally updated. The problem remains
to define what exactly applied config is. So are we only talking about
implementations that do not initiate the 'IPC' immediately or that
initiate the 'IPC' but may not wait for a completion of the 'IPC'?

Since this all sounds somewhat academic, do we have operational
evidence that NC/RC implementations do exhibit significant delays
between ACKing a config change and the config change becoming
communicated to the subsystems affected?

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Oct 2015 15:23:19 -0700

> initiate the 'IPC' but may not wait for a completion of the 'IPC'?
I agree -- even closed system code can be layered such that the
server does not know if the command has been fully applied or not.

> communicated to the subsystems affected?
I have asked this question several times, but no answer so far.
It certainly depends on the data models and the implementation.
How slow is slow enough so that retrieving a lot of meta-data is
worthwhile?
5 sec? 5 min? 5 hours?  Nobody seems to know.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Oct 2015 15:51:35 +0100

On 13/10/2015 23:23, Andy Bierman wrote:
>
>
> On Tue, Oct 13, 2015 at 12:13 PM, Juergen Schoenwaelder 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     to define what exactly applied config is. 
>

>     So are we only talking about
In essence, yes, that is my opinion.

> server does not know if the command has been fully applied or not.

I'm not sure whether it really matters whether the configuration is 
fully applied in this scenario.  The server can only do the best that it 
can and mark the config as being applied once it thinks that it has been.

To me the key question is what can the NMS observe.  If after querying 
(or being notified) that the configuration has been applied it can 
detect that configuration isn't fully in effect then I could imagine 
that might be raised as an issue.

Personally, I don't mind if the description of the applied configuration 
is defined as something like "as far as possible the configuration has 
been applied everywhere."  But equally, I believe that the previously 
proposed descriptions (below) are good enough for the requirements draft.

So, is this text above acceptable, or does it need to be refined?

>     communicated to the subsystems affected?
I don't have any operational evidence for NC, but I do for directly 
equivalent CLI configuration commits:

For some of the larger routers, the configuration files can be over 1M 
lines long, potentially programming 100K+ interfaces/sub-interfaces, all 
of which can take 10's of minutes to apply the configuration depending 
on the configuration complexity and feature set.

Today, the NC reply would be near the end of applying that 
configuration, but it is depends on the affected subsystems.  Some 
subsystems process the configuration synchronously, some asynchronously, 
and for some it is a combination of the both (but normally with the bulk 
of it being synchronous).

The time for one of those asynchronous subsystems processing a config 
change could be several minutes, e.g if the membership of a LAG 
interface was changed to add physical member interfaces on a new 
linecard.  I.e. the side effect of a 1 line configuration change could 
result in the programmatic equivalent of 100,000's lines of 
configuration to be sent to a linecard.

> 5 sec? 5 min? 5 hours?  Nobody seems to know.
OK, I don't know, but my guess would be 5 sec or less, on the following 
premise:

  - the large data centre providers and service providers want to be 
able to re-configure and optimize their networks on the fly in a similar 
way that virtual servers can be spun up/down today.

  - for such systems where lots of independent services may be being 
provisioned/unprovisioned it is easier for the management clients to 
program the configuration changes in an asynchronous fashion, by 
streaming config updates to the network elements.  I.e. ideally the 
client does not want an NC server to block on each config change, it 
wants them to be processed concurrently, with the guarantee that the 
config updates are logically applied in order.

  - as soon as config is handled in an async fashion the client needs a 
way to determine when the configuration has actually been applied, so 
that it knows whether a particular service has been configured and is 
ready for the next stage of provisioning.

Perhaps your question could also be asked in tomorrow's interim?

Thanks,
Rob

>     [email-masked] <mailto:[email-masked]>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Oct 2015 15:59:17 +0100

Hi All,

Are there any more comments on the following proposed descriptions, or
are these descriptions sufficiently clear to update the requirements
draft and resolve issue #6?

Synchronous configuration operation - A configuration request to update
the running configuration of a server that is applied synchronously with
respect to the client request.  The server MUST fully effect the
configuration change to all impacted components in the server, updating
both the server's intended and applied configuration (see terms), before
replying to the client.  The reply to the client indicates whether there
are any errors in the request or errors from applying the configuration
change.

Asynchronous configuration operation - A configuration request to update
the running configuration of a server that is applied asynchronously
with respect to the client request.  The server MUST update its intended
configuration (see term) before replying to the client indicating
whether the request will be processed.  The server's applied
configuration state (see term) is updated after the configuration change
has been fully effected to all impacted components in the server.  The
reply to the client only indicates whether there are any errors in the
original request.

Synchronous configuration server - a configuration server that processes
all configuration operations as synchronous configuration operations.

Asynchronous configuration server - a configuration server that processes
all configuration operations as asynchronous configuration operations.

Thanks,
Rob

---

From: Kent Watsen <[email-masked]>
Date: Wed, 14 Oct 2015 15:25:35 +0000

>am sure the market will sort this out.

This goes to the solution - that it should allow servers to opt-in to
support applied config.

I have also been thinking about market response.  It is a bit frustrating
to even think that solution might not be widely popular, but the reality
is that I don't see how it can be widely implemented - and so maybe it
will only be popular for datacenter operators?

Kent // as a contributor

---

From: Kent Watsen <[email-masked]>
Date: Wed, 14 Oct 2015 22:11:20 +0000

Hi Robert, 

One comment, it seems that the asynchronous configuration operation should
have one more sentence at its end saying that an asynchronous notification
must be used to report any errors produced from when applying the
configuration.

What do you think?

Kent  // as a contributor

---

From: Gert Grammel <[email-masked]>
Date: Thu, 15 Oct 2015 11:17:26 +0000

Kent, Rob,

Comparing the cases, the definition of the asynchronous case doesn’t look complete. The way it stands for the synchronous operation, the client knows that it's intended config was good AND it has been effected to the server. In the Asynchronous case, the client only knows that the intended config was good – and that’s not enough.

So the proposal is to refine the asynchronous case definition as follows:

Asynchronous configuration operation - A configuration request to update the running configuration of a server that is applied asynchronously with respect to the client request.  The server MUST update its intended configuration (see term) before replying to the client indicating whether the request will be processed.  The reply to the client only indicates whether there are any errors in the  original request.
The server's applied configuration state (see term) is updated after the configuration change has been applied to all impacted components in the server.  Once applied, the client MUST be notified whether the intended config is now fully effective or there are any errors from applying the configuration change.

In addition I would suggest to require a verify operation which a client can request from the server to obtain above information on an on-demand basis. Would that somehow go in the opstate-reqs #3 then?

Best

Gert

From: netmod <[email-masked]<mailto:[email-masked]>> on behalf of Kent Watsen <[email-masked]<mailto:[email-masked]>>
Date: Thursday 15 October 2015 00:11
To: Robert Wilton <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Hi Robert,

What do you think?

Kent  // as a contributor

On 10/14/15, 10:59 AM, "Robert Wilton" <[email-masked]<mailto:[email-masked]>> wrote:

Hi All,

Thanks,
Rob

Feeding in the various input, I think that this is the best
refinement
that I've come up with:

Synchronous configuration operation - A configuration request to
update
the running configuration of a server that is applied synchronously
with
respect to the client request.  The server SHOULD ensure that the
request is valid, and MUST fully effect the configuration change to
all
impacted components in the server, updating both the server's
intended
and applied configuration (see terms), before replying to the client.
The reply to the client indicates whether there are any errors in the
request or errors from applying the configuration change.
What does "SHOULD ensure that the request is valid" mean? RFC 6020
says:

     When datastore processing is complete, the final contents MUST
obey
     all validation constraints.  This validation processing is
performed
     at differing times according to the datastore.  If the datastore
is
     <running/> or <startup/>, these constraints MUST be enforced at
the
     end of the <edit-config> or <copy-config> operation.  If the
     datastore is <candidate/>, the constraint enforcement is delayed
     until a <commit> or <validate> operation.

description and also describe the behaviour in a way that isn't
specific

Any further comments?

Based on the feedback from Andy and others, it seems that the
prevailing
view is that existing NETCONF and RESTCONF should be regarded as
being
asynchronous in their behaviour in that the config nodes in the
running
datastore only represent the intended configuration and not the
applied
configuration.
Depends on the definition of applied configuration - where is the
latest
version of it?
The last proposed text for intended/applied is as below:

        configuration state which is currently being being used by
system

/js

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: Kent Watsen <[email-masked]>
Date: Thu, 15 Oct 2015 13:42:01 +0000

Thanks Gert.  I've incorporated these suggestions into my notes for today's interim meeting.

From: Gert Grammel <[email-masked]<mailto:[email-masked]>>
Date: Thursday, October 15, 2015 at 7:17 AM
To: Kent Watsen <[email-masked]<mailto:[email-masked]>>, Robert Wilton <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Kent, Rob,

Best

Gert

Hi Robert,

What do you think?

Kent  // as a contributor

Hi All,

Thanks,
Rob

Any further comments?

/js

---

From: Robert Wilton <[email-masked]>
Date: Thu, 15 Oct 2015 14:59:37 +0100

Hi Gert, Kent,

I think that notifying the client is one possible way that an 
asynchronous request could be completed, but not necessarily the only 
way.  E.g. if the information as to whether a particular intended leave 
had been successfully applied was available (e.g. as per github issue #2).

So, I wonder whether we shouldn't weaken the last sentence, changing 
MUST to COULD, or perhaps reword it.:

E.g. replacing:

Once applied, the client MUST be notified whether the intended config is 
now fully effective or there are any errors from applying the 
configuration change.

Perhaps with:

Once applied, the client COULD be notified whether the intended config 
is now fully effective or there are any errors from applying the 
configuration change.

Or:

Once applied, there MUST be a mechanism available for the client to be 
able to determine whether the intended config is now fully effective or 
whether there are any errors from applying the configuration change.

Thanks,
Rob

On 15/10/2015 12:17, Gert Grammel wrote:
> Kent, Rob,
>
> Comparing the cases, the definition of the asynchronous case doesn’t 
> look complete. The way it stands for the synchronous operation, the 
> client knows that it's intended config was good AND it has been 
> effected to the server. In the Asynchronous case, the client only 
> knows that the intended config was good – and that’s not enough.
>
> Asynchronous configuration operation - A configuration request to 
> update the running configuration of a server that is applied 
> asynchronously with respect to the client request.  The server MUST 
> update its intended configuration (see term) before replying to the 
> client indicating whether the request will be processed.  The reply to 
> the client only indicates whether there are any errors in the 
>  original request.
> The server's applied configuration state (see term) is updated after 
> the configuration change has been applied to all impacted components 
> in the server.  Once applied, the client MUST be notified whether the 
> intended config is now fully effective or there are any errors from 
> applying the configuration change.
>
> In addition I would suggest to require a verify operation which a 
> client can request from the server to obtain above information on an 
> on-demand basis. Would that somehow go in the opstate-reqs #3 then?
>
> Best
>
> Gert
>
>
>
>
> From: netmod <[email-masked] 
> <mailto:[email-masked]>> on behalf of Kent Watsen 
> <[email-masked] <mailto:[email-masked]>>
> To: Robert Wilton <[email-masked] <mailto:[email-masked]>>, 
>     processes
>     processes

---

From: Gert Grammel <[email-masked]>
Date: Thu, 15 Oct 2015 14:35:02 +0000

Rob,

>From a client perspective a server has three states:

  1.  intended != applied
  2.  Funny-state
  3.  Intended == applied

Irrespective of synchronous or asynchronous processing in the server, the third state MUST be reported to the client. Else (from a client perspective) the server stays in funny-state forever.

Gert

From: Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Thursday 15 October 2015 15:59
To: Gert Grammel <[email-masked]<mailto:[email-masked]>>, Kent Watsen <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Hi Gert, Kent,

I think that notifying the client is one possible way that an asynchronous request could be completed, but not necessarily the only way.  E.g. if the information as to whether a particular intended leave had been successfully applied was available (e.g. as per github issue #2).

So, I wonder whether we shouldn't weaken the last sentence, changing MUST to COULD, or perhaps reword it.:

E.g. replacing:

Once applied, the client MUST be notified whether the intended config is now fully effective or there are any errors from applying the configuration change.

Perhaps with:

Once applied, the client COULD be notified whether the intended config is now fully effective or there are any errors from applying the configuration change.

Or:

Once applied, there MUST be a mechanism available for the client to be able to determine whether the intended config is now fully effective or whether there are any errors from applying the configuration change.

Thanks,
Rob

Best

Gert

From: netmod <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>> on behalf of Kent Watsen <[email-masked]<mailto:[email-masked]>>
Date: Thursday 15 October 2015 00:11
To: Robert Wilton <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Hi Robert,

What do you think?

Kent  // as a contributor

On 10/14/15, 10:59 AM, "Robert Wilton" <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>> wrote:

Hi All,

Thanks,
Rob

Any further comments?

/js

---

From: Kent Watsen <[email-masked]>
Date: Thu, 15 Oct 2015 17:03:33 +0000

These terms were edited on today's call, resulting in the following text:

    Synchronous configuration operation - A configuration request to update
    the running configuration of a server that is applied synchronously with
    respect to the client request.  The server MUST fully attempt to apply
    the configuration change to all impacted components in the server,
    updating both the server's intended and applied configuration (see terms),
    before replying to the client.  This may be transactional or non-
    transactional (need terms!).  The transactionality of the operation
    may be a server property or specified on as a property.
    The reply to the client indicates whether there
    are any errors in the request or errors from applying the configuration
    change.

    Asynchronous configuration operation - A configuration request to update
    the running configuration of a server that is applied asynchronously
    with respect to the client request.  The server MUST update its intended
    configuration (see term) before replying to the client indicating
    whether the request will be processed.  This reply to the client only
    indicates whether there are any errors in the  original request.  The
    server's applied configuration state (see term) is updated after the
    configuration change has been fully effected to all impacted components
    in the server.  Once applied, there MUST be a mechanism for the client to
    determine when the request has completed processing and whether the
    intended config is now fully effective or there are any errors from
    applying the configuration change, which could be from an asynchronous
    notification or via a client operation.

We have consensus on the above, but are hoping for some word-smithing before committing it to the draft.  Anybody want to take a crack at it?

BTW, do we still need to define the last two terms anymore?  - they're not used elsewhere in the draft...

Kent

From: Gert Grammel <[email-masked]<mailto:[email-masked]>>
Date: Thursday, October 15, 2015 at 10:35 AM
To: Robert Wilton <[email-masked]<mailto:[email-masked]>>, Kent Watsen <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Rob,

  1.  intended != applied
  2.  Funny-state
  3.  Intended == applied

Gert

Hi Gert, Kent,

E.g. replacing:

Perhaps with:

Or:

Thanks,
Rob

Best

Gert

Hi Robert,

What do you think?

Kent  // as a contributor

Hi All,

Thanks,
Rob

Any further comments?

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 15 Oct 2015 23:59:27 +0200

>     may be a server property or specified on as a property.

I do not understand the transactionality blurb. What is it and why is
it relevant? (The last sentence above also seems incomplete.)

> BTW, do we still need to define the last two terms anymore?  - they're not used elsewhere in the draft...

If we do not need the terms sync/async config server, simply remove
them. If we keep them, I suggest to add the mixed configuration server
- a configuration server that processes some configuration operations
as synchronous configuration operations and some configuration
operations as asynchronous configuration operations. Pure sync/async
servers are likely rare.

/js

---

From: Gert Grammel <[email-masked]>
Date: Thu, 15 Oct 2015 23:07:39 +0000

Kent,

I don't see the need for defining synchronous/asynchronous config servers.

The new definitions look good. Later in the discussion there was a common sentiment that the requirement for synchronous operations contained some crisp wording we could use here too.
I particularly liked the mentioning of blocking requests for some time,

Nevertheless the proposed wording works for me.

Gert

Sent from my Apple ][

On 15 Oct 2015, at 19:03, Kent Watsen <[email-masked]<mailto:[email-masked]>> wrote:

Kent

Rob,

  1.  intended != applied
  2.  Funny-state
  3.  Intended == applied

Gert

Hi Gert, Kent,

E.g. replacing:

Perhaps with:

Or:

Thanks,
Rob

Best

Gert

Hi Robert,

What do you think?

Kent  // as a contributor

Hi All,

Thanks,
Rob

Any further comments?

/js

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 13:08:37 +0100

Hi Kent, Gert,

Balazs pointed out, and I agree, that the text about transaction/not 
transactional can equally apply to both synchronous and asynchronous 
configuration operations.

So rather than reproducing this text twice, once for each configuration 
definition, I propose keeping the transactional text out of the 
definition for synchronous/asynchronous configuration operations and 
instead include it as a sub-bullet for the section 3 requirements.  I'll 
send my proposed text for the section 3 requirements on that separate 
email thread.

Hence, I propose that we revert to this text for synchronous 
configuration operation:

     Synchronous configuration operation - A configuration request to update
     the running configuration of a server that is applied synchronously 
with
     respect to the client request.  The server MUST fully attempt to apply
     the configuration change to all impacted components in the server,
     updating both the server's intended and applied configuration (see 
terms),
     before replying to the client.  The reply to the client indicates 
whether there
     are any errors in the request or errors from applying the configuration
     change.

I agree with the existing proposed text for asynchronous configuration 
operation and also support the proposal to drop the definition of 
synchronous/asynchronous configuration servers if they are not being used.

Thanks,
Rob

On 15/10/2015 18:03, Kent Watsen wrote:
> We have consensus on the above, but are hoping for some word-smithing 
> before committing it to the draft.  Anybody want to take a crack at it?
>
> From: Gert Grammel <[email-masked] <mailto:[email-masked]>>
> To: Robert Wilton <[email-masked] <mailto:[email-masked]>>, Kent 
> Watsen <[email-masked] <mailto:[email-masked]>>, 
> Irrespective of synchronous or asynchronous processing in the server, 
> the third state MUST be reported to the client. Else (from a client 
> perspective) the server stays in funny-state forever.
>
>
> Gert
>
>
> To: Gert Grammel <[email-masked] <mailto:[email-masked]>>, 
> Kent Watsen <[email-masked] <mailto:[email-masked]>>, 
> way.  E.g. if the information as to whether a particular intended 
> leave had been successfully applied was available (e.g. as per github 
> issue #2).
>
> Once applied, the client MUST be notified whether the intended config 
> able to determine whether the intended config is now fully effective 
> or whether there are any errors from applying the configuration change.
>
> Thanks,
> Rob
>
>
>> client indicating whether the request will be processed.  The reply 
>> to the client only indicates whether there are any errors in the 
>>  original request.
>> From: netmod <[email-masked]> on behalf of Kent Watsen 
>> To: Robert Wilton <[email-masked]>, "[email-masked] 
>> <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>>                     are any errors in the

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 13:19:01 +0100

Hi Juergen,

> it relevant? (The last sentence above also seems incomplete.)

It was added in the meeting because some interpretations of the text 
assumed that the text implied that the server would always 
rollback-on-error, so the proposed text was intended to clarify that.

However, I think that this clarification applies equally to both sync 
and async operations and hence I have proposed (on a separate thread) 
that it be documented as part of the requirement 3 "Support for both 
synchronous and asynchronous configuration operations" instead.

Thanks,
Rob

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 13:35:39 +0000

>synchronous and asynchronous configuration operations" instead.

So the proposal is to remove the last two sentences? - I'm OK with that.

I think that the new word "attempt" in the previous sentence fixed it to
be (IMO) a very good definition for what is meant by a "synchronous
configuration operation" without it getting mired in details.

Kent

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 13:45:13 +0000

> I don't see the need for defining synchronous/asynchronous config servers.

Thank you (and Juergen) for the confirmation.   Again, if no objections, these two terms will not be removed.

> The new definitions look good. Later in the discussion there was a common
> sentiment that the requirement for synchronous operations contained some
> crisp wording we could use here too.  I particularly liked the mentioning of
> blocking requests for some time,

[Note: the following removes the last two sentences on "transactions", as being discussed elsewhere on this thread]

OLD:

    before replying to the client.

NEW

    Synchronous configuration operation - A configuration request to update
    the running configuration of a server that is applied synchronously with
    respect to the client request (i.e. a blocking call).  The server MUST fully attempt to apply
    the configuration change to all impacted components in the server,
    updating both the server's intended and applied configuration (see terms),
    before replying to the client.

What do you think?

Kent

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 14:12:19 +0000

Juergen,

Rob summarized the discussion well. Since I also have some strange feelings about transactions here, my proposal in the other thread was to define the state of the config at the time the client is notified.

Gert

Sent from my Apple ][

> On 16 Oct 2015, at 14:19, Robert Wilton <[email-masked]> wrote:
> 
> Hi Juergen,
> 
> It was added in the meeting because some interpretations of the text assumed that the text implied that the server would always rollback-on-error, so the proposed text was intended to clarify that.
> 
> However, I think that this clarification applies equally to both sync and async operations and hence I have proposed (on a separate thread) that it be documented as part of the requirement 3 "Support for both synchronous and asynchronous configuration operations" instead.
> 
> Thanks,
> Rob
>

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 14:16:51 +0000

Kent,

The new one looks much better. However the last sentence is confusing with respect to intended config. Why is there a need to update the intended config?

Proposal:
The server MUST fully attempt to apply
    the configuration change to all impacted components in the server,
    updating the server's applied configuration (see terms),
    before replying to the client.

Sent from my Apple ][

On 16 Oct 2015, at 15:45, Kent Watsen <[email-masked]<mailto:[email-masked]>> wrote:

OLD:

NEW

What do you think?

Kent

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 15:19:47 +0000

> Why is there a need to update the intended config?

Because that is what happens via requests like <edit-config> and PATCH.  The intended (running) config gets updated first and then config is distributed to internal components, ultimately updated the applied config.

Kent  // as a contributor

From: Gert Grammel <[email-masked]<mailto:[email-masked]>>
Date: Friday, October 16, 2015 at 10:16 AM
To: Kent Watsen <[email-masked]<mailto:[email-masked]>>
Cc: Robert Wilton <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #6: clarify impact of synchronous vs asynchronous (esp. wrt intended and applied)

Kent,

Proposal:

Sent from my Apple ][

OLD:

NEW

What do you think?

Kent

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 17:36:07 +0100

On 16/10/2015 14:45, Kent Watsen wrote:
> Thank you (and Juergen) for the confirmation.   Again, if no 
> objections, these two terms will not be removed.
>
>
> [Note: the following removes the last two sentences on "transactions", 
> as being discussed elsewhere on this thread]
>
> OLD:
>
Yes, looks good.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 16:52:28 +0000

Ok,

The term 'intended config is updated' is probably correct but sounds a bit odd to me.    I associate with "update" a process of alignment. In the case of intended config it is more like a config that is imposed by the client. Later the update of the applied config is started.

Certainly a terminology issue, but I wanted to bring it up maybe others apply similar semantics.

Gert

Sent from my Apple ][

Kent  // as a contributor

Kent,

Proposal:

Sent from my Apple ][

OLD:

NEW

What do you think?

Kent

---

From: Kent Watsen <[email-masked]>
Date: Mon, 19 Oct 2015 13:49:22 +0000

Thanks you all for the word smithing.  The final draft text follows:

   This document defines the following terms:

   Synchronous configuration operation:   A configuration request to
       update the running configuration of a server that is applied
       synchronously with respect to the client request (i.e. a blocking
       call).  The server MUST fully attempt to apply the configuration
       change to all impacted components in the server, updating both
       the server's intended and applied configuration (see terms),
       before replying to the client.  The reply to the client indicates
       whether there are any errors in the request or errors from
       applying the configuration change.

   Asynchronous configuration operation:   A configuration request to
       update the running configuration of a server that is applied
       asynchronously with respect to the client request.  The server
       MUST update its intended configuration (see term) before replying
       to the client indicating whether the request will be processed.
       This reply to the client only indicates whether there are any
       errors in the original request.  The server's applied
       configuration state (see term) is updated after the configuration
       change has been fully effected to all impacted components in the
       server.  Once applied, there MUST be a mechanism for the client
       to determine when the request has completed processing and
       whether the intended config is now fully effective or there are
       any errors from applying the configuration change, which could be
       from an asynchronous notification or via a client operation.

Issue #6 will be moved to the REVIEW state - as the draft will be out shortly.

Thanks again,
Kent  // as a contributor

Date: Friday, October 16, 2015 at 12:52 PM

Ok,

Gert

Sent from my Apple ][

Kent  // as a contributor

Kent,

Proposal:

Sent from my Apple ][

OLD:

NEW

What do you think?

Kent