Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Oct 2015 23:06:32 +0100

On 28/09/2015 09:40, Juergen Schoenwaelder wrote:
> On Wed, Sep 23, 2015 at 03:03:57PM +0000, Kent Watsen wrote:
>> Popping the stack on this issue, the issue remains as to what to do with requirement 3:
>>
>>     3.  Support for both transactional, synchronous management systems
>>          as well as distributed, asynchronous management systems
>>
> I fail to understand 'transactional' and 'distributed' here. And
> frankly, I am not sure why the management _systems_ are classified to
> be synchronous or asynchronous - I think we are talking about
> protocols between a management system and a device. In NETCONF as well
> as RESTCONF are, you can pipeline YANG operations but all operations
> are processed sequentially in order.
I basically agree, and hence I think that we can just delete requirement 
3 above.

I'm not convinced that "transactional" or "distributed" are relevant to 
the opstate requirements, I expect that these terms were originally used 
to help provide a distinction between Netconf style interactions and 
whatever management systems the operators are currently considering of 
building with proprietary protocols.

Thanks,
Rob

>
> There are apparently other proprietary protocols that do things
> differently but so far we have not seen a specification of how this
> works.
>
> Anyway, I am not sure 3. is properly worded until someone defines
> 'transactional', 'distributed', 'synchronous management systems' and
> 'asynchronous management systems'.

> /js

---

From: Kent Watsen <[email-masked]>
Date: Wed, 14 Oct 2015 23:51:34 +0000

On 9/28/15, 1:40 AM, "Juergen Schoenwaelder"
<[email-masked]> wrote:

>I fail to understand 'transactional' and 'distributed' here.

I hope that these terms will be clarified on tomorrow's call.   There is
also a chance that these terms will be removed from the text altogether,
as they may be viewed as unnecessarily qualify the
synchronous/asynchronous terms.

> And
>protocols between a management system and a device.

Aye, I didn't see that before.

First off, elsewhere in the draft the term "system" is used 7 times to
refer to the device (e.g., NC/RC server).  The term "system" is otherwise
not defined.

But to your main point, we have been discussing the terms a/synchronous to
have to do with internal server processing of an edit request, but in '3'
the terms are being used to qualify a management system, which can't be
right.  I think that '3' should be rewritten to be a statement about
devices, not a statement about management systems.

>'asynchronous management systems'.

The agenda for tomorrow's interim!  :)

Kent

---

From: Nadeau Thomas <[email-masked]>
Date: Wed, 14 Oct 2015 20:00:51 -0400

> On Oct 14, 2015:7:51 PM, at 7:51 PM, Kent Watsen <[email-masked]> wrote:
> 
> 
> 

	It might be better to frame this in terms of a client and a
server.

	—Tom

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Kent Watsen <[email-masked]>
Date: Thu, 15 Oct 2015 17:22:37 +0000

Requirement #3 was discussed on today's call.   We agreed to remove the
words "distributed" and "transactional" and to reword it in terms of
"configuration operations".  The resulting text follows:

   3.  Support for both synchronous and asynchronous configuration
operations (see terms)

       A. A server may only support synchronous configuration operations,
or may only support
          asynchronous configuration operations, or may support
synchronicity to be client
          specified on a per-operation basis.

       C. Support for synchronous configuration operations requires the
server to block
          sending a response to the client until it is able to able to
determine whether
          there are any errors in the request or errors from applying the
configuration
          change.
    
       D. Support for asynchronous configuration operations requires the
server to send
          a response to the client immediately indicated that the request
was accepted
          and send a notification to the client when the intended config
is fully 
          effective or there are any errors from applying the
configuration change.

       E. Support for asynchronous configuration operations MAY also
provide a verify
          operation which a client can request from the server to obtain
information
          regarding the difference between the intended and applied
configurations.

We have consensus on the above, but wanted to rewrite it relying more on
the terms from the Terminology section, and also potentially merge E into
D. 

Anybody want to take a stab at it?

Thanks,
Kent

On 10/14/15, 8:00 PM, "Nadeau Thomas" <[email-masked]> wrote:

>	‹Tom

---

From: Kent Watsen <[email-masked]>
Date: Thu, 15 Oct 2015 17:32:55 +0000

Again, with better formatting for the list:

       A. A server may only support synchronous configuration
          operations, or may only support asynchronous configuration
          operations, or may support synchronicity to be client
          specified on a per-operation basis.

       C. Support for synchronous configuration operations
          requires the server to block sending a response to
          the client until it is able to able to determine whether
          there are any errors in the request or errors from
          applying the configuration change.
    
       D. Support for asynchronous configuration operations
          requires the server to send a response to the client
          immediately indicated that the request was accepted
          and send a notification to the client when the intended
          config is fully effective or there are any errors from
          applying the configuration change.

       E. Support for asynchronous configuration operations MAY
          also provide a verify operation which a client can request
          from the server to obtain information regarding the
          difference between the intended and applied configurations.

Kent

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 13:12:35 +0100

Hi Kent,

Here is my attempt at word smithing section 3:

The old D and E have been merged together (now labelled as C).  A new D 
has been added to try and define transactional error handling semantics 
without introducing the term transactional.

        operations

        A. A server may choose to support only synchronous configuration
           operations, or only asynchronous configuration operations, or
           both synchronous and asynchronous configuration operations in
           a client specified per-operation basis.

        B. Support for synchronous operations as per the definition of
           "synchronous configuration operation".

        C. Support for asynchronous operations as per the definition of
           "asynchronous configuration operation".  Servers that support
           asynchronous configuration operations MAY also provide a
           verify operation that a client can request from the server to
           return information regarding the difference between the
           intended and applied configurations.

        D. Support for best effort and rollback-on-error error handling
           semantics.  The configuration protocol, or default server
           behavior, MUST specify whether the configuration is applied
           in a best effort fashion, or using "rollback on error"
           semantics - where all configuration changes in the request are
           undone if processing of any part of the configuration update
           failed.  A configuration protocol, or server, SHOULD provide
           support for rollback-on-error behavior and MAY choose to
           provide support for best effort semantics as well.

Any comments?

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Oct 2015 14:23:15 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> The old D and E have been merged together (now labelled as C).  A new
> D has been added to try and define transactional error handling
> semantics without introducing the term transactional.
> 
> 

I think the most common mode *today* is a mix of sync/async, on a
per-object basis.  Is this mode no longer valid?

>           "synchronous configuration operation".

Doesn't this follow from A?

>           provide support for best effort semantics as well.

/martin

> >>>> But to your main point, we have been discussing the terms
> >>>> a/synchronous
> >>>> to

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 12:58:34 +0000

Hi Rob,

The text looks good. With respect to D we probably need more wordsmithing:

 D. Support for best effort and rollback-on-error error handling
          semantics.  The configuration protocol, or default server
          behavior, MUST specify whether the configuration is applied
          in a best effort fashion, or using "rollback on error"
          semantics - where all configuration changes in the request are
          undone if processing of any part of the configuration update
          failed.
--> how would the client know that a certain operation is supported in an atomic or non-atomic manner by the server. In my view the default should be the client assumes best effort and the server is allowed to do better.
Hence no need for a MUST.

A configuration protocol, or server, SHOULD provide
          support for rollback-on-error behavior and MAY choose to
          provide support for best effort semantics as well.

Wondering how much we need here. Assuming the server supports rollback, then do we need protocol extensions for that?
Assuming the server doesn't support rollback. Then do we need anything special in the protocol to re-configure the config to emulate something like a rollback?

My point is: given that transaction is a requirement, would we need to require here anything more that is not a consequence?

Perhaps it is better to limit or even avoid text about transactions and just define the state of the applied config after error. I see 3 cases so far:
1) atomic: after error the applied config is identical to the config before the error
2) sequence: if the config is applied in a sequence and a specific leaf fails, the leafs that have been configured before will keep the intended config, leafs not yet processed keep the applied config and the failed leaf is undefined
3) all leafs in error are undefined, all error free leafs use the (new) intended config as their applied config.

Thoughts?

Gert

Sent from my Apple ][

On 16 Oct 2015, at 14:13, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

Hi Kent,

The old D and E have been merged together (now labelled as C).  A new D has been added to try and define transactional error handling semantics without introducing the term transactional.

Any comments?

Thanks,
Rob

Kent

On 10/15/15, 1:22 PM, "Kent Watsen" <[email-masked]><mailto:[email-masked]> wrote:

Thanks,
Kent

On 10/14/15, 8:00 PM, "Nadeau Thomas" <[email-masked]><mailto:[email-masked]> wrote:

On Oct 14, 2015:7:51 PM, at 7:51 PM, Kent Watsen <[email-masked]><mailto:[email-masked]>
wrote:

On 9/28/15, 1:40 AM, "Juergen Schoenwaelder"
<[email-masked]><mailto:[email-masked]> wrote:

Popping the stack on this issue, the issue remains as to what to do
with requirement 3:

I hope that these terms will be clarified on tomorrow's call.   There
is
also a chance that these terms will be removed from the text
altogether,
as they may be viewed as unnecessarily qualify the
synchronous/asynchronous terms.

And

First off, elsewhere in the draft the term "system" is used 7 times to
refer to the device (e.g., NC/RC server).  The term "system" is
otherwise
not defined.

But to your main point, we have been discussing the terms a/synchronous
to
have to do with internal server processing of an edit request, but in
'3'
the terms are being used to qualify a management system, which can't be
right.  I think that '3' should be rewritten to be a statement about
devices, not a statement about management systems.

        ‹Tom

Kent

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 14:09:11 +0100

Hi Martin,

On 16/10/2015 13:23, Martin Bjorklund wrote:
> per-object basis.
Yes, I agree.

It might also be useful to have some text somewhere in the draft that 
makes this point clear (i.e. that existing NC/RC implementations are 
neither sync or async).

>    Is this mode no longer valid?
I don't think that we can or should invalidate existing netconf server 
implementations.

However, to sensibly support the opstate requirements, I think that the 
client has to know whether a particular request is, or all requests are, 
being handled by the server in a sync or async fashion.

There has been a suggestion that existing NC implementations could be 
regarded as being async, but that isn't going to work if there ends up 
needing to be a separate "async config apply has completed" notification 
since no existing NC/RC implementations are going to generate such a 
notification.

So, I think that ultimately operations need to be regarded as one of:
  (i) netconf current behavior
  (ii) explicit sync
  (iii) explicit async

It isn't clear to me whether only servers that support (ii) or (iii) can 
meet the opstate requirements, or whether servers supporting (i) can 
also be supported.  What do you think?

> Doesn't this follow from A?
Yes, possibly.  I don't mind if B is deleted.  If we do this then I 
would suggest that we also delete the analogous first sentence of C, and 
re-introduce the "(see terms)" text in the headline description.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 14:58:43 +0100

Hi Gert,

On 16/10/2015 13:58, Gert Grammel wrote:
> --> how would the client know that a certain operation is supported in 
> an atomic or non-atomic manner by the server. In my view the default 
> should be the client assumes best effort and the server is allowed to 
> do better.

Either the client would need to know what error semantics a particular 
server will always use to apply configuration, or the operation needs to 
indicate what semantics are in use (perhaps also being controllable by 
the client request).  But unless the client clearly knows what the error 
semantics are when it applies the config it cannot know what the value 
of the remaining leaves are if some configuration has failed.

The default error handling semantics for NC appear to be "stop on 
error", with "continue on error" and "rollback on error" being the other 
allowed options.  I wasn't aware of the "stop of first error" being the 
default option for netconf and hence hadn't considered it my text above.

So, perhaps we should lift the definitions of "stop on error", with 
"continue on error" and "rollback on error" from section 7.2. of NETCONF 
(rfc6241) into the requirements draft, and then rephrase the 1.D text in 
the context of that.

E.g. change the 1.D text to:

         D. The configuration protocol, or default server behavior, MUST
            specify how configuration errors are handled.   Errors can
            be handled by "stop on error", "continue on error" or
            "rollback on error" (see terms).  Support for "rollback on
            error" SHOULD be provided.

Lifted terms (from NETCONF):

          stop-on-error:  The configuration operation is aborted on the first
             error.

          continue-on-error:  Continue to process configuration data on
             error; error is recorded, and negative response is generated
             if any errors occur.

Enns, et al. Standards Track [Page 39]

------------------------------------------------------------------------

<https://tools.ietf.org/html/rfc6241#page-40>
RFC 6241 <https://tools.ietf.org/html/rfc6241> NETCONF Protocol June 2011

          rollback-on-error:  If an error condition occurs such that part
             of applying the configuration fails, the server will stop
             processing the configuration operation and restore the
             specified configuration to its complete state at the start
             of this configuration operation.

Thoughts?

> Wondering how much we need here. Assuming the server supports 
> rollback, then do we need protocol extensions for that?
Yes, NC already has a rollback-on-error option.

> Assuming the server doesn't support rollback. Then do we need anything 
> special in the protocol to re-configure the config to emulate 
> something like a rollback?

I would suggest no.  Either the server supports this (which is the most 
likely outcome), or the client can emulate it.

>
> My point is: given that transaction is a requirement, would we need to 
> require here anything more that is not a consequence?
>
> Perhaps it is better to limit or even avoid text about transactions 
> and just define the state of the applied config after error. I see 3 
> cases so far:
> 2) sequence: if the config is applied in a sequence and a specific 
> leaf fails, the leafs that have been configured before will keep the 
> intended config, leafs not yet processed keep the applied config and 
> the failed leaf is undefined
> 3) all leafs in error are undefined, all error free leafs use the 
> (new) intended config as their applied config.
I agree with (1) and (2) but wasn't expecting (3).  By best effort, I 
was thinking of the equivalent of Netconf's  "continue on error".

Thanks,
Rob

>>> On 10/15/15, 1:22 PM, "Kent Watsen"<[email-masked]>  wrote:
>>>
>>>> On 10/14/15, 8:00 PM, "Nadeau Thomas"<[email-masked]>  wrote:
>>>>
>>>>>> On Oct 14, 2015:7:51 PM, at 7:51 PM, Kent Watsen<[email-masked]>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>> [email-masked] <mailto:[email-masked]>

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 13:59:04 +0000

>> I think the most common mode *today* is a mix of sync/async, on a

Wait, I think we're talking about different things.  Martin, I'm sure that
internal to a NC/RC server, parts of the intended configuration is
actually applied synchronously (e.g., a hostname) whereas other parts are
not (e.g., config for data plane).   But that nuance is not currently
exposed in any way today -right?

What we're talking about here is an ability for a client to request a
protocol operation (PATCH) to block, or for the "done-applying"
notification to not be sent, until all that processing of the request is
complete.  This regardless if the request contains a mix of nodes that are
applied internally sync/async.  Makes sense? - or it is something else?

>re-introduce the "(see terms)" text in the headline description.

+1

Kent  // as a contributor

---

From: Andy Bierman <[email-masked]>
Date: Fri, 16 Oct 2015 08:44:25 -0700

> per-object basis.  Is this mode no longer valid?

good question.
I am curious how this synchronous return will be deployed.
Is the intent to rewrite NETCONF and RESTCONF so they have
some mechanism like "don't return until intended == applied".

Andy

> operations,
> requires
> > >>            the
> config
> > >>>> On Oct 14, 2015:7:51 PM, at 7:51 PM, Kent Watsen <
> [email-masked]>
> > >>>> wrote:
> > >>>>
> > >>>>
> > >>>>

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 16:01:08 +0000

> some mechanism like "don't return until intended == applied".

Yes.  And by example, imagine a new optional-to-implement query parameter called "block":

    PATCH https://example.com/restconf/data/foo?block=true

Which means don't send the HTTP response until done *attempting* to apply the configuration change to all impacted components in the server, updating both the server's intended and applied configuration.

Kent

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 16:05:03 +0000

Rob,

Yes, we should pull the error modes of netconf into the definition. The available text looks ready to be used.
For the rest we may need to define a requirement that the client should be able to retrieve the error handling information from the server. In the end the client needs to know if it has to clean up the mess in the server or not.

In that sense, I'd define a transaction as a configuration change that can be rolled back by the server.

Gert

Sent from my Apple ][

On 16 Oct 2015, at 15:59, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

Hi Gert,

Either the client would need to know what error semantics a particular server will always use to apply configuration, or the operation needs to indicate what semantics are in use (perhaps also being controllable by the client request).  But unless the client clearly knows what the error semantics are when it applies the config it cannot know what the value of the remaining leaves are if some configuration has failed.

The default error handling semantics for NC appear to be "stop on error", with "continue on error" and "rollback on error" being the other allowed options.  I wasn't aware of the "stop of first error" being the default option for netconf and hence hadn't considered it my text above.

So, perhaps we should lift the definitions of "stop on error", with "continue on error" and "rollback on error" from section 7.2. of NETCONF (rfc6241) into the requirements draft, and then rephrase the 1.D text in the context of that.

________________________________

 <https://tools.ietf.org/html/rfc6241#page-40>
RFC 6241<https://tools.ietf.org/html/rfc6241>                    NETCONF Protocol                   June 2011

Thoughts?

I would suggest no.  Either the server supports this (which is the most likely outcome), or the client can emulate it.

I agree with (1) and (2) but wasn't expecting (3).  By best effort, I was thinking of the equivalent of Netconf's  "continue on error".

Thanks,
Rob

Thoughts?

Gert

Sent from my Apple ][

On 16 Oct 2015, at 14:13, Robert Wilton <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>> wrote:

Hi Kent,

Any comments?

Thanks,
Rob

Kent

Thanks,
Kent

And

        ‹Tom

Kent

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 17:43:54 +0100

> exposed in any way today -right?
I think that today, from what a client sees/experiences, many replies 
from netconf servers fits neither the definition of "synchronous 
configuration operation" nor "asynchronous configuration operation", but 
instead, the reply is somewhere between the two extremes.

So the question I was trying to raise is whether servers that need to 
meet the opstate requirements have to change to strictly comply with the 
defined async or sync config operation semantics?  E.g. not just adding 
a "done-applying" option, but perhaps also adding something along the 
lines of a "done-verifying" option.

Thanks,
Rob

> .

---

From: Gert Grammel <[email-masked]>
Date: Fri, 16 Oct 2015 17:14:57 +0000

Rob,

Current implementations are incomplete asynchronous, because they didn't verify the config as operational and applied.

What is frequently done is to perform additional checks on the intended config that go beyond a syntax check. That is fine, but I have a hard time to consider this to be "hybrid" which suggests something in between asynchronous and synchronous.

>From a client perspective an asynchronous server and a hybrid server look the same. The difference is that the asynchronous one should convey a "finished" information to the client, while the hybrid would remain in a "trust me babe" mode forever.

Another way to look at hybrids is to consider them "cheating synchronous". Given that the new config may not have been applied at the time of the response to the client. This is worse than the situation before where a missing verify lets the client know that something may still go on. Clients can't tell if servers are cheating :-)

Gert

Sent from my Apple ][

> On 16 Oct 2015, at 18:44, Robert Wilton <[email-masked]> wrote:
> 
> 
> 
> I think that today, from what a client sees/experiences, many replies from netconf servers fits neither the definition of "synchronous configuration operation" nor "asynchronous configuration operation", but instead, the reply is somewhere between the two extremes.
> 
> So the question I was trying to raise is whether servers that need to meet the opstate requirements have to change to strictly comply with the defined async or sync config operation semantics?  E.g. not just adding a "done-applying" option, but perhaps also adding something along the lines of a "done-verifying" option.
> 
> Thanks,
> Rob
> 
> 
>>

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 17:17:37 +0000

>instead, the reply is somewhere between the two extremes.

I don't think it matters if the server is anywhere between the two
extremes.  My point is that clients currently have no choice but to handle
it as we define "asynchronous" today.  This came up a couple years ago and
it was confirmed then that <ok/> does not mean "applied".

>lines of a "done-verifying" option.

NC/RC servers (other protocols may vary) have to change only to the extent
they want to support some new features.  For instance, if they want to
support blocking, they can advertise a module defining the extensions
needed.  Likewise, if they want to support transactional/atomic behavior
(wrt applying the config), they can advertise a module
defining the extensions needed to do that too.

Where do you derive there being a requirement for a server to notify when
done verifying?  - has that been discussed before?

Also not that whilst a done-verifying notification would be possible, the
synchronous equivalent isn't (AFAICT), as we define it as not returning
until done-applying (long after the done-verifying step).

Kent  // as a contributor

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 19:50:32 +0100

Hi Gert,

> What is frequently done is to perform additional checks on the intended config that go beyond a syntax check. That is fine, but I have a hard time to consider this to be "hybrid" which suggests som…
I'm talking about netconf servers that effectively apply most of their 
configuration before they reply.  E.g. they might take 10 minutes to reply.

Perhaps this isn't a valid Netconf implementation, and all Netconf 
server implementations are expected to be asynchronous w.r.t to when 
they apply their configuration - but I can't see where this is stated in 
the NETCONF RFC (but possibly I'm not looking in the right place).

> >From a client perspective an asynchronous server and a hybrid server look the same. The difference is that the asynchronous one should convey a "finished" information to the client, while the hybr…
Yes, but this isn't the only difference:
  - a client could expect that an asynchronous config operation response 
should be reasonably expedient.  E.g. if a client was to update 10 
devices each with a reasonable size configuration (that takes minutes to 
apply) in an asynchronous way then it might reasonably consider 
sequencing the async config requests to each server on one thread.  If 
each server acknowledged the requests in a couple of seconds then all 
the servers would broadly end up applying the configuration concurrently.
- however, if the servers were hybrid, and their replies were sent much 
closer to when the configuration was actually applied then initiating 
the requests sequentially would effectively mean that the devices end up 
applying the configuration serially which would end up taking much 
longer.  I.e. it seems reasonable that a client may want to 
differentiate between the behaviour of these two servers even if how it 
handles the resultant config changes is the same.

> Another way to look at hybrids is to consider them "cheating synchronous". Given that the new config may not have been applied at the time of the response to the client. This is worse than the situ…
Yes, but clients also need to be able to determine if the server is 
likely to be slow to response, because then the client would probably be 
designed to interact with the server in a different fashion (e.g. use a 
pool of threads to initiate the updates concurrently).

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Fri, 16 Oct 2015 20:15:34 +0100

Hi Kent,

> it was confirmed then that <ok/> does not mean "applied".
I think that it matters only in how long a client might reasonably 
expect to wait for a reply from the server.

> done verifying?  - has that been discussed before?
Sorry, a loose/bad description.

I think that you are saying that there could be an optional "block" 
protocol extension to allow a client to specify that it wants 
synchronous semantics.

I was trying to specify an explicit asynchronous equivalent, e.g. an 
optional "don't block at all" protocol extension to allow a client to 
specify that it wants the server to exhibit asynchronous configuration 
request semantics (and hence reply to the request sooner than an 
existing Netconf server might otherwise have done).

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Oct 2015 21:56:51 +0200 (CEST)

Kent Watsen <[email-masked]> wrote:
> extremes.

Will there ever be a server that operates in synchronous mode, given
that applied will not match intended if hardware is missing?

Will a client ever use "block" mode if it means that it might hang
forever (or at least until some hw is plugged in)?

/martin

---

From: Kent Watsen <[email-masked]>
Date: Fri, 16 Oct 2015 21:32:01 +0000

>forever (or at least until some hw is plugged in)?

I think the key is in the phrase "The server MUST fully *attempt* to
apply..."

In my view, the server does not have to "fully apply", it merely has to
have attempted to do so.  Essentially, a request comes in resulting in a
flurry of activity, that ultimately stabilizes, at which point the
response can be sent.

If a line card is missing, then (as I understand it), the server would not
wait for the line-card to show up.  That said, if the client requested
transactional/atomic update, a missing line-card would cause an immediate
failure/rollback.

Makes sense?

Kent  // as a contributor

---

From: Phil Shafer <[email-masked]>
Date: Fri, 16 Oct 2015 17:49:19 -0400

>transactional/atomic update, a missing line-card would cause an immediate

We have to avoid the scenario when a box boots with a missing FRU.
In past model has always been that missing hardware does not cause
a configuration to fail validity checks.  This is enforced by the
inability of config=true constraints from referring to config=false
nodes.

Thanks,
 Phil

---

From: Gert Grammel <[email-masked]>
Date: Mon, 19 Oct 2015 08:06:14 +0000

Rob,

I think we are describing the same problem from a little different perspective:

Sent from my Apple ][

> On 16 Oct 2015, at 20:50, Robert Wilton <[email-masked]> wrote:
> 
> Hi Gert,
> 
> I'm talking about netconf servers that effectively apply most of their configuration before they reply.  E.g. they might take 10 minutes to reply.
> 
Exactly, that's why those servers are running asynchronously. The trouble is that in contrast to the synchronous operation there is no "done" information. Current implementations often try to work around that point by applying more than just a syntax check in the first phase. This way they are still quick in replying are more reliably responding, but still fail from time to time

> Perhaps this isn't a valid Netconf implementation, and all Netconf server implementations are expected to be asynchronous w.r.t to when they apply their configuration - but I can't see where this is stated in the NETCONF RFC (but possibly I'm not looking in the right place).
> 
> 
>> 
> - a client could expect that an asynchronous config operation response should be reasonably expedient.
Agree, but this is true for hybrid and asynchronous and the term "reasonable" is not well defined.

>  E.g. if a client was to update 10 devices each with a reasonable size configuration (that takes minutes to apply) in an asynchronous way then it might reasonably consider sequencing the async config requests to each server on one thread.  If each server acknowledged the requests in a couple of seconds then all the servers would broadly end up applying the configuration concurrently.
Yes, that's why we need asynchronous mode in the first place.
> - however, if the servers were hybrid, and their replies were sent much closer to when the configuration was actually applied then initiating the requests sequentially would effectively mean that the devices end up applying the configuration serially which would end up taking much longer.  I.e. it seems reasonable that a client may want to differentiate between the behaviour of these two servers even if how it handles the resultant config changes is the same.

It seems your definition of hybrid is something like "asynchronous but with a longer than reasonable response time".

This is similar to a bloated synchronous operation which takes too much time to respond.  Both cases are undesired and in both cases the solution is to make them truly asynchronous. So that we can have a reasonable response time. 

> Yes, but clients also need to be able to determine if the server is likely to be slow to response, because then the client would probably be designed to interact with the server in a different fashion (e.g. use a pool of threads to initiate the updates concurrently).

How would you do this in practice? Even servers don't know if they're likely to slow a response when they get a request. That's why a quick first response is required that assures the client the requests will be processed, followed by a potentially slow feedback that it is successfully executed (=verified) or failed.

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Oct 2015 11:14:33 +0100

> I think the key is in the phrase "The server MUST fully *attempt* to
+1.
> flurry of activity, that ultimately stabilizes, at which point the
I agree.
> wait for the line-card to show up.
I agree.

>    That said, if the client requested
Yes, I think that this is probably right, given that the applied leaves 
cannot match the intended config leaves.  But this might need some 
further thought to check that the behaviour is sane with regards devices 
that support pre-configuration.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Mon, 19 Oct 2015 10:44:30 +0000

+1
From: netmod <[email-masked]<mailto:[email-masked]>> on behalf of Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Monday 19 October 2015 12:14
To: Kent Watsen <[email-masked]<mailto:[email-masked]>>, Martin Bjorklund <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

Thanks,
Rob

Makes sense?

Kent  // as a contributor

.

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 13:06:28 +0200 (CEST)

> > I think the key is in the phrase "The server MUST fully *attempt* to

So today we have this situation:

   intended                                         operational
    config                                             values
      X--------------------------------------------------X
      (time ->)

When the edit-config returns, running (intended) has been updated, and
the operational state may or may not have been updated.  Internally to
the system, there may be a chain of things that need to happen before
the intended config has been pushed out to all software/hardware
components.

The NMS/operator writes to intended config, ang verifies by checking
the operational state.

With applied config we have this situation:

   intended                       applied           operational
    config                         config              values
      X------------------------------X-------------------X     

In some cases, the applied config is an approximation of the
operational values (if e.g., a single config leaf is used by two
different components), and in other cases the applied config is just
one of the internal stages the config value flows through before
reaching the end component.

With async/sync systems we now have:

                      attempted
   intended            applied     applied           operational
    config              config     config              values
      X-------------------X----------X-------------------X     
                          ^
                   this is when "block"
                   would return

To me it seems we're exposing another internal state to the
NMS/operator.  But in reality nothing has changed from the first
picture - the NMS/operator still needs to check the operational values
in order to know that the system behaves as it should.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Oct 2015 13:06:16 +0100

Hi Martin,

> in order to know that the system behaves as it should.
I see it slightly differently.

I think that "attempted applied config" is effectively the same time 
point as "applied config" above.  As in the server has attempted to 
apply all the configuration and for every it has either succeed, failed, 
or can't be applied because the hardware isn't present.

So using your time line above:
1. If you have an async config operation then the server returns at time 
"intended config".
2. If you have a sync (blocking) config operation then the server 
returns at time "applied config".
3. If you have an existing netconf config operation that is neither 
explicitly sync nor async then the server may return at any time between 
"intended config" and "applied config".

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 19 Oct 2015 14:18:08 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> point as "applied config" above.

But if I pre-configure an interface for which the hw is not present,
it has been said that this config will exist in intended but not in
applied.  And you said that you didn't want "block" to wait for the hw
to be present.  So the server will return before this data shows up in
applied.

> As in the server has attempted to
> apply all the configuration and for every it has either succeed,
> failed, or can't be applied because the hardware isn't present.

Yes.

> 1. If you have an async config operation then the server returns at
> time "intended config".
> explicitly sync nor async then the server may return at any time
> between "intended config" and "applied config".

Sure.  My time line shows how the new value trickels down from
intended to oper.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Oct 2015 13:25:24 +0100

Hi Gert,

> I think we are describing the same problem from a little different perspective:
Perhaps, but I'm not sure that we are on the same page yet.

> Exactly, that's why those servers are running asynchronously. The trouble is that in contrast to the synchronous operation there is no "done" information. Current implementations often try to work…
I don't follow why you think that these servers are running 
asynchronously.  They are much closer to handling all config operations 
as synchronous blocking calls.  This is a just a small amount of 
programming that is being performed asynchronously.

> Agree, but this is true for hybrid and asynchronous and the term "reasonable" is not well defined.
This is true for asynchronous but not hybrid, since a hybrid operation 
might return just before a proper synchronous operation would.

> It seems your definition of hybrid is something like "asynchronous but with a longer than reasonable response time".
My definition of hybrid (i.e. the default existing netconf config 
operations) is along the lines of "asynchronous in behaviour, but may 
reply any time between when the intended configuration has been updated 
and the applied configuration has been completely updated ".

> This is similar to a bloated synchronous operation which takes too much time to respond.  Both cases are undesired and in both cases the solution is to make them truly asynchronous. So that we can…
I agree with the goal, but it may be difficult to change some existing 
NETCONF operational handling to be either strictly sync or async.

> How would you do this in practice?
Roughly, I would suggest:

Assuming that this is going to be a new optional to implement extension:
  - There would be a capability to indicate whether a server supports an 
explicit sync config operation.
  - There would be a capability to indicate whether a server supports an 
explicit async config operation.
  - It would be implicitly assumed that the server would support the 
existing default config operation.

In Netconf/Restconf when an edit-config request is made, an extension 
would include an option to indicate whether the request should be 
processed as sync, async, or default (i.e. existing behaviour).

Thanks,
Rob

>   Even servers don't know if they're likely to slow a response when they get a request. That's why a quick first response is required that assures the client the requests will be processed, followed by a potentially slow feedback that it is successfully executed (=verified) or failed.
>> Thanks,
>> Rob
>>
>>
>>> Gert
>>>
>>>
>>>
>>> Sent from my Apple ][
>>>

---

From: Gert Grammel <[email-masked]>
Date: Mon, 19 Oct 2015 12:39:05 +0000

Martin,

Attempting to apply a config appears to me as a process, not a state. So in my mind the little drawing should look like this then:

receive   updated       Attempting to
intended  intended      apply intended    applied           operational
config    config         config            config              values
X----------X-------------------------------------X-------------------X
     ^
 this is when "block"
   would return

Gert

From: netmod <[email-masked]<mailto:[email-masked]>> on behalf of Martin Bjorklund <[email-masked]<mailto:[email-masked]>>
Date: Monday 19 October 2015 13:06
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:
+1.

   intended                                         operational
    config                                             values

                      attempted

/martin

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Oct 2015 13:54:26 +0100

> to be present.  So the server will return before this data shows up in
My thinking is:

For a synchronous operation (with continue-on-error option):
  - the hw dependent config is updated to intended, probably an 
error/warning is returned for those leaves to indicate that it can't be 
applied because the hardware isn't present.
  - at some given point in the future, i.e. as a completely separate 
event, the hardware may be inserted and the intended config is applied 
at that time.

For a synchronous operation (with rollback-on-error option):
  - the configuration would fail because the hw isn't present (same 
errors/warnings returned as above) and hence the entire config operation 
is completely undone due to the rollback-on-error semantics.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Mon, 19 Oct 2015 13:57:05 +0100

Hi Gert,

I'm not sure whether your diagram might have been mangled by email, but 
please can you explain in words when you think that a synchronous 
blocking config event would return.

Thanks,
Rob

On 19/10/2015 13:39, Gert Grammel wrote:
> Martin,
>
> Attempting to apply a config appears to me as a process, not a state. 
> So in my mind the little drawing should look like this then:
>
>
> From: netmod <[email-masked] 
> <mailto:[email-masked]>> on behalf of Martin Bjorklund 
> <[email-masked] <mailto:[email-masked]>>
> To: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> Cc: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for 
> asynchronous systems to provide a blocking config update?
>
> Robert Wilton <[email-masked] <mailto:[email-masked]>> wrote:
>

---

From: Gert Grammel <[email-masked]>
Date: Mon, 19 Oct 2015 13:14:45 +0000

Hi Rob,

From: Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Monday 19 October 2015 14:25
To: Gert Grammel <[email-masked]<mailto:[email-masked]>>
Cc: Kent Watsen <[email-masked]<mailto:[email-masked]>>, Martin Bjorklund <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

Hi Gert,

I think we are describing the same problem from a little different perspective:
Perhaps, but I'm not sure that we are on the same page yet.
Didn’t claim that we’d be on the same page, but at least we appear to look at the same problem.

Sent from my Apple ][

On 16 Oct 2015, at 20:50, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

Hi Gert,

If a configuration would only take some second or so, everyone would probably be happy to apply synchronous config. Asynchronous config come in, when the server would need a long time to apply the config or if the time is unpredictable. So when we talk about a config that needs 10 min to be applied, I would consider a synchronous configuration the wrong design choice to start with. A server should reply shortly after receiving the intended config to indicate that they are accepting the config and after e.g. 9:59 minutes come back with an information that the config has successfully been applied. If you mix both information into one you either create a black hole in the beginning of the process (request an intended config and getting feedback 10min after) or you end up with a grey hole in the end where you know the server is working on it but have no idea when it has finished, so that you can reasonably expect an operational state in line with your config.

What is the level of information the client has before and after this ‘return’? Before the return, it doesn’t know if the config is being processed, and after the return it doesn’t know if it has been applied. Just like asynchronous.

Understand, but the information available to the client is just the same as for asynchronous: before responding, the client doesn’t know if the config is being processed, and after the response it doesn’t know if it has been applied. The only difference is that in a proper asynchronous implementation the time between a config request and response is short, while the time between response and applied config is long. Hybrid tends to move the response time out. I don’t see how in principle it is different from asynchronous.

Please help me  to understand what you mean with "strictly sync or async”. And in particular why a hybrid (according to your definition) is not asynch.

How would that help the client to figure out how slow the response is likely to be? The fact that a request is synch, asynch or hybrid wouldn’t tell him how fast it is.

Thanks

Gert

Thanks,
Rob

Gert

Sent from my Apple ][

On 16 Oct 2015, at 18:44, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

          A. A server may choose to support only synchronous
configuration
             operations, or only asynchronous configuration operations,
or
             both synchronous and asynchronous configuration operations
in
             a client specified per-operation basis.
I think the most common mode *today* is a mix of sync/async, on a
per-object basis.
Yes, I agree.
Wait, I think we're talking about different things.  Martin, I'm sure that
internal to a NC/RC server, parts of the intended configuration is
actually applied synchronously (e.g., a hostname) whereas other parts are
not (e.g., config for data plane).   But that nuance is not currently
exposed in any way today -right?
I think that today, from what a client sees/experiences, many replies from netconf servers fits neither the definition of "synchronous configuration operation" nor "asynchronous configuration operation", but instead, the reply is somewhere between the two extremes.

Thanks,
Rob

Kent  // as a contributor

.

---

From: Gert Grammel <[email-masked]>
Date: Mon, 19 Oct 2015 13:19:58 +0000

Hi Rob,

Guess we have a terminology issue here, hope that one cuts it:
                      attempt to
   intended            apply       applied           operational
    config              config     config              values
      X-------------------X----------X-------------------X
                          ^
                   this is when "block"
                   would return

From: Robert Wilton <[email-masked]<mailto:[email-masked]>>
Date: Monday 19 October 2015 14:57
To: Gert Grammel <[email-masked]<mailto:[email-masked]>>, Martin Bjorklund <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

Hi Gert,

I'm not sure whether your diagram might have been mangled by email, but please can you explain in words when you think that a synchronous blocking config event would return.

Thanks,
Rob

Gert

From: netmod <<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>> on behalf of Martin Bjorklund <[email-masked]<mailto:[email-masked]>>
Date: Monday 19 October 2015 13:06
To: "<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: "<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] opstate-reqs #3: Is there a requirement for asynchronous systems to provide a blocking config update?

+1.

   intended                                         operational
    config                                             values

                      attempted

/martin

_______________________________________________
netmod mailing list
<mailto:[email-masked]>[email-masked]<mailto:[email-masked]>
<https://www.ietf.org/mailman/listinfo/netmod>https://www.ietf.org/mailman/listinfo/netmod