Subject: Re: [Netconf] draft-cole-netconf-robust-config-01

From: Balazs Lengyel <[email-masked]>
Date: Wed, 15 Jul 2009 18:49:12 +0200

Hello,
A trivial way to invoke tests would be to define them as an rpc in yang and specify the 
relevant rpc (with inputs and expected outputs) in the test-template.

I think the idea of scripts is definitely needed. Maybe we should restrict the scripts to local 
(to the netconf server scripts)

Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 15 Jul 2009 18:51:00 +0200

Hello,
For tests you describe just 2 results: OK or Failed, the latter leading to rollback. IMO 
warning type results would be very important. If after a config change I don't have any traffic 
on an interface, that might be because I lost connectivity or because its the middle of the 
night and that specific office is closed. In such cases better warn the operator, but not 
roll-back.

When configuring multiple nodes if the 5th of 10 nodes fails in some case you will want to 
roll-back the first 4 in some other cases not. Both needs to be handled.

Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 15 Jul 2009 18:53:27 +0200

Hello,
Are you allowed to modify the candidate datastore between start-verified-commit and 
complete-verified-commit? (AFAIK you can modify it between commit and the confirming commit.) 
This would mean that the Netconf server will need a temporary datastore, that can be used for 
rollback.

Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 15 Jul 2009 18:53:45 +0200

Hello,
Very interesting draft. Some comments, questions:

General:
If we introduce some new testing/verification mechanisms, I would like to see how it can be 
used for to make sure it will be available for writable-running based nodes as well.

About this version of the draft:
1) I think your examples are not the best. Today confirmed commit only checks whether OAM 
connectivity is still available. For any config-change, that is not related to this 
connectivity, you need extra checks. Examples could be:
- configuring an License server. OAM is OK, but the License server does not serve half of the 
License-client nodes. It is possible to confirm, but obviously extra tests between the License 
server and its License clients are needed.
- Configuring a SIP server. While the config looks nice and it can be committed, will it still 
serve all the SIP user agents
- In telecoms often the OAM connection and the "traffic" connections are on completely 
different networks for security reasons. So even though OAM is up and running, the configured 
node might have lost some or all of the traffic links

2) The easiest way to start checks would be to define in the specific YAMs rpcs or relevant 
state data that the manager can check. We already have these possibilities. Why not let the 
manager invoke these test. (Could be one big generic-health-check rpc or many small specific 
tests) Maybe this should go into the YANG usage guide.

3) I asked earlier,but again: What does test-then-set mean for the running config? According to 
YANG the running is always valid. Which set of checks are run for test-then-set against a 
candidate?

Balazs

About the existing commit:
I) As Andy pointed out today the commit operation is overloaded (original commit, confirming 
commit). This can lead to the mistake when two operators both want to send just an 
original-commit but and up confirming the commit. Broken-> fix it! At least provide a separate 
confirm-commit operation (compatible change), or maybe also remove the possibility to use a 
second commit operation for confirming the commit (incompatible change).

II) The cancel-commit operation is missing. Add it!

III) Netconf should allow the Netconf server itself to rollback a commit, when it detects 
problems (and issue a corresponding notification). Add the possibility!

Balazs

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 15 Jul 2009 19:00:29 +0200

I would not argue that a warning result may be a useful thing, but in
the example that you are providing a rollback would not result from just
a measurement of traffic on interfaces, but the some more complete tests
(like loopbacks, or OAM continuity messages) would be activated before
deciding on a rollback. This is where scripts on the server can play a
role. 

Dan

> -----Original Message-----
> From: Balazs Lengyel [mailto:[email-masked]] 
> Sent: Wednesday, July 15, 2009 7:51 PM
> To: Martin Bjorklund
> Cc: [email-masked]; [email-masked]; Romascanu, 
> Dan (Dan); [email-masked]
> Subject: robust-config not so clear-cut test-results
> 
> Hello,
> For tests you describe just 2 results: OK or Failed, the 
> latter leading to rollback. IMO warning type results would be 
> very important. If after a config change I don't have any 
> traffic on an interface, that might be because I lost 
> connectivity or because its the middle of the night and that 
> specific office is closed. In such cases better warn the 
> operator, but not roll-back.
> 
> When configuring multiple nodes if the 5th of 10 nodes fails 
> in some case you will want to roll-back the first 4 in some 
> other cases not. Both needs to be handled.
> 
> 
> Balazs
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 15 Jul 2009 19:10:20 +0200

Good points concerning the examples. 

Dan

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Balazs Lengyel
> Sent: Wednesday, July 15, 2009 7:54 PM
> To: Andy Bierman
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Netconf] draft-cole-netconf-robust-config-01
> 
> Hello,
> If we introduce some new testing/verification mechanisms, I 
> would like to see how it can be used for to make sure it will 
> be available for writable-running based nodes as well.
> 
> 
> License-client nodes. It is possible to confirm, but 
> obviously extra tests between the License 
> manager invoke these test. (Could be one big 
> generic-health-check rpc or many small specific 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>

---

From: "Cole, Robert G." <[email-masked]>
Date: Fri, 17 Jul 2009 00:42:58 -0400

Balazs,

How is this different from our example of defining a ping.yang test which is executed locally from the server?

Bob
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Balazs Lengyel [[email-masked]]
Sent: Wednesday, July 15, 2009 12:49 PM
To: Martin Bjorklund
Cc: [email-masked]; [email-masked]
Subject: Re: [Netconf] draft-cole-netconf-robust-config-01

Hello,

Balazs

---

From: "Cole, Robert G." <[email-masked]>
Date: Fri, 17 Jul 2009 00:45:46 -0400

Balazs,

Our original intent was to define the Verification tests as a set of active tests generated from the server.  In this case, the traffic will exist.  However, I think you raise some interesting points; passive tests could be useful, as are warning notifications.

Bob
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Balazs Lengyel [[email-masked]]
Sent: Wednesday, July 15, 2009 12:51 PM
To: Martin Bjorklund
Cc: [email-masked]; [email-masked]
Subject: [Netconf] robust-config  not so clear-cut test-results

Hello,

Balazs

---

From: "Cole, Robert G." <[email-masked]>
Date: Fri, 17 Jul 2009 00:58:07 -0400

Balazs,

I have not given this much thought.  But the intent of the Verification tests is to check the candidate configuration once it has been moved to running.  If the tests succeed then the device commits to the current running.  If fails, it should backout to the previous running.  Changing the candidate between the start-verified-commit and complete-verified-commit should have no affect on the previous procedure.

Bob 
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Balazs Lengyel [[email-masked]]
Sent: Wednesday, July 15, 2009 12:53 PM
To: Martin Bjorklund
Cc: [email-masked]; [email-masked]
Subject: [Netconf] Changing the candidate during commit [was:   draft-cole-netconf-robust-config-01]

Hello,

Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 17 Jul 2009 15:09:58 +0200

You are right it is similar, but there is a slight difference.
We should allow the modeler to define his own rpcs to be used for verification, not just 
standard ones. I think this aspect is missing from the text. But maybe this was your original 
intent, and we want the same thing.
Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320                  email: [email-masked]

---

From: "Cole, Robert G." <[email-masked]>
Date: Fri, 17 Jul 2009 14:52:42 -0400

Balazs,

We did have some text in the draft outlining various options for test definitions, one of which was to have the modeler embed tests with configuration objects in the device model.  I believe this to be a very attractive option.  However, to move things along in the draft we choose to work out the details of a simpler, more straightforward option of passing reference to a test module test instance.

Bob
________________________________________
From: Balazs Lengyel [[email-masked]]
Sent: Friday, July 17, 2009 9:09 AM
To: Cole, Robert G.
Cc: Martin Bjorklund; [email-masked]; [email-masked]
Subject: Re: [Netconf] draft-cole-netconf-robust-config-01