Subject: [Netconf] draft-cole-netconf-robust-config-01

From: Martin Bjorklund <[email-masked]>
Date: Fri, 26 Jun 2009 14:29:18 +0200 (CEST)

Hi,

I have read this document, and I think the ideas presented are very
interesting.

But I have a comment on the solution you have chosen to trigger the
tests.  It seems you have re-invented confirmed commit, with some
additional parameters.  IMO, the existing confirmed commit operation
is well suited for the things you want to do (maybe it was even
designed to be able to handle this kind of use case).  Why don't you
simply add a new operation <start-test> that can be used to trigger
the tests at the server when the conrimed commit is active?  This
would look like:

            +------------------------------>
             Sets up <candidate> config

            +------------------------------>
             Sets up test control

   ---      +------------------------------>
    |        Sends <commit> <confirmed>
   (set             - timeout
    timeout)
    |        <-----------------------------+
    |                              reply(OK)
    |      +------------------------------>
    |        Sends <start-test>
    |         - test-template:instanceID
    |        <-----------------------------+
    |                              reply(OK)
   (running                                  (run server-side tests)
    client-side
    tests)
    |                                        (server-side test success)
    |        <-----------------------------+
    |         <verifiedCommitStatus = good> notification
    |
    |
    |        +----------------------------->
    |         Sends <commit> (confirming)
    |
    |        <-----------------------------+
    |                              reply(OK)
    |
   ---

One advantage of this is that is can also be used on servers that have
startup instead of candidate, (with the normal disadvantages of this
design):

   make changes to running
   set up test control
   <start-test>
   if test-ok: copy running to startup
   if test-fail: revert running

/martin

---

From: Phil Shafer <[email-masked]>
Date: Fri, 26 Jun 2009 09:52:26 -0400

Martin Bjorklund writes:
>Why don't you
>the tests at the server when the conrimed commit is active?

Another thing to consider that the nature of the test are typically
interwoven with the architecture of the network and the characteristics
of the change.  What test do you perform when you:

(a) change a BGP export filter
(b) change an AS number
(c) add a BGPVPN site

(a) would only affect your peer(s), (b) could have serious impact (but
that's what you wanted), and (c) should have no impact on existing
peers/sites/etc.

So given that the application is making the change, and that testing
the change is integrated with both the change and the customer
network architecture, why not use the existing "commit confirmed"
mechanism and have the application drive the testing of changes
using remote RPCs to get information off the box re: the impact
of the change?  This seems much more straight forward and was
what we had in mind from the start.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 26 Jun 2009 08:01:50 -0700

> the tests at the server when the conrimed commit is active?  This

I prefer the design of the verified commit over the confirmed commit.

  1) using the same command multiple times to do different things
     is confusing and prone to errors (user B's first <commit>
     can actually be used as user A's 2nd commit.  That's broken.)

  2) there is no way to cancel a confirmed-commit, other than
     by dropping the session.  This is not very clean.

  3) the lack of notifications makes confirmed commit harder to
     use in a network-wide config change.

  4) overloading the <commit> operation with more modes is more
     confusing than using a new operation.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 26 Jun 2009 21:25:07 +0200 (CEST)

>      by dropping the session.  This is not very clean.

If you think the current confirmed commit is broken, it should be
fixed.  I don't like the idea of introducing new partly overlapping,
partly different operations in an ad-hoc manner.  I think it would be
better to fix confirmed commit (if necessary), and use it for this
function as well.  [this should be discussed in a separate thread]

>      use in a network-wide config change.

But you have solved that with your new operation.  The <start-test>
operation would still generate notifications as in your proposal.  I
think this is a good idea.

>      confusing than using a new operation.

The idea is that <commit> is not modified at all.  The new
<start-test> operation is orthogonal to <commit>.  The idea is that
the client uses <start-test> in the confirm time window, just like it
can do other tests before issuing the confirming commit.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 26 Jun 2009 13:04:17 -0700

> function as well.  [this should be discussed in a separate thread]

I don't want to introduce an incompatible version of
the protocol.  The charter already assumes in advance
that there are no problems worthy of a new protocol version.

But you will tell me that we could just introduce :confirmed-commit.1.1,
so I guess it doesn't matter.  But I am not so sure the complexity on
the NMS is less in the long run if we do this.  A lot of capabilities
interact.  So now we have to document how every version of every
capability interacts with every other, for all time.  Sounds hard.
This is probably why most protocol WGs just version the entire protocol,
and not arbitrarily small subsets of it. (hint)

> think this is a good idea.

I kept saying I wanted some notification content. :-)
The details of the partial status updates and how
a real template gets used instead of the 'anyxml'
for <extendedStatus>, depending on the test type.

The idea is that a test could return whatever it wants,
even the direct output of unix ping or traceroute commands,
if that was all that was available.

There should probably be distinct <commitStarted> and <commitComplete>
notifications.  There is no 'start' now, and the current complete is
for the tests (but if it worked, it can be assumed the 2nd commit
worked too.  But maybe not, so that's why a <commitComplete>
notification will tell the operator for sure.

The notification replay log could be searched for

  <commitStarted>
    ... bunch of verification tests reporting status and completion ...
  <commitComplete>

That way, different commits in the log could be determined easily.

> can do other tests before issuing the confirming commit.

Well, using a <complete-verified-commit> operation instead of <commit> again,
and adding a <cancel-verified-commit> operation are certainly different.
Except for that part, it is the same as confirmed-commit,
with some tests on the side to help the operator decide
whether to save or rollback the changes.

The tests need to be independent to support case (2)
I mentioned before.  The operator needs to get baseline
numbers (for example), so running the verification tests
outside the context of a commit is important.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 26 Jun 2009 13:34:37 -0700

> Martin Bjorklund wrote:
....
> and not arbitrarily small subsets of it. (hint)

At this point, the NETCONF WG cannot introduce any new version
of a single capability, because the current capability exchange
is not documented to support it.

RFC 4741 says very clearly what capabilities the manager
MUST sent (netconf-base) and what an agent MUST send
(everything it has).  So manager implementations do
not send all the capabilities (like confirmed-commit)
that they support.  They know a 4741 agent will just
ignore them, so why bother.

Except:
   1) if the agent advertises just :foo.1.1, then
      old managers break, since they only know about :foo.1.0

   2) if the agent advertises :foo.1.0 and :foo.1.1,
      and they are not backward-compatible, then
      the agent may not know which one the manager expects

If I were thinking about a solution to the problem
that is optimal, instead of worrying about the charter,
I would suggest that capability exchange needs to
be rewritten in 4741-bis so both side always send
everything they know, and they automatically pick
the highest capability version in common.

But a charter-compatible solution would be to put
CLRs in place that a new capability version MUST follow
all the same revision rules as YANG modules.

For example, adding 'test-only' follows the rules.
An agent could advertise both :validate1.0 and :validate.1.1,
and both old and new managers will work.

I don't think the same is true (so far) wrt/ :confirmed-commit.1.1.

> .....

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 27 Jun 2009 23:11:37 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> > so I guess it doesn't matter.

Yes!  See below.

> >  But I am not so sure the complexity on

Right.

>       the agent may not know which one the manager expects

There is nothing that prevents us from requiring the client to
advertise foo:1.1, if it is necessary.  So if foo:1.1 changes some
semantics of foo:1.0, then we can say that the client MUST advertise
foo:1.1, otherwise the agent will use the semantics of foo:1.0.

> I don't think the same is true (so far) wrt/ :confirmed-commit.1.1.

I think it can be made to work this way, without requiring the client
to advertise :confirmed-commit:1.1.

For example, :confirmed-commit:1.1 could add one rpc method
<cancel-confirmed-commit/>.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 27 Jun 2009 15:31:28 -0700

> <cancel-confirmed-commit/>.

Yes -- but a verify-commit.1.0 would also require all the
behind-the-scenes integration with the verification tests,
in order to send the mandatory notifications.  I don't see
the log-term benefits of creating lots of tiny little
protocol capabilities, even down to adding enums to
parameters.  It seems expedient now, when they are
still on the order of O(10).  Add one zero
to that and you end up with spaghetti YANG.

There is nothing in RFC 4741 that says the same session has
to issue the 2 <commit> operations.  I really don't like this
at all.  It clearly indicates the <commit> is overloaded,
since (even with all its CLRs) it still requires global locks
to work correctly.

I know we don't really have a proper database transaction model in
NETCONF, but the pseudo-transaction model can be
much cleaner that whatever it is we have now.  Just bolting
more parameters and modes onto this supposedly documented transaction
model is just asking for trouble later.  IMO, the NETCONF database
transaction model (including partial locking) is somewhat amateurish.
It's got more holes in it than Swiss cheese.

I don't really care what the capability is called,
as long as there are no restrictions to the design.
Whatever works best -- not whatever band-aid fits the quickest.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Sun, 28 Jun 2009 00:05:06 -0400

>foo:1.1, otherwise the agent will use the semantics of foo:1.0.

Brief history:  when capabilities were introduced, the idea was
that both sides would always advertise all there capabilities and
the overlapping set of capabilities would be what defines the
capabilities of the session.  So if both sides support the a new
revision of a capability, both sides can operate with an understanding
that the other side will do The Right Thing WRT this capability.

Useful, simple, elegant.

But someone objected and this idea turned into a rat's nest, so
during the Great Netconf Purge, this idea was discarded.

The post-Purge idea was that if you want the server to behave
differently, then the new capability should introduce a new RPC or
parameter to explicitly request this new behavior.

Personally, I'd be happy if 4741 reverts this to an "advertise all
your capabilities" strategy, so capabilities can more easily evolve
over time.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Sun, 28 Jun 2009 00:09:50 -0400

Andy Bierman writes:
>IMO, the NETCONF database

Not sure how to parse this, since IMHO the NETCONF database transaction
model does not include partial locking.  If you do partial locking,
then you can't use the 4741 transaction model.  There's no overlap.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Sat, 27 Jun 2009 22:05:47 -0700

> then you can't use the 4741 transaction model.  There's no overlap.

Well, partial-lock works fine for editing <running>,
but if <candidate> or <startup> is supported
on the agent, then you cannot use it.  Batting .333
is good in baseball, but maybe not here :-)

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 28 Jun 2009 21:44:48 +0200 (CEST)

> your capabilities" strategy, so capabilities can more easily evolve

But I don't think 4741 needs to be changed in order to handle this.
As I wrote above, I think this can be handled on a capability basis -
i.e. if a particular capability's semantics is dependent on e.g. which
version of the capability the client implements, then that capability
can be defined to require that the client advertises it.  There is one
such capability already - the base capability.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Sun, 28 Jun 2009 21:39:59 -0400

>such capability already - the base capability.

Would it be possible to make this example more explicit somehow
so that it becomes a conscious model which capabilities are
allowed (encouraged?) to follow?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 29 Jun 2009 09:33:32 +0200 (CEST)

> allowed (encouraged?) to follow?

I am not convinced that it should be encouraged.  Or rather, I believe
that for most capabilities it will not be necessary.  For example, why
should a client advertise any of the capabilities in 4741?  They just
add rpcs that a client can send.  If the client doesn't understand
:url, it will never send an rpc with a url.

/martin