Subject: [Netconf] too many optional components?

From: Andy Bierman <[email-masked]>
Date: Thu, 06 Aug 2009 08:45:31 -0700

Hi,

I am starting to wonder if so many interacting optional components
in the NETCONF protocol is such a good idea.

We are still early in the technology curve with NETCONF,
and already it is obvious that just the variants
from the 10 or so capabilities already defined are leading
to lots of interoperability issues.

The interactions grow with N^^2, so 20 capabilities will
be 4 times worse, not twice as bad as now.  Every new NETCONF
draft introduces at least 1 (sometimes 2) new capabilities,
and we are on track to reach 20 capabilities in less than 3 years.

Assuming a 15 - 20 year standards track lifetime, we could have
100,000 valid variants of the NETCONF standard by the
time we're done.  That's great for an agent developer,
who just wants to use NETCONF buffet-style.
But it looks like 'worst nightmare ever' to an NMS developer
who has to support lots of different agents.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Aug 2009 08:19:32 -0700

Randy Presuhn wrote:
> Hi Andy -
> 
> I think you're absolutely right on this.  And it's not just the
> capabilities, but also the way module versioning and reuse
> will work that have this property.  I'm soooo glad I don't
> have to implement this.  :-)
> 

Hi Randy,

Sorry for posting your private reply to the WG,
but you have a lot of NM and standards experience
across several SDOs and decades, so I think your perspective
and insights on interoperability need to be
considered by the WG.  The intent is that
many people will attempt implementations.

> Randy

Andy

> 
> ----- Original Message ----- 
> From: "Andy Bierman" <[email-masked]>
> To: "NETCONF" <[email-masked]>
> Sent: Thursday, August 06, 2009 8:45 AM
> Subject: [Netconf] too many optional components?
> 
> 
>> Hi,
>>
>> _______________________________________________
>> Netconf mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/netconf
> 
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 7 Aug 2009 20:40:18 +0200

> many people will attempt implementations.

Well, to be fair, SNMP allows lots of variability in implementation as
well and SNMP did not provide proper ways to announce things - so
manager implementations either had to do quite some extensive probing
to figure out what an agent supports or they settled on expecting the
lowest common denominator. Given my SNMP experience, I believe NETCONF
is on a good track since it provides means to announce the properties
of a given server implementation and I assume this really makes things
simpler. I believe we need to acknowledge implementation differences
instead of claiming implementations are all the same even though in
reality they are not.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Aug 2009 11:57:35 -0700

> instead of claiming implementations are all the same even though in

It is simply a design choice whether to make the conformance
for every little bit of new functionality to be optional, or whether
to bundle them together into a much larger conformance package,
sometimes called a "protocol version".

Simply announcing that you support combination #42 out of 1000
does not help NMS code developers who have to keep rewriting code
because another new capability changes the way 2 other capabilities work.

This is not the same thing as an unknown mix of MIB modules
that may be present on each agent.  Some capabilities (like rollback-on-error)
add 1 enum to 1 parameter to 1 RPC operation.  IMO, this micro-partitioning
of functionality is a way to avoid real standards and real interoperability,
not improve it.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 7 Aug 2009 21:13:33 +0200

> because another new capability changes the way 2 other capabilities work.

I also does not make sense to have one protocol versions while in
reality implementations can implement that one version in several
different ways. I am with you in the cases where capabilities change
semantics of operations - this is simply bad design.
 
> This is not the same thing as an unknown mix of MIB modules that may
> be present on each agent.  Some capabilities (like
> rollback-on-error) add 1 enum to 1 parameter to 1 RPC operation.
> IMO, this micro-partitioning of functionality is a way to avoid real
> standards and real interoperability, not improve it.

Oh, I am talking about MIB modules. Just think of pretty fundamental
things like SNMP's RowStatus - do not expect all agents to handle
things in the same way; some only support createAndGo, others only
dribble mode; some even only single varbind sets while yet others
prefer to get one row in a single set - but they might freak out if
you try multiple rows in a single set. I believe differences like
these do not go away by pretending they do not exist.

And whether we will achieve interoperability will be seen by doing the
experiment of getting this out of the door and good interoperability
tests in place. Real interoperability comes from implementation
interoperability testing and open reference implementations. Good open
specifications help with that but only up to a certain point.

/js

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 7 Aug 2009 12:57:49 -0700

Hi -
 
> From: "Juergen Schoenwaelder" <[email-masked]>
> To: "Andy Bierman" <[email-masked]>
> Cc: "Randy Presuhn" <[email-masked]>; "NETCONF" <[email-masked]>
> Sent: Friday, August 07, 2009 12:13 PM
> Subject: Re: [Netconf] too many optional components?
...
> these do not go away by pretending they do not exist.
...

For an implementation to "freak out if you try multiple rows in a single set"
is a bug, plain and simple.  The variations that in my experience were
most difficult to cope with in manager code were not differences in behaviour
permitted by the protocol or MIB designs, but rather the astonishing array
of misbehaviour in violation of the protocol or MIB specifications.

RowStatus is horrifically complicated, largely due to SNMP and the SMI's
lack of a real object model and ad hoc approach to life-cycle.  (But wait,
Netconf doesn't have one either.) Still, we should not confuse implementation
errors with variability permitted by the specifications.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Aug 2009 16:11:18 -0700

> semantics of operations - this is simply bad design.

There has always been a tug-of-war between NMS and agent developers
over this issue.  The NMS developers call it 'variability' and it's a bad thing,
and the agent developers call it 'flexibility', and it's a good thing.

Sometimes, real standards requires real work to get
the right running code.

In the old days, before if-feature and capabilities,
working groups had to make tough decisions about
what to put in the protocol (or MIB module).
If 1 or 2 vendors could not convince everybody else
they needed to implement this new thing, then it got left
out of the standard.  That is not true anymore.

> these do not go away by pretending they do not exist.

OK, the first indicator that we had a that problem is that
it takes 12 pages to define the RowStatus TC.  But we ignored that sign.
RowStatus is so complicated that even standard MIBs started defining
hacks in DESCRIPTION clauses to leave out the parts that were too hard.

> specifications help with that but only up to a certain point.

is this where we say amen! ?
running code makes better specifications makes better running code makes...

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 07 Aug 2009 21:37:57 -0700

> errors with variability permitted by the specifications.

Since you brought up RowStatus...

Is the NETCONF WG going to ignore 'not-in-service' forever?
This could be considered the same as the 'pre-provisioning'
feature that NETCDNF does not support.  Does it matter that
active config looks exactly the same as pending config
(e.g., waiting for specific hardware to be installed)?

Andy

---

From: Phil Shafer <[email-masked]>
Date: Sat, 8 Aug 2009 10:12:09 -0400

>(e.g., waiting for specific hardware to be installed)?

FWIW, I follow an "ephemeral" model, where the configuration is
valid based only on the configuration, not the current state of the
world.  This means that the installed hardware is not considered
when judging validity.  Pre-provisioning means only that you create
the config for missing hardware, and when the hardware is hotplugged
in, the device sees it and applies the proper config.  Not doing
this model means that someone can remove a FRU and suddenly your
config will not commit (and in ways the committer may not be able
to repair).

So 'pre-provisioning' is already handled, in this respect.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 Aug 2009 07:42:35 -0700

> So 'pre-provisioning' is already handled, in this respect.

good -- I do the same thing.
It does not matter if the the callback code 'arrives' before
or after the config.  Since callbacks are optional,
there is currently no way for the stack to even know
if any instrumentation is missing.

This is an area where specialized RPCs would really help,
such as get-status().

Something like the /etc/rc.d/init.d/serverd <command>
stuff in Fedora would really help in NETCONF.  But now I'm
talking about structured APIs for network services,
which is so far off the charter radar, that I'll shut up now...

Andy