Subject: Re: [netmod] operational knobs

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Sun, 1 Nov 2009 11:30:17 +0100

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Saturday, October 31, 2009 11:57 PM
> To: Joel M. Halpern
> Cc: NETMOD Working Group
> Subject: Re: [netmod] operational knobs
> 
> Joel M. Halpern wrote:
> > While I do not know what the right answer is for YANG, 
> let's not get 
> > too distracted by the SNMP answer.  The mechanism used for 
> this sort 
> > of thing by SNMP is a kludge designed to work around the limitaiton 
> > where the only operations were get and set.
> > 
> 
> Agreed -- but I view the IETF SNMP experience as more than a 
> distraction.  We are theoretically asking IETF WGs to stop 
> using SMIv2 and start using YANG to design data models.
> I think the differences, and guidelines wrt/ those 
> differences, are relevant to a significant portion of the 
> IETF community.
> 

I agree. Is not this something that netmod-arch should deal with? 

Dan

---

From: Phil Shafer <[email-masked]>
Date: Sun, 1 Nov 2009 11:01:01 -0500

"Joel M. Halpern" writes:
>While I do not know what the right answer is for YANG, let's not get too 
>distracted by the SNMP answer.  The mechanism used for this sort of 
>thing by SNMP is a kludge designed to work around the limitaiton where 
>the only operations were get and set.

[Clearly an aside, but as a non-snmp dude.....]

I've always been curious why new verbs weren't added to SNMP
instead of this kludge.  Poking bytes into control registers
clearly wasn't the example to follow and the distaste for it
seems universal, but I've never grok'd why verbs like CREATE,
DELETE, LOCK, CHANGE, PRE-CREATE, DISABLE, &etc weren't made.

Thanks,
 Phil

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 01 Nov 2009 11:22:32 -0500

That would indeed seem an appropriate place to deal with the topic.
Joel

---

From: "Randy Presuhn" <[email-masked]>
Date: Sun, 1 Nov 2009 09:02:03 -0700

Hi -

> From: "Phil Shafer" <[email-masked]>
> To: "Joel M. Halpern" <[email-masked]>
> Cc: "NETMOD Working Group" <[email-masked]>
> Sent: Sunday, November 01, 2009 9:01 AM
...
> DELETE, LOCK, CHANGE, PRE-CREATE, DISABLE, &etc weren't made.
...

Adding them would have been a change to the protocol's grammar,
which would have been a non-starter for several reasons:

   -  re-starting the clock at "Proposed Standard" each time one of
      these was added would have been politically unpalatable (CMIP
      was a full IS, and, since it already had verbs for Create, Delete, etc.,
      adding these to SNMP would also have been seen as an admission
      of inadequacy)

   - changing the grammar would have adversely affected the deployed base
      of management systems and agents, as well as the various vendors'
      development tools, and would have invalidated the interoperability suites,
      bake-offs, and test events.  The ability to work with generic "dumb" MIB
      browsers was demanded by users.

    - by the time SMP, Secure SNMP, SNMPv2, SNMPv2*, etc, started appearing,
      access control models were being formulated in terms of object identifiers.
      If an "Action" facility were to be added, we'd have had to formulate the policies
      in ways similar to how VACM processes notifications, which would severly
      constrain how people could formulate actions.  (Not that that would have been
      a bad thing)

FWIW the CMIP experience, while it did establish that having distinct verbs for
Create and Delete was a Good Thing, and had provisions for synchronization
built into the object model and the protocol, also demonstrated that a generic
Action facility, while useful, needed to be used *Very* carefully.  Like netconf RPCs,
it provided the ability to shovel around rather arbitrary syntax.  Mating this with
fine-grained access control expressed in terms of the object model is
hard.  Later discussions of "we wish we had" in CMIP-land supported limiting
the payload to things that would look like variable bindings, rather than arbitrary
ASN.1, to ease matching up the bits and pieces of actions with the objects'
data models.

Randy

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Sun, 1 Nov 2009 18:01:37 +0100

-----Original Message-----
From: [email-masked] on behalf of Phil Shafer
Sent: Sun 11/1/2009 6:01 PM
To: Joel M. Halpern
Cc: NETMOD Working Group
Subject: Re: [netmod] operational knobs

Thanks,
 Phil
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

The short answer is that SNMP was designed to be the light (and temporary!) alternative to CMIP which had 11 'verbs' (if I remember well) including some of the ones you mentioned. 

The long answer requires a few beers :-)

Dan

---

From: Jon Saperia <[email-masked]>
Date: Sun, 1 Nov 2009 12:07:49 -0500

Randy,

Good points.

In the context of the current conversation, this history argues for  
getting as much of NETCONF/YANG correct the first time.  I mean not  
only for the immediate application to which the protocol and language  
will be put, but also, where people want it to go.

To the degree NETCONF/YANG are successful, it will be difficult to  
change them to make adjustments/corrections that the WG chooses to  
punt now.

/jon

---

From: Andy Bierman <[email-masked]>
Date: Sun, 01 Nov 2009 11:35:49 -0800

Jon Saperia wrote:
> getting as much of NETCONF/YANG correct the first time.  I mean not only
> for the immediate application to which the protocol and language will be
> put, but also, where people want it to go.
> 

I disagree.
We have a lot of theory, but not 1 config=true
object in any I-D or RFC.  We have a description-stmt
for a reason.

It is much easier for a WG to reach consensus
on advanced features if there is common experience
based on well-known data models in use.

There also needs to be a YANG Doctors eventually, like MIB Doctors,
so sufficient review resources are available throughout the
IETF YANG data modeling experience.  This is a far more effective
process than working forever on YANG 1.0, because we know it
is not perfect.

> change them to make adjustments/corrections that the WG chooses to punt
> now.

But now we have capabilities and a <hello> message.
Incremental additions are part of the plan.

> /jon

Andy

---

From: Jon Saperia <[email-masked]>
Date: Sun, 1 Nov 2009 15:19:14 -0500

On Nov 1, 2009, at 2:35 PM, Andy Bierman wrote:

>> only

I was not suggesting chrome polishing or advanced features.    
Yesterday you asked about operational knobs and said current drafts do  
not support them.  My point is that, if as you suggest this is to be a  
requirement, that there is consensus  that it can be addressed  
incrementally, if not right now.
jon
>> punt
>>> FWIW the CMIP experience, while it did establish that having  
>>> distinct
>>> verbs for
>>> built into the object model and the protocol, also demonstrated  
>>> that a
>>> generic

---

From: Andy Bierman <[email-masked]>
Date: Sun, 01 Nov 2009 13:28:52 -0800

Jon Saperia wrote:
> I was not suggesting chrome polishing or advanced features.   Yesterday
> you asked about operational knobs and said current drafts do not support
> them.  My point is that, if as you suggest this is to be a requirement,
> that there is consensus  that it can be addressed incrementally, if not
> right now.

The consensus seems to be that 'action' (rpc-stmt) is
the correct choice, not a config=true leaf.

One question that keeps coming up: 'what is config?'
There seems to be consensus that config is not
a leaf that the server changes, once the client has set it.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Sun, 1 Nov 2009 17:06:55 -0500

>the correct choice, not a config=true leaf.

"actions" were already discussed and the concensus was not to add them:

  http://trac.tools.ietf.org/wg/netmod/trac/wiki/Issues_yang#issue-00221

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Sun, 1 Nov 2009 17:08:51 -0500

>change them to make adjustments/corrections that the WG chooses to  

I think our choice is more open, saying that the world changes
and that we want to be more flexible and open in both NETCONF
and YANG.  Both have extension mechanisms that are meant to make
the technologies more correctable, adjustable, and future proof.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Sun, 1 Nov 2009 17:11:35 -0500

>which would have been a non-starter for several reasons:

Adding new verbs would not have been so destructive.  A MIB variables
listing known verbs would have allowed a device to advertise to the
application the verbs it supported, and verb-less clients would
simply have ignored the advanced options available on enlghtened
devices.

Thanks,
 Phil

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 01 Nov 2009 17:29:07 -0500

To be specific, it was to "not add them now."
I believe Andy said that he does not want to add them before YANG 1.0.

His point, which I think is well taken, is that we will need to figure 
out what variation on action / rpc definition we want to use, and define 
it.  Text pointing in the right direction should go somewhere (the arch 
draft seems sensible) so that folks do not simply re-implement the SNMP 
mechanism.

Also note, the question you pointed to was actually much narrower, as it 
was about defining actions in lists, which introduces many powerful and 
complicated issues.

Yours,
Joel

---

From: "Randy Presuhn" <[email-masked]>
Date: Sun, 1 Nov 2009 16:50:24 -0700

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Sent: Sunday, November 01, 2009 3:11 PM

I disagree.  The *grammar* of the SNMP protocol has no provision for this
kind of extensibility.   There simply is no way to add a new verb without
changing the grammar of the protocol.  I am certain this limitation was
deliberate; CMIP, developed during the same period, made extensive
use of ASN.1's extinsibility mechanisms (ANY DEFINED BY).

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 2 Nov 2009 06:36:19 +0100

> use of ASN.1's extinsibility mechanisms (ANY DEFINED BY).

This is really off topic, but RFC 1157, RFC 1901, and RFC 3412 all use
"ANY -- e.g., PDUs". Yes, it is not ANY DEFINED BY but if people would
have wanted to add create/delete operations, the grammar would not
have been the show stopper.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Randy Presuhn" <[email-masked]>
Date: Sun, 1 Nov 2009 22:50:26 -0700

Hi -

> From: "Juergen Schoenwaelder" <[email-masked]>
> Sent: Sunday, November 01, 2009 10:36 PM
...
> have been the show stopper.

The "ANY" in the envelope was just a hack to allow an encrypted payload.
The elements of procedure require it to be treated as a parse error if it
doesn't match the grammar for "PDUs".  Putting anything else there
would still require it to be treated as a new protocol version.

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 2 Nov 2009 09:52:13 +0100

> would still require it to be treated as a new protocol version.

Yes, because the WG did choose to forbit this sort of extension and
this was a political decision, not a grammar problem.

/js

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 2 Nov 2009 12:51:31 +0100

> 
> There also needs to be a YANG Doctors eventually, like MIB 
> Doctors, so sufficient review resources are available 
> throughout the IETF YANG data modeling experience.  

Yes. This team will need the YANG Usage Guidelines document at hand as a
reference as soon as possible. I have seen to little attention being
paid to it, which is maybe normal until we send the core YANG document
to the IESG, but then we need to start focusing on making it happen. 

Dan

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 2 Nov 2009 12:56:51 +0100

> [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
> Sent: Monday, November 02, 2009 10:52 AM
> To: Randy Presuhn
> Cc: NETMOD Working Group
> > > This is really off topic, but RFC 1157, RFC 1901, and RFC 
> 3412 all 
> > > use "ANY -- e.g., PDUs". Yes, it is not ANY DEFINED BY 
> but if people 
> > > would have wanted to add create/delete operations, the 
> grammar would 
> > > not have been the show stopper.
> > 
> > The elements of procedure require it to be treated as a 
> parse error if 
> > it doesn't match the grammar for "PDUs".  Putting anything 
> else there 
> Yes, because the WG did choose to forbit this sort of 
> extension and this was a political decision, not a grammar problem.
> 
> /js
> 

It was a political decision which resulted in parsing errors :-) 

Dan

---

From: Phil Shafer <[email-masked]>
Date: Mon, 2 Nov 2009 09:42:02 -0500

>It was a political decision which resulted in parsing errors :-) 

But couldn't the political decision have been reversed when the
tide of "not so simple variables" began to flow in?

Thanks,
 Phil

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 2 Nov 2009 16:53:56 +0100

> From: Phil Shafer [mailto:[email-masked]] 
> Sent: Monday, November 02, 2009 4:42 PM
> To: Romascanu, Dan (Dan)
> Cc: Juergen Schoenwaelder; Randy Presuhn; NETMOD Working Group
> But couldn't the political decision have been reversed when 
> the tide of "not so simple variables" began to flow in?
> 
> Thanks,
>  Phil
> 

It was not that easy, and the success of the protocol was part of the
reason. It took a few years to realize that CMIP / CMOT will never
happen. By that time SNMP was working at least for some applications the
IETF needed, with security being the principal flaw the whole community
was told to focus for another few good years. You can consider SNMCONF
and SMIng as tentative to finesse the issue without rocking the boat of
the protocol. And then somebody said X-M-L !!

Dan
(apologizing for historical soapboxes)

---

From: Andy Bierman <[email-masked]>
Date: Mon, 02 Nov 2009 08:53:20 -0800

> the protocol. And then somebody said X-M-L !!

There were some of us that were tried to get SNMP to
change so that SNMP Set would be relatively implementable.
The 'gumball machine effect' in SNMP, and the lack of complex data
structures in SMIv2, made distributed control tables a nightmare
to code in the agent.  CLI accepts 1 command at a time. SNMP
accepts an arbitrary number of leafs from an arbitrary number
of commands, and reconstructs commands as they arrive in Set PDUs.

Something as dirt-simple in NETCONF as <make-toast> is a nest of kludges
in SNMP.

For awhile, many OPS people did not really believe operators when
they said they liked CLI because they could read it.  Think OID.
Then go as far in the opposite direction as possible.  Some of the
many benefits of XML -- it can be read, cut-and-pasted, and emailed.

But we don't need to replay the XMLCONF discussions that led to
the formation of the NETCONF WG.  SMIv2 action knob --> NETCONF <rpc>.
End of problem (for now).

> (apologizing for historical soapboxes)
me too!

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 2 Nov 2009 10:20:04 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Romascanu, Dan (Dan)" <[email-masked]>
> Sent: Monday, November 02, 2009 9:53 AM
...
...

That's what we though we had accomplished with CMIP's "Action".  :-(

There are serious lessons to be learned from the failures there, especially
with regard to access control, how one constructs information models,
and formalizing the description of side-effects, at least to the point where
action X can be said to interoperably accomplish "the same thing" on
both implementations Y and Z.

But you're right - this group has already set its direction, and a Cassandra-
like rehash isn't helpful.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 03 Nov 2009 09:26:00 -0800

> tide of "not so simple variables" began to flow in?

Bad decisions can still benefit from Newton's first law of motion :-)

At 10,000 feet, SNMP was succeeding at monitoring.
But on the ground, it was obvious (SNMPCONF, DIFFSERV-MIB,
APM-MIB, TPM-MIB) that we had reached the
upper bound of SMIv2/SNMP utility.

IMO, new verbs and nested data structures in notifications
are the best parts of YANG.  The config database may
turn out to be a 'legacy' concept, and complex verbs may
replace get/set operations on an XML tree someday.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 3 Nov 2009 09:38:26 -0700

Hi -

> To: "Phil Shafer" <[email-masked]>
> Sent: Tuesday, November 03, 2009 10:26 AM
...
> are the best parts of YANG.

Initially, we thought the same thing about these capabilities in CMIP.
Implementation and deployment experience demonstrated that they
came at a high cost, though in areas that netconf hasn't really started
to dig into.

> The config database may

If that's the case, then you may as well discard the whole notion
that configuration management is about bringing a system to a
reasonably known state.

Randy

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 3 Nov 2009 18:43:46 +0100

> [mailto:[email-masked]] On Behalf Of Randy Presuhn
> Sent: Tuesday, November 03, 2009 6:38 PM
> To: NETMOD Working Group
> > IMO, new verbs and nested data structures in notifications are the 
> > best parts of YANG.
> 
> Implementation and deployment experience demonstrated that 
> they came at a high cost, though in areas that netconf hasn't 
> really started to dig into.
> 

What changed in the two decades dramatically is the cost of the
resources needed for implementation on a network device. As an
implementer at that time I can bare direct witness that the principal
reason we adopted SNMP at that point in time was simply that it was
implementable in network devices (hubs, bridges, routers) while CMIP was
not. 

> > The config database may
> > turn out to be a 'legacy' concept, and complex verbs may replace 
> > get/set operations on an XML tree someday.
> 
> If that's the case, then you may as well discard the whole 
> notion that configuration management is about bringing a 
> system to a reasonably known state.

I would agree. I do not see yet how interoperable management is possible
without a config database shared between different vendors of devices
and applications but maybe it's just that I am not imaginative enough. 

Dan

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 3 Nov 2009 10:07:05 -0700

Hi -

> From: "Romascanu, Dan (Dan)" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>; "NETMOD Working Group" <[email-masked]>
> Sent: Tuesday, November 03, 2009 10:43 AM
> Subject: RE: [netmod] operational knobs
...
> implementable in network devices (hubs, bridges, routers) while CMIP was

That depends on the implementation approach.  Yes, at that time there
was a freely-available implementation which was spectacularly inefficient
in terms of code size, memory use, and processing time.  I think that
implementation did more to kill CMIP than anything else.  We implemented
it from scratch, and found it could be implemented compactly and efficiently,
working quite well in 68000/68302-based network gear.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 03 Nov 2009 12:11:22 -0800

> and applications but maybe it's just that I am not imaginative enough. 

I'm not saying the technology is in place to
allow a command-oriented approach to completely replace the
data-oriented approach, but...

The problem with pure data structures is that the high-level
procedures built on top of the data structures are not uniform,
and usually not even formally defined.

It may not make sense to adjust knobs arbitrarily once a network
service has been created from the XML representation
of the desired state.  This was often the case with MIBs,
and there are many DESCRIPTION clauses that say "This object
may not be modified if the associated RowStatus is equal to 'active'".

There are high implementation, interoperability, and training
costs due to the data-approach that are not immediately obvious.
Operators like to think in terms of sending commands to the device.
Translating between YANG data structures and high-level commands
is ad-hoc and usually non-trivial.

IMO, we would reach faster, better interoperability by standardizing
basic operations, and not mandating complete adoption of
complex data structures.

For example, an operation to start a DHCP service instead of a complex
data structure:

   start-dhcp (start-address=192.168.1.10, end-address=192.168.1.100, ...)

There are no internal data structures that can be
altered in arbitrary ways with edit-config and copy-config.
There is no parameter changing on the fly to support.

Obviously DHCP config would be way more complicated
than this, but my point is that RPC operations allow the WG to
standardize procedures, which can be a subset of
all possible protocol behavior.  This allows some forward progress,
in highly directed and interoperable steps.  I am not
convinced that the forklift data-model (ala ipfix-psamp.yang)
has much chance of widespread deployment.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 3 Nov 2009 15:41:14 -0500

>   start-dhcp (start-address=192.168.1.10, end-address=192.168.1.100, ...)

This is fine if the customer doesn't care about any of the details
of how dhcp is deployed, or it your RPC contains all the parameters
they need.  But each customer is different; beware of underestimating
the diversity of the marketplace.

IMHO the world's just not that simple.  Eventually, you'll need
access to other config data (interfaces) and datastore-wide constraint
checking.   The good news is that YANG handle do all this.

>I am not

My hope is that I can define simple transformations between IETF
models and JUNOS models.  I think it's unlikely that JUNOS will
adopt the DHCP module directly, but if we can generate data that
is ietf-dhcp.yang with junos augmentations.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 03 Nov 2009 14:14:53 -0800

> checking.   The good news is that YANG handle do all this.

Maybe it's a better-than-nothing approach, but nothing
is what we have now wrt/ standard configuration.
For 20 years, vendors have been claiming they
cannot agree on every detail of any sort of config.

YANG has some formal clauses, bit no new optionality features
that have not existed before, so I do not think 'if-feature'
will be enough to facilitate standard data models.

It is a total myth that there is any sort of procedural interoperability
with NETCONF. When it comes to applying arbitrary
explicit and implicit nc:operation attributes throughout
the conceptual config tree -- especially for copy-config and commit,
it is very implementation-dependent.

What constitutes an edit?
Is it just the data that changed?
What about applying defaults?
For 'explicit default' servers, a 'client-replacing-server-default'
is an edit, but it is not an edit anywhere else.
Do non-edits get saved to NV-storage?  Does confirmed-commit
still proceed for NO-OP edits?  What about nested nc:operation
attributes?  Which combinations does the server allow?
Does the server support alteration of every parameter for an
existing list entry?

> is ietf-dhcp.yang with junos augmentations.

An analogy from another industry is General MIDI.
Even though a fancy keyboard has 1000 details to create
an instrument, not everybody wants to spend 5 years
studying the manual before they could just make the thing
sound like a piano, organ, trombone, etc.

So all the keyboards support a standard set of instruments,
which can be used as-is, or edited like the 'from-scratch'
instruments.  It is understood that the Classical Piano
from 2 different vendors may not sound exactly the same,
and certainly will not be configured the same.

So back to Junos -- in this analogy, you would support
the general IETF RPC operations that do limited things,
and these RPCs would map directly to your vendor data structure(s),
for experts to use.

Summary:

  ietf-start-dhcp(P1, P2, P3)

      [server implementation]
       --->  Junos 'dhcp' container;  P1 --> Junos(P1), etc.

Your expert customers will just use the complex vendor config data,
but if they do not care about the details, they may use ietf-start-dhcp.

IMO, the current IETF processes have produced no positive results,
so maybe the all-or-nothing, giant data-model or bust approach
is the reason why.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 3 Nov 2009 21:18:06 -0500

>but if they do not care about the details, they may use ietf-start-dhcp.

I'd rather see:

   <edit-config>
     <config>
       <ietf>
         <dhcp>
           <thing>
             <p1>v1</p1>
             <p2>v2</p2>
             <p3>v3</p3>
          </thing>
        <dhcp>
      <ietf>
    </config>
  </edit-config>

Then the incremental cost is adding a new parameter (vendor specific or not)
is just adding the appropriate element under <thing>.

   <edit-config>
     <config>
       <ietf>
         <dhcp>
           <thing>
             <p1>v1</p1>
             <p2>v2</p2>
             <p3>v3</p3>
             <acme:hardware-accelerated-dhcp>
               <acme:fusion/>
               <acme:bandwidth>10g</acme:bandwidth>
               <acme:cool>
                 <acme:rating>very</acme:rating>
                 <acme:target-audience>18-25</acme:target-audience>
               </acme:cool>
             </acme:hardware-accelerated-dhcp>
          </thing>
        <dhcp>
      <ietf>
    </config>
  </edit-config>

In the end, you'll build your content and make a call to your
netconf library to say "add this chunk of xml to this device's
configuration", and the library will hide all the netconf-layer
stuff, like :candidate, :distinct-startup, :confirmed-commit, etc.

So the upper-level call will be "add this config", which is close
enough to your "command-oriented interface".

Thanks,
 Phil