Subject: [netmod] module conformance vs. agent-variance

From: Andy Bierman <[email-masked]>
Date: Sat, 03 May 2008 14:19:23 -0700

Hi,

I'm not sure if I'm reading all the emails right,
but it seems that sometimes we are mixing 2 very different
concepts and language mechanisms.

1) module conformance

For each revision, it must be clear what is mandatory-to-implement,
what is conditional (e.g., when-capability) and what is simply
optional-to-implement.  Unless otherwise stated, this refers
to the agent implementation, not the manager implementation.

2) agent-variance

For each release of each agent implementation, there may be
aspects which are not compliant to some part
of the module conformance.  In SNMP, an AGENT-CAPABILITIES
document can be stored offline and used to figure out the
differences between what the agent claims to support,
and what it is willing to admit it does not support.

There has never been any support for retrieving
this data from the agent.  Vendors want to keep this sort
of info as hard to know as possible.

Q1: What to do about module conformance in NETMOD?

Q2: What to do about agent variance in NETMOD?

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 03 May 2008 23:57:07 +0200 (CEST)

Hi,

Andy Bierman <[email-masked]> wrote:
> optional-to-implement.

I'm not sure I understand the distinction between
conditional-to-implement and optional-to-implement - couldn't
capabilities be used to cover them both?

I think that capabilities would be the NETMOD version of
OBJECT-GROUP.

> of info as hard to know as possible.

Currently in YANG we just have conformance to entire modules.  With
'with-capability' we would have conformance to complete capabilities.

Could this be good enough for now?  Maybe we need some experience of
these constructs before we do (if we do) more detailed agent-variance,
esp. if the SMI experience is that AGENT-CAPABILITIES rarely is used.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 03 May 2008 16:18:08 -0700

> capabilities be used to cover them both?

yes -- the derivative 'condition' is 'none'.

Conditional means you MUST implement this object if the
condition is met.

Optional means there are no conditions under which the
object is required to be implemented.

> 'with-capability' we would have conformance to complete capabilities.

I don't really like the capability/with-capability form
of conformance documentation (as much as it is defined at all).
There can be arbitrarily nested clauses within the data structures
in your example.

This seems in the same vein as 'import-by-revision-date' to me.
Something that seems like a rather minor feature takes
on massive complexity, then more complexity to cover up
some flaws, then a little more, then a little more.

By the time these minor features are fully documented,
inter-operable, and able to pass IESG inspection,
half the language could be done.

> esp. if the SMI experience is that AGENT-CAPABILITIES rarely is used.

I am not so willing to dismiss the 20 years experience we have in
the IETF with MODULE-CONFORMANCE in SMIv2.

I am not so willing to start the 'experience meter' over, just
because we are using XML now, instead of ASN.1/BER encoding.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 03 May 2008 16:38:58 -0700

Andy Bierman wrote:
>..

IMO, it would be better to start off simple with
minor features like import and conformance, rather
than spend a long time designing something by committee.

That would mean, for the current issues:

  -- you can only import the latest version of the module

  -- there is no concept of a partial conformance level.
     Capabilities can be defined outside the language.
     Conditions based on capabilities can be mentioned in description
     clauses. (This is what NETCONF has now.)

  -- there is no documentation of agent variance.

To be fair, these features were not added on Day One of SMI either.

We probably need to consider conformance to an XML namespace
as well as module conformance.  Is there a difference?
We probably need to consider augments, groupings, choices,
and deep tables when designing conformance and variance
mechanisms for NETCONF/YANG.

I think the design team got it right by leaving
these items off the feature list in the first place.

Andy

---

From: "David Harrington" <[email-masked]>
Date: Mon, 5 May 2008 14:07:01 -0400

Hi,

I never asked for agent-variance, nor has anybody else.

My concern is that Netconf is adding support for schema discovery, and
for imports of sub=schemas that could change, and it is unclear what
rules must be followed in reporting what schemas are supported.

If my agent says during schema discovery that it supports the FOO
module, what does that mean? What if I actually implement one object
froma a large module. My agent is obviously not compliant; does this
mean I should not report this schema during discovery?

The point came up when talking about versiuoning and the
all-or-nothing compliance to a module. In many cases, as we know from
SMI experience, a module definition may contain more objects than it
makes sense to support by an implementation. This is known to be true
for monitoring purposes, and likely to be common in the case of
configuration data (I cannot help but think of the IPSec Policy MIBs).
Often a WG will define a standard module that contains a bunch of
tables, some of which model functionality that is optional in the
underlying technology, or really cannot be used to model a vendor's
actual implementation. Vendors often "cherry-pick" MIB modules to
support the objects they consider important, and ignore those they
consider unimportant. 

This raises two questions:
1) whether we should allow for some types of partial compliance to
support underlying optional functionality, and
2) if an agent only partially supports a module standard, what should
it report during schema discovery? 

Support for multiple compliace clauses in standard MIBs was added
based on operational experience. If partial compliance is not allowed,
then should we (try to) ensure that all modules are broken down into
small enough chunks that full compliance is likely in the real world,
or do we make compliance so hard to achieve in real world
implementations that any "full compliance" edict effectively becomes
meaningless?

Schema discovery is in the process of being defined. Is there any
implication for an agent reporting a module during discovery that the
agent is fully compliant? If that is not the case, then I think the
schema discovery capability needs to be very clear that no such
assumption should be made by the querying application. This implies
that using the reported schema to validate data might prove
inaccurate. Are there any clearly defined rules about when a schema
should or should not be reported in schema discovery, and does NETMOD
need to provide anything to support those rules?

dbh

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Saturday, May 03, 2008 7:39 PM
> To: Martin Bjorklund
> Cc: [email-masked]
> Subject: Re: [netmod] module conformance vs. agent-variance
> 
> Andy Bierman wrote:
> > Martin Bjorklund wrote:
> >..

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 May 2008 11:33:47 -0700

> I never asked for agent-variance, nor has anybody else.

Scratch it off the list then ;-)

> rules must be followed in reporting what schemas are supported.

The problem is that "uses foo:someGrouping;" is not static.
We want module bar, version 2008-05-05 to have the same
contents, no matter when it is compiled.  That is not true in YANG.

If 'someGrouping' changes in module foo over time, then
each recompile of cast-in-stone "foo, version 2008-05-05"
will be different.  IMO, this is somewhat broken.

What does it mean to advertise that your agent
supports "foo, version 2008-05-05" if the actual contents of foo
may depend on some arbitrary subset of all other module versions
advertised?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 May 2008 23:27:56 +0200 (CEST)

Hi,

> rules must be followed in reporting what schemas are supported.

Agreed.

> mean I should not report this schema during discovery?

Isn't what you're asking for here agent variance?

> it report during schema discovery? 

There are two parts to the schema discovery process.  One part is the
capability list which is already supported in the base NETCONF
protocol.  The netconf monitoring draft adds a mechanism to find the
actual schema files (or YANG module files).  The schema list already
supports multiple versions, and it might include "type libraries"
which are not advertised as separate capabilities.

Currently in YANG, w/o any MODULE-COMPLIANCE equivalent, the server
advertises, for each module it supports, the capability
<namespace-uri>?<revision>, e.g.

   http://example.com/foo-module?2008-05-01

With a finer grained MODULE-COMPLIANCE equivalent, the same syntax
could be extended:

   http://example.com/foo-module?revision=2008-05-01&compliance=full

But the equivalent of AGENT-CAPABILITIES (if we want to do it) would
probably be reported in some other way...

> need to provide anything to support those rules?

I think NETMOD should provide rules for how YANG modules /
capabilities are reported during schema discovery.  As long as the
schema discovery capability is DML-agnostic it cannot have too
specific rules.

/martin

---

From: "David Harrington" <[email-masked]>
Date: Mon, 5 May 2008 18:30:32 -0400

> Isn't what you're asking for here agent variance?

No. Agent variance in SNMPv3 is a external document (similar to
release notes) that **details** all the little variances in your
implementation:

"The agent supports counter ABC, but the spec says it should have a
range of 1..4B and we only support up to 2B.
The agent does not support column X and Y in the WXYZ table.
The agent does not support the QP4 interface elements, so therefore
does not support the QP4 table."
 and so on.

If you report a capability, and that capability is related to a
specific data model, and you do not support every element in the data
model, can you claim that you support the capability? 

How much can an NMS application actually rely upon the capability
negotiation (or the schema provided) to imply that the whole data
model has been implemented? Does the capability advertisement mean a)
it absolutely supports everything in the data model, or b) it supports
at least part of the data model? 

>    http://example.com/foo-module?revision=2008-05-01&compliance=full
...
> schema discovery capability is DML-agnostic it cannot have too

In SMI, we found it necessary to go to a finer-grained compliance
clause than just "full". I don't think a binary "everything or you're
not-compliant" rule makes sense given our previous experience.

And a binary "everything or you're not-compliant" rule makes no sense
if imported groups can change underneath the importing module.

I think we need to lock down "uses" and we need to have
"protcol-sense" granular compliance clauses.

dbh

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 06 May 2008 10:07:38 +0200 (CEST)

Hi,

> release notes) that **details** all the little variances in your

Yes I know.

> at least part of the data model? 

Unless we claim a), isn't some kind of agent variance needed?  If we
say b then w/o agent variance anyone can claim conformance to anything
by implementing some small subset of each capability.  That will not
help interoperability.

> not-compliant" rule makes sense given our previous experience.

Right.  What I meant was something like this:

  module foo {
     namespace "http://example.com/foo-module";
     revision 2008-05-01;

     ...

     compliance basic { ... }
     compliance feature-x { ... }
     compliance feature-y { ... }
     compliance full { ... }

  }

Then a server that conforms to the 'full' compliance would advertise
the capability above.

> I think we need to lock down "uses"

Agreed.  IMO this is a must (and we have 3-4 proposed solutions).

>  and we need to have

Yes this would be useful.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 06 May 2008 11:39:16 -0400

>Then a server that conforms to the 'full' compliance would advertise

What about doing this with the when-capability mechanism?  "basic"
would be the base capability for the module, "x" and "y" would be
additional capabilities, and "full" may or may not be needed, if
it's just "x + y".

Using the "when-capability" statement has the nice effect of keeping
the condition with the node being defined, and not having to repeat
nodes in a "compliance" statement.

    module foo {
        ...
        capability feature-x { ... }
        capability feature-y { ... }

        container ahh {
            container beh {
                container say {
                    container day {
                        when-capability feature-x;
                        ....
                    }
                    container night {
                        when-capability feature-y;
                        ...
                    }
                }
            }
        }
    }

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 May 2008 10:03:09 -0700

>         capability feature-x { ... }

What goes in the { ... } (description, reference, status?)

>                         when-capability feature-y;

1) Conformance

I like 'when-capability' as a variant of the 'when-stmt'.
This is better than adding a meta-containment layer.

It seems 'all-or-nothing' conformance isn't winning the day.
In that case, I strongly favor a conformance model based
on capabilities, since the protocol already supports it.

A separate conformance section requires data to be entered
more than once, but may be easier to read, as it keeps the
data definitions uncluttered with conformance meta-data.

Entering data more than once can lead to human-introduced errors.
Cluttered confusing data models can lead to human-introduced errors.
IMO, the inline approach (Phil's proposal, also in netconf.yang)
is best because:
   - it works for groupings and inline data
   - reproducing the object tree by hand can be nearly impossible
     for deep data models, almost ensuring human-introduced errors

2) Revisions

Note that the 'grouping revision issues' are still unsolved
by both approaches.

IMO, the simplest approach is to mandate (outside the YANG language)
that standard YANG groupings MUST NOT be changed in certain ways,
just like SMIv2 does with TEXTUAL-CONVENTIONs.  That way, import
is simple, and conformance-by-capability (or by-anything) will
be stable.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 06 May 2008 22:44:00 +0200 (CEST)

Hi,

> What goes in the { ... } (description, reference, status?)

And uri and when-capability:

  capability confirmed-commit {
    uri  urn:ietf:params:netconf:capability:confirmed-commit:1.0;
    when-capability  nc:candidate;
    description  "...";
  }

> on capabilities, since the protocol already supports it.

I agree.  But the capability list reported in each hello message might
get long... 

> data definitions uncluttered with conformance meta-data.

Right.  With when-capability, you have to scan the entire module to
see if there are any more occurances of 'when-capability foo'.  So for
a reader it takes some work to figure out exactly what constitutes a
capability.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 06 May 2008 17:15:50 -0400

>a reader it takes some work to figure out exactly what constitutes a

I see it the opposite.  When I'm looking at a node, I want to see
if that node is affected by capabilities.  I don't want to go off
and check a list (hierarchy?) of nodes in another section to see
that this is conditional.  We put things inline so the reader can
see what's required directly, with the information as localized as
possible.

If you like the 'distinct section' approach, would you apply this
approach  to "status"?  A distinct "status" section containing a
list of nodes that are deprecated?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 06 May 2008 23:27:43 +0200 (CEST)

> see what's required directly, with the information as localized as

I agree with this.

> list of nodes that are deprecated?

I didn't say I like it ;-)

What I *would* like is a syntax w/o repetition which made it easy to
see what is contained in a capability, and at the same time when
looking at a node made it clear if the node was part of a capability
or not.  I don't think this is possible though, and of the
alternatives we have now I think the when-capabiltiy is least bad.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 May 2008 14:34:59 -0700

> list of nodes that are deprecated?

I agree with Martin.
An implementor or an operator is going to want to know
what new data is going to be on the agent if this capability
is supported.  Tracking down this info in one or more modules
would be quite a challenge, if when-capability clauses were
scattered through the data model.  Nested combinations
are not that obvious either.

It is also useful to know that 'when-capability' applies,
when reading the data model inline.

How are changes to the capability tracked over time? (Poorly.)

One solution is 'external text', like in RFC 4741.
There are sections that describe what the capability changes
in the protocol, meant for humans.  There are no machine-readable
sections for this purpose, and maybe that's not a big deal.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 07 May 2008 08:41:20 -0400

>alternatives we have now I think the when-capabiltiy is least bad.

Agree.  Generating a list of capability-enabled nodes
would be a simple xslt script that can be run on the YIN
format of a YANG module.

Thanks,
 Phil