Subject: [netmod] WGLC comments on deviation-stmt

From: Andy Bierman <[email-masked]>
Date: Thu, 07 May 2009 10:34:51 -0700

Hi,

IMO, the deviation-stmt as currently designed
is not implementable.  I know of 2 developers
who have tried, but both of us needed to indicate
the deviations file(s) to the compiler along
with the module-under-test (MUT).

There is no indication whatsoever in the MUT
that any external deviations exist.  Unlike
augment (the only other way this can happen),
deviations are destructive to the MUT.
An external augment cannot possibly alter
the validity of the MUT by itself (by design).

In order to make use of deviations, they must be implementable.
The current approach in the standard is that deviations
can appear in any module, and alter the contents
of any other random module.  IMO, this is not implementable.

A compiler needs perfect knowledge of all the deviation files
in advance, and standard ignores this fact completely.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 07 May 2009 20:33:38 +0200 (CEST)

Hi,

> with the module-under-test (MUT).

But this is the intention.  As you describe below, this is how
deviations work.

In general of course YANG does not *require* any tool chain at all -
you are free to write your YANG models and code them by hand.  If you
do, you can write down in your deviations module how you deviate from
the main module.

But if you have tool support, you will first write your deviation
module how you will deviate from the main module, then feed the main
module and the deviations module to your tools, which will generate
code for the "patched" module.

In both cases, your agent will advertise to the client the main
module, and along with it also the deviation module.

> that any external deviations exist.

Just like in SMIv2 where there is no indication in the IF-MIB that
vendor X has a AGENT-CAPABILITIES module describing how it deviates
from the IF-MIB.

> Unlike

It does not ignore this.  In fact, this is the reason that the
deviation module is advertised *in the same capability string* as the
main module.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 07 May 2009 11:56:36 -0700

> But this is the intention.  As you describe below, this is how

This is not specified in the standard.

There is a difference between what a user MAY choose to
do with the tools and what a tool MUST support.
Assuming that tools will have mechanisms to identify
all the deviations files associated with a module should
be written in the standard (or avoided completely).

I think the IESG and other reviewers will be
interested in knowing how interoperability for
deviations-stmt is going to be measured
in the future, when YANG either advances on the standards
track or moves to Historic status.

I have already posted emails demonstrating that implementation
of deviations depends on the order the deviations are applied.

The point of a standard data modeling language is to
provide a set of constructs that will be implemented
consistently across all devices and tools.  That means,
given the same YANG module (in isolation), two independent
implementations should agree on the validity of that module.

I do not think deviations-stmt currently meets that minimum
level of interoperability.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 8 May 2009 09:16:34 -0400

>implementations should agree on the validity of that module.

And the point of deviations is to admit that, for a variety of
reasons, the constructs detailed in a standard module cannot be
implemented consistently across all devices.

Historically, an implementation that is not able to fully implement
that standard has two choices: (a) don't implement that standard,
or (b) lie and say they implement the full standard, fudging input
and output when needed.  Given marketing/customer/plm pressures,
(a) may not be an option.  If the device support 99% of the standard,
lying about the one bit that cannot be implemented may well be the
lesser of two evils.

Deviations give implementations a third option, allowing them to
detail the places where they fail to fully implement the standard.
This in no way reduces the need to push for interoperability by
maximizing the ability of an implementation to fully statisfy the
constraints of the standard, but by allowing the truth to be known,
applications can better deal with implementations that aren't fully
compliant.

So interoperability does not benefit from implementations that
deviate from the standard, but interoperability _does_ benefit from
allowing implementations to express (and applications to intergrate
the knowledge of) the places where the standard has not been fully
implemented.

We've not encouraging deviations, but we are wise enough to admit
that they occur and should not be ignored.

The working group has repeatedly decided that this feature is useful
and worthwhile.  Nothing in the above comments is new.  Do we really
need/want to rehash this again?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 May 2009 08:29:03 -0700

> need/want to rehash this again?

You misunderstood my point.
I am not trying to remove deviations-stmt.

Every single language construct in YANG must be implementable
in a manner that will allow data modelers to reuse their
YANG files in different tool-sets.

Every compliant YANG tool should be able to
agree that a specific YANG file contains well-formed YANG.
But arbitrary combinations of deviation modules can
result in invalid YANG.  Unless 2 validation tools
use the same YANG files, the results will not likely be the same.

The section on deviations needs to have some text that explains
that tools MAY require that the 'deviations modules' be specified
at YANG module compile time, since they cannot be easily
identified any other way.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 8 May 2009 12:50:55 -0400

>identified any other way.

Do we need to specify what tools may or may not need?  This will
vary among tool chains, and isn't constant nor part of the language.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 May 2009 10:57:26 -0700

> vary among tool chains, and isn't constant nor part of the language.

Don't we already do that by declaring that XML order
does not matter at all, except for a couple corner-cases
that are specifically intended to allow implementations
to use a non-buffering design?

The "keys MUST be encoded first" CLR isn't part of the
language either.  There is a presumption that this is
so important to tools, that all managers and all agents
MUST special code this little rule.  (BTW, what is
the error code that the agent is supposed to return?
"YANG-forced-reject-of-valid-XML" ;-)

I suspect that market forces will weed out any lame
implementations, but it is always frustrating to
consumers of a standard when implementors of
the standard do very different things and all claim
to be following the standard to the letter.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 8 May 2009 14:04:16 -0400

>to use a non-buffering design?

Nope, this part of the spec refers to over-the-wire behavior,
which belongs in the spec.  Saying that some tool chains may
need deviations at compile time .vs. run time is strictly a
tools issue.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 May 2009 11:29:00 -0700

> need deviations at compile time .vs. run time is strictly a

OK.
The reason I agree with you is that we already have
optional import/include by revision.  This is so
extremely implementation-dependent that there is
little chance that 2 tool-chains will find the
same version (if more than 1 exists) unless the
revision date is specified.  Since nobody seems to
actually use revision dates in their import-stmts,
this will become a real problem.  It is fixed
by relying on out-of-scope configuration parameters
for the YANG tools themselves.  (What irony :-)

Since we are currently on version 1 of everything,
this is not a problem yet.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 May 2009 21:25:57 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> extremely implementation-dependent

How do you mean?  If you import/include by revision there is no doubt
which module you're importing/including.  If you *don't* use this
optional feature, things are unclear.  Maybe you augement something in
the module you import, but the object you augment was added in version
2 of the module.  

 that there is

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 May 2009 21:29:51 +0200 (CEST)

[resend - I accidentally hit send too early]

How do you mean?  If you import/include by revision there is no doubt
which module you're importing/including.  What exactly is
implementation-dependent?  If you *don't* use this optional feature,
things are unclear.  Maybe module A augements something in the
imported module B, but the object module A augments was added in
version 2 of module B.  Unless module A imports B revision, one tool
set might give an error, and another tool might say it is ok.

But as has been said before, there are use cases where you do want
import without revision (IANAifType-MIB e.g.)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 08 May 2009 12:59:00 -0700

> import without revision (IANAifType-MIB e.g.)

I do not believe import-any-revision will be
used only in this special case.  It seems to
be used this way every time.  Maybe that will
change as YANG stabilizes.

If external files (imports/includes) are used
without a revision-date (very common practice) then it
is completely up to the tool implementation what file
will actually be found to match the import or include.

Since we are comfortable with this very basic level
of tool-specific behavior for an issue that shows
up almost 100% of the time, then we should also
accept the 1% corner-cases that show up wrt/ deviation-stmt.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 May 2009 15:32:09 +0200

> change as YANG stabilizes.

There is also a possible interaction with module namespace URI: A
developer of a module might decide to avoid the separate revision
mechanism of YANG and encode the revision info into the namespace URI
instead. This way, the revision would have to be always specified.

According to the draft, such an approach seems to be allowed, but
together with YANG revisions it could result in an awful mess. YANG
should IMO choose between these two options:

1. Keep the revision mechanism and state that the namespace must not
change during the module lifetime.

2. Use the namespace URI for encoding both module and revision info, and
remove the revision statement (or use it only as editorial
documentation).

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 11 May 2009 15:49:00 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> According to the draft, such an approach seems to be allowed

Section 10 lists the legal modifications to a module, and changing the
namespace URI is not listed.  In fact, the current text says:

   Furthermore, the "namespace" statement MUST NOT be changed,

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 May 2009 16:20:13 +0200

>    Furthermore, the "namespace" statement MUST NOT be changed,

It depends on how you interpret the word "module", whether it is just a
given YANG text or an abstract entity with a certain life cycle. The
first interpretation is supported by many formulations in the text,
e.g., in sec. 4.2.1.: "A module contains three types of
statements: ...".

What if the said developer encodes the revision in the module name as
well?

module foo-1.2.3 {
    namespace "http://example.com/ns/foo/1.2.3";
    ...
}

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 11 May 2009 16:25:34 +0200 (CEST)

> e.g., in sec. 4.2.1.: "A module contains three types of

I don't understand your point.  Could you propose some text that could
go into section 10 to make sure we use the word 'module'
unambiguously?

>     namespace "http://example.com/ns/foo/1.2.3";

This doesn't change anything.  If this developer ever publishes a new
revision of the module foo-1.2.3, the rules from 10 should be
followed.

If the developer publishes a module called foo-1.2.4 then that's a
completely different module than foo-1.2.3.  The module name is just
an opaque string.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 May 2009 17:40:56 +0200

> go into section 10 to make sure we use the word 'module'

If you say that a module has revisions, it means it is more complicated
than a simple content of a text file. It is important that the manager
and agent always use the same data model, so I'd prefer to always
specify (or be able to unambiguously resolve) the revision. What is
actually the exact procedure for resolving the module name and/or
namespace URI to a concrete YANG file if the revision is not specified?
What if the manager has access to the newest revisions of a module but
the agent doesn't?

> completely different module than foo-1.2.3.  The module name is just

Right, so this could indeed be used as a brute-force alternative to YANG
revisions.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C