Subject: [netmod] import-by-revision not used

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 12:07:49 -0700

Hi,

If import-by-revision is so easy to use, how come
not 1 single YANG module in a Internet Draft is using it?
Is the plan to start using it in the future, or just
set it in RFC versions?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 04 Sep 2009 23:29:37 +0200 (CEST)

> Is the plan to start using it in the future, or just

Maybe because the YANG drafts we currently have only import one of the
type modules?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 15:03:18 -0700

> Maybe because the YANG drafts we currently have only import one of the

But the author of module 'X' does not have change control
over module 'Y' in another RFC/I-D, right?
So an 'import Y' can break at any time in the future.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 05 Sep 2009 00:12:45 +0200 (CEST)

> So an 'import Y' can break at any time in the future.

How do you mean break?  The ipfix module uses yang:object-identifier,
yang:counter64 and some more types. 

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 04 Sep 2009 15:32:58 -0700

> yang:counter64 and some more types. 

In general, the importing module does not know for sure
that a the imported module will change in the future
in a way that alters the previous definitions in use.
Not even the yang-ietf-types module is immune to changes
and bug-fixes.

I think that's why import-by-revision was added.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 06 Sep 2009 23:12:33 +0200 (CEST)

> I think that's why import-by-revision was added.

Yes, but in some cases maybe you don't care.  If a bug is fixed in
ietf-yang-types, do we really want to re-publish all importers?  As an
importer, you know what kind of changes you can expect in new
revisions, and you will have to decide if you need to import by
revision or not.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 06 Sep 2009 19:30:37 -0400

The problem I have with the description below is that it is not possible 
for the person writing the import statement to know if he will care 
about changes that may  or may not be made to a document he is importing.

For various (mostly good) reasons, we do not have substitutability 
requirements on revisions.  This means that some revisions "fix bugs" or 
otherwise make changes that will not break a user of the module.
Other revisions can change very basic assumptions, and break uses of the 
module.

When writing an import statement, there is no way to know what will 
happen in the future.  It seems to follow that one of two things has to 
be the case for an import statement to be writeable.
Either the import is by revision, with th restriction that if you want 
to work with a newer revision, you have to say so.  (Which leads to 
wanting to specify a range of revisions.)
Or there needs to be a way to indicate that for some modules only 
compatible revisions will ever be made, so that an import without a 
revision can be done reliably.  Unfortunately, there is currently no way 
to say that.

The current situation is a recipe for trouble.  A module A without 
revision can only be imported without revision.  But if module B imports 
that, there is no way to have any check as to whether the module A it 
gets actually meets the properties it expects.  I don't see how we can 
claim this gives interoperability.
As currently defined, an RFC using an import without revision is 
completely unreliable.  Hence, at least for RFCs, it seems that we need 
to mandate (not recommend in the guidelines document, mandate) revisions 
and importing by revisions.  (And having done that, I don't see why we 
would permit anyone else an escape.)

Yours,
Joel

Martin Bjorklund wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Andy Bierman <[email-masked]>
Date: Sun, 06 Sep 2009 16:34:24 -0700

> revisions, and you will have to decide if you need to import by

I don't think the data modeler can be sure
a default-stmt will never be added to module X
during the lifetime of module Y (which imports X),
especially if the IETF controls the updates to X.
If all the data types used already have defaults,
then it is safe.

If floor(server-requirements) == floor(client-requirements)
then the rules for client-derived defaults at least
make sense.  At a conceptual level, if everything has
a MUST or SHOULD conformance level, then deriving
a default value SHOULD work.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 07 Sep 2009 07:45:18 +0200 (CEST)

If the ipfix module imports ietf-inet-types and uses inet:ip-address,
it knows that it safe to import that module without revision.  So it
depends on what type of information you use from a module.

"Joel M. Halpern" <[email-masked]> wrote:
> The problem I have with the description below is that it is not possible for
> the person writing the import statement to know if he will care about changes
> that may or may not be made to a document he is importing.

Yes it is possible - the imported module follows the rules in section 10.

[...]

> Or there needs to be a way to indicate that for some modules only compatible
> revisions will ever be made, so that an import without a revision can be done
> reliably.  Unfortunately, there is currently no way to say that.

The intention of section 10 is to provide rules that protects clients
and importers.  Of course, clients that want to be future-proof will
have to be written in a way that takes those rules into account (just
as an SNMP manager).  And importers that cannot risk e.g. a new
optional leaf in a grouping will have to use import by revision.

> The current situation is a recipe for trouble.  A module A without revision can
> only be imported without revision.  But if module B imports that, there is no
> way to have any check as to whether the module A it gets actually meets the
> properties it expects.  I don't see how we can claim this gives
> interoperability.
> As currently defined, an RFC using an import without revision is completely
> unreliable.  Hence, at least for RFCs, it seems that we need to mandate (not
> recommend in the guidelines document, mandate) revisions and importing by
> revisions.  (And having done that, I don't see why we would permit anyone else
> an escape.)

See above.  I don't think is desirable for type library module.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 07 Sep 2009 07:53:35 +0200 (CEST)

> If all the data types used already have defaults,

I think maybe you argue that the upgrade rules in section 10 should
not allow to add a "default" statement.  Maybe we should fix this.

But I do not understand why this is a problem.  A client that uses an
optional leaf w/o a default value may choose not to set the leaf to
any value.  Depending on the semantics of the object, the server may
use some locally derived value operationally.  This operationally used
valu e might even be available for retrieval in an -oper leaf.  Now,
in the next revision of the module, a default statement is added which
sets the value "X" as default.  How does this affect the client?
Previously it made no assuptions on the value, so if the server always
used "X" it would have worked.  Now all servers use the value "X", so
why is this a problem for the client?

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 07 Sep 2009 02:07:10 -0400

Taking a quick look at section 10, it looks to me like the things 
permitted there are too generous.  The following things look 
:interesting" if the server is using a version of an included module 
which is newer than the client is using.

Allowing the addition of a default statement changes the meaning of the 
absence of configuration information.
Extending the maximum number of elements, or range of values, means that 
a reader may be presented with a value they consider invalid.  In 
particular, extending the number of elements may cause array bounds 
errors if the client trusts the server to comply with the model.

It is hard to tell whether omitting MUST statements can cause problems. 
  While one is always inclined to relax constraints, this seems to allow 
room for problems if the client is relying on those constraints being met.

Clients will certainly need to be careful in their modeling mechanisms 
to allow for the fact that mandotyr or min-element constraints may 
disappear.

Similar problems would seem to occur if a portion of a device (along the 
lines of SNMP subagents) were using a newer revision than the parent.

Even more problematic would be what happens if the client were using a 
newer version of the imported module than the device being managed.  It 
would seem that no errors would be detectable.  But the client would 
attempt to access fields that do not exist, set elements to values that 
are not permitted, violate MUST and mandatory constraints, etc.
Similar problems would occur if there are internal module users in a 
device, and the master is using a newer version than the subsidiary part.

One could, with somewhat tighter rules than are given in section 10, and 
with sufficient exchange of revisions for imported modules, write rules 
so that devices could use newer versions that specified in the import 
clause, but clients had to use version taht were at least as old as what 
the managed device used.

Claiming that the section 10 rules will make everything work without 
revisions does not match my experience in trying to model things.
Versioning is tricky.  Compatible versioning is tricky.  Compatible 
versioning with specifications as to which side has seen which change, 
and without any explicit indication of the versions is likely not to work.

Yours,
Joel

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 07 Sep 2009 08:27:36 +0200 (CEST)

"Joel M. Halpern" <[email-masked]> wrote:
> Even more problematic would be what happens if the client were using a newer
> version of the imported module than the device being managed.

We never intended to cover this case.

> One could, with somewhat tighter rules than are given in section 10, and with
> sufficient exchange of revisions for imported modules, write rules so that
> devices could use newer versions that specified in the import clause, but
> clients had to use version taht were at least as old as what the managed device
> used.

If tighter rules are needed, we should discuss them.  

FWIW, we have some implementation experience with the current rules
(on the client side).  Again, no magic is going on.  As a client
developer, you know that e.g. max-elements may be releaxed, so be
prepared to get more elements (which BTW you need to be prepared for
anyway if you ever read the candidate).

> Claiming that the section 10 rules will make everything work without revisions
> does not match my experience in trying to model things.

I do not think I claimed that.

> Versioning is tricky.  Compatible versioning is tricky.  Compatible versioning
> with specifications as to which side has seen which change, and without any
> explicit indication of the versions is likely not to work.

This whole scheme relies on the fact that revisions are used.  So I am
in favor of the change that Andy suggested:

OLD:

 For any published change, a new "revision" statement (Section 7.1.9)
   SHOULD be included in front of the existing revision statements.

NEW:

 For any published change, a new "revision" statement (Section 7.1.9)
   MUST be included in front of the existing revision statements.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sun, 06 Sep 2009 23:47:21 -0700

> See above.  I don't think is desirable for type library module.

But all imported typedefs become critical because they can contain
a default-stmt.  Knowing which revision of a typedef
is just as important as which revision of a leaf.

Andy