Subject: Re: [netmod] review of draft-bierman-netmod-yang-conformance-04

From: Andy Bierman <[email-masked]>
Date: Wed, 3 Dec 2014 13:55:28 -0800

On Wed, Nov 19, 2014 at 7:20 AM, Phil Shafer <[email-masked]> wrote:
> Martin Bjorklund writes:
>>(*) section 5 might be needed unless we make include-by revision
>>mandatory.  But I think I am the only one that thinks that would be a
>>good solution.
>
> I wouldn't say make it mandatory, but certainly mandatory for
> anything externally published, including IETF work.  Using
> revision agnostic imports might be suitable for internal and
> development work, but once a spec is supposed to be fixed and
> hardened, all imports should be by-revision.
>
>>+1.  A lot of the confusion comes from this.  Obviously, the spec is
>>not clear on this - it doesn't specify what it means to "implement" a
>>module.
>
> Yes, this should be clarified.
>

This might be the least painful solution, except it needs some clarification.
The most common use of typedefs and groupings is to use them in
the module they are defined, so conformance for meta-data and real data
has to be separated.

I think this simple set of rules might work:
  1) Import-by-revision and include-by-revision everywhere
  2) imported meta-data from A is from the specified revision
       when used in B
  3) for conformance of real objects in A, the most recent advertised
    revision must be implemented
  4) even if multiple revisions of A are advertised (B->A.1, C->A.2)
      any protocol accessible data must use the most recent
      advertised revision
  5) not clear if the most recent revision of any identities and extensions
     needs to be used or not

>>It must be ok to advertise b and a@2014-05-01.
>
> Yes, since the contract for 'a' remains in place.
>
>>A very simple alternative would be to not allow updated typedefs and
>>groupings at all.
>
> This shifts the version tracking task on to the author and
> reader.  "foo_v2", "foo_v3", "foo_v2014-04-01", etc.  The
> updating costs and possibility of errors would be significant.
>
> Thanks,
>  Phil
>

Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Phil Shafer <[email-masked]>
Date: Thu, 4 Dec 2014 11:24:14 -0500

>the module they are defined, so conformance for meta-data and real data

We should be careful of the term "meta-data" here.  Metadata is
data that describes data, so all YANG is meta data.  I think you
are talking about typedefs and groupings which might be better
termed "macro data".

>    revision must be implemented

We should clarify that advertising module A at revision X means
that all top-level containers, lists, leafs, leaf-lists and augments
are supported.  Imported modules used by module A do not need to
be advertised.  Advertising imported modules mean that you are
supporting all top-level containers, lists, leafs, leaf-lists and
augments from that module, without regard to their use in A.

If module A support a top-level container, and module B augments
that top-level container, advertising B without advertising A
is a logic error.

>      any protocol accessible data must use the most recent

Only one revision of any module should ever be advertised.  By advertising,
you are saying that you will be bound by the contract given in that module,
so saying you support multiple revisions would be a logic error.

>  5) not clear if the most recent revision of any identities and extensions

This is trickier, but still clear.  If A-2014-01-01 lists an extension
and B-2014-01-02 imports and uses it, then advertising B-2014-01-02
means that anything in B-2014-01-02 that uses that extension means
the A-2014-01-01 implementation of that extension.  If A-2014-02-01
change the meaning of the extension (is that even allowed?) and
C-2014-02-02 uses it, then anything in C-2014-02-02 wants the
A-2014-01-01 implementation of that extension.  B-2014-01-02 would
continue to want the older definition.

Thanks,
 Phil

---

From: Kent Watsen <[email-masked]>
Date: Thu, 4 Dec 2014 21:53:20 +0000

>augments from that module, without regard to their use in A.

+1

>that top-level container, advertising B without advertising A

Why an error?  It seems more like a MAY than a MUST to me...

>so saying you support multiple revisions would be a logic error.

Yes, since the passed namespace doesn't qualify revision

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Dec 2014 14:39:06 -0800

>>augments from that module, without regard to their use in A.

The term "advertise" is no longer accurate.
For example, RESTCONF uses the ietf-yang-library to specify all the
YANG modules it uses.

There is a leaf called conformance to distinguish between a module
listed for complete module inventory vs. conformance:

       leaf conformance {
           type boolean;
           mandatory true;
           description
             "If 'true', then the server is claiming conformance to
              the YANG module identified in this entry.

              If 'false', then the server is not claiming any
              conformance for the YANG module identified by this
              entry. The module may be needed for reusable definitions
              such as extensions, features, identifies, typedefs,
              or groupings.";
         }

So the server will list all the revisions of module A, but only
set the 'conformance' flag to true on 1 of them.

> Why an error?  It seems more like a MAY than a MUST to me...

What happens if B augments /A:foo in R1 but the server implements R2
of the /A:foo container? What if /A:foo is changed to obsolete in R2?
How can the server stick to its original A.n contract it it changes
any status to obsolete in A.n+i?

> Yes, since the passed namespace doesn't qualify revision

YANG 1.1 is not going to advertise modules, just an opaque
identifier to indicate the module set ID.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 4 Dec 2014 18:00:03 -0500

>Why an error?  It seems more like a MAY than a MUST to me...

module A {
    container a { ... }
}

module B {
    augment /a:a {
        container b { ... }
    }
}

If you don't support the /a:a container, how can you augment it
with /a:a/b:b?  To implement any sort of meaningful augmentation,
you'd have to implement the contract the base node appears in.
Otherwise, you'd be implementing /a:a without implementing any
sort of content under it.

Thanks,
 Phil

---

From: Kent Watsen <[email-masked]>
Date: Fri, 5 Dec 2014 00:10:02 +0000

>sort of content under it.

What if A contains more than one top-level container, but B only augments
one of them?  Does the server still have to implement all of A?

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Fri, 5 Dec 2014 15:50:19 +0000

>one of them?  Does the server still have to implement all of A?

Never mind, I see it now.  Yes, the server needs to implement all of A.

Kent

---

From: Andy Bierman <[email-masked]>
Date: Fri, 5 Dec 2014 08:06:53 -0800

> Never mind, I see it now.  Yes, the server needs to implement all of A.

So what happens when module X augments rev. 1 of  /A:foo,
module Y augments rev. 2 of  /A:foo, and the server advertises
support for module A, rev. 3?

What happens when module A is using typedefs from rev 3 in
the /A:foo subtree, and modules X and Y are using the rev 1 and rev 2
versions of the typedefs?

IMO the possible ways that multiple revisions can mix with import-by-revision
is probably unmanageable.  There are too many statements that can reference
other nodes, so mixing revisions of macro-data will occur.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 5 Dec 2014 16:07:26 -0500

>one of them?  Does the server still have to implement all of A?

Implementing half a module is equivalent to following half
a contract.  The client wouldn't be able to know what is
and isn't there.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Sun, 7 Dec 2014 10:30:44 -0800

> a contract.  The client wouldn't be able to know what is

You are suggesting that if B augments A, then the server MUST
claim conformance to A if it claims conformance to B?
If the augmented object requires YANG features, then the server MUST
support the required features in order to claim conformance to B?

The proposal in RESTCONF/ietf-yang-library is that if the
"conformance" flag is false, then the server does not need to
advertise deviations for that module.  It would be possible for
the server to claim conformance to B and not A.

There are currently no rules in NETCONF/YANG requiring a server
to claim conformance to any specific modules (except maybe
ietf-netconf.yang for NETCONF 1.1).

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 8 Dec 2014 16:30:15 -0500

>claim conformance to A if it claims conformance to B?

Yes.  Augmenting something a server doesn't support makes little sense.

>support the required features in order to claim conformance to B?

No.  Features are a per-server issue; modules aren't.

>advertise deviations for that module.

I need to go read this draft, but it doesn't really sound clear.

>It would be possible for

This would make no sense.

>ietf-netconf.yang for NETCONF 1.1).

Seems like it's time to think this thru and add some.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 08 Dec 2014 23:11:08 +0100 (CET)

> >  1) Import-by-revision and include-by-revision everywhere

Import-by-revision everywhere quickly leads to problems if we also say
that a server only advertises one revision of a module.

> are supported.

And identities.  But actually, you shouldn't have to support all
identities.  This use case is handled by the combination of the
'conformace' leaf in ietf-yang-library and the rpc
'get-allowed-identities' in ietf-yang-conformance.

> Imported modules used by module A do not need to

Yes, this was the original intent in YANG 1.0.

But it leads to problems.  Suppose module A imports (not by-revision)
module B, and uses some typedef from module B.  Which version of the
typedef does the server implement?

One way to solve this is to mandate import-by-revision.  But that
leads to problems if the server only can advertise one revision of a
module it implements.

Another solution is to use the 'conformance' leaf in
ietf-yang-library.

A third solution could be to make the update rules even more stricter,
and make any change to typedefs and groupings illegal.

> that top-level container, advertising B without advertising A

Yes, again this was always the intent, and I think it works.  But, it
is overly restrictive in one case.  Suppose I want to augment
/sys:system with tacacs config.  Now, a server that wants to implement
tacacs also is forced to implement /system/clock.  It would be nice to
be able to support np-containers w/o advertising the whole module.

> so saying you support multiple revisions would be a logic error.

Agreed.

/martin

---

From: Kent Watsen <[email-masked]>
Date: Tue, 9 Dec 2014 20:53:52 +0000

>that a server only advertises one revision of a module.

Why is this?  Can you provide an example?

IIUC, a server only needs to advertise (via hello or yang-library) the
top-level modules it implements, not any modules imported by those
top-level modules, unless a container they declare is being augmented.
For instance, a device supporting ietf-system does not need to also
advertise support for ietf-yang-types, ietf-inet-types, ietf-netconf-acm,
or iana-crypt-hash (assuming we file errata against RFC7317 to import
these modules by revision).

If the above it true, then it should also be possible for the server to
advertise more than one module that imports different revisions of some
underlying module.  For instance, in the following example, it would be
valid for the server to only advertise "b@2014-06-01" and "c@2014-07-01":

      module a {
        revision 2014-04-01;
      }

      module a {
        revision 2014-05-01;
      }

      module b {

        revision 2014-06-01;

        import a {                   // not imported for purpose of
augmentation
          revision-date 2014-04-01;
        }
      }

      module c {
        revision 2014-07-01;
        import a {                   // not imported for purpose of
augmentation
          revision-date 2014-05-01;
        }
      }

Now assume a newer version of 'a' is released and the server wants to
support it explicitly as a top-level module.  For instance, in the
following example, what can break if the server advertises "b@2014-06-01"
and "c@2014-07-01" and "a@014-08-01"?

      module a {
        revision 2014-04-01;
      }

      module a {
        revision 2014-05-01;
      }

      module a {
        revision 2014-08-01;
      }

      module b {
        revision 2014-06-01;
        import a {                   // not imported for purpose of
augmentation
          revision-date 2014-04-01;
        }
      }

      module c {
        revision 2014-07-01;
        import a {                   // not imported for purpose of
augmentation
          revision-date 2014-05-01;
        }
      }

Playing devil's advocate, let's assume the next version of module 'c'
decides to augment a container provided by a@2014-05-01 (see below).  I'm
assuming that this would not be allowed (right?), that the next version of
'c' would have to import the current version of 'a' (2014-08-01) and thus
the server advertising a@2014-08-01 and c@2014-09-01 is OK.

      module c {
        revision 2014-09-01;
        import a {                   // not imported for purpose of
augmentation
          revision-date 2014-08-01;
        }
      }

Thanks,
Kent

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 09 Dec 2014 22:10:24 +0100 (CET)

> Why is this?  Can you provide an example?

  module a {
    ...
    revision 2014-04-01;
    container x;
  }

  module a {
    ...
    revision 2014-05-01;
    container x;
    container y;
  }
  
  module b {
    ...
    import a {
      prefix a;
    augment /a:x {
      ...
    }
  }

  module c {
    ...
    import a {
      prefix a;
    augment /a:y {
      ...
    }
  }

Now, my server wants to support both b and c.  Which version of a
should it implement?  The only reasonable choice is 2014-05-01, but
that means that for b, import-by revision really means
import-by-min-revision.  This may or may not work.

/martin

---

From: Kent Watsen <[email-masked]>
Date: Tue, 9 Dec 2014 22:00:10 +0000

>import-by-min-revision.  This may or may not work.

Yes, "import-by-min-revision" seems to resolve it, given the backwards
compatibility rules defined in 6020 section 10.  Also, since 'a' is having
a container augmented, the server would have to explicitly advertise
support for 'a' and, in this case, it would have to be a@2014-05-01.

BTW, you're back the augmenting-a-container problem.  My post was mostly
trying to explore if there is anything else besides this.  Is there?

Kent