Subject: Re: [netmod] Y26 again, sorry

From: Randy Presuhn <[email-masked]>
Date: Thu, 10 Sep 2015 18:19:20 -0700 (GMT-07:00)

Hi -

>From: Ladislav Lhotka <[email-masked]>
>Sent: Sep 10, 2015 2:02 AM
>To: Randy Presuhn <[email-masked]>, [email-masked]
>Subject: Re: [netmod] Y26 again, sorry
>
>Randy Presuhn <[email-masked]> writes:
>
>> Hi -
>>
>>> From: Andy Bierman
>>> Sent: Sep 9, 2015 12:10 PM
>>> To: Ladislav Lhotka
>>> Cc: Robert Wilton , Randy Presuhn , "[email-masked]"
>>> Subject: Re: [netmod] Y26 again, sorry
>> ...
>>> I don't think it really addresses the design pattern very well.
>>> You want to claim that M and Q are both being developed at the
>>> same time,so it's OK that Q adds mandatory nodes to M.  YANG
>>> has no such rule.YANG says a server can implement M and never
>>> implement Q.YANG says a server may implement M and then add Q
>>> in a future release.These conformance mechanisms do not align
>>> with your expectationsof how YANG can/should be used.
>>
>> I agree with Andy that there seems to be a mismatch in expectations.
>
>I wonder if we can explain these differences. Is your and Andy's
>expectation that the configuration schema has to reflect actual hardware
>configuration, perhaps even dynamically adjust to the changes?

I can't speak for Andy, but I would expect that a server's
schema could change dynamically, as a result, for example,
of new hardware or software being introduced into a running
system.  We were doing this in the late 1980's - it's one
of the things that pulled us into the subagent technology space.

>In my view, the supported (and advertised) data model and hardware
>configuration of the device are two different things.

They are different, but there are relationships.  When a new
line card is inserted, I would expect the system to acquire
any necessary schema knowledge in order to manage that new
resource.  Likewise, if non-present hardware is being provisioned,
part of that provisioning process (possibly implicit) is for
the system being provisioned to acquire the necessary schema
knowledge so that it can perform at least a preliminary validation of
the provisioning data.

>> Let's look at a slightly more complex hypothetical case to tease
>> out how folks *want* things to work.  Assume the following have
>> been defined:
>>
>>   - base module M
>>   - augmentation Q
>>   - augmentation R
>>
>> On a server claiming to supporting the modules containing M, Q,
>> and R, respectively, which of the following might one encounter
>> concurrently?
>>
>>      - plain M
>>      - M+Q
>>      - M+R
>>      - M+Q+R
>
>It depends on what you mean by "encounter" but in terms of datastore
>validity the only possible answer IMO is M+Q+R.

By "encounter" I mean if a client asks the server for all of its Ms,
and the server also supports Q and R, are all of the Ms *guaranteed*
to be M+Q+R, or is it possible that some of the Ms might lack Q or
lack R of lack both?  If what netmod gives us is strictly M+Q+R,
how would one model a system inhabited by a mixture of the four
kinds of Ms?

Randy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 11 Sep 2015 11:20:30 +0200

> of the things that pulled us into the subagent technology space.

My (faint) understanding of CMIP/GDMO is that it was about management
objects and prototypes, which indeed makes such a plug-in architecture
easier. NETCONF/YANG deals with documents and schemas instead, and I
assume the data model is in mostly (always?) fixed by NETCONF/RESTCONF
server implementation. Am I wrong?

> knowledge so that it can perform at least a preliminary validation of

I would expect that schema parts related to the new hardware have to be
present beforehand. Encapsulation in YANG models is just a matter of
conventions, so I think a general solution to dynamically changing
schemas is next to impossible, also because parts of data model
semantics may be expressed in prose (descriptions). 

> how would one model a system inhabited by a mixture of the four

This very much depends on how M, Q and R are designed. In a typical
case, such as ietf-interfaces or ietf-routing, the augmenting modules
just add new alternatives (interface types, address families, routing
protocols).

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 14 Sep 2015 16:12:06 +0200

> how would one model a system inhabited by a mixture of the four

Retrieval is easy since a client is going to ignore data it does not
understand and the server is expected to report what makes sense from
the server's perspective. The question is relevant for writing: Is a
client programmed based on M going to work even if a server supports
M+Q, M+R, or M+Q+R? I believe the rule stated in section 7.15 of RFC
6020 wanted to achieve this by forbidding mandatory nodes in
augmentations.

Y26-02 says that a presence container may be used to avoid breaking an
old client. Frankly, it seems not totally clear to me what the
sentence in section 7.15 of RFC 6020 really means:

    If the target node is in another module, then nodes added by the
    augmentation MUST NOT be mandatory nodes (see Section 3.1).

Does this rule only apply to nodes directly added under the target
node or does it apply to the whole hierarchy added? In the later case,
this would still cause a problem with the presence container work
around.

The recent suggestion to replace MUST NOT with SHOULD NOT in this
sentence may be seen as a softer variant of Y26-01. However, I think
we would, in addition, have to describe when it is OK to have
mandatory nodes in augmentations and when not. Simply saying SHOULD
NOT instead of MUST NOT will not help people to understand the issues
around this.

When this issue was discussed in the past, it turned out to be
difficult to find someone to write the necessary text explaining when
augmenting mandatory nodes is safe and when not. Are we doing better
now?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Mon, 14 Sep 2015 18:22:51 +0100

Hi Juergen,

> augmenting mandatory nodes is safe and when not. Are we doing better
I'm happy to give this a try - I have got a vested interest in relaxing 
this rule for the sub-interface model that I'm putting forward.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 15 Sep 2015 10:00:46 +0200

> this would still cause a problem with the presence container work

Yes, it's unclear but IMO it is about mandatory nodes directly below the
target node - a similar rule for module updates is in sec. 11.

> NOT instead of MUST NOT will not help people to understand the issues

The definition in RFC 2119 is:

4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that
   there may exist valid reasons in particular circumstances when the
   particular behavior is acceptable or even useful, but the full
   implications should be understood and the case carefully weighed
   before implementing any behavior described with this label.

I think it should suffice if data model designers are aware of the
fact that certain augments may break old clients, and give some
reasoning if they use them anyway. 

> augmenting mandatory nodes is safe and when not. Are we doing better

I don't think we can enumerate all cases where it is allowed because
YANG doesn't follow any rigid structure and there may be a variety of
different designs.

I'd prefer to explain the ramifications of defining mandatory nodes in
augments (and also in module updates) and leave it to the model designer
to judge whether there are valid reasons to override the "SHOULD NOT" -
and YANG doctors should verify it in IETF models.

I think the point is that parameters are mandatory not because a
spiteful module author defines them so but rather because the system
cannot work without them.

Lada

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2015 10:26:30 +0200

> cannot work without them.

Still some text needs to be written explaining that breaking old
clients by adding new mandatory nodes is a no go. I did not ask to
enumerate all situations where it is allowed, I am looking for text
that clearly says what people have to look out for if they add
mandatory nodes.

/js

---

From: Robert Wilton <[email-masked]>
Date: Wed, 23 Sep 2015 14:50:50 +0100

Hi Juergen,

> that clearly says what people have to look out for if they add
Would it be sufficient to just change the MUST NOT to SHOULD NOT and 
then reference section 5.17.2 of draft-ietf-netmod-rfc6087bis which 
seems to provide the explanation that you are requesting?

E.g. something like:

draft-ietf-netmod-rfc6020bis, section 7.17.  The augment Statement

From:

To:

If the target node is in another module, then nodes added by the
augmentation SHOULD NOT be mandatory nodes (see Section 3.1).  The
safe scenarios to augment with mandatory nodes are described in
[draft-ietf-netmod-rfc6087bis] section 5.17.2.

For reference, I've also reproduced the text for 
draft-ietf-netmod-rfc6087bis-04] section 5.17.2 below:

5.17.  Augment Statements

    The YANG "augment" statement is used to define a set of data
    definition statements that will be added as child nodes of a target
    data node.  The module namespace for these data nodes will be the
    augmenting module, not the augmented module.

    A top-level "augment" statement SHOULD NOT be used if the target data
    node is in the same module or submodule as the evaluated "augment"
    statement.  The data definition statements SHOULD be added inline
    instead.

5.17.1.  Conditional Augment Statements

    The "augment" statement is often used together with the "when"
    statement and/or "if-feature" statement to make the augmentation
    conditional on some portion of the data model.

    The following example from [RFC7223] shows how a conditional
    container called "ethernet" is added to the "interface" list only for
    entries of the type "ethernetCsmacd".

         augment "/if:interfaces/if:interface" {
             when "if:type = 'ianaift:ethernetCsmacd'";

             container ethernet {
                 leaf duplex {
                     ...
                 }
             }
         }

5.17.2.  Conditionally Mandatory Data Definition Statements

    YANG has very specific rules about how configuration data can be
    updated in new releases of a module.  These rules allow an "old
    client" to continue interoperating with a "new server".

    If data nodes are added to an existing entry, the old client MUST NOT
    be required to provide any mandatory parameters that were not in the
    original module definition.

    It is possible to add conditional augment statements such that the
    old client would not know about the new condition, and would not
    specify the new condition.  The conditional augment statement can
    contain mandatory objects only if the condition is false unless
    explicitly requested by the client.

    Only a conditional augment statement that uses the "when" statement
    form of condition can be used in this manner.  The YANG features
    enabled on the server cannot be controlled by the client in any way,
    so it is not safe to add mandatory augmenting data nodes based on the
    "if-feature" statement.

    The XPath "when" statement condition MUST NOT reference data outside
    of target data node because the client does not have any control over
    this external data.

    In the following dummy example, it is OK to augment the "interface"
    entry with "mandatory-leaf" because the augmentation depends on
    support for "some-new-iftype".  The old client does not know about
    this type so it would never select this type, and therefore not be
    adding a mandatory data node.

      module my-module {
        ...

        identity some-new-iftype {
           base iana:iana-interface-type;
        }

        augment "/if:interfaces/if:interface" {
           when "if:type = 'mymod:some-new-iftype'";

           leaf mandatory-leaf {
              mandatory true;
              ...
           }
        }
      }

    Note that this practice is safe only for creating data resources.  It
    is not safe for replacing or modifying resources if the client does
    not know about the new condition.  The YANG data model MUST be
    packaged in a way that requires the client to be aware of the
    mandatory data nodes if it is aware of the condition for this data.
    In the example above, the "some-new-iftype" identity is defined in
    the same module as the "mandatory-leaf" data definition statement.

    This practice is not safe for identities defined in a common module
    such as "iana-if-type" because the client is not required to know
    about "my-module" just because it knows about the "iana-if-type"
    module.

Rob

---

From: Qin Wu <[email-masked]>
Date: Thu, 24 Sep 2015 09:00:37 +0000

I see it is useful to relax the restriction on mandatory augment given several use cases that have been discussed on the list.
The proposed changes get in line with draft-ietf-netmod-rfc6087bis-04 and look good to me.

-Qin
-----邮件原件-----
发件人: netmod [mailto:[email-masked]] 代表 Robert Wilton
发送时间: 2015年9月23日 21:51
收件人: Ladislav Lhotka; [email-masked]; [email-masked]
主题: Re: [netmod] Y26 again, sorry

Hi Juergen,

> that clearly says what people have to look out for if they add 
Would it be sufficient to just change the MUST NOT to SHOULD NOT and then reference section 5.17.2 of draft-ietf-netmod-rfc6087bis which seems to provide the explanation that you are requesting?

E.g. something like:

From:

If the target node is in another module, then nodes added by the augmentation MUST NOT be mandatory nodes (see Section 3.1).

To:

If the target node is in another module, then nodes added by the augmentation SHOULD NOT be mandatory nodes (see Section 3.1).  The safe scenarios to augment with mandatory nodes are described in [draft-ietf-netmod-rfc6087bis] section 5.17.2.

For reference, I've also reproduced the text for draft-ietf-netmod-rfc6087bis-04] section 5.17.2 below:

5.17.  Augment Statements

             container ethernet {
                 leaf duplex {
                     ...
                 }
             }
         }

      module my-module {
        ...

           leaf mandatory-leaf {
              mandatory true;
              ...
           }
        }
      }

Rob