Subject: Re: [netmod] Y26 again, sorry

From: Robert Wilton <[email-masked]>
Date: Mon, 7 Sep 2015 11:05:28 +0100

Hi Lada,

I've just take a quick look at your DNS model referenced below.  For 
your containers under the choice rdata-content you have used must 
statements to enforce the constraint.

E.g.

...

|choice rdata-content { mandatory "true";|

...

|container A { must "../../type = 'ianadns:A'";|

I was wondering what the reason was that you choose to use must 
statements rather than equivalent when statements?

The reason that I'm asking is that I had used when statements for a 
similar construct that I had used for VLANs encapsulation, and I guess I 
would normally choose a when statement out of preference over a must 
statement, hence the desire to understand the rationale for the alternative.

Thanks,
Rob

On 26/08/2015 09:28, Ladislav Lhotka wrote:
> Hi,
>
> I changed the DNS zone data model to use this pattern - it looks
> good and works great:
>
> - module:
>    https://gitlab.labs.nic.cz/llhotka/zone-data-yang/blob/master/dns-zones.yang
>    
> - tree:
>    https://gitlab.labs.nic.cz/llhotka/zone-data-yang/raw/master/model.tree
>
> Lada
>
> Martin Bjorklund <[email-masked]> writes:
>
>> Hi,
>>
>> I think this could work (thanks Robert; maybe this is what you
>> meant!):
>>
>>    container zones {
>>      list zone {
>>        ...
>>        list rrset {
>>          ...
>>          leaf type {
>>            type identityref { ... }
>>          }
>>          list rdata {
>>            key id;
>>            ...
>>            choice type-specific-params {
>>              mandatory true;
>>              // to be augmented with type-specific nodes
>>            }
>>          }
>>        }
>>      }
>>    }
>>
>> And then in another module:
>>
>>    augment "/dnsz:zones/dnsz:zone/dnsz:rrset/dnsz:rdata"
>>          + "/type-specific-parameters" {
>>      when "derived-from-or-self(../dnsz:type,'iana-dns-parameters',"
>>         + "'TLSA')";
>>
>>      leaf certificate-usage {
>>        mandatory true;
>>        ...
>>      }
>>    }
>>
>> The empty mandatory choice formally tells the client that additional
>> cases are expected.
>>
>> (If the empty choice looks fishy, it is probably often possible to
>> define at least one case inline...)
>>
>> This pattern is nice to the client, since there is no way an
>> additional augmenting module can break a working client.
>>
>>
>> /martin
>>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 7 Sep 2015 12:52:56 +0200

Hi Rob,

I generally prefer must to when, except in augments, because it is IMO a much cleaner concept that also has analogies in standard XML schema languages. When manipulates the schema based on arbitrary values in an instance document (that is supposed to be validated with the schema), and because of that it requires extra rules and sometimes even temporary manipulation with the instance document to work correctly, see

https://tools.ietf.org/html/draft-ietf-netmod-rfc6020bis-06#section-7.21.5

In contrast, must expressions are evaluated in the context of an instance document and there are no concerns about circular references, order of evaluation etc.

Lada

> On 07 Sep 2015, at 12:05, Robert Wilton <[email-masked]> wrote:
> 
> Hi Lada,
> 
> I've just take a quick look at your DNS model referenced below.  For your containers under the choice rdata-content you have used must     statements to enforce the constraint.
> 
> E.g.
> 
> ...
>           choice rdata-content {
>             mandatory "true";
> ...
>             container A {
>               must "../../type = 'ianadns:A'";
> 
> 
> I was wondering what the reason was that you choose to use must statements rather than equivalent when statements?
> 
> The reason that I'm asking is that I had used when statements for a similar construct that I had used for VLANs encapsulation, and I guess I would normally choose a when statement out of preference over a must statement, hence the desire to understand the rationale for the alternative.
> 
> Thanks,
> Rob
>  
> 
>> Martin Bjorklund 
>> <[email-masked]>
>>  writes:

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Robert Wilton <[email-masked]>
Date: Mon, 7 Sep 2015 17:37:42 +0100

Hi Lada,

Thanks for the explanation.

Initially I was concerned that there would be a higher server 
implementation cost of using a must statement over a when statement, but 
given that the container/leaves are under a choice statement then it 
seems that the must statement can presumably be implemented as 
efficiently as a when statement in this scenario.

Thanks,
Rob

On 07/09/2015 11:52, Ladislav Lhotka wrote:
> .

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 7 Sep 2015 18:50:48 +0200

> On 07 Sep 2015, at 18:37, Robert Wilton <[email-masked]> wrote:
> 
> Hi Lada,
> 
> Initially I was concerned that there would be a higher server implementation cost of using a must statement over a when statement, but given that the container/leaves are under a choice statement then it seems that the must statement can presumably be implemented as efficiently as a when statement in this scenario.

Well, if “when” was limited to the most common case like

container foo {
    when “../type = ‘foo’”;
}

then it would indeed be possible to have a more efficient (generic) implementation of “when”. But since it may contain essentially arbitrary XPath expressions, implementation cost of “when” should be at least as high as that of “must”.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 7 Sep 2015 10:04:51 -0700

On Mon, Sep 7, 2015 at 9:50 AM, Ladislav Lhotka <[email-masked]> wrote:

> given that the container/leaves are under a choice statement then it seems
> that the must statement can presumably be implemented as efficiently as a
> when statement in this scenario.
>
> then it would indeed be possible to have a more efficient (generic)
> implementation of “when”. But since it may contain essentially arbitrary
> XPath expressions, implementation cost of “when” should be at least as high
> as that of “must”.
>
>
when-stmt is much harder to implement correctly than must-stmt.

The use of 'must' vs. 'when' depends on which data structure is primary
and which is secondary.  must-stmt is for use in the primary data node.
It says "if I exist then you must pass this validation test".  when-stmt
is for secondary data structures.  It says "I exist if your settings pass
this test".

> Lada

Andy

> >> I generally prefer must to when, except in augments, because it is IMO
> a much cleaner concept that also has analogies in standard XML schema
> languages. When manipulates the schema based on arbitrary values in an
> instance document (that is supposed to be validated with the schema), and
> because of that it requires extra rules and sometimes even temporary
> manipulation with the instance document to work correctly, see
> >>
> >>
> >> In contrast, must expressions are evaluated in the context of an
> instance document and there are no concerns about circular references,
> order of evaluation etc.
> >>
> >> Lada
> >>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Robert Wilton <[email-masked]>
Date: Mon, 7 Sep 2015 18:13:59 +0100

Hi Andy,

Picking up a slightly old thread after PTO ...

On 24/08/2015 22:50, Andy Bierman wrote:
>
>
> On Mon, Aug 24, 2015 at 2:24 PM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi Randy,
>
>     On 24/08/2015 20:20, Randy Presuhn wrote:
>
>         Hi -
>
>             From: Ladislav Lhotka <[email-masked] <mailto:[email-masked]>>
>             Sent: Aug 24, 2015 11:44 AM
>             To: Andy Bierman <[email-masked]
>             <mailto:[email-masked]>>
>             Cc: "[email-masked] <mailto:[email-masked]>"
>             <[email-masked] <mailto:[email-masked]>>
>             Subject: Re: [netmod] Y26 again, sorry
>
>         ...
>
>                 YANG does not provide any mechanism to REQUIRE modules
>                 A and B
>                 to both be implemented on a server.  You may think it
>                 should, but
>                 currently the YANG conformance is for an individual
>                 module.
>
>             There are sections on conformance and conformance
>             announcement,
>             and they say nothing like this. In my view, the data model
>             comprises
>             *all* modules advertised by the server. I think your
>             interpretation
>             of conformance might be an extrapolation from SNMP/SMI
>             times, but,
>             for better or worse, it really has no support in the YANG
>             spec.
>
>         It sounds as though you might be talking past each other.
>         I believe part of Andy's point is that clients will need to deal
>         with servers that do not implement (and thus do not advertise)
>         the augmenting module.  But there's (I think) a more interesting
>         issue beneath this.
>
>         Let's start with module M.  Let's say M is for "modem" (to pick
>         an obsolete but widely understood resource).
>         Two different augmenting modules, F (for FSK - frequency
>         shift keying) and Q (for QAM - quadrature amplitude modulation)
>         are developed.  Let us say that F and Q are mutually incompatible.
>
>         A system with multiple Ms could well have both M+F and M+Q modems,
>         but (if we claim F & Q are incompatible) could not have M+F+Q.
>         If naked M is to be prohibited in systems (also) supporting F or Q
>         or both, we don't currently have a mechanism to do so.
>
>     Could this be achieved by augmenting from a choice statement?
>
>     M defines the choice statement but with no case statements.
>
>     F and Q both implement separate case statements that augment the
>     choice statement in M.  The case statements have containers which
>     hold the parameters related to F or Q.
>
>     M without F or Q is meaningless.
>     M+F or M+Q works, but the choice statement means that you cannot
>     have M+F+Q.
>
>
> nice solution
>
> This is perhaps the cleanest way to add mandatory nodes to a module.
> The old client will not attempt to create the new case.
>
> As I said before, I am OK with changing MUST NOT to SHOULD NOT
> add mandatory nodes, and then add MAY when X, Y, Z conditions are met.
>
> Two conditions so far:
>
>     (1)  augment + when <new flag set that old client will not set>
>
>     (2) augment choice with a new case-stmt
>
> (1) is hard to define, but not (2)

So, Lada is using (2) for DNS zones which works.  Does the Y26 text need 
to be updated to explicitly allow this, or is this implicitly allowed 
anyway?

Unfortunately (2) won't work for my model augmentation issue, of wanting 
to enforce that a sub-interface has to have a parent interface-ref.   As 
a recap, the yang from my interfaces-common draft is:

   augment "/if:interfaces/if:interface" {
     when "if:type = 'ianaift:l2vlan' or
           if:type = 'ianaift:atmSubInterface' or
           if:type = 'ianaift:frameRelay'"  {
       description
         "Any ianaift :types that represent sub-interfaces";
     }
     if-feature "sub-interfaces";
     description "Add a parent interface field to interfaces to model
                  sub-interfaces";
     leaf parent-interface {
       type if:interface-ref;
       /*
        * TODO - How to make this mandatory without using the
        * mandatory keyword.
        * - Current options appear to be:
        *   - Create a sub-interface container with presence.
        *   - Enforce the constraint with a must statement.
        */
       //mandatory true;
       description
         "This is the mandatory reference to the parent interface of
          this sub-interface.";
     }
   }

One suggestion that I've heard is based on a specific instance of your 
first condition above, where the when statement only uses identities 
defined by the same augmenting module:
I.e. don't use the existing "ianaift:l2vlan" for a VLAN sub-interface, 
but define a new interface type identity "vlan-sub" in my interface 
extensions module which would inherit from "iftype:l2vlan".   Similarly 
for atmSubInterface and frameRelay. Obviously, at the moment, this is 
not allowed, but potentially it could be, and it is still safe to 
existing clients (since they can't be using the new type).

However, I'm not really sure whether fragmenting the list of iftypes 
into separate modules would be a good idea ...

Thanks,
Rob

>
>     I can point you to a concrete example if it helps.
>
>     Thanks,
>     Rob
>
>
>
>
>
> Andy
>
>         Randy
>
>         [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 7 Sep 2015 10:41:39 -0700

On Mon, Sep 7, 2015 at 10:13 AM, Robert Wilton <[email-masked]> wrote:

> On Mon, Aug 24, 2015 at 2:24 PM, Robert Wilton <[email-masked]> wrote:
>
>> Hi Randy,
>>
>>> From: Ladislav Lhotka <[email-masked]>
>>>> To: Andy Bierman <[email-masked]>
>>>> Cc: "[email-masked]" <[email-masked]>
>>>> YANG does not provide any mechanism to REQUIRE  modules A and B
>>>>> to both be implemented on a server.  You may think it should, but
>>>>> currently the YANG conformance is for an individual module.
>>>>>
>>>> There are sections on conformance and conformance announcement,
>>>> and they say nothing like this. In my view, the data model comprises
>>>> *all* modules advertised by the server. I think your interpretation
>>>> of conformance might be an extrapolation from SNMP/SMI times, but,
>>>> for better or worse, it really has no support in the YANG spec.
>>>>
>> F and Q both implement separate case statements that augment the choice
>> statement in M.  The case statements have containers which hold the
>> parameters related to F or Q.
>>
>> M+F or M+Q works, but the choice statement means that you cannot have
>> M+F+Q.
>>
>>
> nice solution
>
> to enforce that a sub-interface has to have a parent interface-ref.   As a
> recap, the yang from my interfaces-common draft is:
>

Your example shows the YANG conformance problems fairly well.
Clearly the IETF (and others) want to use advanced design patterns
in which conformance to the base module (M) is insufficient to describe
the actual API requirements.

YANG uses revision dates to identify versions.
There is no such thing as a major vs. minor revision.
I agree with Lada that it is possible to have major revision update
where the old clients should not be used anymore.

I already suggested relaxing the MUST NOT to a SHOULD NOT,
wrt/ adding mandatory nodes.

 Andy

> I.e. don't use the existing "ianaift:l2vlan" for a VLAN sub-interface, but
> define a new interface type identity "vlan-sub" in my interface extensions
> module which would inherit from "iftype:l2vlan".   Similarly for
> atmSubInterface and frameRelay.  Obviously, at the moment, this is not
> allowed, but potentially it could be, and it is still safe to existing
> clients (since they can't be using the new type).
>
> However, I'm not really sure whether fragmenting the list of iftypes into
> separate modules would be a good idea ...
>
> Thanks,
> Rob
>
>
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 08 Sep 2015 10:16:25 +0200

> to be updated to explicitly allow this, or is this implicitly allowed 

It is allowed in YANG 1.0.

> to enforce that a sub-interface has to have a parent interface-ref.

ietf-interfaces could also use the same mandatory choice pattern but
it seems too late now. This is an example why the strict module update rules
are counter-productive at this stage - instead if adopting the best current
practice we have to resort to kludges.

>         *   - Create a sub-interface container with presence.

... which doesn't make the parent-interface mandatory anyway.

>         *   - Enforce the constraint with a must statement.

Yes, but this design offers no good place for it, you'd need to use an
extra np-container and attach the must statement to it. 

> existing clients (since they can't be using the new type).

I think it would be best to make backward old-client compatibility a
general guideline rather than a hard language rule. Currently it is
enforced in some cases (augments, module update rules) but a mere
guideline in other cases (must statements, extensions, defaults).

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 08 Sep 2015 10:50:44 +0200 (CEST)

> practice we have to resort to kludges.

Can you explain what you would like to do with ietf-interfaces, and
why that is not allowed according to the upgrade rules?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 8 Sep 2015 15:25:01 +0200

> On 08 Sep 2015, at 10:50, Martin Bjorklund <[email-masked]> wrote:
> 

list interface {
    key name;
    leaf name { … }
    leaf type { … }
    choice interface-parameters {
        mandatory true;
    }
}

Lada

---

From: Robert Wilton <[email-masked]>
Date: Tue, 8 Sep 2015 15:09:24 +0100

Hi Lada,

> It is allowed in YANG 1.0.

I was thinking that this would still fall foul of the RFC 6020 section 
7.15 rule:

    If the target node is in another module, then nodes added by the
    augmentation MUST NOT be mandatory nodes (seeSection 3.1 <#section-3.1>).

But presumably, this statement is only intended to refer to the top 
level nodes that are by added by the augmentation statement rather than 
all nodes that are being added (which the current wording seems to imply 
to me).

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Tue, 8 Sep 2015 16:48:55 +0100

>      choice interface-parameters {

I'm not sure that it helps for my sub-interface parent leaf problem, 
since I would ideally like the same parent-if-ref leaf to exist for 
multiple if:types (e.g. Frame, ATM, VLANs) with the same namespace. But 
unless VLANs  + ATM + Frame Relay are all defined in the same module 
(which wouldn't seem a very natural thing to do) then they would end up 
in different namespaces, and hence not be so usable.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 8 Sep 2015 19:47:51 +0200

> On 08 Sep 2015, at 17:48, Robert Wilton <[email-masked]> wrote:
> 
> 
> 
> I'm not sure that it helps for my sub-interface parent leaf problem, since I would ideally like the same parent-if-ref leaf to exist for multiple if:types (e.g. Frame, ATM, VLANs) with the same namespace. But unless VLANs  + ATM + Frame Relay are all defined in the same module (which wouldn't seem a very natural thing to do) then they would end up in different namespaces, and hence not be so usable.

This could probably be done by defining an interface type as kind of superclass for all interface types that need parent-if-ref and then derive all concrete interface types (Frame Relay, ATM, VLANs) from it. A corresponding augment then would be, e.g.,

augment “/if:interfaces/if:interface/if:interface-parameters/“ {
    when “derived-from(../if:type, ‘mymod’, ‘child-interface’)”;
    container child-interface {
        leaf parent-if-ref {…}
    }
}

Concrete interface types could then add specific nodes by augmenting the child-interface container.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Wed, 9 Sep 2015 16:25:24 +0100

Hi Andy,

> wrt/ adding mandatory nodes.
I support that - it seems to strike the right pragmatic balance to me.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 9 Sep 2015 20:56:42 +0200

> On 09 Sep 2015, at 17:25, Robert Wilton <[email-masked]> wrote:
> 
> Hi Andy,
> 

I support that, too.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 9 Sep 2015 12:10:44 -0700

> > I support that - it seems to strike the right pragmatic balance to me.

I don't think it really addresses the design pattern very well.

You want to claim that M and Q are both being developed at the same time,
so it's OK that Q adds mandatory nodes to M.  YANG has no such rule.
YANG says a server can implement M and never implement Q.
YANG says a server may implement M and then add Q in a future release.
These conformance mechanisms do not align with your expectations
of how YANG can/should be used.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 9 Sep 2015 22:44:23 +0200

> On 09 Sep 2015, at 21:10, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

There seems to be an impedance mismatch between us at this point. You are saying that a server can implement ietf-interfaces only. That’s correct but the server implementor can easily see that it doesn’t make much sense because nothing very useful can be done with it. So what?

It is similar to the situation before OS software packages were introduced: One had to know (or read in the documentation, or later in the output of a configure script) that package X needs either Y or Z to work correctly, and install one of them beforehand. It was more laborious but one could make things work even then.

It would be great to have a formalism that allows for stating that module M requires Q, but I think multi-module data models can be designed and effectively used without it.

Issue Y26 is about something else, namely allowing an augmenting module Q to introduce mandatory nodes, and if the server advertises both M and Q, the client should get the message that the nodes from Q are mandatory.

The formulation “A module SHOULD NOT augment another with mandatory nodes” is indeed an acceptable compromise - nodes that positively need to be mandatory can be tagged as “mandatory true”.

Lada