Subject: [netmod] Augmentable Groupings

From: <[email-masked]>
Date: Tue, 12 Aug 2008 18:26:46 +0300

Hi All, 

in the Friday session of NETMOD WG, I proposed to 
support the concept of augmentable groupings. 

module A { 
  grouping concept { 
    ... 
  }. 
} 

module B { 
  import A {prefix a;} 
  augment a:/concept { 
    ... 
  } 
} 

In particular that would mean: 

- A groupings can be the target (node) of an 
  augment statement. 
- Augmentation of a grouping becomes only effective
  in case the module containing the augment statement
  is advertized.
- The nodes added to the grouping are therefore 
  implicitly added to all places the grouping is used 
  in the set of modules supported by an agent.
- The augmented nodes are in the XML namespace 
  of the module of the augment statement. 

For example assume a module for providing a reusable 
definition of network interface parameters 

module Interfaces { 
  grouping NetworkInterface { 

    leaf ifName {type string;}
    leaf ifDescription {type DisplayString;}
    leaf ifMtu {type int32;}
    // . . .
    leaf ifSpeed {type Gauge;}
    leaf ifInOctets {type Counter;} 
    // . . .
 }
 
which is then used to define one or more interface 
lists (tables). 
If now high capacity support should be added 
consistently, this could be done by augmenting 
this group with HC attributes: 

module HCSupport {
  import NetworkInterfaces {prefix if; } 
  augment /if:NetworkInterface {
    leaf ifHCInOctets {type Counter64;}
    leaf ifHCOutOctets {type Counter64;}
    // . .
   } 
} 

Of course, the module HCSupport should only
be used and advertized if HC support
is available for all interfaces controlled
by the advertizing agent.

There are different benefits of augmentable 
groupings:

- Supports consistent and YANG-like extension 
  of a grouping by using an existing language 
  construct, i.e. it provides additional capabilities 
  without introducing new keywords. 

- Augmentable groupings maximize re-use of 
  model code. In that sense augmentable 
  groupings introduce a modeling mechanism, 
  which is based on "true re-use" and is 
  technically better compared to the existing 
  grouping concept, which requires copying 
  and pasting augmentation many times. 

- When combined with the when statement, 
  augmentable groupings even allow selective, 
  content based augmentation in a consistent 
  manner. 

- Augmentable groupings adopt every benefit 
  but also obstacle of augmentations in the YANG 
  language. As it already has been suggested by 
  Martin B., we would like propose that an augment 
  MUST NOT add any mandatory nodes (i.e. 
  mandatory leaf, list or leaf-list w/ min-elements > 0 
  etc). An augment must not break a client that 
  doesn't understand the augmentation.

- Augmenting all places where a grouping is used
  is error-prone and causes unnecessary development
  costs.
  The problem is actually worse than this. 
  When someone writes an extension module, 
  he/she might not know all places where the
  grouping must be augmented.

BR,
Bernd

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 12 Aug 2008 08:50:12 -0700

> support the concept of augmentable groupings.

Actually groupings are not in the same namespace as objects,
so this Xpath is broken.

    grouping foo;

    container foo;

Both of these are legal YANG.

If I nest the groupings, then what happens to the augment:

module C {

   grouping new-concepts {
      uses concepts;
   }

Does new-concepts contain the augmentations from concepts?

How is somebody reading 'uses C:new-concepts;' in module D
know if this is augmented or not?

What if I want to use the 'concept' grouping with the augment and
sometimes without it in the same system?  Advertising module B
means that every use of concepts is now augmented.
What if some existing 'uses concepts;' statements
do not support the new nodes?

There are also conformance issues, depending on what the WG
does about 'if-feature' and capabilities documentation.

How is a YANG compiler supposed to know that M of N modules
will be augmenting groupings in other modules, which changes
the uses expansion in all the modules.  Do you think the
compiler is going to 'redo' N-1 modules, as it processes
the Nth module?

Have you implemented this feature in a complete NETCONF system?
I am not convinced it is even implementable, let alone deployable.

IMO, this approach has more unintended consequences than benefits.

Andy

> ------------------------------------------------------------------------
>

---

From: Phil Shafer <[email-masked]>
Date: Tue, 12 Aug 2008 15:06:29 -0400

>  grouping must be augmented.

The reverse is also true, and I think this is the problem
with this proposal.  When you augment a grouping, you have
no idea who has used it and what havoc you are wrecking on
those other users.

Imagine the case where I have taken your NetworkInterface
grouping and used it in a list of low speed interfaces:

    list slow-dudes {
        use NetworkInterface;
        leaf how-slow {
            type enumeration {
                enum very-slow { ... }
                enum horribly-slow { ... }
                enum 2g { ... }
            }
        }
    }

When you augment NetworkInterface in your module, I suddenly get
your ifHCIn/OutOctets leafs, which are completely inappropriate.
I can't turn them off and I can't tell nm apps "hey he didn't
mean my list".  Sure, I can choose not to implement/announce
your module, but if both slow-dudes and HCSupport are IETF
standards, I'm stuck.

I see the attraction of "say it once and it happens everywhere",
but the down side far overways the benefit.

Thanks,
 Phil

---

From: "Lahdensivu, Mikko (NSN - FI/Tampere)" <[email-masked]>
Date: Wed, 13 Aug 2008 19:07:08 +0300

Hi,

>know if this is augmented or not?

If module A defines a list "a", and module B augments list "a", how is
somebody reading module C which contains reference to "a" know whether
"a" is augmented or not?

Assuming mandatory content cannot be added in an augmentation, you can
in both cases (list, grouping) not provide values for the mandatory
part, and ignore the values in reading.

>
>What if I want to use the 'concept' grouping with the augment 
>and sometimes without it in the same system?  Advertising 
>module B means that every use of concepts is now augmented.
>What if some existing 'uses concepts;' statements do not 
>support the new nodes?

What if I want that, from one list, elements referred in one place of
the model can be augmented, but those referred in other places cannot be
augmented?

Let's say I have a list of "server" (name and IP address pair), which I
can reuse in several places in my model (to avoid redundant entering of
the same address in multiple places of the model). Now an application
processing "connection" (which refers to a server) would encounter the
augmentation, like it would have if I would have used grouping "server"
instead. In both cases the application reading the "connection",
including the related address, need to be able to handle situation with
unexpected optional leaves.

I don't think we should assume that a grouping always has multiple
implementations in an application/server, but a list always has one
implementation in an application/server.

>compiler is going to 'redo' N-1 modules, as it processes the 
>Nth module?

Let's say each module corresponds to a header file. If module augments
another module, the compiler would update the header file. When you
would compile the Yang compiler output, the compiler would be able to
include the modified, instead of the original, header file in every
place it is used.

There would not be any difference on whether the header file would be
modified by an augmentation of a grouping or augmentation of a list.
Even without augmentation of groupings the compiler needs modify the
original header file, once it encounters a module with the augmentation.

Mikko

---

From: "Lahdensivu, Mikko (NSN - FI/Tampere)" <[email-masked]>
Date: Wed, 13 Aug 2008 19:16:22 +0300

Hi, 

>The reverse is also true, and I think this is the problem with 
>this proposal.  When you augment a grouping, you have no idea 
>who has used it and what havoc you are wrecking on those other users.
>
>When you augment NetworkInterface in your module, I suddenly 
>get your ifHCIn/OutOctets leafs, which are completely inappropriate.

And if concept NetworkInterface is mapped to a list NetworkInterface.
Now when you refer the list NetworkInterface from other places of your
model (where otherwise you could use a grouping), and you add your
slow-dudes elements to the list, they will again have the high capacity
leaves. What is the difference from application point of view whether it
reads the data from NetworkInterface list or NetworkInterface grouping?
In both cases your application can have a generic routine for reading
NetworkInterface data from the element, and on both cases you can have a
condition on the type of the NetworkInterface so that augmented leaves
are not applicable to incorrect type of NetworkInterface.

Mikko

---

From: Phil Shafer <[email-masked]>
Date: Wed, 13 Aug 2008 12:52:36 -0400

"Lahdensivu, Mikko (NSN - FI/Tampere)" writes:
>leaves.

The problem remains:  if I use a grouping that you can randomly
augment, then I can never be sure of the set of nodes I'm getting
when I use a grouping.

On a theoretical level, it may not matter to your application, but
on the practical level, when your application sends my box content
that it doesn't understand, things break.

If I made a list of interfaces and reused a standard grouping, and
you augment that grouping with your new nodes, how do you know
whether I implemented your augmented nodes in my list?  I can't come
in after that fact and say "he didn't mean me".   Yes, you can put
a "when" on your augment, but that condition may not match my
reuse of the grouping.

>are not applicable to incorrect type of NetworkInterface.

The generic solution doesn't exist on the device side, where the config
that your application ships to my device must match the nodes I've
implemented in my software.

Thanks,
 Phil

---

From: "Lahdensivu, Mikko (NSN - FI/Tampere)" <[email-masked]>
Date: Tue, 19 Aug 2008 10:36:22 +0300

Hi,

>The problem remains:  if I use a grouping that you can 
>randomly augment, then I can never be sure of the set of nodes 
>I'm getting when I use a grouping.
>
>On a theoretical level, it may not matter to your application, 
>but on the practical level, when your application sends my box 
>content that it doesn't understand, things break.

Right, but the problem is the same already with augmenting a list. When
your applications needs to use the referred data, it needs to be handle
it. In both cases the data structure (grouping or list element) could
have been augmented. While use of grouping and reference to a list look
different in the payload, I don't consider them fundamentally different
from device/application logic point of view; in both cases I use a
generic routine in both device and application to process the data,
including any augmentations I support for such structure.

>
>If I made a list of interfaces and reused a standard grouping, 
>and you augment that grouping with your new nodes, how do you 
>know whether I implemented your augmented nodes in my list?  I 
>can't come

Right, and if I put a reference to your list, and you augment the list
with some new nodes, I cannot say later that I didn't mean that the
reference applies also to elements which have augmented nodes.

>>NetworkInterface data from the element, and on both cases you 
>can have 
>>a condition on the type of the NetworkInterface so that augmented 
>>leaves are not applicable to incorrect type of NetworkInterface.
>
>The generic solution doesn't exist on the device side, where 
>the config that your application ships to my device must match 
>the nodes I've implemented in my software.

If you augment a grouping within your device (either in your device
specific model, or by supporting some standard model), you can create a
routine to handle processing of that grouping regardless of where in the
model it is used, concept of sub-routine does exist in devices as well.

Mikko