Subject: [netmod] deviation-stmt

From: Andy Bierman <[email-masked]>
Date: Tue, 16 Dec 2008 08:58:47 -0800

Hi,

I've already commented on the text needing a bit of work,
but as I'm trying to implement it, I notice the text and
ABNF are extremely sloppy and need to be redone.

As with augment and refine, there is no text wrt/
multiple deviation statements that refer to the same target.

There is no normative text to declare which deviate-arg-str
values can be used with which deviate-stmt sub-clauses.

   deviation /foo/bar {
      deviate not-supported {
         type string;
      }

      deviate delete {
         type string;
      }
   }

There needs to be separate ABNF sections for each of the 4
argument types.  Too many nonsense permutations exist for
it to be ignored.

Is it an error to specify an illegal sub-clause for a target
object type, or just a warning, or a silent NO-OP?

    leaf some-leaf {
       type float32;
    }

    deviation /some-leaf {
       deviate add {
          min-elements 4;
       }
    }

Is it an error to specify an legal sub-clause for a target
object type that doesn't exist, or just a warning, or a silent NO-OP?

    deviation /some-leaf {
       deviate delete {
          units "seconds";
       }
    }

What does it even mean to not support or delete the units clause?

What operations are supported for type-stmt, besides the
one I strongly objected to all along?

    deviation /some-leaf {
       deviate replace {
          type enumeration {
             enum on;
             enum off;
          }
       }
    }

I fail to accept that devices that do this sort of
thing should be tolerated and supported by the standard.
One sentence that shouts in caps that this is STRONGLY DISCOURAGED
is not good enough.

I know we need to strike a balance between standards-value
and vendors' freedom to innovate, but this is out of balance.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 Dec 2008 23:31:17 +0100 (CET)

> multiple deviation statements that refer to the same target.

I would like to state that an object MUST NOT occur in more than one
deviation statement in a module, and that a property MUST NOT be
deviated more than once.

> argument types.  Too many nonsense permutations exist for

Ok.  See below for a restructured version.

> object type, or just a warning, or a silent NO-OP?

Error.

> object type that doesn't exist, or just a warning, or a silent NO-OP?

Error.

> What does it even mean to not support or delete the units clause?

I don't know.  I removed the possibility to deviate it.

/martin

New grammar for deviations:

deviation-stmt         = deviation-keyword sep
                         deviation-arg-str optsep
                         "{" stmtsep
                             ;; these stmts can appear in any order
                             [description-stmt stmtsep]
                             [reference-stmt stmtsep]
                             (deviate-not-supported-stmt /
                               1*(deviate-add-stmt /
                                  deviate-replace-stmt /
				  deviate-delete-stmt))
                         "}"

deviation-arg-str      = < a string which matches the rule
                           deviation-arg >

deviation-arg          = absolute-schema-nodeid

deviate-not-supported-stmt =
                         deviate-keyword sep
                         not-supported-keyword optsep
                         (";" /
                          "{" stmtsep
                          "}")

deviate-add-stmt       = deviate-keyword sep add-keyword optsep
                         "{" stmtsep
                             *(must-stmt stmtsep)
                             *(unique-stmt stmtsep)
                             [default-stmt stmtsep]
                             [config-stmt stmtsep]
                             [mandatory-stmt stmtsep]
                             [min-elements-stmt stmtsep]
                             [max-elements-stmt stmtsep]
                         "}"

deviate-delete-stmt    = deviate-keyword sep delete-keyword optsep
                         "{" stmtsep
                             *(must-stmt stmtsep)
                             *(unique-stmt stmtsep)
                         "}"

deviate-replace-stmt   = deviate-keyword sep replace-keyword optsep
                         "{" stmtsep
                             [type-stmt stmtsep]
                             [default-stmt stmtsep]
                             [config-stmt stmtsep]
                             [mandatory-stmt stmtsep]
                             [min-elements-stmt stmtsep]
                             [max-elements-stmt stmtsep]
                         "}"
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 Dec 2008 16:09:14 -0800

> Ok.  See below for a restructured version.

much much better

> deviate-delete-stmt    = deviate-keyword sep delete-keyword optsep

This one is tricky.
How are the comparisons of two must-stmts or two unique-stmts done?
Is whitespace normalized somehow? Are prefix differences ignored?
Are different representations of the same object ignored?
(e.g., ../foo vs. /top/foo).

Does unique order matter?
(e.g., does unique "foo bar" match unique "bar foo")

Implementing this in a interoperable way seems difficult.

>                              [max-elements-stmt stmtsep]

Are there going to be any rules or guidelines on type-stmt?
This one looks real easy in the 1 line ABNF,
but the type-stmt is massive.

In the worst use-case, a vendor could claim partial conformance
to a standard data model by implementing the shells of the
data nodes, and then using deviations to replace all the guts
of all the objects with their proprietary versions.

As far as figuring out the true contents of the module
that the agent implements -- that gets much harder to
do with just eyeballs, thanks to this deviation-stmt.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 17 Dec 2008 10:17:49 +0100 (CET)

> (e.g., ../foo vs. /top/foo).

The spec says:

  the argument's string MUST be equal to the corresponding keyword's
  argument string in the target node.

I.e. the string values (after normal YANG string interpretation rules,
concatenation etc) must match exactly, including prefixes etc.

This is IMO the only workable solution - yes it might be cumbersome
for the writer of the 'deviate' statement, but this is a corner case
that is not recommended anyway, so I don't think we need to optimize
for this case.

> of all the objects with their proprietary versions.

But a vendor can claim anything, regardless of the deviation
statement.  Just b/c they specify how broken they are does not make
them more compliant.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 17 Dec 2008 08:32:09 -0800

> statement.  Just b/c they specify how broken they are does not make

This is the first time an IETF standard would be helping
the vendor 'claim anything'.

Vendors are always looking to lower costs.
If I'm implementing some YANG CM module,
and I tell my boss "it will take 15 weeks until code-complete,
because it took 12 weeks to add our proprietary version,
and I have to come up with some way that they can work
together now, because we can't remove our module."

When my boss asks me if there's anything we can do
to bring in the schedule (never heard that one before! ;-)
I have to mention the deviation-stmt.  "It will
take me 2 weeks to 'plug in' our DM into the standard one,
but most of the standard knobs won't really do what
they are supposed to do".

We all know that the vendor is going to take the easy path.

This issue is especially important for configuration
because multiple write objects for the same thing requires
coordination.  The standard model always lags the proprietary
model.  The temptation to just plug in part or all of
the proprietary model will always be compelling to the
vendor, and of no value whatsoever to the customer
of IETF standards.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 17 Dec 2008 22:54:11 +0100 (CET)

> the vendor 'claim anything'.

As David S. pointed out in another thread, this concept is supported
today in SMIv2 with AGENT-CAPABILITIES.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 17 Dec 2008 14:03:46 -0800

> today in SMIv2 with AGENT-CAPABILITIES.

There are many constraints placed on AGENT-CAPABILITIES
sub-clauses such as WRITE-SYNTAX.  You cannot replace
the data type with a completely different type like YANG.
YANG allows all the semantics to be changed, not just constrained.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 17 Dec 2008 23:17:35 +0100 (CET)

> YANG allows all the semantics to be changed, not just constrained.

I think it would be appropriate to have the same kind of restrictions
in YANG.  If you really want to completely change the type you will
have to specify that you don't support the standard object, and
add a brand new object (through augmentation).

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Dec 2008 05:42:46 -0800

> Andy Bierman <[email-masked]> wrote:
...
> deviation statement in a module, and that a property MUST NOT be

I do not see how this helps, since an arbitrary subset
of modules can have deviation-stmts, for which the
targets may overlap.   So the agent has to combine
deviation-stmts anyway (just like augment across submodules).
Restricting the DM writer to 1-per-module does not
solve any problem.  It just creates a CLR.

It is more work for the compiler to generate errors
for this, than it is to just combine the sub-clauses.

It is fine to be strict about adding existing
subclauses, deleting nonexistent subclauses,
and declaring the same subclause twice.
But if one submodule changes min-elements and
another one changes the default-stmt, why should
this be illegal?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 18 Dec 2008 14:52:29 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> I do not see how this helps

Ok.

Should the deviations be done in sequence?

So would this (stupid) example be legal:

  deviation /foo/bar {
      deviate add {
          must ". > baz";
      }
      deviate delete {
          must ". > baz";
      }
  }          

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Dec 2008 06:11:37 -0800

> So would this (stupid) example be legal:

No,  because the compiler cannot possibly
satisfy both the 'add' and 'delete' requests.
According to the text, one of them MUST fail.
The existing target node either has the must-stmt or not.
You can only add if it is not there.
You can only delete if it is there.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Dec 2008 06:32:51 -0800

> add a brand new object (through augmentation).

This will make it much less controversial.
The builtin type MUST NOT change if you
replace the type-stmt.  (1 little CLR :-)

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 18 Dec 2008 18:16:13 +0100

Hello,
Reading the thread some comments:

1) I hope that the order of evaluating the deviation statements does not influence the end 
result or the success/acceptability of deviation statements. However today it does. This could 
be a problem if the statements are in different modules, so the evaluation order is uncertain.

E.g. add the subtree myInterfaces later add subtree myInterfaces/specialCounters
Is this valid?
If myInterfaces is evaluated first, it looks fine.
If myInterfaces/specialCounters is evaluated first then it is trying to add something to a 
non-existing node .../myInterfaces, so it must fail.
You could come up with a lot of other examples as well.

2) I think we should have a generic statement somewhere: "The model changed according to the 
deviation statements MUST be a valid YANG model." That cuts down a lot of corner cases.

3) Why is it not allowed to delete the default statement?

4) If we don't advertise models that only provide typedefs and groupings, why do we need to 
advertise pure deviation modules? (Actually I can live with advertising them.)

Balazs

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Dec 2008 09:28:42 -0800

Balazs Lengyel wrote:
> 1) I hope that the order of evaluating the deviation statements does not 
> influence the end result or the success/acceptability of deviation 
> statements. However today it does. This could be a problem if the 
> statements are in different modules, so the evaluation order is uncertain.
> 
> If myInterfaces/specialCounters is evaluated first then it is trying to 
> add something to a non-existing node .../myInterfaces, so it must fail.

It does not matter.
Just like augment, the targets can appear in any order,
including out of order.

The deviation-stmt is validated wrt/ the real
object specified in the XPath target.  The other
deviation statements have nothing to do with it.
The XPath target node will exist or not, regardless of
how many deviation-stmts are present.

Also, you cannot add objects via deviation-stmt.
You can add sub-clauses to existing objects.

> 2) I think we should have a generic statement somewhere: "The model 
> changed according to the deviation statements MUST be a valid YANG 
> model." That cuts down a lot of corner cases.
> 

The new ABNF from Martin fixes that

> 4) If we don't advertise models that only provide typedefs and 
> groupings, why do we need to advertise pure deviation modules? (Actually 
> I can live with advertising them.)
> 

You should advertise all the modules, especially when
import-by-revision is added.

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 19 Dec 2008 09:55:11 +0100

Andy Bierman wrote:
>> 1) I hope that the order of evaluating the deviation statements does 
>> not influence the end result or the success/acceptability of deviation 
>> If myInterfaces/specialCounters is evaluated first then it is trying 
>> to add something to a non-existing node .../myInterfaces, so it must 
>> fail.
> object specified in the XPath target.  
You are right about the previous example, but there are other cases, when the order of 
evaluation could matter. E.g.:

leaf foo {
	type int8;
	mandatory true;
}

deviation ../foo {
    deviate replace {
	mandatory false;
    }
}

deviation ../foo {
    deviate add {
	default 5;
    }
}

If the replace is evaluated first it is fine. If the add (default) is evaluated first, it 
fails. Or do you have to collect all deviation and evaluate them together?

Similar conflicts can arise with type and default, type(leafref/keyref) and config(false), must 
and config(false).

> The other

IMHO it is good to state basic ideas, like the one I proposed, clearly and not just imply them. 
(Writing a perfect ABNF is a tough job.)

>> 3) Why is it not allowed to delete the default statement?

Please make this possible, otherwise switching mandatory to true will often not be possible.

>> groupings, why do we need to advertise pure deviation modules? 
>> (Actually I can live with advertising them.)
>>
> 
Earlier IMHO the consensus was that if you only use typedefs from an imported module you should 
not advertise it. Do we have a way of differentiating between:
- I only use types from module bar
- I use types from module bar, but I also implement it's data nodes

Balazs