Subject: Re: [netmod] YANG 1.1 virtual interim 2014-11-19

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Nov 2014 14:37:42 -0500

Juergen Schoenwaelder writes:
>we are having the next YANG 1.1 virtual interim tomorrow, 2014-11-19,
>at 4pm CET. Below you will find the webex information.

I reviewed the issue list yesterday.  The following is my opinions
on these issues.  Apologies for the "bulk transfer mode".

Thanks,
 Phil

---------------

* REVIEW :Y01: allow boolean if-feature

No objection.  Not sure why we wouldn't use a real XPath expression.
Language describing the context would be needed, but it would allow
a better set of logic operations.

* REVIEW :Y02: allow must in input, output, and notification

No objection.

* REVIEW :Y03: allow if-feature in refine

No objection.

Does this mean the 'refine' is done if that expression is true,
or that the refined target gets an new 'if-feature' statement?

Does this if-feature replace an existing if-feature under the
target or add to it?  How does one remove an if-feature?

* REVIEW :Y04: accessible tree in xpath in notifs and rpc

document("data") has an existing meaning in XSLT.  Are we supporting
document() for arbitrary paths?

http://www.w3.org/TR/xslt#document

An alternative would be to make an XPath variable available that
contains the notication, like:

    must "$notification/a or $notification/b";

Variables would also be useful for feature logic, like:

    when "$features/max-connections > 10";

* OPEN :Y05: unprefixed path in top-level typedef

I'm unclear on the difference between y05-01 and y05-03.
It seems like making typedefs work like groupings would be
the path of least astonishment.

* REVIEW :Y06: escaped characters in double quoted strings

The convention that "\w" means "w" is very common and
is what folks will expect (Y06-03).

* REVIEW :Y07: do not allow when or if-feature on keys

No objection, though I'm not sure why the exception for
keys with identical conditions to their parents is worth
the time/trouble to explain.  Forbidding it is simpler.

* DEAD :Y08: make leaf-list of type empty illegal

Agree.

* OPEN :Y09: introduce optional keys

I don't understand the 'resolution' here.

My concern is that optional keys make interaction with databases
more difficult.  Keys are mandatory for most databases.

The usage of 'not(key)' in the second solution is not compatible
with xpath expressions, which defines not() as follows:

 Function: boolean boolean(object)
    The boolean function converts its argument to a boolean as follows:
    - a number is true if and only if it is neither positive or negative zero nor NaN
    - a node-set is true if and only if it is non-empty
    - a string is true if and only if its length is non-zero
    - an object of a type other than the four basic types is converted
    to a boolean in a way that is dependent on that type
 Function: boolean not(boolean)
    The not function returns true if its argument is false, and false otherwise.

not(optional-key) would be true when the value is a numeric zero
or a zero length string.

I would be against this change.  Keeping keys mandatory is
simpler and more straight forward.  Those wanting an optional
key can make a union type with a suitable 'missing' optional.

* VRFY :Y10: allow restrictions on enumerations

This is needlessly confusing.  The YANG looks like it's defining
a new type and the user is left to do the legwork to find out.

Are you really looking for refinement for types?  Seems like a
type is an atomic idea, not suitable for refinement.

I would be against this change.  It's confusing to the reader.
The same result can be done using a distinct type.

* REVIEW :Y11: allow if-feature on enums

Agree.

* EDIT :Y12: initialized-by system

Agree.  We left this out originally and I do not recall the
reasoning.  It's something we use in JUNOS.

* VRFY :Y13: allow multiple inheritance for identities

No strong objection, aside from the fact that it makes identities
(an already opaque subject) even more confusing.

* REVIEW :Y14: clarify the string value for identities when used in must/when

Agree.

* DEAD :Y15: identity advertisement problem

One can import modules without implementing them.  Importing allows
reuse of groupings and types, but doesn't require one to implement
the objects in a module, unless they are augmented or referenced.
If my monitoring/reporting module imports the bgp module to get
access to the groupings and typedefs, my monitoring system does not
need to implement bgp or its module to implement the monitoring/reporting
one.

I think there's some confusion regarding import-by-revision and the
need for this new conformance mechanism.

* OPEN :Y16: module advertisement optimization

No disagreement in principle.  I'd be happy to toss the capibility
exchange completely, especially given that it's drifted so amazingly
far from the original concept.  A yang:get-capabilities RPC is a fine
answer.

* DEAD :Y17: support "augment" of unique

Dead.

* OPEN :Y18: fix "when" expression context node problem

I think the problem is that the order in which various
leaf 'when' statements are resolved is not defined.  We
can't say 'document order' since augment allows inconsistent
ordering.  And we can't say "optimial ordering", since that
increases complexity and opens us to loops.  Imagine if
C had 'when ../A; default 2;'.  Or 'when not(../A);'.

I don't have an answer, but I can see adding words like
"leafs constraints are evaluated in document order where
possible" and "module authors need to avoid loops and logic
errors".

* DEAD :Y19: we need a better unique

Dead.

* REVIEW :Y20: new set of built-in XPath functions

Will review draft-bjorklund-netmod-yang-xpath-extensions-00.txt
separately.

* DEAD :Y21: statement to define new XPath functions in a module

It's possible to have a default value for the function in the
function definition, or to use function-available() in when/must
expressions that refer to it, but without a ternary operator, that's
fairly ugly.

I can certainly see the use for modules to define xpath functions,
e.g. ipv4:host-part(addr) ipv6:canonical-format(addr).

* DEAD :Y22: support constraints on unions

Dead.

* REVIEW :Y23: support negative patterns as string restrictions

A fine addition.

* DEAD :Y24: YIN -> YANG translation breaks text formatting

Dead.  Unavoidable cost of formating .vs. XML.

* OPEN :Y25: make enum numbering purely informative and optional

There's no contraction.  The rule makes it possible for
others to use these values, since values that change would
be unusable.

I think that enum numbering is _already_ purely informative and
optional.

* OPEN :Y26: allow mandatory nodes in augment

The restriction against having mandatory nodes in augments
is to prevent a client that understands the base model from
breaking the constraints introduced in an augmented module.
I think this is an important feature.

Intra-module augments are already permitted.

If we're seeing a need for 'abstract base models', then
there should be a means of declaring in the base module
that this is the permitted.  Without such a mechanism,
there's just too much room for abuse.

* DEAD :Y27: allow mandatory nodes in an updated module

Dead.

* VRFY :Y28: support default values in leaf-lists

The presece of multiple default statements looks like a
typo, especially if they weren't adjacent in the file,
I don't have a better suggestion.

* REVIEW :Y29: allow choice as a short-hand case

No objection.

* REVIEW :Y30: allow leafref in union

Will this create some interesting interactions?

* REVIEW :Y31: allow require-instance in leafref

No objection.

* DEAD :Y32: allow boolean combinations of pattern

Agree.

* DEAD :Y33: add 'attribute' statement

New attributes can be semi-defined via extension, like
"my:inactive true".

* OPEN :Y34: remove/deprecate/replace the 'anyxml' statement

I'm not following why anyxml is being considered for removal.
I consider it an important feature and strongly object.

* REVIEW :Y35: allow empty in union

No objection.

* DEAD :Y36: associate a notification with a data node

Dead.

* DEAD :Y37: allow notifications to be derived from other notifications
* DEAD :Y38: add 'mandatory' to NP-container
* DEAD :Y39: define the terms system/user-controlled instances
* DEAD :Y40: add floating point base types

All dead.

* REVIEW :Y41: clarification of "must" in NP-container

Agree.

* VRFY :Y42: a better model for configuration versus state data is needed

Dead, right?

* DEAD :Y43: add exception statement
* DEAD :Y44: add a mechanism to parameterize error-message

Agree.

* OPEN :Y45: better conformance mechanism

Will review draft-bierman-netmod-yang-conformance-02.txt separately.

* DEAD :Y46: data-specific meta-data
* DEAD :Y47: bit name too restrictive
* DEAD :Y48: enabled leaf design pattern

Dead.

* OPEN :Y49: clarify nested submodule behavior

Part of the list-everything-in-the-module was supposed to help the
reader know what the working parts are.

I'm not convinced this is really an issue, nor that it's
an important feature.  I've no strong opinion.

* DEAD :Y50: additional extension grammar validation
* DEAD :Y51: support for long running commands
* DEAD :Y52: add a shorthand statement for 'mandatory'
* DEAD :Y53: allow deep keys

Agree.

* OPEN :Y54: remove the advertisement of conformance information in NETCONF hello message

This issue seems pretty dead.

* EDIT :Y55: clarify type in union

I'm guessing the confusion is around the term "the first type that
validates is actually used", but yang says:

   When a string representing a union data type is validated, the string
   is validated against each member type, in the order they are
   specified in the "type" statement, until a match is found.

"actually used" .vs. "until a match is found"?  Are we looing for
words like "When a data value can be validated against a member of
a union, the value is considered to be of that type."?

* REVIEW :Y56: UTF8 non-characters

Making characters illegal is _always_ risky.  Consider that XML
made control characters illegal, and the grief that's caused.  We
should say UTF-8, and let I-JSON live with it's own problem.

* OPEN :Y57: non-unique leaf-list

While this is a problem, the normal fix is to introduce a user-specified
handle.  Making duplicate leaf-list entries introduces a number of
problems, including ordering (which "3" are you referring to?) and
deleting (remove both "3"s?).

* VRFY :Y58: associate an actions with a data node

I strongly disagree with the need for a second mechanism for
performing RPCs.  There's nothing one can do with this second
method that can't be done with the existing one.

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 10:32:38 +0100 (CET)

> target or add to it?  How does one remove an if-feature?

This issue is in the REVIEW state, which means that the current draft
has text addresing this issue.  Please check the text and let me know
if anything is unclear.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 10:46:51 +0100 (CET)

> more difficult.  Keys are mandatory for most databases.

Implementation detail.  Most databases do not support hierarchical
structures; yet we have that in YANG.

> with xpath expressions

[...]

> not(optional-key) would be true when the value is a numeric zero

Whoops.  You are right.  This would work: not(count(key)) - but looks
extremely weird.

> key can make a union type with a suitable 'missing' optional.

That's what people are using currently - but it is an ugly
workaround.   This affects the end user of the system, who has to
provide these "NULL" values.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 10:56:38 +0100 (CET)

> type is an atomic idea, not suitable for refinement.

Types in general can be "refined":

  typedef port {
    type uint16;
  }
  typedef system-port {
    type port {
      range "0..1023";
    }
  }

It is only enumerations that cannot - this is the CLR we want to
remove.

> The same result can be done using a distinct type.

Yes, that's the workaround.  But that implies copy&paste, which is
bad, and you loose the fact that the two types share semantics.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 11:01:20 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> leaf 'when' statements are resolved is not defined.

No this is not the problem.  The problem is that in this:

  leaf foo {
    when ../bar;
    ...
  }

the node 'foo' needs to exist in order to evaluate the XPath
expression - but if the expression evaluates to false it means the
node doesn't exist!

The proposed solution is to state that when the expression is
evaluated, the node 'foo' exists.

[a backwards-incompatible change would be to let the context node be
the parent, but that would be highly confusing:
   
   leaf foo {
     when "bar";
     must "../bar > 1";
     ...
   }

noone would ever get that right.]

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 11:08:55 +0100 (CET)

> I think that enum numbering is _already_ purely informative and

Explicit numbering is optional.  The proponents for this issue wants
to remove the implicit auto-numbering.  One argument is that even with
the auto-numbering the module designer has to be aware of it when the
module is updated; you cannot add enums in the middle w/o giving
explicit values.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 11:11:54 +0100 (CET)

> I think this is an important feature.

Agreed.

But the current rule is overly restrictive.  For example, if a module
defines an identity, and at the same time an augment that is
conditional on that identity, it would be fine:

   identity ethernet {
     base if:interface-type;
   }

   augment /if:interfaces/if:interface {
     when "if:type = 'ex:ethernet'";
   }

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 11:15:24 +0100 (CET)

> I consider it an important feature and strongly object.

The problem is that it is encoding-specifc.  In most case where it is
used, what we really mean is "any data modelled with YANG".  Hence,
the proposal Y34-02 suggests we add 'anydata' for this purpose.  Such
data can be encoded in XML or JSON.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 11:21:36 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> performing RPCs.

It is more than a second mechanism for performing rpcs.  From a data
modelling perspective it is a matter of encapsulation.  Define the
data structure and operations on the structre in one place (in the
hierarchy).

Several implementations already implement actions and it is very
common by the users to make use of them.

> There's nothing one can do with this second

Why do we have hieararcical data models; we could as well have all
lists and leafs at the top-level.  Why do you (presumably) not store
all your files in your home directory?

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 12:23:37 +0100 (CET)

> the path of least astonishment.

Yes, that is the idea.  I have merged Y05-03 with Y05-01; so my
proposal is to make them behave as groupings, see the updated proposal
Y05-01 at
https://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/issues.html#sec-6.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 10:55:31 -0500

FWIW, the answers were:

>> or that the refined target gets an new 'if-feature' statement?

if-features are added, not a conditional of the refine.

>> target or add to it?

refine adds new if-feature expressions.

>>How does one remove an if-feature?

You cannot.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 11:59:32 -0500

>     when "if:type = 'ex:ethernet'";

So since only one that implements this module would be
able to add the nodes in this module.  Any client not
understanding this module would not be able to create
these nodes, and so would not need to know what is
mandatory.

Ok, I can see this, but how does one limit it to cases like this
without opening clients to breakage?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 12:08:08 -0500

>data can be encoded in XML or JSON.

I take is as a given that there's no realistic say to round trip
between arbitrary XML and arbitrary JSON (or whatever else the
future may hold for other potential non-NETCONF uses of YANG).
So the choices are to keep anyxml or replace it with a string.
I would be strongly against the latter.  anyxml allows one to
carry opaque chunks of XML in a YANG data model.

Consider the recent proxy feature as spec'd in the "peer mount"
draft and how easily anyxml allows the encapsulation of subhierarchies
of data within a parent hierarchy.  This is just the sort of problem
anyxml was meant to address.

Or consider the need for undocumented content.  In JUNOS, we put
undocumented configuration under an "anyxml undocumented" member
so we can transmit our secrets without documenting them.  Clients
can validate the known parts of their config without tripping on
the unknown parts.

anyxml is a useful feature and should not be removed.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 12:11:31 -0500

>all your files in your home directory?

Why do you not say "/etc/motd.copy(/tmp/foo)"?

YANG data is the target of operations.  YANG doesn't define a
hierarchy of objects, but a hierarchy of data.   There's nothing
object-oriented in our approach.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 18:52:44 +0100 (CET)

> without opening clients to breakage?

Right, this is why we have:

  The proposal is to add general guiding rules
  that backwards compatibility needs to be maintained. Lets see
  whether someone can write more concrete rules when mandatory nodes
  in augment are allowed.

:)

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 18:54:31 +0100 (CET)

> anyxml is a useful feature and should not be removed.

The proposal is (Y34-02):

  Keep 'anyxml'.  Introduce 'anydata'

  'anyxml' would still be used to represent unrestricted XML, as is
  done in NETCONF.

  'anydata' would be an unknown chunk of data that can be modelled
  with YANG.  Can be encoded as xml or json.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Nov 2014 21:31:26 +0100

>  with YANG.  Can be encoded as xml or json.

IMO anydata should represent a chunk of data that is well-formed in the given encoding but for which no schema is known or applied. There is no need to exclude mixed content or attributes in XML, or things like [1, null, false] in JSON.

This means that for XML anydata and anyxml would mean the same. The question remains what anyxml could mean for JSON. My preference would be to declare anyxml as obsolete but equivalent to anydata.

Lada

> 
> 
> 
> /martin
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Kent Watsen <[email-masked]>
Date: Wed, 19 Nov 2014 20:43:01 +0000

>IMO anydata should represent a chunk of data that is well-formed in the
>given encoding but for which no schema is known or applied. <snip/>
>
>This means that for XML, anydata and anyxml would mean the same.

This is my understanding as well.

>The question remains what anyxml could mean for JSON. My preference would
>be to declare anyxml as obsolete but equivalent to anydata.

Could we alias anyxml to mean the same as anydata so that, despite its
name, we get the same semantic?

Kent

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Nov 2014 13:25:21 -0800

> This is my understanding as well.

Our tools have supported anyxml for years in all YANG contexts
and I do not see any changes at all needed except to accept
2 names for the exact same thing.

IMO, just add a note to YANG1.1 explaining the restricted and
simple mapping to YANG:

   complex node --> container
   simple node w/non-zero length content --> string leaf
   simple node w/zero length content --> empty leaf
   All attributes processed (or ignored) but implementation-specific
   All other XML ignored

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 22:35:44 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> IMO anydata should represent a chunk of data that is well-formed in
> the given encoding but for which no schema is known or applied. There
> is no need to exclude mixed content or attributes in XML, or things
> like [1, null, false] in JSON.

I object to this interpretation.  It's even worse than anyxml, since
it means that the data is different depending on which encoding was
used.

My interpretation of anydata is an unknown chunk of data where the data
is modelled w/ YANG, but the schema is unknown to us.

But this doesn't really work in the sense that roundtripping from JSON
to XML to JSON won't return the same result, since the XML
representation might have prefixes used in the content (i-i and
identyref again), and the JSON encoding is type-sensitive.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 16:44:39 -0500

>identyref again), and the JSON encoding is type-sensitive.

+1 to all.

"data that is well-formed in the given encoding" means that you
are making a modeling language that is suddenly _tightly_ bound
to some encoding.  I don't might RESTCONF as an alternative access
method for NETCONF datastores, but strongly object to anything
that breaks compatibility between YANG and NETCONF.

Since round tripping is not possible, anyxml would have to rendered
in JSON as a string.  "anydata" would not work, unless it was an
opaque string in both XML and JSON renderings.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Nov 2014 23:00:42 +0100 (CET)

> opaque string in both XML and JSON renderings.

Actually, the existing anyxml has its own set of roundtripping
problems.  Consider this:

  module a {
    namespace urn:a;
    ...
    container x {
      anyxml y;
    }
  }

and this payload:

  <rpc xmlns:p0="http://example.com">
    <edit-config xmlns:p1="http://acme.net">
      ...
      <config>
        <x xmlns="urn:a">
          <y>
            <p0:foo>
              <p1:bar>p0:baz</p1:bar>
            </p0:foo>
          <y/>
       ...
  </rpc>

You cannot simply store the string

            "<p0:foo>
              <p1:bar>p0:baz</p1:bar>
            </p0:foo>"

and return it in get-config, since the prefixes won't be declared
properly.

Also you cannot parse the prefixes and store the uris, since the
prefixes are used in the content.

Fortunately, you *can* store the string, and all prefixes in effect,
and in the get-config return:

   <y xmlns:p0="http://example.com"
      xmlns:p1="http://example.com">
     ... stored string goes here ...
   </y>

This also shows that "store anyxml as a string" in json doesn't work.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 19 Nov 2014 14:00:50 -0800

> opaque string in both XML and JSON renderings.

Maybe I agree with you, except...
Encoding anyxml as a string in JSON would be horrible.
What can it be used for?

Our running code for anyxml is just fine without any fixes
to theoretical problems. Some details....

1) parsing YANG data, not JSON data;
     If there is a conflict between the YANG and JSON type, YANG wins.
  2) the XML or JSON does not follow any schema, so it doesn't
       matter if multiple container siblings are parsed (instead of list)
       or multiple leaf siblings (instead of leaf-list)
  3) since there is no public schema, the client cannot expect much
       interoperability.  Basic structure w/ containers and leafs may not
        even be supported.

Stop trying to create rules for a chunk of data
that intended to be without rules.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 19 Nov 2014 17:31:33 -0500

>     ... stored string goes here ...

If we were smart, we would have been more pedantic about the XML
being stand-alone.

What's meant by this text (from XML Mapping rules in anyxml)?

   Note that any prefixes used in the encoding are local to each instance
   encoding.  This means that the same XML may be encoded differently by
   different implementations.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 09:11:49 +0100

> it means that the data is different depending on which encoding was

Well-formedness guarantees that the server will be able to safely parse
the data. Of course, well-formedness is encoding-specific.

A different issue is whether the server is able to interpret the data,
apply them, or translate them to another encoding.

> is modelled w/ YANG, but the schema is unknown to us.

The requirement that the data conform to *any* (even unknown) YANG model
is insufficient: for

anyxml configlet;

the server expects data conforming to a very specific schema. If it
receives BGP configuration instead, it is stuck. And if the module is
not known, the server may be unable to translate the data to a different
encoding, as you write below and elsewhere. So this restriction doesn't
really aid interoperability.

On the other hand, if the server does know the schema, it should be able
to work easily with data that are not modelled with YANG, too. One use
case that comes to my mind is exchanging YANG modules (or their parts)
in the YIN format.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 09:23:44 +0100

> are making a modeling language that is suddenly _tightly_ bound

I don't get the tight binding: anydata has to be well-formed (legal) in
whatever encoding that happens to be used.

> that breaks compatibility between YANG and NETCONF.

NETCONF uses XML, and for XML anydata in my interpretation is exactly
what anyxml is in YANG 1.0. Where does this break interoperability?

> opaque string in both XML and JSON renderings.

I disagree. Why should a JSON-only system be forced to work with
serialized XML data?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Nov 2014 09:38:37 +0100 (CET)

> I disagree. Why should a JSON-only system be forced to work with

The problem with encoding-specific anydata is that if you have such an
instance in the datastore, the contents depends on which encoding was
used to set it.  Once set with one encoding, it cannot be retrieved
using some other encoding.  For example, one json-based restconf
client sets it to some json structure, then another xml-based restconf
client cannot read it.

IMO, we should try to *remove* encoding specifics (i-i etc as we said
before), not add to the problem.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 09:41:23 +0100

> Whoops.  You are right.  This would work: not(count(key)) - but looks

No, Phil is not right:

$ cat test.xsl
<?xml version="1.0" standalone="yes"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
		version="1.0">
  <xsl:output method="text"/>
  <xsl:strip-space elements="*"/>

  <xsl:template match="/">
    <xsl:if test="not(key)">
      <xsl:text>NOT PRESENT.
</xsl:text>
    </xsl:if>
  </xsl:template>
</xsl:stylesheet>

$ cat test.xml
<?xml version="1.0" encoding="utf-8"?>
<key/>

$ cat test2.xml
<?xml version="1.0" encoding="utf-8"?>
<nokey/>

$ xsltproc test.xsl test.xml
$ xsltproc test.xsl test2.xml
NOT PRESENT.

not(key) evaluates to true only if no <key> element is present. The
value doesn't matter.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Nov 2014 09:44:31 +0100 (CET)

> If we were smart, we would have been more pedantic about the XML

Yes.  But this goes back to the original NETCONF spec.  It would have
been nice if the different protoocl layers in NETCONF were constrained
to being stand-alone:

  <rpc ...>
    [stand-alone operation here]
  </rpc>

  <rpc ...>
    <edit-config ...>
      <config ...>
         [stand-alone content here]
      </config>
    </edit-config>
  </rpc>

Since we use anyxml to model things like edit-config content, we
couldn't put the self-contained restriction on anyxml.

>    different implementations.

It means you cannot rely on the prefixes being the same after
roundtripping.  For example, you might set:

   <foo xmlns="urn:foo">42</foo>

and get back

   <x:foo xmlns:x="urn:foo">42</x:foo>

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Nov 2014 09:55:44 +0100 (CET)

[...]

> not(key) evaluates to true only if no <key> element is present. The

Good!  Sorry for contributing to the confusion.

Actually, Phil wrote:

which *is* true, but irrelevant.

What is important for Y09-02 is that we have a representation of an
instance-identifier for instances with unset optional keys, and that
this representation gives the correct instance when evaluated as an
XPath expression.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 10:04:31 +0100

> client sets it to some json structure, then another xml-based restconf

This is not necessarily true. If the server receives data it understands, it should also be able to render it in a different encoding. And if the server doesn’t understand the (structured) data, what’s the point of sending them? If it is an opaque string, the server can use it in all encodings (modulo escaping rules).

> before), not add to the problem.

XML, JSON or any other encoding are necessarily specific - for example, lists are rendered differently. Dealing with differences in scalar leaf values is nothing special or particularly difficult. Value-space values need to be common but lexical-space values may be encoding-specific.

I-I values are useful if the instance they point to can be easily identified, and this IMO requires them to be close to the form how the data tree is encoded. 

Lada

---

From: Phil Shafer <[email-masked]>
Date: Thu, 20 Nov 2014 04:10:35 -0500

>which *is* true, but irrelevant.

Doh!  'key' is a node-set, not a value:

% slaxproc -d /tmp/foo.slax /tmp/test.xml
sdb: The SLAX Debugger (version 0.18.1)
Type 'help' for help
(sdb) b 7
Breakpoint 1 at file /tmp/foo.slax, line 7
(sdb) c
Reached breakpoint 1, at /tmp/foo.slax:7
foo.slax:7:     if (not(key)) {
(sdb) p key
[node-set] (1)
<key>0</key>

(sdb) p boolean(0)
[boolean] false

(sdb) p boolean(1)
[boolean] true

(sdb) p boolean(key)
[boolean] true

(sdb)

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 10:13:07 +0100

> which *is* true, but irrelevant.

I don’t think it is true: in my example with test.xml, the <key/> element is empty (it’s value is a zero-length string), yet not(key) is false. The value of zero works the same:

$ cat test3.xml

$ xsltproc test.xsl test3.xml

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 10:27:28 +0100

>   whether someone can write more concrete rules when mandatory nodes

I think it is in the interest of a module writer to avoid backward
compatibility problem as much as possible. Maybe it needn't be enforced
in the language but just give appropriate guidelines in 6087bis.

Sometimes it might be desirable to break the backward compatibility,
e.g. if the old module has a security hole and a fix is provided through
an augment.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Nov 2014 10:38:05 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> I don’t think it is true

I think we agree, but just to be clear: The statement is actually true
in the sense that if X is a numeric zero or zero length string then
not(X) would be true.  But this is irrelevant, since in our case X is
always a node-set.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 10:41:39 +0100

> not(X) would be true.  But this is irrelevant, since in our case X is

OK, got it.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Nov 2014 10:45:21 +0100 (CET)

Hi,

I have added Y09-03, as decided in the meeting yesterday.

However, I think the arguments given for Y09-03 are all moot.

The arguments were:

  o  The instance-identifier syntax proposed in Y09-02 doesn't work,
     b/c empty strings and 0-values integers would also return "true"
     in the expression "not(key)".

  -- But this turned out to be a false claim; the syntax proposed in
     Y09-02 is correct.

  o  Y09-02 is tricky to implement in "common databases", since they
     require values for all keys.

  -- But the idea for how to handle Y09-03 in these databases was to
     store additional info that the default value is set.  The same
     technique can be used to handle purely optional keys.

Thus, I think there are no reasons for doing Y09-03.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 11:07:55 +0100

> Thus, I think there are no reasons for doing Y09-03.

+1, I think Y09-02 should be fine.  

Lada

---

From: Kent Watsen <[email-masked]>
Date: Thu, 20 Nov 2014 14:49:53 +0000

>   <x:foo xmlns:x="urn:foo">42</x:foo>

This is because the XML format allows alternate mechanisms for declaring
namespaces.  I like it the JSON encoding has no encoding variations.

AFAIK, the current JSON encoding is different than anything supported by
XML.  While it's scoping rules behave a lot like XML's default namespaces
(albeit with a prefix instead on an xmlns), the URI for each namespace is
known only implicitly from context.  That is, the execution environment
must have previously loaded the YANG modules, there is no global or local
xmlns used to declare the namespace that the environment can use to
dynamically load the schema.  Is this an issue?

Kent

---

From: Kent Watsen <[email-masked]>
Date: Thu, 20 Nov 2014 15:06:47 +0000

>client sets it to some json structure, then another xml-based restconf

My first thought was that the reader should be able to support JSON too
but, by extension, this leads to a statement along the lines of "Readers
MUST support all current encodings", which seems difficult to guarantee.

This is related to the XML-JSON MUST/SHOULD/MAY encoding debate, in which
we were debating if a RESTCONF server could only support JSON, to which
the WG consensus was have XML as a mandatory encoding.  Then, in our
discussions, we were primarily talking about the impact on NMSs (e.g. NMS
MUST support both XML and JSON).

This is a similar discussion, but there seems to be a difference in
granularity.  That is, the NMS use-case appears to be tolerant of
supporting new encodings as they are defined in time.  In contrast,
low-level database reader/writers appear to be more sensitive.  Is that
right?

Kent

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 20 Nov 2014 16:17:30 +0100

> dynamically load the schema.  Is this an issue?

Namespace URIs (and prefixes) are needed only for XML encoding, so unless you need to transcode JSON to XML the module name is good enough as a namespace ID.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Nov 2014 07:34:13 -0800

> +1, I think Y09-02 should be fine.

RFC 6536 (NACM) does not support optional keys.
The new instance-identifier syntax is not accepted by NACM.
Simply leaving out the optional key predicate in NACM will cause
the data-rule to match all values of that key.

I do not support this change to YANG 1.1.
The impact on implementations is pervasive and severe,
and the apparent use-case is for a small number of lists
that could have optional keys, but no value is available
to indicate "not-in-use" for 1 or more optional keys.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 21 Nov 2014 16:42:27 +0100 (CET)

> The new instance-identifier syntax is not accepted by NACM.

Correct.  In order to be able to use NACM for these entries, we need
an updated version of NACM.  The good news is that the changes are
minimal for this particular issue.  Essentially we just need to refer
to the YANG 1.1 spec instead of 6020 - the text as written works for
the new syntax for i-i as well.  It is probably a good idea to clearly
state this in the description of 'node-instance-identifier' though.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 21 Nov 2014 16:42:06 +0100

> the data-rule to match all values of that key.

NACM is YANG 1.0, so it would have to be updated.

> to indicate "not-in-use" for 1 or more optional keys.

Isn’t it the proposed [not(optkey)]?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 21 Nov 2014 08:11:57 -0800

> state this in the description of 'node-instance-identifier' though.

Somehow this minor maintenance release of YANG has a lot of
new bells and whistles, some of which have a huge impact
on existing implementations.  It is turning into a forklift upgrade,
which lowers the probability it will be implemented.

I don't know of any databases that support optional keys.
It is not supported in SNMP either.
This is a new feature, not any sort of bug-fix.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 23 Nov 2014 08:24:08 -0800

> e.g. if the old module has a security hole and a fix is provided through

The solution we have in YANG is to change the status to "obsolete",
if an existing definition is found to be harmful.

If a server update breaks old clients, the vendor is likely
to hear about it from unhappy developers and operators.
This is the sort of CLR that gets followed.

If an external augment add mandatory nodes that break
any exiting clients, then it will be visible to the vendor,
and no amount of IETF wordsmithing is going to help
the server developer who added the mandatory augment
win that argument.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Nov 2014 15:27:03 +0100

> if an existing definition is found to be harmful.

This solution won't work if the fix is based on adding stuff rather than
removing.

> This is the sort of CLR that gets followed.

Strict backward compatibility rules - for augments or new module
revisions - could be useful if the set of YANG modules were relatively
stable. However, this is not the current situation because everything is
in flux. Insisting on compatibility rules means that either

- we will end up with awkward modules having too loose constrains and
  containing a lot of obsolete stuff, or

- old modules will be abandoned and new ones started instead.

I am not sure either of these will make developers or operators any
happier in the long run.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Nov 2014 06:49:53 -0800

> This solution won't work if the fix is based on adding stuff rather than

Sure it will, just change the names.
The new client can code to the new names.
The old client is supposed to go away because the data model is harmful.

> in flux. Insisting on compatibility rules means that either

That is no excuse -- design and implement first, publish later

> - old modules will be abandoned and new ones started instead.

If the first pass is so bad that they need to be thrown out
and started over, then so what?  Mandatory nodes can be added
in a different table that uses leafref indexing. There are options
in YANG available besides augmenting an external module
with mandatory nodes.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Nov 2014 16:51:26 +0100

> The old client is supposed to go away because the data model is harmful.

This is not a good strategy for standard models, where readability should be the priority.

> That is no excuse -- design and implement first, publish later

It is not how the current modules are being developed, at least in the IETF.

> in YANG available besides augmenting an external module

They needn’t be so bad, they just might need to be updated in ways that are incompatible with the YANG update rules.

I understand that backward compatibility is an absolute priority for software producers but quite often it leads to seriously sub-optimal products that are full of cruft.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Nov 2014 06:47:19 -0800

Hi,

If we do not follow the update rules, we just add to the churn.

When a client app is written, the set of modules at the time
are available for use.  Modules written or modified in the future
are not available for use.  This should seem rather obvious,
and point out why future modules cannot break existing contracts.
The apps cannot be rewritten and deployed constantly or instantly.

Andy