Subject: [netmod] Client validation text [was RE: YANG Versioning Weekly Call Minutes - 2021-04-06]

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 7 Apr 2021 09:19:15 +0000

// As a contributor

Hi Jason, all,

In yesterday's meeting there was a discussion on this text, and Joe pointed out that any sensible client must validate its input.

   While incoming configuration data is checked according to YANG
   constraints, constraints on state data sent by the server MAY or MAY
   NOT be enforced.  The following guidelines are provided for client
   application designers to allow a smooth interworking with servers.

   o  A client MUST tolerate any data received (or not received) without
      crashing.

   o  A client MUST be able to discard any data that is not part of the
      model but is sent by the server additionally (e.g.  XML elements
      or attributes, JSON properties).

   o  A client SHOULD be able to handle valid parts of a received data
      set even if it discards other parts as invalid.

   o  A client SHOULD be able to handle data that is outside the
      valuespace defined, as long as it is of the same basic type.

   o  A client SHOULD be prepared to handle more items for a list or
      leaf-list than what is defined by the model.

Based on Joe's comments, I suggest that this text could potentially be written as something like:

   Client applications are expected to perform sanity checking of data

   received from a server and to handle unexpected or missing data

   gracefully, e.g., this could include ignoring unexpected data, or

   logging unexpected values for further analysis.  Clients SHOULD NOT

   discard an entire response from a server because some data contained

   within the response is not expected.  Examples of well-encoded but

   unexpected data received from a server may include:

   o  Values that are outside the value space of a data node defined

      in the YANG schema, but that are within the value space of the

      underlying base type, e.g., if the value represents an unexpected

      error condition on the server.

   o  Additional data nodes, e.g., if the server implements a

      different, but compatible, version of a YANG module.

   o  A greater or lesser number of list or leaf-list items than the

      permitted range defined in the YANG module.

   o  Non mandatory data nodes that are sometimes missing from the

      response.  Noting that the server is expected to deviate any data

      nodes for which it will never return values for.

   o  Values that do not conform to the semantic constraints of the schema.

   o  Additional YANG meta data in the encoding (e.g., XML elements or

      attributes, JSON properties).

   NMDA [RFC 8342], section 5.3, provides additional constraints on the

   data that a server can return from the operational state datastore.

Thanks,
Rob

From: netmod <[email-masked]> On Behalf Of Sterne, Jason (Nokia - CA/Ottawa)
Sent: 06 April 2021 15:10
To: [email-masked]
Subject: [netmod] YANG Versioning Weekly Call Minutes - 2021-04-06

YANG Versioning Weekly Call Minutes - 2021-04-06

Focus for this meeting was going through Jason's review comments for section "3.1.2 Backwards-compatibility rules for config false and output data" of https://tools.ietf.org/html/draft-ietf-netmod-yang-module-versioning-02.

(A)
Valuespace:
- value space (with a space between the words): use 7950 meaning/definition (remove the definition in our draft)
- make "must" its own bullet
- don't particularly address "description"
- Balazs propose updated text

(B)
replace this:

"an additional state leaf can easily be discarded"

with this:

"the presence of an unexpected state leaf is not typically a problem and may be ignored by the client"

(C)
replace "config=false data" in the 1st paragraph with the following (and keep the quotes - that is how RFC8342 presents it):
                "config false" data

(D)
Lots of debate about the "client" bullets in 3.1.2.  Didn't conclude.  Perhaps just summarize and say clients need to sanitize data (give examples of data they might get, values outside range)

ACTION: focus on reviewing section 3.1.2

----------------------------------------------
Weekly webex call details:
Meeting number (access code): 171 069 0374
Meeting password: semver?
Occurs every Tuesday effective Tuesday, September 1, 2020 until Tuesday, August 24, 2021 from 9:00 AM to 10:00 AM, (UTC-04:00) Eastern Time (US & Canada)
9:00 am  |  (UTC-04:00) Eastern Time (US & Canada)  |  1 hr
https://ietf.webex.com/ietf/j.php?MTID=ma7627a2ae7b770537cff5f5b89293c70
Tap to join from a mobile device (attendees only)
+1-650-479-3208,,1710690374## Call-in toll number (US/Canada)

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Thu, 8 Apr 2021 17:58:58 +0000

Hi Rob,
I like the reformulation of this.
Jason

From: Rob Wilton (rwilton) <[email-masked]>
Sent: Wednesday, April 7, 2021 5:19 AM
To: Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>; [email-masked]
Subject: Client validation text [was RE: YANG Versioning Weekly Call Minutes - 2021-04-06]

// As a contributor

Hi Jason, all,

Thanks,
Rob

From: netmod <[email-masked]<mailto:[email-masked]>> On Behalf Of Sterne, Jason (Nokia - CA/Ottawa)
Sent: 06 April 2021 15:10
To: [email-masked]<mailto:[email-masked]>
Subject: [netmod] YANG Versioning Weekly Call Minutes - 2021-04-06

(A)
Valuespace:

(B)
replace this:

with this:

(C)

(D)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 8 Apr 2021 20:51:27 +0200

Some of these make no sense to me and I wonder whether we really need
to define these.

>    o  A client MUST tolerate any data received (or not received) without

What is 'tolerate', what is 'without crashing'? Are we talking about
buggy parsers or buggy backend logic or backend logic fooled by
inconsistent values?

>       set even if it discards other parts as invalid.

In all cases?

>       valuespace defined, as long as it is of the same basic type.

Really? What does 'handle' mean? If a value makes no sense at all,
what should the client do other than discarding the value (and logging
an error)? Or is this 'handling'?

>       leaf-list than what is defined by the model.

Again, what does 'handle' mean???

>    data that a server can return from the operational state datastore.

It might be necessary to write the security considerations text for
this.  From a security perspective, being lenient is often a
mistake. For example, if a leaf is of type inet-address, I rather not
have clients accepting any garbage strings as IP addresses.

I am not sure why we go into this. Which problem are we solving by
requiring clients to accept any garbage?

/js

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 9 Apr 2021 13:32:15 +0000

Thanks for taking a look Juergen.

Note that the first part of Rob's email was the *old* text that he suggests replacing with the bullets in the second part.  But some of your comments in the old text probably also apply to the new text.

The root of this section in our draft is that we wanted to clarify how to decide if a change is BC vs NBC for *state* data (config false).  Our feeling is that we may want to refine some of the rules in 7950 for state.  

One key example is this:  7950 says that adding another enum to an enumeration leaf is NBC (and that applies to state). But that may not really be how most implementations would want to treat state. Would we really want to flag a module as non backwards compatible when a state leaf gets an additional enum?  Wouldn't that create a lot of unnecessary noise?

Jason 

> -----Original Message-----
> From: Juergen Schoenwaelder <[email-masked]>
> Sent: Thursday, April 8, 2021 2:51 PM
> To: Sterne, Jason (Nokia - CA/Ottawa) <[email-masked]>
> Cc: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
> Subject: Re: [netmod] Client validation text [was RE: YANG Versioning
> Weekly Call Minutes - 2021-04-06]
> 
> > In yesterday's meeting there was a discussion on this text, and Joe pointed
> out that any sensible client must validate its input.
> >
> > From: netmod <[email-masked]<mailto:netmod-
> [email-masked]>> On Behalf Of Sterne, Jason (Nokia - CA/Ottawa)
> > Focus for this meeting was going through Jason's review comments for
> section "3.1.2 Backwards-compatibility rules for config false and output data"
> of https://tools.ietf.org/html/draft-ietf-netmod-yang-module-versioning-02.
> >
> > (A)
> > Valuespace:
> > - value space (with a space between the words): use 7950
> meaning/definition (remove the definition in our draft)
> > replace "config=false data" in the 1st paragraph with the following (and
> keep the quotes - that is how RFC8342 presents it):
> >                 "config false" data
> >
> > (D)
> > Lots of debate about the "client" bullets in 3.1.2.  Didn't conclude.  Perhaps
> just summarize and say clients need to sanitize data (give examples of data
> they might get, values outside range)
> >
> > Occurs every Tuesday effective Tuesday, September 1, 2020 until Tuesday,
> August 24, 2021 from 9:00 AM to 10:00 AM, (UTC-04:00) Eastern Time (US &
> Canada)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Apr 2021 15:39:01 +0200

> One key example is this:  7950 says that adding another enum to an enumeration leaf is NBC (and that applies to state). But that may not really be how most implementations would want to treat state…

I read this in RFC 7950:

   o  An "enumeration" type may have new enums added, provided the old
      enums's values do not change.  Note that inserting a new enum
      before an existing enum or reordering existing enums will result
      in new values for the existing enums, unless they have explicit
      values assigned to them.

What do you want this to change to?

/js

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 9 Apr 2021 13:43:09 +0000

Urghh.  I reversed my example.  I should have said removing an enum.  Let me reword:

One key example is this:  7950 says that removing an enum from an enumeration leaf is NBC (and that applies to state). But that may not really be how most implementations would want to treat state. Would we really want to flag a module as non backwards compatible when a state leaf has an enum removed?  Wouldn't that create a lot of unnecessary noise?

> Sent: Friday, April 9, 2021 9:39 AM
> > One key example is this:  7950 says that adding another enum to an
> enumeration leaf is NBC (and that applies to state). But that may not really
> be how most implementations would want to treat state. Would we really
> want to flag a module as non backwards compatible when a state leaf gets an
> additional enum?  Wouldn't that create a lot of unnecessary noise?
> 
> I read this in RFC 7950:
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Apr 2021 15:53:08 +0200

I do not recall that removing an enum was ever an issue in
practice. If an enum value is not used anymore, you leave the old enum
value and it will slowly but surely not be used anymore. (An enum
statement even has a status statement, so you can deprecate or
obsolete enum values.) That said, if the module owner decides to
remove the value, then this is indeed non-backwards compatible. (And
removing an enum paves the way to reallocate the associated number,
and be it by accident later again. I suggest people think twice
before removing enums.)

/js

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 9 Apr 2021 14:00:42 +0000

The key focus here is *if* the author does remove the enum, then how should we label the revision -> BC or NBC ?

RFC7950 does indeed say that is NBC.  But do we actually want that for state for:
- removing an enum
- shrinking a range
- changing a pattern in a manner that reduces the value space

We're worried that will create too much "NBC noise" when it really in practice won't be an issue at all for clients.  Client just won't receive the old values from the larger value space anymore.  So why flag this as NBC and make people do work to analyze it ?

Jason

> Sent: Friday, April 9, 2021 9:53 AM
> > One key example is this:  7950 says that removing an enum from an
> want to flag a module as non backwards compatible when a state leaf has an
> enum removed?  Wouldn't that create a lot of unnecessary noise?
> >
> > > On Fri, Apr 09, 2021 at 01:32:15PM +0000, Sterne, Jason (Nokia -
> CA/Ottawa)
> > > wrote:
> > >

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Apr 2021 16:19:35 +0200

Creating lots of special rules makes me feel uncomfortable. Is there
evidence that people reduce state value spaces a lot and in isolation,
i.e., they just rev a module to reduce some state value spaces?

/js

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Fri, 9 Apr 2021 14:32:25 +0000

Hi Juergen,

Yes - it isn't obvious and we've been struggling with what to do about state. It just feels like maybe the 7950 rules were more focussed on config and it might be worthwhile adapting some of them a bit for state for this YANG Revision work we're doing.

The big one is this value space issue. It feels like expanding/reducing value space has different impacts on clients for config vs state.

I think a fairly common use case for this is bug fixes in vendor modules. And note that at least two major router vendors have separate modules for state (Nokia and Cisco).

Jason

> Sent: Friday, April 9, 2021 10:20 AM
> > The key focus here is *if* the author does remove the enum, then how
> should we label the revision -> BC or NBC ?
> >
> > We're worried that will create too much "NBC noise" when it really in
> practice won't be an issue at all for clients.  Client just won't receive the old
> values from the larger value space anymore.  So why flag this as NBC and
> make people do work to analyze it ?
> >
> > Jason
> >
> > > On Fri, Apr 09, 2021 at 01:43:09PM +0000, Sterne, Jason (Nokia -
> CA/Ottawa)
> > > wrote:
> > > > Urghh.  I reversed my example.  I should have said removing an enum.
> Let
> > > me reword:
> > > >
> > > > >    o  An "enumeration" type may have new enums added, provided
> the
> > > old

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Apr 2021 10:00:15 -0700

> i.e., they just rev a module to reduce some state value spaces?
The IETF does not typically focus this much on implementation details.
(Like I need the IETF to write "MUST NOT crash" in an RFC to get better
code. ;-)

Do we need 20 pages of rules on how to increment a SEMVER?
I sure hope not. The client app still needs to access specific objects and
it depends on
the implementation details how to handle NBC changes.  The SEMVER value is
not
very useful at the YANG object level, where the code needs to focus.

/js

Andy

> practice won't be an issue at all for clients.  Client just won't receive
> the old values from the larger value space anymore.  So why flag this as
> NBC and make people do work to analyze it ?
> >
> > Jason
> >
> > > > Urghh.  I reversed my example.  I should have said removing an
> enum.  Let
> > > me reword:
> > > >
> > > > > enumeration leaf is NBC (and that applies to state). But that may
> not
> > > really
> > > > > Fax:   +49 421 200 3103         <https://www.jacobs-university.de/
> >
> > >
> > > --