Subject: Re: [netmod] Canonical value representations in typedefs [was RE: 6021 ipv4-prefix]

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 02 May 2019 15:12:58 +0200

"Rob Wilton (rwilton)" <[email-masked]> writes:

> Forking the thread title to avoid further polluting the original issue, and because these comments really apply to YANG.Next.
>
>> -----Original Message-----
>> From: Juergen Schoenwaelder <[email-masked]>
>> Sent: 29 April 2019 18:26
>> To: Rob Wilton (rwilton) <[email-masked]>
>> Cc: Ladislav Lhotka <[email-masked]>; NETMOD WG <[email-masked]>
>> Subject: Re: [netmod] 6021 ipv4-prefix
>> 
>> On Mon, Apr 29, 2019 at 05:08:41PM +0000, Rob Wilton (rwilton) wrote:
>> >
>> > If YANG allows a typedef to refine the canonical definition of a base
>> > type, then I think that the YANG RFC should be explicit on this (e.g.
>> > in YANG Next).  Particularly, because this requires a server
>> > implementation to read/understand the description associated with a
>> > leaf/typedef in case they have to add specific canonicalization code
>> > to implement the leaf/typedef.
>> 
>> Description statements in general are expected to be read and understood and
>> implemented where necessary. But I now see that the fact that this section 9.1 is
>> under section 9 which is titled built-in types is causing the confusion. This is,
>> indeed, unfortunate.
>
> Yes, OK.
>
> But this really means that YANG has two sorts of typedefs:
>
> (i) A regular typedef which is just an alias for a base type, perhaps with some additional refinements that can all be handled automatically (e.g. pattern statements, etc).  In this scenario, I think that a compiler can ignore the description, and process the typedef automatically.
> (ii) An enhanced typedef that has additional semantics embedded into the description statement that requires custom implementation of the typedef.
>
> Being able to distinguish these two cases in a programmatic way seems
> useful to me, but perhaps it is just unnecessary noise.

YANG assumes "powerful" descriptions in general. An option would be to find a way for specifying the canonical
format in a machine-readable form.

>
> For the second case, I do wonder whether this is much closer to adding new concrete type to the language, just in a slightly backdoor way.  There have been discussions about adding binary representations for some of these types (e.g IPv4 address, IPv6 address ,etc).  Perhaps that would be easier if they were somewhat closer to base types than derived types, and maybe this class of type definitions shouldn't be using typedef at all.
>
>> 
>> > I'm not sure that we have really got a simple solution for either clients or
>> servers:
>> >  1) Clients may use non canonical format on configuration input
>> >  2) But clients must still use canonical format for xpath expressions
>> >  3) Clients must also handle canonical format being returned on any get
>> requests.
>> >  4) Servers must perform normalization of any type to a canonical format, as
>> defined in the type/typedef/leaf description.
>> 
>> Exactly. Note that clients only send xpath for filtering (if they want to filter via
>> xpath). What is more important is that module authors can predict the format of
>> values when they write when or must expressions. And as Lada points out,
>> having predictable key values also is kind of desirable.
>
> I wasn't suggesting that we lose uniqueness of list keys, but instead
> don't allow typedefs to define their own canonical format.  I.e. that
> would always require IPv6 addresses to be provided in the canonical
> form.  However, the benefits of this are probably marginal given that

This just shifts the burden to the client side, with pretty much the same
issues. I think that the client should be allowed to send IPv6
addresses in any form permitted by RFC 4291.

Lada

> a robust server would want to check that the type is in the canonical
> form anyway, and hence the server still has to write very similar
> canonicalization code regardless.

>
>
>> 
>> The reality is that there are many different ways to write an IPv6 address and the
>> idea was to accept them on input but to subsequently work with the normalized
>> canonical representation. And this is in ietf-yang-types and ietf-inet-types since
>> day one. But yes, the text in section 9.1 seems to be misplaced.
>
> I've raised https://github.com/netmod-wg/yang-next/issues/83 to clarify the expected YANG behaviour here, and perhaps to also consider whether an extra statement would be beneficial.
>
> Thanks,
> Rob
>
>
>> 
>> /js
>> 
>> --
>> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
>> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
>> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Robert Varga <[email-masked]>
Date: Thu, 2 May 2019 17:42:34 +0200

> format in a machine-readable form.

The way I understand typedefs (and use of restricted types in
leaf/leaf-list) as that the first such use defines a value domain and
any further references must strictly refer to a subset of that domain.

Notably it does not seem to be sane to redefine the canonical format
once it has been set.

I think the entire canonical format problem comes from the fact
ipv4-address is, unlike case (i), only a representation of a value
domain which is semantically 32 bits.

That value domain has two functions that are different from plain old
string:
- equality
- total ordering

Canonical representation deals with the first one, while also providing
interoperability with XPath. There is no current mechanism to address
the second one.

I find having a machine-readable marker (such as an extension) that a
value domain is being defined immediately useful -- it allows the
compiler to warn that a specific typedef needs additional work to get it
working as intended. This one's really easy.

Having some (other) machine-readable marker of how a canonical value
looks like (i.e. similar to pattern) is also useful: it allows the
runtime to check if an incoming value is in canonical format and accept
it irrespective whether specific value domain support (from above) is
present or not. This one is moderately hard.

Finally, some third marker, which would describe how to transform a
non-canonical string into a canonical string would obviously be awesome,
as now the runtime can work automatically. This one is really hard,
especially since the operations probably involve operating not on a
string, but some other representation.

As in implementer, I really care about the first two -- they are easy to
do and provide immense value in making things work 'out of the box' and
automatically catching cases when they do not.

Regards,
Robert

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 03 May 2019 10:02:32 +0200

> interoperability with XPath. There is no current mechanism to address

Yes. XML schema languages use the concept of lexical vs. value space,
it is more general and more robust that the concept of canonical
format.

> string, but some other representation.

I liked the idea developed by Jeni Tennison for DSDL:

https://slideplayer.com/slide/15172219/

(tl;dr: see slide 33). It seems though that this idea didn't materialize.

Lada

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod