Subject: [netmod] instance file format and etags/timestamps, xml attributes

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 28 Nov 2018 11:01:16 +0100

draft-ietf-netmod-yang-instance-file-format-00.txt says:

   The JSON format SHALL follow the format of the reply returmed for a
   RESTCONF GET request directed at the datastore resource:
   {+restconf}/data.  ETags and Timestamps SHOULD NOT be included, but
   if present SHOULD be ignored.

I assume that you mean the JSON content in the message-body of the
HTTP Response message for GET message. My understanding is that ETags
and Timestamps (what are these precisely?) are carried in the HTTP
header. So how could the ETag or 'Timestamps' be in the JSON data?  We
should not mess up the HTTP difference between header data and payload
data.

I also do not fully understand the text concerning the XML format.

   The XML format SHALL follow the format returned for a NETCONF GET
   operation.  The <data> anydata (which is not part of the real data
   itself) SHALL contain all data that would be inside the <data>
   wrapper element of a reply to the <get> operation.  XML attributes
   SHOULD NOT be present, however if a SW receiving a YANG Based
   Instance data file encounters XML attributes unknown to it, it MUST
   ignore them, allowing them to be used later for other purposes.

It is unclear what exactly is the instance data - the entire reponse?
Everything inside <data>? Everything inside and including <data>? I
assume the second sentence is trying to say the later but I do not
find it very clear not does it seem to be right. The examples show to
content of the NETCONF <rpc-reply><data/></rpc-reply> inside a <data>
container that belongs to the instance data format (two times <data>
but in different namespaces).

It is also unclear to me why XML attributes are to be removed. Why is
that? If I snapshot <operational>, why should I remove important
information such origin annotations? And removing attributes is
actually plain wrong if you consider that attributes carry XML
namespaces.

/js

PS: I am also concerned about the revision being not fine grained
    enough to be useful. I would love to have a much more precise
    timestamp telling me when the instance data was recorded. I would
    probably replace 'revision' with simply a 'timestamp' or add next
    to a 'revision' a more fine grained 'timestamp'.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Balázs Lengyel <[email-masked]>
Date: Fri, 30 Nov 2018 12:28:09 +0000

Hello Jurgen,

Thanks for the comments. See answers below.

regards Balazs

On 2018. 11. 28. 11:01, Juergen
Schoenwaelder wrote:

BALAZS: OK, I will correct it. Yes, it is the
payload that I want to include in the instance data.

I thought that lower level etags/timestamps are returned inside
the get-response payload. On re-reading RFC8040 I see my mistake.

BALAZS: I will try to reword it to clarify
the issue. How about:

An instance data set is made up of header
part and content-data. The content-data is all data inside the
anydata data node.

The header part is defined by the
-ietf-instance-data module while the content-data is defined by
the target YANG modules. The content-data  SHALL contain all
data that would be inside the <data> wrapper element of a
reply to the <get> operation .

I hope this conveys that content data
excludes the <data> wrapper element from the get-reply.

BALAZS: You are right, although some
attributes might be absent in some use cases.  E.g. namespace as
you pointed out is always needed. However e.g. origin may be
present if the instance data is a snapshot of the operational
datastore, but it may be absent if the instance data is used to
document readOnly server capabilities.

So I propose to change the text to:

Some XML attributes (e.g. metadata like origin) MAY be absent.
SW handling YANG Instance data MUST ignore
XML attributes unknown to it, allowing them to be used
later for other purposes.

/js

BALAZS: I agree that in some use cases a
timestamp would be useful e.g. diagnostic data from a real live
YANG server.

However in other use cases like documenting factory default,
defining default configuration to be preloaded or documenting
server capabilities I see no need for the timestamp. It is not
interesting exactly at which hour/minute/second the server
capabilities were documented.

So while I would not like to add the timestamp in the draft, the
draft documents, that additional metadata like a timestamp may
be added to the instance data set.

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 30 Nov 2018 13:48:01 +0100

On Fri, Nov 30, 2018 at 12:28:09PM +0000, Balázs Lengyel wrote:
>    On 2018. 11. 28. 11:01, Juergen Schoenwaelder wrote:
> 
>    BALAZS: OK, I will correct it. Yes, it is the payload that I want to
>    include in the instance data.
>    I thought that lower level etags/timestamps are returned inside the
>    get-response payload. On re-reading RFC8040 I see my mistake.

The more I think of it, defining the format by refering to specific
protocol operations really seems to be broken and causing complex
maintenance problems in the future. If you fetch <operational>, you do
not do a GET on {+restconf}/data.

>    BALAZS: I will try to reword it to clarify the issue. How about:
> 
>    An instance data set is made up of header part and content-data. The
>    content-data is all data inside the anydata data node.
> 
>    The header part is defined by the -ietf-instance-data module while the
>    content-data is defined by the target YANG modules. The content-data�
>    SHALL contain all data that would be inside the <data> wrapper element of
>    a reply to the <get> operation .� �
> 
>    I hope this conveys that content data excludes the <data> wrapper element
>    from the get-reply.

As explained above already, I meanwhile believe it is wrong to define
the format by refering to protocol operations. A NETCONF <get> (using
NETCONF writing style) is wrong for any NMDA datastore content. We
should define the instance data format by refering to the XML and JSON
YANG encoding rules and not by refering to specific protocol
operations.

>    BALAZS: You are right, although some attributes might be absent in some
>    use cases.� E.g. namespace as you pointed out is always needed. However
>    e.g. origin may be present if the instance data is a snapshot of the
>    operational datastore, but it may be absent if the instance data is used
>    to document readOnly server capabilities.
> 

I disagree and I would prefer to be silent. The YANG models and the
encoding rules define the content. The instance format document has no
right to mess around with that. Whether origin is there or not is well
defined elsewhere. It is not the job of this document to repeat or
even change what is said elsewhere.
 
>    BALAZS: I agree that in some use cases a timestamp would be useful e.g.
>    diagnostic data from a real live YANG server.
>    However in other use cases like documenting factory default, defining
>    default configuration to be preloaded or documenting server capabilities I
>    see no need for the timestamp. It is not interesting exactly at which
>    hour/minute/second the server capabilities were documented.
>    So while I would not like to add the timestamp in the draft, the draft
>    documents, that additional metadata like a timestamp may be added to the
>    instance data set.
> 

Having an (optional) timestamp with proper granularity seems rather
basic for me and I fail to see why we would not define this.

/js

---

From: Balázs Lengyel <[email-masked]>
Date: Fri, 30 Nov 2018 16:41:30 +0000

On 2018. 11. 30. 13:48, Juergen
Schoenwaelder wrote:

BALAZS: Don't really agree, but I will add the optional timestamp
as you think its important.