Subject: [Netconf] RESTCONF media type issue

From: Andy Bierman <[email-masked]>
Date: Mon, 20 Jun 2016 17:11:26 -0700

Hi,

The media type definitions in RESTCONF-13 are incorrect.
We cannot make subtypes for application/yang.
Each RESTCONF (and YANG Patch) media type has
to be registered.

Approach 1) hard-wired types

RESTCONF would register 10 media types and YANG Patch 4 media types
if we use the hardwired approach.

   application/yang-api+xml
   application/yang-api+json
   application/yang-datastore+xml
   application/yang-datastore+json
   application/yang-data+xml
   application/yang-data+json
   application/yang-errors+xml
   application/yang-errors+json
   application/yang-operation+xml
   application/yang-operations+json
   application/yang-patch+xml
   application/yang-patch+json
   application/yang-patch-status+xml
   application/yang-patch-status+json

Any new media types would need to be registered.
No usage of the restconf-media-type is possible besides
this list of media types for RESTCONF and YANG Patch
without additional registrations.

Approach 2) generic types with a parameter

A parameter approach would allow YANG to be used to define any
message content schema.  E.g., a mandatory "type" parameter
that specifies the module and name of the restconf-media-type.

Only 2 media type need to be registered for every media-type
that could ever be specified (SDO or vendor)

   application/yang-data-xml;type=<module>:<name>
   application/yang-data-json;type=<module>:<name>

   application/yang-data-xml;type=ietf-restconf:api
   application/yang-data-json;type=ietf-restconf:api
   application/yang-data-xml;type=ietf-restconf:datastore
   application/yang-data-json;type=ietf-restconf:datastore
   application/yang-data-xml;type=ietf-restconf:data
   application/yang-data-json;type=ietf-restconf:data
   application/yang-data-xml;type=ietf-restconf:errors
   application/yang-data-json;type=ietf-restconf:errors
   application/yang-data-xml;type=ietf-restconf:operation
   application/yang-data-json;type=ietf-restconf:operation
   application/yang-data-xml;type=ietf-yang-patch:yang-patch
   application/yang-data-json;type=ietf-yang-patch:yang-patch
   application/yang-data-xml;type=ietf-yang-patch:yang-patch-status
   application/yang-data-json;type=ietf-yang-patch:yang-patch-status

If you have a preference how this issue is resolved, send comments.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Jun 2016 09:39:25 +0200

> On 21 Jun 2016, at 02:11, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 

I prefer #1, as media types aren't supposed to carry schema information (and we do have it elsewhere). One thing to consider though is to reduce the number of media types. For example, it's IMO not necessary to have separate media types "application/yang-data" and "application/yang-operation" because they can be easily discriminated via the URI.

Lada

> 
> 
> 
> Andy
> 
> 
> 
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:16:26 -0700

On Tue, Jun 21, 2016 at 12:39 AM, Ladislav Lhotka <[email-masked]> wrote:

> I prefer #1, as media types aren't supposed to carry schema information
> (and we do have it elsewhere). One thing to consider though is to reduce
> the number of media types. For example, it's IMO not necessary to have
> separate media types "application/yang-data" and
> "application/yang-operation" because they can be easily discriminated via
> the URI.
>
>

I think #1 is simpler to implement.
I don't agree we should make operation and data resources the same media
type.
It is useful in implementation to tell POST data from POST operation before
parsing
the message-body.

Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Jun 2016 14:23:06 +0200

> On 21 Jun 2016, at 14:16, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> I don't agree we should make operation and data resources the same media type.
> It is useful in implementation to tell POST data from POST operation before parsing
> the message-body.

Parsing and resolving the RequestURI should be sufficient to tell them apart, no?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:32:12 -0700

> Parsing and resolving the RequestURI should be sufficient to tell them
no...

container foo {
   container input {
      leaf X { type string; }
   }
   action bar {
      input {
         leaf X { type string; }
      }
    }
 }

POST /restconf/data/foo

{ "input" : {
     "X" : "so is this an invocation of action bar or a creation of
container input?"
   }
}

What problem are you solving by combining them?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:35:18 -0700

>> Parsing and resolving the RequestURI should be sufficient to tell them

oops -- the action would include input in the URI

POST /restconf/data/foo  (data)

POST /restconf/data/foo/input  (operation)

IMO is is still clearer to use separate media type for operations and data.

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Jun 2016 14:49:31 +0200

> On 21 Jun 2016, at 14:32, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
>      "X" : "so is this an invocation of action bar or a creation of container input?"
>    }
> }

Shouldn't the action be invoked with

POST /restconf/data/foo/bar

Section 3.6 says

   An action is invoked as:

   POST {+restconf}/data/<data-resource-identifier>/<action>

> What problem are you solving by combining them?

Media types should not be multiplied unnecessarily.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:49:32 -0700

> ...

Andy

> POST /restconf/data/foo/input  (operation)

I was right the first time.
The URI does not contain "input", just the message.
So the media type is needed to distinguish actions from data.

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Jun 2016 14:56:02 +0200

> On 21 Jun 2016, at 14:49, Andy Bierman <[email-masked]> wrote:
> 
> 
> ...
> no...
> 
> 
> container foo {
>    container input {
>       leaf X { type string; }
>    }
>    action bar {
>       input {
>          leaf X { type string; }
>       }
>     }
>  }
> 
> 
> POST /restconf/data/foo
> 
> 

IMO it should contain the action name.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 05:56:59 -0700

On Tue, Jun 21, 2016 at 5:49 AM, Ladislav Lhotka <[email-masked]> wrote:

> > > I prefer #1, as media types aren't supposed to carry schema
> information (and we do have it elsewhere). One thing to consider though is
> to reduce the number of media types. For example, it's IMO not necessary to
> have separate media types "application/yang-data" and

yes -- the URIs will be different.

I don't see what problem it solves to change it now.
Either we think RESTCONF-specific media types help or they should all be
removed.

To take this argument to the logical conclusion, we could dump all the
media types
and just use application/xml and application/json.

In fact, we support that in our server, proving the RESTCONF media types
can be ignored
because there is enough context in the request to deduce the schema.

I think the media types add some strong typing.
Not sure if that is important or not.

Andy

Section 3.6 says

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Jun 2016 15:08:09 +0200

> On 21 Jun 2016, at 14:56, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> Either we think RESTCONF-specific media types help or they should all be removed.
> 
> To take this argument to the logical conclusion, we could dump all the media types
> In fact, we support that in our server, proving the RESTCONF media types can be ignored

The granularity of media types seems to be a controversial topic (feed Google with "media type explosion").
I'd suggest to consult APPS folks.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 07:31:03 -0700

Hi,

I forgot to CC: the media-types list ask Benoit asked.
Maybe they have an opinion on removing the operation media type.

Andy

---

From: [email-masked] (Dale R. Worley)
Date: Tue, 21 Jun 2016 10:45:19 -0400

> Parsing and resolving the RequestURI should be sufficient to tell them

That does seem to be the natural way to me -- We are trying to
distinguish what the request is going to do, and the resource it is
going to do it to.  That's the role of the URI and the method.  The
media type should be to tell how the body is "encoded" (in the
highest-level semantic sense).

One test we can apply is this:  Can you imagine the media type being
used in requests used in entirely unrelated applications?  Most of these
media types seem to be intended to be used in only one or a few methods
applied to only one URI of a Restconf instance.  And if we define
further things Restconf can do, there will be many more media types
added.

It seems to me the natural media types are just "XML" and "JSON".

Dale

---

From: [email-masked] (Dale R. Worley)
Date: Tue, 21 Jun 2016 10:47:09 -0400

> So the media type is needed to distinguish actions from data.

It seems to me this is an argument for changing the URI so it tells that
the request is an action, and *what* the action is (since the methods
is fixed as POST).

Dale

---

From: Sean Leonard <[email-masked]>
Date: Tue, 21 Jun 2016 09:40:23 -0700

On 6/21/2016, people said:

    application/yang-api+xml
[etc.]

[etc.]

> Not sure if that is important or not.

Hello, I wanted to take the opportunity to address the Netconf list...

First of all: The way that 2) is proposed has a typo. It should be 
application/yang-data+xml and application/yang-data+json, not -xml and 
-json. + is for "structured syntax suffixes", which applies here.

Officially (from the media-types perspective) either 1) or 2) could 
work, depending on various engineering and administrative 
considerations. In this case, those considerations are about module 
routing, protocol advertisement, and effort to register new types.

You could, theoretically, dump all media types and use application/xml 
and application/json. But that invites a performance impact in that all 
data received as /xml or /json has to be routed through a generic XML or 
JSON processor that has to content-sniff for the data type inside the 
payload. XML and JSON are generic data formats; they apply as a 
structured syntax to innumerable content types that semantically have 
nothing to do with each other. That is one reason why + structured 
syntax suffixes were created. There ought to be a performance increase 
by moving the data routing to a different part of the parsing chain, 
discriminating based solely on the media type (a single case-insensitive 
string comparison operation).

Registering additional media types means you have to fill out (and have 
approved) a registration template for each new media type. Therefore, 
there is additional administrative overhead. Sometimes it's a formality 
and sometimes it invites nitpicking. As there are more than 4 types, 
this weighs against including specific schema information inside the 
media type name.

Whether you put the schema in application/yang-NAME or in a parameter 
application/yang-data; module=NAME (or whatever), it's clear that the 
module name is duplicated inside the payload. This invites conflict. 
What if the response is labeled application/yang-operation+json but the 
content is "ietf-restconf:errors"? What is the proper behavior?

Having reviewed the RESTCONF drafts closely, my engineering opinion is 
in favor of 2), making the type parameter OPTIONAL (unless it can be 
shown that the payload is not sufficient to discriminate the YANG data 
type).

The registration of application/yang-data is well-justified because it 
is instance data while application/yang is (apparently) schema data.

Once you know it's "YANG Data", you know that a YANG processor has to 
grok the data anyway; a robust YANG processor ought to be able to handle 
all of the proposed types (or conversely, be aware of its own 
limitations). By making the YANG type/sub-type optional, you're saying 
that it's a hint. At the same time, your workflow can do some 
preliminary routing based on the parameter without needing to invoke XML 
or JSON parsing, so there is operational value in defining the 
parameter. Compare with application/cms, RFC 7193 (and predecessor, 
application/pkcs7-mime).

In some formats, a type parameter might be REQUIRED. For example, I 
understand that CBOR is being considered as a data format. If you decide 
not to put the discriminator in the CBOR payload, then the type 
parameter would be REQUIRED for +cbor, rather than OPTIONAL.

Regards,

Sean

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Jun 2016 14:20:15 -0700

On Tue, Jun 21, 2016 at 9:40 AM, Sean Leonard <[email-masked]> wrote:

> Officially (from the media-types perspective) either 1) or 2) could work,
> depending on various engineering and administrative considerations. In this
> case, those considerations are about module routing, protocol
> advertisement, and effort to register new types.
>
> You could, theoretically, dump all media types and use application/xml and
> application/json. But that invites a performance impact in that all data
> received as /xml or /json has to be routed through a generic XML or JSON
> processor that has to content-sniff for the data type inside the payload.
> XML and JSON are generic data formats; they apply as a structured syntax to
> innumerable content types that semantically have nothing to do with each
> other. That is one reason why + structured syntax suffixes were created.
> There ought to be a performance increase by moving the data routing to a
> different part of the parsing chain, discriminating based solely on the
> media type (a single case-insensitive string comparison operation).
>
> approved) a registration template for each new media type. Therefore, there
> is additional administrative overhead. Sometimes it's a formality and
> sometimes it invites nitpicking. As there are more than 4 types, this
> weighs against including specific schema information inside the media type
> name.
>
> module name is duplicated inside the payload. This invites conflict. What
> if the response is labeled application/yang-operation+json but the content
> is "ietf-restconf:errors"? What is the proper behavior?
>
> Having reviewed the RESTCONF drafts closely, my engineering opinion is in
> favor of 2), making the type parameter OPTIONAL (unless it can be shown
> that the payload is not sufficient to discriminate the YANG data type).
>
> The registration of application/yang-data is well-justified because it is
> instance data while application/yang is (apparently) schema data.
>
> Once you know it's "YANG Data", you know that a YANG processor has to grok
> the data anyway; a robust YANG processor ought to be able to handle all of
> the proposed types (or conversely, be aware of its own limitations). By
> making the YANG type/sub-type optional, you're saying that it's a hint. At
> the same time, your workflow can do some preliminary routing based on the
> parameter without needing to invoke XML or JSON parsing, so there is
> operational value in defining the parameter. Compare with application/cms,
> RFC 7193 (and predecessor, application/pkcs7-mime).
>
> not to put the discriminator in the CBOR payload, then the type parameter
> would be REQUIRED for +cbor, rather than OPTIONAL.
>
>

I just had a meeting with our developer who wrote our RESTCONF server code
that processes the HTTP request.  Only the suffix is relevant in the
Content-Type.
There are no cases where the RESTCONF media type is needed because the
request URI and/or message-body is ambiguous.  It is only used to return an
error if it is wrong.

Option 3:

   Register 2 media types:

   No parameters are needed

The +xml is not equivalent to application/xml. It is usually "anydata", not
"anyxml".
Unless the YANG node really is from an anyxml-stmt.
(Valid application/yang-data+xml is also valid XML)

The +json is not equivalent to application/json.
It requires draft-ietf-netmod-yang-json and draft-ietf-netmod-yang-metadata
syntax.
(Valid application/yang-data+json is also valid JSON)

Regards,

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Jun 2016 13:29:27 +0200

>        application/yang-data+json

Whilst I proposed a reduction of media types, this seems too much to
me. For one, "application/yang-patch" is IMO needed because (presumably)
other patch format may be used with the PATCH method, and without
getting the proper media type the receiving application can only try to
guess the format.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Jun 2016 07:00:40 -0700

On Wed, Jun 22, 2016 at 4:29 AM, Ladislav Lhotka <[email-masked]> wrote:

> work,
> and
> there
> What
> in
> is
> grok
> parameter
I agree application/yang-patch is needed.

  Option 3:

      Register 4 media types:

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Jun 2016 17:02:45 +0200

> On 22 Jun 2016, at 16:00, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

OK, this looks good to me.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Jun 2016 18:25:33 -0700

> OK, this looks good to me.

Almost...

Remember that we created the hack called "restconf-media-type" because you
complained
that we could not just declare a grouping.  The goal here is to use YANG to
define message content,
so that it is easy to write but also precise and encoding-independent.
This saves a lot of cut-and-paste
every time a new serialization format is created for YANG. (e.g., CBOR).

So now the deleted media types do not have YANG templates.

I propose changing the "restconf-media-type" extension so it is called
"restconf-data".

It will just have a unique name as the argument, not a media-type

OLD:

  rc:restconf-media-type "application/yang-errors" {
    uses errors;
  }

  rc:restconf-media-type "application/yang-api" {
    uses restconf;
  }

NEW:

  rc:restconf-data "yang-errors" {
    uses errors;
  }

  rc:restconf-data "yang-api" {
    uses restconf;
  }

(Similar change in YANG Patch)

Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 23 Jun 2016 13:15:39 +0200

> every time a new serialization format is created for YANG. (e.g.,

I agree it is convenient to be able to use YANG as an
encoding-independent schema language, but I'd argue this hack (or
similar) should be specified elsewhere because it has nothing to do with
RESTCONF the protocol - even if RESTCONF spec uses it for specifying
some schemas.

> I propose changing the "restconf-media-type" extension so it is called

Due to the above reason, I think the name of such an extension shouldn't
contain "restconf". After all, it could be used for other purposes, too.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 23 Jun 2016 19:10:27 -0700

On Thu, Jun 23, 2016 at 4:15 AM, Ladislav Lhotka <[email-masked]> wrote:

> .....

We were not really chartered to solve the general problem of using YANG to
define encoding-independent data abstractions.  This is how the IETF turns
a 1 year project into 3+ years.

> contain "restconf". After all, it could be used for other purposes, too.

How about changing the extension name to "yang-data"?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 24 Jun 2016 09:34:25 +0200

> On 24 Jun 2016, at 04:10, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

Yes, I know, but I suspect a casual reader of the RESTCONF spec may be wondering what the extension is about and whether data of that media type is to be expected on the wire.  

> How about changing the extension name to "yang-data"?

Sounds good.

Lada