Subject: [apps-discuss] JSON Schema considered harmful

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 13:30:33 -0400

One of the biggest mistakes that was made in the XML world was to allow one
group of authors to develop a specification and call it 'XML Schema'. I
think we are at risk of making the same mistake here.

The more I hear about this effort, the less I like it. The approach seems
to be needlessly complex for a start. There should be no need to have a
pointer structure in a modern data modelling language, none. C# and Java
don't use pointers, so why are they needed in JSON Schema?

I think people should be allowed to go off and develop a spec but they
should not be allowed to grab a false sense of authority through an
inappropriate name grab. That had bad results in the XML world and I can't
see this effort working out any better.

Using a schema to develop a protocol is a very good idea but each and every
schema language I have had to use has been botched. XML schema has two
separate type systems. ASN.1 schema is arcane. We must not let that happen
to JSON.

It should be possible to have a schema language that is essentially
language neutral and encoding neutral. Pretty much every programming
language in current use has the same set of intrinsic types (bytes, int16,
int32, int64, strings (unicode), chars (unicode), real32, real64, boolean),
and at least one form of enumerated collection (arrays, lists, sets). For
protocol design it is very useful to add in standard representations for
Binary and DateTime intrinsict types as base64 encoded strings are easier
to deal with than arrays of decimal bytes and we already have an IETF
string representation for time.

It should be possible to use a schema to map a data structure that can be
represented conveniently in any sensible modern programming language (i.e.
anything from Perl to Java, C# but not necessarily FORTRAN, COBOL or the
like) to at least a subset of any sensible encoding (JSON, XML, ASN.1)

Where the XML Schema effort went wrong was they tried to support every
feature of DTDs and they gave the schema designer the ability to map a data
structure onto multiple different encodings. Which is pretty weird. Making
standards is the process of making design choices that don't matter.
Providing multiple ways to serialize the same data structure is a harmful
option.

At any rate I suggest that either

1) the authors of JSON Schema start using a different name or
2) Everyone else with ideas for a schema for JSON busilly pollute the name
space by introducing their own plans named JSON Schema until the authors
back down.

Since there is no consensus that JSON needs a schema, I can't see how there
can be a consensus for one particular proposal.

In fact I don't think JSON does need a schema either. We need a schema for
describing protocol data structures that can be reified as JSON or other
data formats.

On Wed, Sep 19, 2012 at 1:05 PM, Francis Galiegue <[email-masked]>wrote:

> On Wed, Sep 19, 2012 at 6:59 PM, Phillip Hallam-Baker <[email-masked]>
> wrote:
> [...]
> >
> >
> > Just where is JSON Pointer intended to be used? Who is using it?
> >
>
> JSON Schema, for instance, even though only implicitly for now. The
> next version will mandate JSON Reference support and, as such, JSON
> Pointer (which is the fragment part of a JSON Reference).
>
> >
> > Pretty much every modern programming language has adopted a notation that
> > uses dots for member extraction and square braces for array indices. I
> don't
> > see a particularly good reason to invent another way to extract
> information
> > from a data structure.
> >
> > In C# and Java the notation would be x.y[1].
> >
>
> Again, programming languages do not enter the picture. What is wanted
> is an unambiguous way to address any part of a JSON value.
>
> Oh, and "." is a valid object member name. And so is "/" for that
> matter. And so is "", and so is "\x00".
>
> [...]
> >
> > I can't see why I would want to have a different syntax to extract
> elements
> > after a data object has been JSON encoded. What is the value of such a
> > standard meant to be?
> >
>
> Language-proof, future-proof, covers all corner cases. If these are
> not arguments, I don't know what is.
>
> Oh, and the fact that it can be used in URI fragments.
>
> > This looks like an attempt to reinvent a wheel that has already been
> tried
> > and tested.
> >
>
> Tried and tested _by a few subset of programming languages_.
>
> --
> Francis Galiegue, [email-masked]
> JSON Schema: https://github.com/json-schema
> "It seems obvious [...] that at least some 'business intelligence'
> tools invest so much intelligence on the business side that they have
> nothing left for generating SQL queries" (Stéphane Faroult, in "The
> Art of SQL", ISBN 0-596-00894-5)
>

-- 
Website: http://hallambaker.com/

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 19 Sep 2012 19:48:52 +0200

On Wed, Sep 19, 2012 at 7:30 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> The more I hear about this effort, the less I like it. The approach seems to
> be needlessly complex for a start. There should be no need to have a pointer
> structure in a modern data modelling language, none. C# and Java don't use
> pointers, so why are they needed in JSON Schema?
>

OK, sorry, but that is a _very limited view of JSON Schema_.

JSON Schema was there _far_before the GitHub group came into the
picture, and FYI, JSON Reference and JSON Pointer came into light
_before the GitHub group took over_. Go read both the JSON Reference
and JSON Pointer I-Ds: do you see my name in them as an author? NO.
Only as a side note for JSON Reference because I highlighted the
dangers of a loop.

And please get a grip: before you say the approach is "needlessly
complex", have a look at what is being done. It is _precisely_ because
JSON Schema needs input that no drafts are submitted at the moment.
They are still in the process of being written.

And JSON Schema, even in its latest incarnation, is used: consider the
Google discovery API.

> separate type systems. ASN.1 schema is arcane. We must not let that happen

Go read the proposed specs. Do you see ASN.1 anywhere?

Link: https://github.com/json-schema/json-schema

> It should be possible to have a schema language that is essentially language
> neutral and encoding neutral.

Yes, and that is _precisely_ what the GitHub group is aiming for. And
you contradict yourself ohsomuch by saying that dot should be
preferred to solidus, for one.

> Pretty much every programming language in
> current use has the same set of intrinsic types (bytes, int16, int32, int64,
> strings (unicode), chars (unicode), real32, real64, boolean), and at least
> one form of enumerated collection (arrays, lists, sets). For protocol design
> it is very useful to add in standard representations for Binary and DateTime
> intrinsict types as base64 encoded strings are easier to deal with than
> arrays of decimal bytes and we already have an IETF string representation
> for time.
>

You obvioulsy haven't read neither the past nor the present.

[...]
> Providing multiple ways to serialize the same data structure is a harmful

Oh dear. You really HAVE NOT read anything of the work being done by
the GitHub organization, have you?

Schema, Pointer. Those are just names. Please go carefully read the
current productions at said organization. I certainly DO NOT want to
replicate XSD. XSD and LSD have only one letter apart, and I, and the
GitHub organization, DO NOT want to go there.

Sorry for being so blunt, but I cannot stand this kind of "feedback"
from people who DO NOT EVEN TRY AND SEE WHAT IS BEING DONE.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 13:59:12 -0400

You keep raising JSON Schema here referring to it as if it was a done deal.
In fact you just told me that JSON Pointer is needed by JSON Schema which
is why I said, whooa, if you need a pointer spec for a schema language then
you are doing it wrong.

My observation about XML Schema and ASN.1 schema is that the groups
producing them should not have been allowed a monopoly. The ASN.1 case is
actually understandable as it was a schema design group that developed the
encodings. But even so it was design by a committee.

As for 'Github', isn't that a place where people start up communities? So
calling something the Github community makes it sound as if you have the
endorsement of GitHub which I am pretty sure you don't. In fact that is
precisely what my complaint was about. I don't think you should be
aggrandizing yourself by claiming the endorsement of the JSON community or
the GitHub community.

The only organization that should be accrediting a JSON Schema or JSON
Pointer is the standards body for JSON which seems to now be IETF since
ECMA seem to be citing the IETF RFC as normative.

Until there is an IETF working group set up to develop one you should call
it something else.

On Wed, Sep 19, 2012 at 1:48 PM, Francis Galiegue <[email-masked]>wrote:

> language

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 19 Sep 2012 20:09:17 +0200

On Wed, Sep 19, 2012 at 7:59 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> In fact you just told me that JSON Pointer is needed by JSON Schema which is
> why I said, whooa, if you need a pointer spec for a schema language then you
> are doing it wrong.
>

What I say is that an unambiguous way of accessing ANY path in a JSON
value, whatever that value is, is needed, and JSON Pointer is that.
Try as hard as you want, I fail to see how you can do any better. I
didn't even design that. Out of the light of pure reason, JSON Pointer
_makes sense_.

[...]

> Pointer is the standards body for JSON which seems to now be IETF since ECMA
> seem to be citing the IETF RFC as normative.
>

Read about the history of JSON Schema. Please.

I have NEVER, EVER, pretended to have endorsement from ANYONE. The
GitHub JSON Schema group is an ad-hoc organization aimed at reviving
JSON Schema, and that is all there is to it. And the principles behind
this ad-hoc organization are: collect existing uses (and there are);
propose solutions (and there are); reach a consensus (in process); AND
THEN submit to the IETF.

Did I ever claim being backed by GitHub? No. Did I ever claim being
backed by the IETF (what a foolish notion to begin with)? No. Do I
claim to be working to get JSON Schema into something to be considered
for an I-D (note, an I-D)? Definitely, yes. On behalf of all people
involved with this organization, which is an ad-ho group of
VOLUNTEERS.

But, damnit, I have known the subject for long enough to KNOW, just
KNOW, that addressing JSON today, in an unambiguous way, involves two
things: JSON Reference, and JSON Pointer. However "unlikeable" JSON
Pointer may look to people who are too used to JavaScript. JSON is NOT
Javascript.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 15:14:01 -0400

Excuse me but you just did refer to the 'Github JSON Schema group'

And yes, I have read the purported JSON Schema spec. It is far too complex
with far too many options.

Having written protocols using schemas, I can tell you for example that you
don't need maxOccurs or minOccurs, the only options that make sense 95% of
the time are one or none [0..], exactly one [1..1], at least one
[1..MAXINT] or any [0...MAXINT].

The spec replicates XML Schema in JSON. I don't think that has any value.
If people want to have that type of validation and see value in it then
they are going to use XML Schema. The reason some of us are interested in a
redo is that we want something that is simple like JSON is simple.

On Wed, Sep 19, 2012 at 2:09 PM, Francis Galiegue <[email-masked]>wrote:

> ECMA

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 19 Sep 2012 22:49:57 +0200

On Wed, Sep 19, 2012 at 9:14 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> the time are one or none [0..], exactly one [1..1], at least one [1..MAXINT]
> or any [0...MAXINT].
>

There is no maxOccurs, no minOccurs. Where on earth did you see that?

No, you decidedly DID NOT make even an ATTEMPT to read the proposed
specifications. It's in the README.md on the main page, damnit!

> The spec replicates XML Schema in JSON.

Yet another proof of my statement above.

Read what is proposed before making any comments, and if you have any
_constructive_ criticism, I'd be glad to hear it.

Not before.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 17:28:38 -0400

> There is no maxOccurs, no minOccurs. Where on earth did you see that?

http://tools.ietf.org/html/draft-zyp-json-schema-03

    5.13 <http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.13>.
minItems . . . . . . . . . . . . . . . . . . . . . . . . . 11
<http://tools.ietf.org/html/draft-zyp-json-schema-03#page-11>
     5.14 <http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.14>.
maxItems . . . . . . . . . . . . . . . . . . . . . . . . . 11
<http://tools.ietf.org/html/draft-zyp-json-schema-03#page-11>

5.13 <http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.13>.
 minItems

   This attribute defines the minimum number of values in an array when
   the array is the instance value.
5.14 <http://tools.ietf.org/html/draft-zyp-json-schema-03#section-5.14>.
 maxItems

   This attribute defines the maximum number of values in an array when
   the array is the instance value.

That would seem to be the same as the XML Schema constraints only someone
changed the name.

> specifications. It's in the README.md on the main page, damnit!

I read the Internet draft. If you are refering to a different spec then we
have an even bigger problem.

> _constructive_ criticism, I'd be glad to hear it.

I think that you folk are giving a good idea a bad name here.

You are producing the ADA of schema languages.

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 19 Sep 2012 23:36:11 +0200

On Wed, Sep 19, 2012 at 11:28 PM, Phillip Hallam-Baker <[email-masked]> wrote:
[...]
>     5.13. minItems . . . . . . . . . . . . . . . . . . . . . . . . . 11
>      5.14. maxItems . . . . . . . . . . . . . . . . . . . . . . . . . 11
>
[...]
> That would seem to be the same as the XML Schema constraints only someone

This is the old specification. It is being redone.

And you don't understand JSON at all, do you? Can you tell a JSON
array from a JSON object? From what I read: no...

> have an even bigger problem.

YOU have a problem. You don't know how to read.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 17:49:10 -0400

> This is the old specification. It is being redone.

So when you were attacking me for not reading the specification you were
referring to a version that has not yet been submitted as a draft?

You really are a piece of work.

> array from a JSON object? From what I read: no...

I think that we are reaching the point where I suggest that you no longer
be heard. You keep making these personal attacks. In fact I think they have
somewhat marred your whole approach from the start.

You really don't have standing to lecture other people on what they do or
do not understand.

I didn't say I was referring to objects when I mentioned the maxItems
thing. I used the XML Schema terminology but that hardly changes my case
that the feature is unnecessary in XML Schema and almost certainly
unnecessary here.

Before you dig yourself in deeper I suggest you find out who you are
directing your attacks at and decide whether my being unable to understand
you would be a fault of mine, a defect in your presentation or a confusion
in the case you present.

> YOU have a problem. You don't know how to read.

I don't know how to read documents that have not been submitted.

Argument from undisclosed specification is not exactly convincing.

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 19 Sep 2012 23:49:31 +0200

> array from a JSON object? From what I read: no...

Hint: this is a proposed text for the next version of the specification:

"JSON Schema does not mandate that an instance be of a particular
type: JSON Schema can process any JSON value, including null"

maxOccurs and minOccurs are _meaningless_ to JSON.

---

From: Nico Williams <[email-masked]>
Date: Wed, 19 Sep 2012 16:56:50 -0500

> producing them should not have been allowed a monopoly. The ASN.1 case is

Huh?  What's the complaint here?  That you weren't there?  Or in the
committee?  Or that a committee (or WG, or whatever) was involved?
Why bother standardizing anything if there are to be no
committees/WGs?  How would we do that?  Would there be SDOs that only
rubber-stamp standards proposals brought to them but which can make no
changes to those proposals, just yay/nay?

> encodings. But even so it was design by a committee.

wat?   First you complain about someone having a monopoly, but then
you say that as long as it's the same people that developed some other
related standard it's OK?  But then... you're asking for a monopoly
and thus contradicting yourself.

If you have comments on proposals, or as to how to improve existing
standards, then state them.  If you want a different process to apply
to standardization of JSON Schema, well, say so, but this is the IETF,
and the IETF process will apply at the IETF.  If you want to change
the IETF process then make proposals to that effect.

Nico
--

---

From: Francis Galiegue <[email-masked]>
Date: Thu, 20 Sep 2012 00:04:25 +0200

> You really are a piece of work.

So are you, sir, not to understand that minItems and maxItems relate
to the _number of elements of an array instance_. Had you read the
proposed specifications, you'd see that they say exactly that. To the
letter.

And you'd have remembered, also, that I _explicitly said_ that they
were not submitted yet because feedback was needed.

> I think that we are reaching the point where I suggest that you no longer be
> heard.

I think that we are reaching a point where you should go and read RFC 4627.

> You keep making these personal attacks. In fact I think they have
> You really don't have standing to lecture other people on what they do or do
> not understand.
>

Go see who is on the member board of the GitHub organization. I just
happen to be the one who does most of the work out there.

And standing or not, who cares? You keep going on with senseless
arguments, who is not to be heard? I want to get work done _right_,
and this is _precisely_ for this reason that specs are not submitted
yet.

[...]
> you would be a fault of mine, a defect in your presentation or a confusion

I know the answer to that already.

---

From: Nico Williams <[email-masked]>
Date: Wed, 19 Sep 2012 17:11:43 -0500

> were not submitted yet because feedback was needed.

Well, we submit Internet-Drafts because feedback is needed :)  It's
quite OK to submit a badly incomplete/unfinished I-D; indeed, I'd say
that that's the only way to get started and make progress towards a
Standards-Track RFC.

Nico
--

---

From: James M Snell <[email-masked]>
Date: Wed, 19 Sep 2012 15:17:34 -0700

+1 ... as much as I would hate to jump into the middle of this pissing
match, let's not forget that IDs are not finished products. If there have
been incremental updates to the spec since it was last published, please
post the update so it can be properly reviewed.... even if its not yet done.
On Sep 19, 2012 3:11 PM, "Nico Williams" <[email-masked]> wrote:

> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 19:06:55 -0400

IDs are not finished products but that does not entitle Mr insults-a-lot to
attack me for not having read a version of the ID that has not been
submitted yet. I made comments on the version of the ID they submitted. It
is overly complex and solves a problem that in my experience does not exist.

In response I was attacked as an imbecile and accused of not reading their
spec. Now it might just be me but I think that being a board member of
Github does not entitle people to go round calling people imbeciles if they
are not capable of mind reading.

Now I did overlook the statement in the introduction to RFC 4627 stating
that objects can appear in any order. But that is not where I would expect
to find a normative statement and I don't think many others would. I don't
think that excuses Mr Galiegue's behavior.

A schema is a useful tool for writing specifications and as an input to
coding tools. The spec in question seems to be designed to support schema
validation and nothing else. Which is really weird since schema validation
has not been found to be particularly useful in the XML world. The
constraints that can be described in a schema tend to be a subset of the
constraints that are needed by applications. So applications tend to end up
having to repeat the schema validation.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 19 Sep 2012 19:21:45 -0400

> type: JSON Schema can process any JSON value, including null"

I have no clue as to what you intend to mean with that statement.

> maxOccurs and minOccurs are _meaningless_ to JSON.

They serve the same function as minItems and maxItems.

XML encoding just has a different method of encoding arrays. If you have an
array of 2 foo objects you would have to specify them as something like
<array><foo>1</foo><foo>2</foo> </array>

If you have something like

class fred {
   list <foo> foos;
   list <bar> bars;
   }

This would serialize as

<fred> <foo/> <foo/>... <bar/> <bar/>... </fred>

We have established that the JSON equivalent SHOULD be (and I think it
should be a MUST)

"fred" {
     "foos" [ {} {} {} ]
     "bars" [ {} {} ... ] }

The clunkiness of the XML approach is the reason that XML is not a
particularly good choice for encoding data structures.

---

From: "Tschofenig, Hannes (NSN - FI/Espoo)" <[email-masked]>
Date: Thu, 20 Sep 2012 09:26:57 +0300

My 5 cents to this topic. 

Even though many folks, including Tim Bray, educated the community that
it is not necessary to produce XML schemas when writing specifications
that use XML, everyone did. They felt that the specification was not
complete without the schema.

The consequence was that 

*        The XML replaced text in specifications leaving important
semantic unspecified. Look at many of the OASIS document, for example
CAP, and you know what I am talking about. The authors somehow thought
that the schema is a replacement for a description of what the protocol
actually does. Wrong! Interoperability problems are the consequence.

*        Sometimes the XML schema was provided in addition to a textual
description. Nice idea but then they often get out of sync. 

*        Extensibility thoughts were left behind. XML by itself meant
extensible, at least that's what many thought, and hence there was no
story about how the protocol would get extended in a smooth way and what
the policies are for extensions.

The worst thing was, however, that the entire protocol development was
guided by the (lack of) understanding of the XML schema language rather
than the requirements for the actual protocol. 

Funny enough the schema did not really help anyone: it did not help a
person reading the specification (who does not want to implement it)
since they typically care about an entire level of detail nor the guy
who wants to implements it. Just using the schema for an implementation
was never enough because you had to consider all the other special case
as well (which can typically only be found elsewhere in the
specification). If you managed to encode all the special cases into the
schema then it became unreadable. The instance validation against
concept was also useless once you included the proper extension points. 

In a nutshell, I think that XML (& Relax NG) schemas are harmful.
Unfortunately, it is already too late for fixing that in the IETF. I
believe we should not introduce schemas into JSON in the IETF.
Everything will be fine without using it. 

Ciao

Hannes

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 09:52:49 -0400

Actually I agree with most of what Hannes says here.

I found using a schema helpful in writing the SAML spec but I didn't use
XML schema. I used a tool and generated the XML Schema and the text from my
own schema language. That addressed the problem of the spec and schema
being out of sync which was a real bear as the SAML TC made major changes
to the schema, adding and dropping items at each con-call

What I think we need for the IETF is a tool to help document protocols. Now
that might be a schema done right or it might be a protocol description
tool.

Extensibility is certainly problematic. I have seen pretty much every WG
using XML end up in rat hole after rat hole as people try to consider what
v 1.1 of the spec might look like. XML Schema looks like it should answer
that question but it does not. And not many other tools do so either.

This problem crops up in a JSON protocol as well. Should unknown tags be
ignored or cause an error? Should there be a mechanism that allows
intentional breaking of backwards compatibility in cases where doing so may
cause applications to do the wrong thing?

Even the terminology can be disastrous. There is a feature in X,509v3 that
is designed to allow a certificate to say 'if you don't understand and
process this extension then reject this certificate'. The original
intention was to allow extensions to specify new revocation/status checking
schemes. RPs that did not understand the scheme could not use the
certificate.

Unfortunately that feature was called 'critical' which also means
'important'. And so people started using 'critical' to mean 'I think this
is important' and not 'I think this so important that you should break
backwards compatibility'. This is why I called the same feature
'Conditions' in the SAML specification and disguised it so that it didn't
look like a criticality flag. But Conditions and Criticality are the exact
same thing. The SAML assertion structure began life as part of a design to
re-do X.509 in XML which is why it had to have some equivalent of
criticality.

What we need in my view is a way to identify the places where additional
items can be added into protocol messages and a default that additional
items are prohibited everywhere else. So in my Simple JSON Schema I would
represent a SAML assertion like thing something like:

Type Assertion
    Description "Assertion container yada yada"
    DateTime Issued
    DateTime NotBefore
    DateTime NotOnOrAfter
    List Any Statements
    List Any Conditions
    List Any Advice

[I have left out the descriptions of the member elements but everything is
documented, that is the point in fact.]

The Any type is an intrinsic type that refers to an object that is tagged
with the object type. So an authentication assertion would be tagged with a
"authentication" an authorization assertion with "authorization" and so on.

So the JSON structures compatible with the above signature would be

{ "Assertion" {
    "Issued": "RFC3339Time",
    "NotBefore":  "RFC3339Time",
    "NotOnOrAfter":  "RFC3339Time",
    "Statements" : [
        {"Authentication": {whatever}},
        {"NewAssertion" : {<any valid JSON can go here>}}],
    "Conditions" : [ { "Audience" : {<any valid JSON can go here>}} ],
    "Advice" : [ {"Proof" : {<any valid JSON can go here>}} ]
    }

The explicit extension model here is that any new elements of any type can
be added in to the Statement, Conditions and Advice lists, just tag each
member with its type.

Depending on the circumstance the Assertion object itself might be open or
closed, that is it might be permissible to add in additional members or
receivers might be required to reject them.

Note that my tool does not use JSON syntax to describe the schema. This is
intentional since the primary design goal is to make editing easy. I find
XML to be a tiresome format to edit by hand and JSON looks rather worse. I
am not that fond of C style either, the semicolons are unnecessary and so
are the braces if you use indentation properly. Removing all the clutter
makes the spec much easier to read.

A protocol design schema can be very much simpler than XML schema as it
does not have to support the ability to describe every possible JSON or XML
encoding. It only needs to be sufficiently descriptive to specify the
messages we want to use.

So it is not necessary to consider corner cases like non alphanumeric tags
or tags with spaces. They are legal JSON tags but they are not going to be
accepted as protocol tags by an IETF WG. It is not necessary to specify
every possible encoding of time as we have decided we are only going to use
RFC3339.

We can even build into the compiler additional nits restrictions such as
'don't allow the tags to have the same name as the reserved words in C#,
C++, Java or Python. "for" is a perfectly legal JSON tag but trying to use
"for" as a tag in a protocol message would be an act of stupidity, malice
or both.

On Thu, Sep 20, 2012 at 2:26 AM, Tschofenig, Hannes (NSN - FI/Espoo) <
[email-masked]> wrote:

> My 5 cents to this topic. ****
>
> ** **
>
> Even though many folks, including Tim Bray, educated the community that it
> is not necessary to produce XML schemas when writing specifications that
> use XML, everyone did. They felt that the specification was not complete
> without the schema.****
>
> ** **
>
> The consequence was that ****
>
> **·        **The XML replaced text in specifications leaving important
> semantic unspecified. Look at many of the OASIS document, for example CAP,
> and you know what I am talking about. The authors somehow thought that the
> schema is a replacement for a description of what the protocol actually
> does. Wrong! Interoperability problems are the consequence.****
>
> **·        **Sometimes the XML schema was provided in addition to a
> textual description. Nice idea but then they often get out of sync. ****
>
> **·        **Extensibility thoughts were left behind. XML by itself meant
> extensible, at least that’s what many thought, and hence there was no story
> about how the protocol would get extended in a smooth way and what the
> policies are for extensions.****
>
> ** **
>
> than the requirements for the actual protocol. ****
>
> ** **
>
> person reading the specification (who does not want to implement it) since
> they typically care about an entire level of detail nor the guy who wants
> to implements it. Just using the schema for an implementation was never
> enough because you had to consider all the other special case as well
> (which can typically only be found elsewhere in the specification). If you
> managed to encode all the special cases into the schema then it became
> unreadable. The instance validation against concept was also useless once
> you included the proper extension points. ****
>
> ** **
>
> believe we should not introduce schemas into JSON in the IETF. Everything
> will be fine without using it. ****
>
> ** **
>
> Ciao****
>
> Hannes****
>
> ** **
>

---

From: Nico Williams <[email-masked]>
Date: Thu, 20 Sep 2012 11:02:58 -0500

Hannes,

I find the use of ASN.1, XDR, XML Schema, ... to be useful in
providing a desirable degree of formalism for encoding of complex
structured data.  As you say, these schema languages do not describe
semantics (either at all or to any useful degree), but we often handle
this by either a) embedding comments in the ASN.1/whatever, b) text in
the I-D/RFC outside the ASN.1/..., c) both (a) and (b).  I don't think
there's any problem with this approach, frankly.  And I prefer a
higher degree of formalism to a lesser degree of formalism because the
former promotes the use of automated tools (such as code generators
and validators).

So, while I have no problem with specific complaints about
shortcomings of one or another schema language, I do object to the
idea that because they fail to denote semantics we should not use
them.

Regarding specific complaints...  I am not very familiar with XML
Schema, so I can't say much about it and extensibility.  ASN.1 has
extensibility rules nowadays, and often uses of ASN.1 predating
extensibility rules are nonetheless extensibile (sometimes we need to
resort to negotiation of extensions, but that's OK).  I suspect that
the situation w.r.t. XML Schema extensibility is not as dire as you
paint it.

Nico
--

---

From: Erik Wilde <[email-masked]>
Date: Thu, 20 Sep 2012 09:26:27 -0700

hello all.

> idea that because they fail to denote semantics we should not use

that's my general feeling as well. schemas are not able to and not 
designed to completely capture protocol semantics, but they provide a 
good starting point, and allow some level of formalism which then needs 
to be augmented with documentation.

> the situation w.r.t. XML Schema extensibility is not as dire as you

imho, the real challenge is to *design* a good extension model for a 
language; coding it in a schema language then is just an exercise in 
figuring out what can be done in that specific language. in XSD, all you 
can do is to use wildcards (possibly refined by namespace specifications 
for them), and then you have to add documentation saying how processors 
are supposed to handle wildcards (ignore, interpret, stop, ...). 
extension rules cannot be fully captured (such as saying that extensions 
are not allowed to change the semantics of any fields in the base 
format), but at least the schema describes the base syntax, and also 
validates instances containing extensions.

i have to admit that i am really curious why the seasoned spec writers 
that have voiced general concerns about the usefulness of schema 
languages are feeling this way, but so far i have not really understood 
what they are saying. could somebody please try again, i'd really like 
to understand why the general idea of describing protocol syntax in a 
semi-formal (as opposed to non-formal) way is considered a bad idea.

thanks a lot and kind regards,

dret.

-- 
erik wilde | mailto:[email-masked]  -  tel:+1-510-2061079 |
            | UC Berkeley  -  School of Information (ISchool) |
            | http://dret.net/netdret http://twitter.com/dret |

---

From: Nico Williams <[email-masked]>
Date: Thu, 20 Sep 2012 11:29:42 -0500

On Thu, Sep 20, 2012 at 8:52 AM, Phillip Hallam-Baker <[email-masked]> wrote:
> I found using a schema helpful in writing the SAML spec but I didn't use XML
> schema. I used a tool and generated the XML Schema and the text from my own
> schema language. That addressed the problem of the spec and schema being out
> of sync which was a real bear as the SAML TC made major changes to the
> schema, adding and dropping items at each con-call

OK, we're getting to the meat of the complaint.  It's not about how or
who but about failures of the end product (XML Schema, JSON Schema).

> that might be a schema done right or it might be a protocol description

I agree.  I don't know that this is a realistic expectation (but see
below), and in particular I'm skeptical that enough code can be
generated from such a tool (as compared to merely data
encoding/decoding) that it would be anywhere near as useful as data
description languages have been.  But I'd like to be proven wrong!
(Or even to prove myself wrong.)

The ITU-T would say that SDL should be it, no doubt :), but SDL is
weirdly incompatible with ASN.1 (SDL is case-insensitive(!) while
ASN.1 is case-sensitive), and anyways, I don't think I care for SDL.

That said, and as has been demonstrated before, there's a large degree
of duality between the various data description languages that we
have.  Thus we have XER (XML Encoding Rules for ASN.1), and
FastInfoSet (PER encoding applied to XML) and so on.  And I think the
proposition that XDR is a subset of ASN.1/PER with 4-octet alignment
and non-packed encoding of optional/defaulted fields, is defensible.
Therefore I think at least a unified data description language is
possible.

What is difficult to manage is protocol flows, particularly when there
is cryptography involved.  We might be able to use any of various
high-level programming languages (e.g., Haskell, Python, Scheme, ...)
in combination with a) a generic data description language and b) some
functions left undefined (defined in English language prose rather
than a formal language).  (I would prefer to use a LISP or Scheme with
a powerful macro language so as to make it easier to generate code
from the specification by just defining a suitable set of macros.)

Maybe we could experiment with this?

> using XML end up in rat hole after rat hole as people try to consider what v
> 1.1 of the spec might look like. XML Schema looks like it should answer that
> question but it does not. And not many other tools do so either.
>

This shows up in ASN.1 as well.  ASN.1 lets you say that extensions
"go here" and are to be ignored by decoders that don't expect them,
but, so what, it's not enough.  We always end up having to signal or
negotiate the use of extensions, and I don't find that to be
particularly problematic.  Perhaps XML Schema is particularly
disastrous w.r.t. extensibility?  But I doubt it.  Instead I suspect
that some wish that extensibility were simpler and more automatic, but
I'm afraid that it simply cannot be.

> process this extension then reject this certificate'. The original intention
> was to allow extensions to specify new revocation/status checking schemes.
> RPs that did not understand the scheme could not use the certificate.
>
> 'important'. And so people started using 'critical' to mean 'I think this is
> important' and not 'I think this so important that you should break
> backwards compatibility'. This is why I called the same feature 'Conditions'
> in the SAML specification and disguised it so that it didn't look like a
> criticality flag. But Conditions and Criticality are the exact same thing.
> The SAML assertion structure began life as part of a design to re-do X.509
> in XML which is why it had to have some equivalent of criticality.

Criticality is a big deal.  We don't want to render extant
implementations vulnerable when we extend a protocol, but we don't
want to break interop with them either, and sometimes you just cannot
have both of those.  It mostly behooves us to get things like PKI
right from day zero, and yet that also is wishful thinking.  I wish I
had an actual, better answer here; I don't.

> represent a SAML assertion like thing something like:

OK, like ASN.1 extensibility markers then.  (Really, we only keep
re-inventing the wheel :)  Maybe we need JER -- JSON Encoding Rules
for ASN.1 :)

> "authentication" an authorization assertion with "authorization" and so on.

So, like typed holes in ASN.1 (including the Information Object Set, a
very difficult to use syntax extension to ASN.1 that adds very useful
formalism).

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 12:33:16 -0400

Actually you can do a lot more than Wildcards in XML Schema.

And no, it is not a good idea to try. That is why SAML 2.0 became necessary
as people had told me to use model groups in SAML 1.0.

On Thu, Sep 20, 2012 at 12:26 PM, Erik Wilde <[email-masked]> wrote:

> designed to completely capture protocol semantics, but they provide a good
> starting point, and allow some level of formalism which then needs to be
> augmented with documentation.
>
>
> for them), and then you have to add documentation saying how processors are
> supposed to handle wildcards (ignore, interpret, stop, ...). extension
> rules cannot be fully captured (such as saying that extensions are not
> allowed to change the semantics of any fields in the base format), but at
> least the schema describes the base syntax, and also validates instances
> containing extensions.
>
> that have voiced general concerns about the usefulness of schema languages
> are feeling this way, but so far i have not really understood what they are
> saying. could somebody please try again, i'd really like to understand why
> the general idea of describing protocol syntax in a semi-formal (as opposed
> to non-formal) way is considered a bad idea.
>
> ______________________________**_________________
> https://www.ietf.org/mailman/**listinfo/apps-discuss<https://www.ietf.org/mailman/listinfo/apps-discuss>
>

---

From: Erik Wilde <[email-masked]>
Date: Thu, 20 Sep 2012 09:43:52 -0700

hello phillip.

On 2012-09-20 09:33 , Phillip Hallam-Baker wrote:
> And no, it is not a good idea to try. That is why SAML 2.0 became
> necessary as people had told me to use model groups in SAML 1.0.

thanks for this clarification, i think now i understand better where 
you're coming from. instead of just specifying extension points and 
semantics, you also tried to specify models that people could or maybe 
even would have to use when actually designing extensions. i absolutely 
agree that this is not something one should even try to do in a loosely 
coupled setting such as protocols and possible extensions. also, this 
part of XSD is just riddled with weird design choices and random 
limitations of what's possible and what not. but i think the real 
problem there is that you're trying to define a metamodel for extensions 
(and therefore condemning all of them to use XSD as well), instead of 
just trying to clearly and fully specify the basic protocol format.

assuming you stayed away from defining the extension model itself in XSD 
by providing model groups that people that would have to use for 
defining actual extensions, would you still say that using a schema 
language for specifying the general syntax (including extension points 
using simple wildcards) is a bad approach? and if so, why?

dret.

---

From: Tim Bray <[email-masked]>
Date: Thu, 20 Sep 2012 09:53:15 -0700

> i have to admit that i am really curious why the seasoned spec writers
that
> have voiced general concerns about the usefulness of schema languages are
> feeling this way, but so far i have not really understood what they are

I gave a preso at IETF70 in Vancouver a few years ago which is advertised
as being XML-centric, but has (I think) useful things to say about
plain-text/binary/ASN.1/JSON/XML, and discusses about the proper place of
schemas: http://www.ietf.org/proceedings/70/slides/xmltut-0.pdf

I believe:
- ASN.1 has been bypassed by history and has truly horrible tooling. Just
don’t go there.
- If you’re interchanging documents, use XML; if (much more common) you’re
interchanging data structures, use JSON.
- Do not use multiple syntaxes for the same protocol. Pick one of XML or
JSON and stick with it.
- The most important thing in defining a protocol is good clean
comprehensible human-readable prose.
- All the protocols I’ve ever seen have important constraints that can’t be
expressed usefully in declarative schema languages.
- Therefore, the next most important thing is an automated
validator/test-suite.
- A schema is a nice-to-have.
- If you’re in XML and want schema-ware, use RelaxNG. For an example, see
RFC4287.
- If your protocol is JSON, investment in a validator/test-suite has much
higher ROI than chasing schema unicorns.

---

From: Bob Wyman <[email-masked]>
Date: Thu, 20 Sep 2012 13:08:39 -0400

> - ASN.1 has been bypassed by history and has truly horrible tooling. Just
Perhaps what we need is "Abstract Syntax Notation V2.0 (i.e. ASN.2)" which
would do what ASN.1 was trying to do but benefit from all that we've
learned since the mid-80's...

> - All the protocols I’ve ever seen have important constraints that can’t
> be expressed usefully in declarative schema languages.

---

From: James M Snell <[email-masked]>
Date: Thu, 20 Sep 2012 10:09:44 -0700

> higher ROI than chasing schema unicorns.
+1 on every point.

I would venture one step further in that a consistent RelaxNG-esque model
for describing the data model for JSON structures **in spec documents**
would be helpful as a tool for spec writers to use when describing a
vocabulary. Something that would play the same fundamental role as ABNF,
but that is absolutely as far as I would go with it.

example,

ABNF:
  value := TOKEN

Foo = object {
  bar = string value,
  baz = array int
}

- James

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 20 Sep 2012 13:34:19 -0400

Can we engrave these rules on a big rock somewhere?

Tim Bray <[email-masked]> wrote:

>------------------------------------------------------------------------
>

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 13:42:57 -0400

On Thu, Sep 20, 2012 at 12:43 PM, Erik Wilde <[email-masked]> wrote:

> semantics, you also tried to specify models that people could or maybe even
> would have to use when actually designing extensions. i absolutely agree
> that this is not something one should even try to do in a loosely coupled
> setting such as protocols and possible extensions. also, this part of XSD
> is just riddled with weird design choices and random limitations of what's
> possible and what not. but i think the real problem there is that you're
> trying to define a metamodel for extensions (and therefore condemning all
> of them to use XSD as well), instead of just trying to clearly and fully
> specify the basic protocol format.
>
> by providing model groups that people that would have to use for defining
> actual extensions, would you still say that using a schema language for
> specifying the general syntax (including extension points using simple
> wildcards) is a bad approach? and if so, why?

I think a simple schema is a good thing and a complex one is a bad thing. A
schema does need to be a little more complex than data structure
declarations in C# or Java or any other reasonable typed language as it has
to cope with things like extensions and you want the documentation to be
part of the type definitions. Another big issue is that modern programming
languages have pointers (they may not be exposed in C# and Java but they
are there). So if you have a data structure that indexes the same list of
objects in two different ways you have to consider things like pointer
swizzling or object identifiers.

The differences are small and manageable but they do exist.

A schema does not need to be a validation tool. I have yet to find an
instance where one is useful because the fact that a message has valid
syntax is no guarantee of it having valid semantics. I always check the two
at the same time. So 1000000000000000000000000000000000000000 is a valid
JSON integer but it probably isn't going to be a valid array index.

The other big slam on XML Schema nobody has mentioned yet is that prefixes
are a stupid idea. Schemas should be composable without the stiches showing
like Frankenstien's monster.

If you write a specification that uses XML Encryption then your data
structure is going to have at least three prefixes in use, the prefix for
its own tags, the prefix for the XML Encryption tags and the XML Signature
spec that is called by reference.

But But! If I am writing a STANDARD then I am not going to choose element
tags that overlap anyway. Only a moron is going to write a protocol with a
protocol element that is the same as the XML Signature / Encryption
elements. And in any case XML Schema actually makes it possible to vary the
type of elements according to context.

So given that I am going to write my protocol so that it is not going to be
ambiguous, why can't I simply have a schema that says #include "XML Schema"
and then incorporate all those elements and types as native to my spec. Why
am I forced to introduce a new namespace?

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 13:47:11 -0400

> - ASN.1 has been bypassed by history and has truly horrible tooling. Just

Agree

> interchanging data structures, use JSON.

Agree

> - Do not use multiple syntaxes for the same protocol. Pick one of XML or

Agree but note that this conflicts with the REST notion of using URL
encoded requests and JSON responses. Which I think is yucky but some insist
on.

> comprehensible human-readable prose.

Agree [hence I do not want to see a JSON or an XML syntax]

> be expressed usefully in declarative schema languages.

Agree and furthermore I have found a good use for the constraints they do
provide.

> - Therefore, the next most important thing is an automated

Agree and want to lock that to the schema and the specification (which I
have done in ProtoGen)

> higher ROI than chasing schema unicorns.

Unless you have a tool that will produce the test-suite and the spec (and
other schemas if you really must).

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 13:53:39 -0400

I could try to upload my Goedel and ProtoGen toolset to Github.

this is the specification I am currently working on.

http://tools.ietf.org/html/draft-hallambaker-omnibroker-02

ProtoGen is a tool that is designed to help write specifications that also
emits code that may be used in either a validator or a test suite or
examples or production code.

You would want to use different back ends for each case though.

Now I did one thing which is a bit of a cheat and that is that I pushed the
integrity checking information into the HTTP wrapper rather than try to use
something like a JSON Signature approach. So that means that I do not have
to worry about how to define an integrity checking constraint at the
protocol level, I assume that to be done for me in the HTTP framing.

But looking how simple this approach turns out and how complex the
alternatives are, I think it is the right way forward.

On Thu, Sep 20, 2012 at 1:34 PM, [email-masked] <[email-masked]> wrote:

>> - If you’re interchanging documents, use XML; if (much more common)
>> you’re interchanging data structures, use JSON.
>> ------------------------------
>>

---

From: Julian Reschke <[email-masked]>
Date: Thu, 20 Sep 2012 19:54:37 +0200

On 2012-09-20 19:47, Phillip Hallam-Baker wrote:
> ...
> ...
>     - Do not use multiple syntaxes for the same protocol. Pick one of
>     XML or JSON and stick with it.
>
>
> encoded requests and JSON responses. Which I think is yucky but some
> insist on.
> ...

Where does that notion come from? I don't see what this has to with 
being restful or not (there, I said it; I'll probably regret it :-).

Best regards, Julian

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 14:38:03 -0400

On Thu, Sep 20, 2012 at 1:54 PM, Julian Reschke <[email-masked]>wrote:

> Where does that notion come from? I don't see what this has to with being
> restful or not (there, I said it; I'll probably regret it :-).
>

I asked Roy Fielding that and he gave me much the same answer.

But I am going to have to implement that to use my compiler to do OAUTH and
specs already using that style.

https://tools.ietf.org/html/draft-ietf-oauth-v2-30

As a practical matter, what I like about the schema approach is that it can
help channel people into making the sensible choice or at least a
consistent one. If the default switches on the compiler are for JSON
requests and responses then people have to make a case for messing with
them.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 20 Sep 2012 20:45:56 +0200

On 2012-09-20 20:38, Phillip Hallam-Baker wrote:
>
>
> On Thu, Sep 20, 2012 at 1:54 PM, Julian Reschke <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> But I am going to have to implement that to use my compiler to do OAUTH
> and specs already using that style.

You may or may not enjoy reading 
<https://www.w3.org/Bugs/Public/show_bug.cgi?id=17725>.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 20 Sep 2012 13:05:11 -0700

On 20/09/2012, at 9:53 AM, Tim Bray <[email-masked]> wrote:

> I believe:
> - ASN.1 has been bypassed by history and has truly horrible tooling. Just don’t go there.
> - If you’re interchanging documents, use XML; if (much more common) you’re interchanging data structures, use JSON.
> - Do not use multiple syntaxes for the same protocol. Pick one of XML or JSON and stick with it.
> - The most important thing in defining a protocol is good clean comprehensible human-readable prose.
> - All the protocols I’ve ever seen have important constraints that can’t be expressed usefully in declarative schema languages.
> - Therefore, the next most important thing is an automated validator/test-suite.
> - If you’re in XML and want schema-ware, use RelaxNG. For an example, see RFC4287.
> - If your protocol is JSON, investment in a validator/test-suite has much higher ROI than chasing schema unicorns.

+1; somebody please give this a BCP#.

--
Mark Nottingham
http://www.mnot.net/

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Sep 2012 13:12:36 -0700

> +1; somebody please give this a BCP#.

In its current form, perhaps not.

On the other hand, something along this lines could provide exremely 
useful pedagogy in doing specifications work.

It would, of course, need rather more verbose discussion/justification 
of its guidance.

That is, something that says what the characteristics are of the better 
formal notation choices are and what the characteristics of the less 
better ones are.  Then describes the better choices and their enticing 
features, perhaps in terms of tradeoffs.

Then meta-points, such as "choose exactly one".  Etc.

d/

-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Tim Bray <[email-masked]>
Date: Thu, 20 Sep 2012 13:20:27 -0700

Frankly that notion had never crossed my mind, but is there a case for a
“Tao of network protocols” or an “Informational BCP” or some such?  I could
put some cycles into it.  -T

On Thu, Sep 20, 2012 at 1:12 PM, Dave Crocker <[email-masked]> wrote:

>>> - If your protocol is JSON, investment in a validator/test-suite has
>>> much higher ROI than chasing schema unicorns.
>>>
>>
> It would, of course, need rather more verbose discussion/justification of
> its guidance.
>
> formal notation choices are and what the characteristics of the less better
> ones are.  Then describes the better choices and their enticing features,
> perhaps in terms of tradeoffs.
>

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Sep 2012 13:24:54 -0700

On 9/20/2012 1:20 PM, Tim Bray wrote:> Frankly that notion had never 
crossed my mind, but is there a case for a
 > “Tao of network protocols” or an “Informational BCP” or some such?  I
 > could put some cycles into it.  -T

That's a book, not a BCP.

I was suggesting a small-but-useful subset, which I thought was nicely 
exemplified by the terse list of practical direction of the 
precipitating note about choosing a notation language.

Very narrow topic.  Very important.  Often very religious and very 
confused.

Just the stuff of a BCP, IMO...

d/

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 20 Sep 2012 16:35:27 -0400

My plan was to give people a tool that had the BCP encoded into its DNA...

On Thu, Sep 20, 2012 at 4:24 PM, Dave Crocker <[email-masked]> wrote:

> exemplified by the terse list of practical direction of the precipitating
> note about choosing a notation language.
>

---

From: SM <[email-masked]>
Date: Thu, 20 Sep 2012 13:51:05 -0700

Hi Tim,

[Cc to IAB.  Please drop it if the thread is willfully wild.]

At 09:53 20-09-2012, Tim Bray wrote:
>I gave a preso at IETF70 in Vancouver a few years ago which is 
>advertised as being XML-centric, but has (I think) useful things to 
>say about plain-text/binary/ASN.1/JSON/XML, and discusses about the 
>proper place of schemas: http://www.ietf.org/proceedings/70/slides/xmltut-0.pdf

I found the presentation informative.

>I believe:
>- ASN.1 has been bypassed by history and has truly horrible tooling. 
>Just don't go there.
>- If you're interchanging documents, use XML; if (much more common) 
>you're interchanging data structures, use JSON.
>- All the protocols I've ever seen have important constraints that 
>can't be expressed usefully in declarative schema languages.
>- If you're in XML and want schema-ware, use RelaxNG. For an 
>example, see RFC4287.

It might help to have a discussion of the above as part of an IAB document.

Regards,
-sm

---

From: Francis Galiegue <[email-masked]>
Date: Thu, 20 Sep 2012 22:54:44 +0200

On Thu, Sep 20, 2012 at 10:24 PM, Dave Crocker <[email-masked]> wrote:
[...]
> Very narrow topic.  Very important.  Often very religious and very confused.
>

I-D 3 of JSON Schema has it: a "small-but-useful subset", which can be
used to describe _any_ JSON value. It has sufficient intrinsic value
that it is being used, for instance, by the Google discovery API.

However, there was room for improvement. In particular, the fact that
the core schema keywords were not fully defined, and also section 5.27
which has been a constant source of misinterpretation. And the fact
that some "untold", but important stuff, was lurking behind. And this
is precisely the goal that this ad-hoc organization "of no value" is
pursuing, and why it takes time. But XSD is at best a goal NOT to
pursue.

And yes, all keywords are meant to define the _structure_ of JSON data
anyway. Semantic validation keywords exist, but the goal is to make
them optional.

In any case, I-Ds will be submitted via the official mechanism. Some
refining is needed before that.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 21 Sep 2012 17:44:49 +0900

> put some cycles into it.  -T

What about doing it in the style of http://tools.ietf.org/html/rfc3470, 
or maybe even as an update of tha document?

Regards,    Martin.

> On Thu, Sep 20, 2012 at 1:12 PM, Dave Crocker<[email-masked]>  wrote:
>
>>
>>
>>> On 20/09/2012, at 9:53 AM, Tim Bray<[email-masked]>  wrote:
>>>
>>>   I believe:
>>>> Just don’t go there.

---

From: Hannes Tschofenig <[email-masked]>
Date: Fri, 21 Sep 2012 12:18:36 +0300

On Sep 20, 2012, at 7:26 PM, Erik Wilde wrote:

> i have to admit that i am really curious why the seasoned spec writers that have voiced general concerns about the usefulness of schema languages are feeling this way, but so far i have not really understood what they are saying. could somebody please try again, i'd really like to understand why the general idea of describing protocol syntax in a semi-formal (as opposed to non-formal) way is considered a bad idea.

I don't think that arguing are against the usage of formal or semi-formal ways to describe protocols. Pseudo code may be a useful way to describe an algorithm, for example. The issue is just that they have to be applied selectively and carefully. There has to be a real benefit. 

In the context of this discussion I had been arguing that XML schemas (and Relax NG schemas) had not been useful for the target audience compared to the benefits they claimed to provide. For that reason I argued that we should discourage people from using them since they cause problems for extensibility, specification quality, and readability. 

I like the tool idea that PhB and Tim had brought up. I also think that examples are the most valuable tool for implementers. 

Ciao
Hannes

---

From: Andrew Newton <[email-masked]>
Date: Fri, 21 Sep 2012 09:00:51 -0400

> In the context of this discussion I had been arguing that XML schemas (and Relax NG schemas) had not been useful for the target audience compared to the benefits they claimed to provide. For that r…

As far as extensibility goes, I think that's an issue more with
Namespaces in XML than with the schema languages. However, there could
be improvement. From a spec quality and readability point of view, I
think this is more true of XML Schema than RelaxNG.

Regarding utility for a target audience, from what I have experienced
the problem is that tools vendors convince programmers that as long as
they drop their XSDs or WSDL in spot X, magic will happen and they
don't have to worry about all that nasty protocol crap (including
talking to the network engineers about firewalls)... that is until
they drop an XML document in a string and the whole thing blows up.
But the issue isn't with XML Schema itself, it is with the
magic-happens-here and computers-will-learn grand architecture in
which XML Schema was drafted.

> I like the tool idea that PhB and Tim had brought up. I also think that examples are the most valuable tool for implementers.

+1 on the examples.

Perhaps Phillip could get some time in Atlanta in Apps or maybe get a
room for a BoF to demonstrate his idea. Though tools, including
diagramming languages and schema languages, for writing better
specifications seems like a topic for more than just Apps.

-andy

---

From: Dave Crocker <[email-masked]>
Date: Fri, 21 Sep 2012 07:48:21 -0700

> or maybe even as an update of tha document?

This might be overly elaborate, but I actually had something a level 
above that in mind.

Docs like RFC 3470 are excellent for the /details/ of a particular 
package.  We ought to have one of those for each language in common use 
for IETF docs.

However the one I was suggesting was an /umbrella/ over this class of 
documents.

It would talk about the common issues across such languages (or whatever 
they should be called) and the issues that might help choose among them.

What was triggering my suggestion was that Tim's bullet list was exactly 
the type of semantics I mean:  basic rules that are common across our 
use of xml, json, abnf, whatever.

d/