Subject: [apps-discuss] Announce: JSON Schema, I-D version 4

From: Francis Galiegue <[email-masked]>
Date: Fri, 1 Feb 2013 16:59:45 +0100

Hello,

JSON Schema is now up to version 4.

Links to the specifications:

* http://tools.ietf.org/html/draft-zyp-json-schema-04 (core specification),
* http://tools.ietf.org/html/draft-fge-json-schema-validation-00
(validation specification),
* http://tools.ietf.org/html/draft-luff-json-hyper-schema-00
(hyperschema specification).

It is dependent on two other I-Ds:

* JSON Reference,
* JSON Pointer (as a consequence of the first).

Reviews appreciated. For the recall, JSON Schema has its Google group:
[email-masked]

All the best,
--
Francis Galiegue, [email-masked]
Try out your JSON Schemas: http://json-schema-validator.herokuapp.com

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 1 Feb 2013 08:44:06 -0800

This is a major reorganization of the long-dead -03 draft. Can you say a bit about what you foresee for this document?

I'm also interested in hearing what others on the list feel about a JSON Schema. I had a recent discussion with someone who said that this document was the best of the schema proposals, and it was still not useful. I know "are schema useful" is a religious discussion, but I think it might be useful now for this document.

--Paul Hoffman

---

From: Francis Galiegue <[email-masked]>
Date: Fri, 1 Feb 2013 18:09:56 +0100

> This is a major reorganization of the long-dead -03 draft. Can you say a bit about what you foresee for this document?

The major goals were as follows:

* split the specification into core, validation, hyper schema -- since
draft v3 started to be huge;
* be much more explicit about some keywords which were confusing
people (the first of them being "extends", which is removed and
"replaced" with "allOf");
* add explanations gathered with time on the Google group; be more precise;
* use existing specifications (I believe JSON Reference originated
from JSON Schema at first -- it only made sense to refer to it),
update references to others.

I am primarily a validation person, and my initial goal was to clarify
this part, along with the core mechanics. The hyper schema aspects are
not really my center of interest, but I know this is one "vector"
bringing more and more people on the group. If this is the aspect you
are mainly interested about, David Geraint (the author of the
hyperschema draft, Cc:ed) will be a much more suited interlocutor.

But my initial goal was really to make the core and validation aspects "solid".

Regards,

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 01 Feb 2013 09:23:10 -0800

Hi Paul,

--On February 1, 2013 8:44:06 AM -0800 Paul Hoffman <[email-masked]> 
wrote:

> I'm also interested in hearing what others on the list feel about a JSON
> Schema. I had a recent discussion with someone who said that this
> document was the best of the schema proposals, and it was still not
> useful. I know "are schema useful" is a religious discussion, but I think
> it might be useful now for this document.

I have used 
<https://datatracker.ietf.org/doc/draft-newton-json-content-rules/> for 
describing JSON documents in several drafts and have found that sufficient 
for my needs.

-- 
Cyrus Daboo

---

From: Francis Galiegue <[email-masked]>
Date: Fri, 1 Feb 2013 18:52:32 +0100

Hello,

On Fri, Feb 1, 2013 at 6:23 PM, Cyrus Daboo <[email-masked]> wrote:
[...]
> describing JSON documents in several drafts and have found that sufficient

I didn't know about this specification but imho it has two flaws:

* it fails to describe itself;
* it requires a dedicated parser, since it is not written in JSON.

Those are, to me, two very strong points of JSON Schema.

Regards,

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 01 Feb 2013 10:03:58 -0800

Hi Francis,

> Those are, to me, two very strong points of JSON Schema.

My primary goal is to have something to describe a JSON document format in 
a specification in a clear and understandable manner to any implementor. 
Being able to fully validate a JSON document is a lesser requirement. i.e. 
I am after something more akin to ABNF or a XML DTD, as opposed to 
full-blown XML schema.

-- 
Cyrus Daboo

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 1 Feb 2013 13:27:09 -0500

On Fri, Feb 1, 2013 at 1:03 PM, Cyrus Daboo <[email-masked]> wrote:

> Hi Francis,
>
>
> --On February 1, 2013 6:52:32 PM +0100 Francis Galiegue <
> [email-masked]> wrote:
>

+1

I find 90% of the features in XMLSchema to be completely irrelevant to
protocol design. They are designed to support document markup, not
protocols.

I do not know of any widely used language where people write class or
structure definitions with constraints on the number of elements etc. that
look anything like XML Schema. The only useful values for minOccur are 0
and 1. The only valid values for maxOccur are 1 and infinity.

The constraints that are useful for validating input are invariably more
complex than XML schema can support. The interesting constraints are
constraints that cut across multiple entries in a single structure (X > 3
is not useful, X>Y might be).

-- 
Website: http://hallambaker.com/

---

From: Francis Galiegue <[email-masked]>
Date: Fri, 1 Feb 2013 19:32:05 +0100

On Fri, Feb 1, 2013 at 7:27 PM, Phillip Hallam-Baker <[email-masked]> wrote:
[...]
> protocol design. They are designed to support document markup, not

You know, I hate XML Schema with a passion. And one reason why I
immediately adopted JSON Schema for my needs is that I didn't see
anything remotely as bloated in JSON Schema. It does a very good job
at describing the structure of JSON data, whatever it may be. And it
is relatively easy to grasp.

This is certainly related to the fact that JSON itself is much easier
to grasp than XML for mere mortals.

---

From: Francis Galiegue <[email-masked]>
Date: Fri, 1 Feb 2013 19:40:12 +0100

On Fri, Feb 1, 2013 at 7:32 PM, Francis Galiegue <[email-masked]> wrote:
[snip]
>
> [...] It does a very good job

I might as well include "real life" schemas. Recently, someone on the
group asked whether a schema for GeoJSON (http://geojson.org) existed.

The answer was no. 30 minutes after, it was yes. These schemas only
lack functional checks which cannot be expressed using JSON Schema or,
for that matter, anything other than actual programming:

https://github.com/fge/json-schema-validator-demo/tree/master/src/main/resources/geojson

---

From: Tim Bray <[email-masked]>
Date: Fri, 1 Feb 2013 11:03:35 -0800

Well, wading into the religion...

In the XML space, schemas have probably on balance been actively
harmful, for two reasons:

- XML Schema (XSD) is a turd: Fails to do all sorts of useful stuff
while still being hard to understand and  use
- The existence of schemas tend to distract people into investing a
lot of effort into writing them, which usually has poor ROI, for two
reasons: First, every Internet protocol imposes constraints that
typically cannot be expressed in a declarative schema. And second,
clear, understandable English-language prose is much more important,
and any time invested in things other than getting that right is
questionable.

Having said that, RFC4287 (Atom) has a non-normative schema, which is
in RelaxNG, an XML schema thingie that is much better than XSD; and
people have found that useful.

So... if the JSON schema tool were lightweight and easy to understand
and covered the important use cases, it might not be actively harmful.
But if I were a WG chair doing a JSON-based protocol, I’d be perfectly
happy to ship a schema-less RFC, and I’d probably be fairly harsh on
bikesheddy discussions of schema minutiae.  -T

On Fri, Feb 1, 2013 at 8:44 AM, Paul Hoffman <[email-masked]> wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Dave Cridland <[email-masked]>
Date: Sat, 2 Feb 2013 10:06:41 +0000

I managed to fumble the addressing on this one. I'll blame the tablet.
---------- Forwarded message ----------
From: "Dave Cridland" <[email-masked]>
Date: 2 Feb 2013 10:05
Subject: Re: [apps-discuss] Announce: JSON Schema, I-D version 4
To: "Tim Bray" <[email-masked]>
Cc: "Paul Hoffman" <[email-masked]>

The XSF has a history of non normative agendas in its XEP series, these
tend to be largely used as illustrations of particular limits, such as
whether a publish can have more than one item or not.

As such, I have them about as useful as ABNF. That is, they can imply
things which are not true, and fail to express things which are essential,
but if the reader is aware of the limitations, they still offer some value.

---

From: Andrew Newton <[email-masked]>
Date: Sat, 2 Feb 2013 17:59:29 -0500

> * it fails to describe itself;

Francis,

Can you elaborate more on what you mean by this and why you believe it to
be important?

-andy

---

From: Francis Galiegue <[email-masked]>
Date: Sun, 3 Feb 2013 00:17:28 +0100

Hello,

On Sat, Feb 2, 2013 at 11:59 PM, Andrew Newton <[email-masked]> wrote:
> Can you elaborate more on what you mean by this and why you believe it to be
> important?
>
> -andy

I mean that you can use JSON Schema (the defined meta schemas to be
precise) to validate any other schema, including the meta schema
itself, using the same rules as the ones used to validate instances;
and if you choose to extend it with your keywords, you can easily grab
a copy of the meta schema, modify it, and still use the same rules.

This means you can easily check the validity of schemas not under your
direct control, and also detect whether you can actually use such
schemas (for instance, a schema written for draft v3 will not validate
against v4 unless it uses keywords which have not changed definitions
inbetween). And as you can use the same code for both syntax checking
and instance validation, that leaves less room for errors.

Regards,

---

From: Francis Galiegue <[email-masked]>
Date: Sun, 3 Feb 2013 11:36:27 +0100

On Sun, Feb 3, 2013 at 12:17 AM, Francis Galiegue <[email-masked]> wrote:
[...]
> and instance validation, that leaves less room for errors.

And also, it can easily describe other JSON-based formats. For
instance, it took me 10 minutes to write this:

https://github.com/fge/json-schema-validator/blob/master/src/main/resources/json-patch.json

If you receive a patch, you can use JSON Schema to filter out bad
patches before even attempting to patch.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 5 Feb 2013 11:54:22 +1100

FWIW, since people are talking about it:

I'm about +0.25 on doing this.

On the one hand, XML Schema has created a lot of damage (the more so because many many people still don't understand how harmful it is / can be).

On the other hand, this is not XML Schema. AFAICT it does not try to do nearly as much as XML Schema did. As has been discussed ad nauseum, the major use cases for a schema language are doing QA and documentation, and JSON could use help on both fronts.

So, if it were possible to get enough review / input to assure that it didn't create any of the same problems (in particular, I want to look at how it supports / fails to support format evolution), it'd probably do some amount of good without too much risk. However, we'd have to do that without killing the community around it (i.e., the architects among us would need to restrain themselves).

Just my .02.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Francis Galiegue <[email-masked]>
Date: Tue, 5 Feb 2013 22:22:31 +0100

Hello Mark,

> On the other hand, this is not XML Schema. AFAICT it does not try to do nearly as much as XML Schema did. As has been discussed ad nauseum, the major use cases for a schema language are doing QA an…

I am not sure what you mean by QA here. Anyway, this may be of
interest. It is a schema fully describing JSON Patch. Note that the
"json-pointer" format attribute may be replaced by a regular
expression:

Which means it is possible to map existing JSON representations using
JSON Schema. Answering requests on the list, I also wrote schemas for
GeoJSON and SMD. Links on demand. I am about to write a JSON Schema
for JSON-RPC-2.0 as well. I can write schemas for nearly anything,
within the limitations of JSON Schema. If you want me to write a
schema for a defined format, I'll happily do it -- within the
limitations of what JSON Schema can do.

One thing you will notice about the link above is "notes": it is not
known to JSON Schema and, as such, it is allowed. The specification
explicitly allows for this. The specification asks that keywords
unknown by an implementation be ignored by it -- while not explicitly
rejecting their use between parties if they agree on it.

At least, that was the intent. I thrived to make the wording as
precise as possible. And I would certainly appreciate reviews: the
earlier they come, the earlier we (I and others) can address comments.

Thanks,