Subject: [apps-discuss] The "array vs object" issue in JSON Pointer

From: Barry Leiba <[email-masked]>
Date: Wed, 16 Jan 2013 13:18:00 -0500

As everyone who's been reading the JSON threads knows, there's been
controversy about one aspect of JSON pointer: specifically, that while
JSON itself makes a distinction between arrays and objects, the JSON
Pointer syntax does not.  Users of the pointers have to know what
they're pointing to (or have to be in an application situation where
it doesn't matter).

It's clear that the discussion has been had, that the working group
has rough consensus on the document as it stands, that what we have
meets the current use cases and represents running code, and that
those pushing for a change to this, to make a syntactic distinction
between a pointer to an array element and one to an object member, are
in the rough.

That said, I find the arguments for the change sufficiently compelling
to ask folks to have one more think on it.  I do NOT want to make this
a free-for-all where we have the whole argument all over again.  We
all know what's been said.  Carsten's note about its coming down to
how strong you want the typing to be is a useful one.  So I want to
ask the question a particular way.

Consider these points:

1. Failing to make the distinction now pretty much locks us into it
for good.  What we have might meet the use cases we have now, but are
we really sure we're never going to regret that, that next week or
next month or next year we might not have a new use case that tips the
balance?

2. There's running code, but *if* we're going to make a change, now's
the time, and the running code can be changed.  If, as we say, the
current uses of Pointer know whether they're talking about arrays or
objects, then the code can be changed to use the appropriate syntax
for the appropriate type.  How hard is it, *really*, to change this?

3. On the other side, the heartburn is exacerbated by the fact that
JSON Patch applies the same operations differently for arrays and
objects.  That's an issue with Patch, not with Pointer, but it's hard
not to blend them and say that distinguishing the syntax for Pointer
would at least partially meliorate the issue in Patch.

Given those considerations, the question is this: Does the working
group stand by its rough consensus for the JSON Pointer syntax as it
stands?  Or might it be better to make a syntactic distinction between
arrays and objects now, while we can, to be more robust for the
future?  (Or, on the other hand, for those who have three hands, would
it actually *harm* the protocol to change this?)

Again, repeating:
Of course the rough consensus in appsawg is what wins, and if
everyone's *really sure* that we want to go ahead with things as they
are, then we will.  The IESG has already given its approval for that.
I'd just like people to take a moment to be certain about that before
we move on.

Barry, trying to be responsible as AD

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 16 Jan 2013 20:05:13 +0100

>it actually *harm* the protocol to change this?)

I think there is no reason to force users of JSON Pointer to encode the
distinction between arrays and objects in pointers; it's a feature that
they do not have to do so. But it would be bad if people who want such a
distinction came up with a competing "A/O-sensitive JSON Pointer" speci-
fication where users have to be careful not to use the wrong variant all
the time. I would have liked more discussion of making this an optional
feature.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: James M Snell <[email-masked]>
Date: Wed, 16 Jan 2013 11:32:31 -0800

I believe the current decision is sufficient as is. For the various use
cases that have been presented in favor of making the change, alternative
solutions have been put on the table (e.g. my predicates draft). For other
cases (fully concurrent editing) there is not yet enough implementation
experience to justify making changes (that is, it's not at all clear that
making this change alone would be enough to fully address such cases).

- James

On Wed, Jan 16, 2013 at 10:18 AM, Barry Leiba <[email-masked]>wrote:

> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Nico Williams <[email-masked]>
Date: Wed, 16 Jan 2013 13:34:32 -0600

IMO it's a feature that path components of JSON Pointer don't
distinguish between arrays and dicts.

I can see how it might be useful to do so for lazy schema validation
purposes, even so I'm not sure I care.  It might help to have a very
specific example of what the new syntax might be.

As it happens I was coding JSON Pointer two nights ago, so thinking
about incompatibilities resulting from changes to JSON Pointer now...
I'd have to say that the right way to have done this would have been
to reserve more ~ escapes for future use, and the right way to do this
going forward is to call new syntaxes JSON PointerN and have separate
functions/methods for using one or another syntax.  Given that we can
always do this later, then, what's the compelling rationale for making
a backwards-incompatible change this late?

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 16 Jan 2013 14:33:42 -0600

Also, going back over discussions I missed I'd have to say that I
don't think the lack of array/dict distinction in Pointer sytax is a
problem for JSON Patch either, nor do I think it's a good idea to
forbid leading zeros in otherwise all-digits path components (except
that they must be treated as strings, not numbers, and then they apply
only to dicts).

Nico
--

---

From: Francis Galiegue <[email-masked]>
Date: Wed, 16 Jan 2013 23:18:22 +0100

> they're pointing to (or have to be in an application situation where

There are only two things that an array/object specific syntax can
bring to JSON Pointer:

* easiness for some language (which one? I wonder) to interpret them
and complexity for other languages,
* impossibility to address some otherwise valid members of a JSON Object.

JSON Pointer as it is currently defined guarantees that any part of
any existing JSON text can be reliably addressed by any language
without even encoding/decoding issues. You cannot really ask for much
more than this.

--
Francis Galiegue, [email-masked]
"It seems obvious [...] that at least some 'business intelligence'
tools invest so much intelligence on the business side that they have
nothing left for generating SQL queries" (Stéphane Faroult, in "The
Art of SQL", ISBN 0-596-00894-5)

---

From: Matthew Morley <[email-masked]>
Date: Wed, 16 Jan 2013 23:12:56 -0500

> next month or next year we might not have a new use case that tips the

I find the reverse is be more true. The downside to the syntax change,
would be that pointers which might otherwise have been resolved would now
fail. Even though the only reasonable action would been to attempt to map
the provided key against the provided structure.

Part of this is that pointer itself is a string, so not unlike a numerical
value in middle of a URI, it is already a string and it does not simply map
to a specific structural construct intuitively. Otherwise we would have
seen a syntax like below strongly promoted by the community as the correct
approach?

http://example.com/user:0/name

> for the appropriate type.  How hard is it, *really*, to change this?

Running code should not be a consideration either way on this issue. There
is no time like the present to make the right decision for the right
reasons. :)

> would at least partially meliorate the issue in Patch.

JSON Patch needs to address this issue within its own specification. It is
not clear to me at this point, that anything but optimistic patching is
truly a requirement. Being more specific via layering is often the best
approach, while baking that specificity into the pointer string offers
little actual advantage.

-- 
Matt (MPCM)

---

From: Robert Sayre <[email-masked]>
Date: Thu, 17 Jan 2013 00:31:12 -0800

On Wed, Jan 16, 2013 at 10:18 AM, Barry Leiba <[email-masked]> wrote:
>
> It's clear that the discussion has been had

Agree.

> Consider these points:

All good points, which deserve to be addressed directly and
substantively. According to the metrics in [1], each of the three
should be addressed at level 6.

For me, this means that there better be very concrete reasons for the
type system of JSON Pointer and JSON Patch to diverge from the rules
specified by JSON itself. RFC4627 is short, and section 2 of that RFC
is easy to understand.

> Barry, trying to be responsible as AD

Succeeding. However, we will not be "stuck with" the decision of this
WG. It's actually quite easy to ignore.

- Rob

[1] http://www.paulgraham.com/disagree.html

---

From: Nico Williams <[email-masked]>
Date: Thu, 17 Jan 2013 22:39:01 -0600

> next month or next year we might not have a new use case that tips the

For some purposes this is true, and for some it isn't.  In any case,
this is only a problem if it is a problem to not indicate the type of
a container at any point in the Pointer path.  Let's set #1 aside --
it is not the fundamental issue here.

> for the appropriate type.  How hard is it, *really*, to change this?

If there's a serious bug then "but there's running code" is not a
problem.  But if there isn't then #2 becomes a serious impediment to
making changes to JSON Pointer now.  Changing running code is hard, so
we need strong justification for doing so.

> would at least partially meliorate the issue in Patch.

Many, many programming languages provide generic syntax for indexing
several types of containers where the syntax indicates nothing about
the container type.

This is not new.  I don't think it's a problem here.  I also don't
think it's a problem in any one programming language (so maybe it
prevents type inference, but so what, if I want a language with strong
typing and type inferencing then I'll use one that provides that).

Note that not one of the above issues is serious.  Not one of them
points to something that breaks because JSON Pointer has no way to
indicate the _expected_ type of containers in path.

And by the way, I think there are more serious issues with JSON
Pointer.  Quick, what's ~0, and what's ~1?  Is ~0 / and ~1 ~, or is it
the other way around?!  This is extremely annoying.  It's a wart, but
not a fundamental problem, so let's let it slide.

But maybe we can think of new ~ escapes that can be used to indicate
expected container types.  A specific proposal for how to add type
expectation would help those arguing that we must address that now.
So I'd propose ~A at the beginning of a component means "this had
better be an array or I'll have a kitten" and ~O means "this better be
an object or I'll eat a kitten".

> it actually *harm* the protocol to change this?)

Looking back over this thread the answer to the first question is "yes".

But if someone were to make a specific proposal then maybe the answer
would be different.

Nico
--

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 18 Jan 2013 17:11:43 +0900

Hello Barry, others,

I have thought about this a bit. I'm totally fine with the current state 
of things. The reason for this is that JavaScript itself also allows me 
to be totally ignorant of whether I'm using arrays or objects if I use 
indexing with [] and string values inside.

So given that we would know by now about this being a serious problem in 
JavaScript if it were one, but that I have seen a lot of different 
problems discussed for JavaScript, but not this one, I think we can 
safely ignore it.

This is different e.g. from child elements vs. attributes in XPointer, 
because there can be a child element (actually more than one) and an 
attribute of the same name.

Regards,    Martin.

---

From: Barry Leiba <[email-masked]>
Date: Mon, 21 Jan 2013 17:37:57 -0500

> it actually *harm* the protocol to change this?)

The answer to my set of questions is clear, given the responses I've
seen: the working group is not interested in thinking about this
further and no one has changed her mind.  The consensus stands.  I've
asked Mark to post a final draft update that fixes a minor wording
thing, and I'm going to push the "approval" button.

I personally would have preferred a different outcome, but the working
group has the decision and has made it.  And I thank everyone for
taking the time to re-iterate that clearly.

Barry, Applications AD

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 22 Jan 2013 08:40:48 +0100

On Jan 18, 2013, at 09:11, Martin J. Dürst <[email-masked]> wrote:

> The reason for this is that JavaScript itself also allows me

Right outcome, wrong reasoning.
What is good in a programming language may or may not be good in a protocol.

Grüße, Carsten

PS.: Yes, thinking in analogies can sometimes help elicit good arguments.
But these arguments still need to be validated in the target environment.

I think what caused most people in the WG to want to stick with the current version is that nobody has demonstrated a good enough reason for wanting this feature in the protocol.

(Sorry for being off-topic by contributing to reopening the discussion.
I just think there is something that can be learned from it.)

---

From: Nico Williams <[email-masked]>
Date: Tue, 22 Jan 2013 02:36:35 -0600

> What is good in a programming language may or may not be good in a protocol.

The protocol is built on a subset of *the* language from which JSON is
derived.  (And it may well be implemented using that language in some
cases.)

That there's a protocol involved doesn't change the fundamental
property of the language on which it is modeled: that type errors are
run-time errors.  Indeed, adding type hints to JSON Pointer wouldn't
even change that property: protocol errors are also run-time errors in
this case.

Martin's point was a good one.

Nico
--

---

From: Robert Sayre <[email-masked]>
Date: Tue, 22 Jan 2013 00:45:44 -0800

On Tue, Jan 22, 2013 at 12:36 AM, Nico Williams <[email-masked]> wrote:
> run-time errors.

This is just wrong. Type errors don't occur in JavaScript in the case
at hand (you can set named properties on arrays). Furthermore, the
"leading zero" issue in the pointer syntax shows that JSON pointer
can't accurately point at things.

> Indeed, adding type hints to JSON Pointer wouldn't

Not really. It seems obvious that a pointer syntax that differentiated
between objects and arrays could be simplified by an implementation.
Don't need to differentiate between object types? Don't do it.

This email is not an objection.

- Rob

---

From: Nico Williams <[email-masked]>
Date: Tue, 22 Jan 2013 02:59:28 -0600

> can't accurately point at things.

You're right, I was thinking more of Python since I use that much more
often than JS (but I should change that).  This error of mine doesn't
change my view though.  It's harmless to not have type information in
Pointer (it might not be useless, but it is harmless).

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 22 Jan 2013 10:52:20 -0500

> Don't need to differentiate between object types? Don't do it.

I think the example that makes this very clear is C++ vs C#.

C# is pretty much the most technically rigorous modern language in general
use out there right now. It has more ways to enforce static type checking
than anything else that I know of that can actually be used to write real
programs.

In C+ you ended up with this type of syntax:

instance::pointer->structure.member.submember->referent

In C# you have just:

instance.pointer.structure.member.submember.referent

The compiler knows which is which and there is no possibility of ambiguity
arising. So why have the programmer tell the compiler what it should
already know?

The C# approach is vastly more convenient to use and allows programs to be
refactored easily without affecting the code that calls them.

Note my use of the past tense with respect to C++. I have never used it and
never intend to. In my view C is a vast improvement on C++. The only C++
feature I see as an advantage is the addition of single line comments. The
only case where I would use C++ is as one option for the target language of
a code generator.

Static type checking is good when it distinguishes between cases where a
programmer might otherwise make an error. I don't like using languages like
python or JavaScript because I prefer having integer arithmetic clearly
separated from real and have the compiler warn me if something odd is about
to happen.

But this isn't making that type of check. It is makework type checking of
the type that make Pascal a completely wirthless enterprise (or rather it
would have been better if it had been wirthless).

Ditto

-- 
Website: http://hallambaker.com/