Subject: [apps-discuss] Early Initial Draft: JSON-Test (was Re: JSON Patch: extensibility)

From: James M Snell <[email-masked]>
Date: Mon, 17 Sep 2012 12:39:01 -0700

Here is the early initial draft of JSON-Test ... this is not yet published
as an I-D. I need to fill in some of the additional detail and examples
around the basic predicate operations. However, the key elements are in
place. I will try to get this published either tonight or tomorrow at the
latest

  http://goo.gl/osKeI  <== Links to my Github repo

As always, feedback is more than welcome.

- James

On Mon, Sep 17, 2012 at 12:33 PM, Paul C. Bryan <[email-masked]> wrote:

> **
> On Mon, 2012-09-17 at 17:32 +1000, Manger, James H wrote:
>
> The JSON patch spec looks good: draft-ietf-appsawg-json-patch-04.
>
> There is no explicit mention of extensibility of the JSON patch syntax.
> Presumably new operations (beyond add/remove/replace/move/copy/test) could
> be added, with the understanding that "old" implementations will reject
> such documents [section 4 says an unknown operation is an error]. Adding
> extra fields to an existing operation is less clear (eg
> {"add":"/foo/1","value":"baz","repeat":10}). There is no explicit mention
> that that would be an error. Perhaps implementations will ignore
> unrecognized fields.
>
>
> I had envisioned a container document type, which would enumerate a set of
> conditions that must be met before processing the operations an associated
> (contained) JSON Patch document. Taking this approach means that there is
> no confusion over what's a JSON Patch document vs. the extension document
> type. Since JSON Patch is an array of operation objects, such a structure
> would remain relatively straightforward.
>
> More concretely, perhaps something like:
>
> {    "tests": [        { "less-than": "/some/path", "value": 42 },        { "exists": "/some/other/path" },        { "not-exists": "/yet/another/path" },
>         { "contains": "/foo/bar", "value": "Major-General" }    ], "application/json-patch": [        { "replace": "/some/path", "value": 42 },        { "delete": "/some/other/path" },        { "add": "/yet/another/path", "value": "bar!" }    ]}
>
>
> Thoughts?
>
>
>  Could more sophisticated future test rules define, say, a "test2"
> operation? Or is it the expectation that such a syntax would need a new
> media type?
>
>
> I would say any extension should require its own media type.
>
>
>  Each operation lists a full JSON pointer from the root of the resource.
> If you want to replace a dozen fields in a deeply nested object you need to
> repeat the path to that object a dozen times. This may be a reasonable
> compromise to avoid a more complex patch structure (particularly if
> compression removes much of the overhead), but I wasn't sure if any
> alternative had been considered. For example,
> {"patch":"/fee/fie/fo/fum/foo","value":[{"add":"x","value":"baz"},{"add":"y","value":3}]}.
>
>
> We've considered alternatives in the past, but concluded that given the
> verbosity of the JSON Patch document, the likely strategy to reduce size
> would be through compression. As such, multiple similar paths would
> compress fairly well.
>
>
>  Would I be opening a can of worms (or doing some bike-shedding) if I
> asked why application/json-patch instead of application/patch+json?
>
>
> Probably. I originally proposed application/patch+json. The main problem
> with application/patch+json is that it implies that there are other
> representations of application/patch, not just JSON (e.g.
> application/patch+xml). In this case, JSON Patch is meant to address *only
> * JSON documents in a JSON format. Furthermore, I'm not yet aware of a
> specification for JSON like there is is for XML that promotes the addition
> of +json. Therefore, I'd say that application/json-patch is probably the
> better choice.
>
> Paul
>
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>
>

---

From: James M Snell <[email-masked]>
Date: Mon, 17 Sep 2012 16:56:50 -0700

Ok.. working through some possible ways of manifesting the predicate
operations within a JSON-Patch context... there are two possible options
that I see...

(A) One is to use something like the example syntax Paul suggests in his
previous note...

  {
    "tests": [
        { "less-than": "/some/path", "value": 42 },
        { "test": "/some/other/path" },
        { "contains": "/foo/bar", "value": "Major-General" }
    ], "application/json-patch": [
        { "replace": "/some/path", "value": 42 },
        { "delete": "/some/other/path" },
        { "add": "/yet/another/path", "value": "bar!" }
    ]
  }

(B) Another is to introduce an optional "predicate" member on the
JSON-Patch "test" operation, such that a "test" operation would only
succeed if, and only if, the attached predicate evaluates true:

  [
    {"test": "/",
     "predicate": {
       "and": [
        { "less-than": "/some/path", "value": 42 },
        { "test": "/some/other/path" },
        { "contains": "/foo/bar", "value": "Major-General" }
       ]
     }
    },
    { "replace": "/some/path", "value": 42 },
    { "delete": "/some/other/path" },
    { "add": "/yet/another/path", "value": "bar!" }
  ]

Running through the various cases, option (B) seems to be a cleaner mapping
overall. Note that in this case, the JSON-Pointer value specified by the
"test" member identifies the base reference point for the predicate.

One of the main reasons I prefer (B) is that it allows me a clean way of
interpolating predicate checks throughout the patch document.

Option (B) also allows us to get by without necessarily creating a new base
document type and media type.

Thoughts?

- James

On Mon, Sep 17, 2012 at 12:39 PM, James M Snell <[email-masked]> wrote:

>> I had envisioned a container document type, which would enumerate a set
>> of conditions that must be met before processing the operations an
>> associated (contained) JSON Patch document. Taking this approach means that
>> there is no confusion over what's a JSON Patch document vs. the extension
>> document type. Since JSON Patch is an array of operation objects, such a
>> structure would remain relatively straightforward.
>>
>> application/patch+xml). In this case, JSON Patch is meant to address *
>> only* JSON documents in a JSON format. Furthermore, I'm not yet aware of
>> a specification for JSON like there is is for XML that promotes the
>> addition of +json. Therefore, I'd say that application/json-patch is
>> probably the better choice.
>>
>> Paul
>>

---

From: "Paul C. Bryan" <[email-masked]>
Date: Mon, 17 Sep 2012 18:02:01 -0700

Some comments off the top of my head:

1. I'd probably like to understand more about the interpolation usage,
especially its utility value before defaulting to my preference for
option A. :-)
2. I happen to think that the introduction of a new media type will
probably be necessary.
3. I like your thinking about logical (a.k.a "aggregate" in your
document) operations: "and", "or", "not".
4. Though I have no concrete use case so far, there's something very
attractive to me about having something wrap a set of conditions around
something else that can be processed.

Paul

On Mon, 2012-09-17 at 16:56 -0700, James M Snell wrote:

> operations within a JSON-Patch context... there are two possible
> options that I see... 
> 
> 
> 
> (A) One is to use something like the example syntax Paul suggests in
> his previous note...
> 
> 
>   {
>     "tests": [
> Running through the various cases, option (B) seems to be a cleaner
> mapping overall. Note that in this case, the JSON-Pointer value
> specified by the "test" member identifies the base reference point for
> the predicate.
> 
> 
> One of the main reasons I prefer (B) is that it allows me a clean way
> of interpolating predicate checks throughout the patch document.
> 
> 
> Option (B) also allows us to get by without necessarily creating a new
> base document type and media type.
> 
> 
> Thoughts?
> 
> 
> - James
> 
> 
> 
>         Here is the early initial draft of JSON-Test ... this is not
>         yet published as an I-D. I need to fill in some of the
>         additional detail and examples around the basic predicate
>         operations. However, the key elements are in place. I will try
>         to get this published either tonight or tomorrow at the latest
>         
>         
>         
>                 > There is no explicit mention of extensibility of the
>                 > JSON patch syntax. Presumably new operations (beyond
>                 > add/remove/replace/move/copy/test) could be added,
>                 > with the understanding that "old" implementations
>                 > will reject such documents [section 4 says an
>                 > unknown operation is an error]. Adding extra fields
>                 > to an existing operation is less clear (eg
>                 > {"add":"/foo/1","value":"baz","repeat":10}). There
>                 > is no explicit mention that that would be an error.
>                 > Perhaps implementations will ignore unrecognized
>                 > fields.
>                 
>                 
>                 
>                 
>                 I had envisioned a container document type, which
>                 would enumerate a set of conditions that must be met
>                 before processing the operations an associated
>                 (contained) JSON Patch document. Taking this approach
>                 means that there is no confusion over what's a JSON
>                 Patch document vs. the extension document type. Since
>                 JSON Patch is an array of operation objects, such a
>                         { "exists": "/some/other/path" },
>                         { "not-exists": "/yet/another/path" },
>                 > Could more sophisticated future test rules define,
>                 > say, a "test2" operation? Or is it the expectation
>                 > that such a syntax would need a new media type?
>                 
>                 
>                 
>                 
>                 > Each operation lists a full JSON pointer from the
>                 > root of the resource. If you want to replace a dozen
>                 > fields in a deeply nested object you need to repeat
>                 > the path to that object a dozen times. This may be a
>                 > reasonable compromise to avoid a more complex patch
>                 > structure (particularly if compression removes much
>                 > of the overhead), but I wasn't sure if any
>                 We've considered alternatives in the past, but
>                 concluded that given the verbosity of the JSON Patch
>                 document, the likely strategy to reduce size would be
>                 through compression. As such, multiple similar paths
>                 would compress fairly well.
>                 
>                 
>                 
>                 
>                 > Would I be opening a can of worms (or doing some
>                 > bike-shedding) if I asked why application/json-patch
>                 > instead of application/patch+json?
>                 
>                 
>                 
>                 
>                 Probably. I originally proposed application/patch
>                 +json. The main problem with application/patch+json is
>                 that it implies that there are other representations
>                 of application/patch, not just JSON (e.g.
>                 application/patch+xml). In this case, JSON Patch is
>                 meant to address only JSON documents in a JSON format.
>                 Furthermore, I'm not yet aware of a specification for
>                 JSON like there is is for XML that promotes the
>                 addition of +json. Therefore, I'd say that
>                 application/json-patch is probably the better choice.
>                 
>                 Paul
>                 
>

---

From: James M Snell <[email-masked]>
Date: Mon, 17 Sep 2012 18:13:51 -0700

The interpolation case is a bit abstract and deals more with allowing for
fail-fast (checking to see if an operation could succeed before actually
attempting the operation) as well as ensuring success in cases where the
target resource may not actually be a json document and a preceding change
might potentially trigger an unintended change of state. (This is largely
experimental for now so I can be easily swayed in either direction with a
solid enough argument :-) ...)

For example, suppose I have a resource that can be represented as JSON, but
implements a rule that the "/updated" property is to be modified
automatically whenever any of it's other discreet fields are modified. A
client may not be aware of this semantic and may attempt to set the
"/updated" and "/title" values...

[
  {"replace":"/title","value":"foo"},
  {"replace":"/updated", "value":"2012-12-12T12:12:12Z"}
  {"test":"/updated", "value":"2012-12-12T12:12:12Z"}
]

The test ensures that the patch as a whole will only succeed if the value
of "/updated" is precisely the value intended by the sender.

Yes, this is a fairly contrived example.. like I said, I'm more than
willing to be convinced that I'm just being silly.

- James

On Mon, Sep 17, 2012 at 6:02 PM, Paul C. Bryan <[email-masked]> wrote:

> especially its utility value before defaulting to my preference for option
> A. :-)
> 3. I like your thinking about logical (a.k.a "aggregate" in your document)
> operations: "and", "or", "not".
> mapping overall. Note that in this case, the JSON-Pointer value specified
> by the "test" member identifies the base reference point for the predicate.
>
>
>
>    I had envisioned a container document type, which would enumerate a
> set of conditions that must be met before processing the operations an
>    We've considered alternatives in the past, but concluded that given
> the verbosity of the JSON Patch document, the likely strategy to reduce
> size would be through compression. As such, multiple similar paths would
> compress fairly well.
>
>
>
>    Probably. I originally proposed application/patch+json. The main
> problem with application/patch+json is that it implies that there are other

---

From: Matthew Morley <[email-masked]>
Date: Tue, 18 Sep 2012 08:05:59 -0400

I used a less verbose json structure to do something along the same
lines... calling it 'minimum change communication' (MMC).

This was the use case:

   - Client requests a full json object from the server
      - the object comes with a version field (integer)
   - Client sends a  change request to the server for the object
      - also sending the version field it knows about
   - If a request comes in with a matching version field
      - responses to the request can be easily expressed in a diff
   - if a request comes in and doesn't match
      - additional handling may be needed if other clients have already
      changed the structure significantly
      - the request may be processed anyway, but the UI can then also
      brought into the loop
         - "the last change you requested was prevented due to other
         editors actions"

The one portion where this deviated for me, with MCC, is that I
communicated what values the server set, *not* just the values that
changed. This way, details of the servers actions could be used to trigger
additional actions within the client, even if the field value was the same.

I'm not promoting this approach, but it can be an issue when your client
needs awareness of the action, more than the change in value. Json patch
can be used in this way with a same value replace, but it does impact how
you generate the patch.

On Mon, Sep 17, 2012 at 9:13 PM, James M Snell <[email-masked]> wrote:

> For example, suppose I have a resource that can be represented as JSON,
> but implements a rule that the "/updated" property is to be modified
>> Here is the early initial draft of JSON-Test ... this is not yet
>> published as an I-D. I need to fill in some of the additional detail and
>> examples around the basic predicate operations. However, the key elements
>> are in place. I will try to get this published either tonight or tomorrow
>> at the latest
>>
>>
>>

-- 
Matthew P. C. Morley