Subject: Re: [apps-discuss] [Json] msgpack/binarypack (Re: JSON mailing list and BoF)

From: Sadayuki Furuhashi <[email-masked]>
Date: Mon, 4 Mar 2013 00:01:10 -0800

I'm frsyuki (Sadyuki Furuhashi).

As the initial developer of the MessagePack specification, I am feeling unrest
in bringing MessagePack to IETF as an Internet-Draft at this very moment.

I don't against having a standard itself but I really have difficulty on its downside: incompatibility.

Because at least I already have hundreds TBs of data stored in MessagePack format.
And there're already many other users who don't expect incompatible changes. Here is a list of users:
(This list is quite old, though. There're more users now. You'll see the list includes Pinterest, Redis, etc.)
http://wiki.msgpack.org/display/MSGPACK/PoweredBy

Thus compatibility is an essential problem of msgpack. So I can't help opposing drafts which are
incompatible with MessagePack.

Another problem which makes this decision complicated is that we're discussing on changing
the msgpack spec. We'll likely add string or binary type to msgpack. Prof. Dr. Bormann kindly
joined us to help happening this change. But we have not updated implementations based on
the new spec yet. It's not validated by users. It means that we need to disscuss about the
change of spec, implement it in many languages, validate them on production environments,
and write documents. I don't think we can make them all happen soon, at the same time.

I don't against having a standard itself but we're not prepared to start it now. However, even
writing an individual Internet-Draft under my name is an our possible option. Meaning that
sooner or later, we'll have a document about the new MessagePack spec (which includes
the string/binary types). I can't commit the timing at this time, though. We're focunsing on
the new spec right now.

Let me keep updated about discussion on our issue thread:
https://github.com/msgpack/msgpack/issues/129

--
Sadayuki Furuhashi
http://fluentd.org http://msgpack.org
twitter:@frsyuki

On 2013/02/24, at 5:11, Carsten Bormann <[email-masked]> wrote:

> 
> On Feb 19, 2013, at 17:39, Carsten Bormann <[email-masked]> wrote:
> 
>> On Feb 19, 2013, at 00:47, "Joe Hildebrand (jhildebr)" <[email-masked]> wrote:
>> 
>>> As an individual, I'm +1 on that.  I love msgpack, and don't mind the
>>> addition of UTF8 as a separate type.  Was frsyuki involved in the draft,
>>> or at least know that it happened?
>> 
>> I tried to involve him.
> 
> Well, I did engage the msgpack community some more.
> 
> You can find a transcript of some 275 messages about separating byte and UTF-8 strings at:
> 
> 	https://github.com/msgpack/msgpack/issues/121
> 
> Summary:
> Some members of the msgpack community are very enraged that this change hasn't happened earlier.
> Of course, some have gone off and done their own incompatible forks.
> Others are very enraged that any change is happening at all, and that new people are intruding on their turf.
> (And some probably feel guilty that it took a ****storm from outside to finally make this change.)
> 
> frsyuki is now working on a proposal that solves the problem:
> 
> 	https://gist.github.com/frsyuki/5022569
> 
> The proposal is technically complete (and has already been implemented).
> It already is pretty good at the details, too, but this whole thing is being done in a process that is closer to Japanese consensus processes than to IETF culture.
> 
> My -01 will be fully aligned with whatever the state of frsyuki's proposal will be on Monday's I-D deadline (find today's snapshot at http://www.tzi.de/~cabo/draft-bormann-apparea-bpack-01pre2.txt).
> (frsyuki's proposal may change some more, but those will in all likelihood be minor details.)
> I think his overall thinking is fine, but it is much more dominated by a requirement for backwards compatibility than an IETF process would be.
> 
> So, the larger question on whether the msgpack community is ready to take part (or just endure) in an IETF-style consensus process (including handing over change control) still looms.
> 
> That doesn't diminish from the requirement for a msgpack-like format, and I think we should use Hallway Time in Orlando to discuss potential ways forward.
> 
> I any case, I definitely don't want to disturb the constructive discussion about chartering a very narrow JSON fixup WG with this work.
> (I do want to find a home for it, soon, though: I want to build other specs on top of it.)
> 
> Grüße, Carsten
> 
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 4 Mar 2013 18:36:49 +0000

>its downside: incompatibility.

(as individual)

We had a similar unrest about bringing XMPP to the IETF the first time.
We wrote very strict charter language about backward compatibility, and
became highly involved in the process.  It turned out to be a very
positive experience on the whole.  We got lots of reviews from folks that
had not paid attention to XMPP before, including security,
internationalization, and the clarity of our documentation.  The current
RFCs are so much better than we would have produced on our own that I
can't imagine not having done the work, now.  The authors from our
community got credit in the RFCs for their work; there was no lack of
respect for their previous efforts.

Another reason why msgpack is interesting to consider for standardization
is that I for one would like to use it as the wire protocol for
standards-based protocols.  In order to do this today, I would would
probably need to do one of:

- Include whatever subset of msgpack I want to use into the new protocols.
 This would be  awful, because then each other protocol that wanted to use
msgpack would need to do the same thing, and they would all choose
slightly incompatible approaches.

- Refer to your existing web page as the msgpack specification.  This
would be difficult to use as a "stable reference", since it doesn't come
from a standards body with previously-vetted intellectual property rules,
and we don't know how often it will change.

- Invent something new that tried to solve the same problem, perhaps using
a method that avoids reusing your work.  This would be the worst possible
outcome for the industry, since having two completely different approaches
to the same problem is a recipe for years of incompatibility.

I'm sorry if the communications you've had with people that participate in
the IETF have made your community feel as if we're trying to take over
your idea, have made you feel disrespected, or that we want to force
backward-incompatibilities on you.  At least from my personal point of
view, your work to date has been exemplary, and it is a measure of my
respect that I would like to help enable that work to be used by even more
people.

-- 
Joe Hildebrand