From nobody Mon Jan 1 18:49:53 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 36E84126C89 for ; Mon, 1 Jan 2018 18:49:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fp-iJkx1ZrIA for ; Mon, 1 Jan 2018 18:49:50 -0800 (PST) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 71261120454 for ; Mon, 1 Jan 2018 18:49:50 -0800 (PST) Received: by mail-wm0-x22c.google.com with SMTP id 9so59035262wme.4 for ; Mon, 01 Jan 2018 18:49:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=683QKUrj8vjuyqUOMEiCApvWFZHjJRvhTy1mPLaUyXY=; b=pp7hEitdBGiI/+ZumPQw8cR2d4uz3l8RW+6qyHbG9HoOZWdosFloxRrAfmaO3WxTe+ WmooKh+KEQQfnyodv+n4Vofv5eMSX2Dgc4pGt7Q4EjT9oPsXI8rnkVqFrprtxa6UnLfT KkSYiTAtx4rzcgz9VfgYP4FR7qnwWNZXao2w6TwjXXjUmEs1C+9TYWOfaQXwJnhl8BwF UHOfuq91y2O33w1WkwzqjvEDk3GoxsraoA2s+qcgcYZS3Rn0q7t7EgKPT9NvnXn3yU8h FWy0279F98MVWKG1mWZtEENr/8494p3QCbCCib0QevmduqIepEu5TxmRBz5r8Emzvvke 3Snw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=683QKUrj8vjuyqUOMEiCApvWFZHjJRvhTy1mPLaUyXY=; b=TmxyZsOIBL24Y9MFoikwqhDrDDTawW/HASH6FBEw2iHjeT6QiP+pNTuUIlNjiGZB3n tnbh4bmJb10YByv+Kt1tHk4N7w4Gden67q+40VIVeBj0pBv3hLSCMpasrDZsN2rungKc HATGeV+aX/HdCTcnv/h7U0aJ2yB/EiCx+5OygaINIGAKqIWc9icut5R53AtKMPKTrfmR Ud9rCmUXdWx0ciSiaJOhU11HXN9Q0RgwErMpSZPFQvpy/rfgbylgI028c2Dsza5/rVvV OkynXmWcR8bPYvlOK2tcihOL2BIhFcrArkebmqAlrLWnfckDIR74q0H6gpfD1HKPQGAs QJFA== X-Gm-Message-State: AKGB3mKFppl80FlMM/tewyvN9w5DXHktXyi75UB1NP5Ysn2N98VGs+Iw RyRVUY4mxCjxeMNbUiqNRDPEytDkBGVyLnHcNN0= X-Google-Smtp-Source: ACJfBot88NH8g/gvMfXT2dOVkAYG3DKvq/IP7Oe5jBUlBnNm1EjSpeLY8MQQpMwBDu17HOjBFbnpMeuSmcut4XY2LGM= X-Received: by 10.80.230.143 with SMTP id z15mr61695549edm.106.1514861388640; Mon, 01 Jan 2018 18:49:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.177.233 with HTTP; Mon, 1 Jan 2018 18:49:48 -0800 (PST) In-Reply-To: References: From: Austin Einter Date: Tue, 2 Jan 2018 08:19:48 +0530 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary="089e082b5d5cf435910561c22688" Archived-At: Subject: [Cellar] Fwd: Understanding VP8 data in a MKV file X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 02:49:52 -0000 --089e082b5d5cf435910561c22688 Content-Type: text/plain; charset="UTF-8" I have a mkv file, that contains VP8 encoded data. I need to stream this data to chrome (through RTP), and need to play it in chrome. I have parsed the mkv file, and I have VP8 data. I will be sending this VP8 data in RTP packetc. Looks I need to follow RFC 7741 to put VP8 descriptor / header in RTP packet followed by VP8 data. Now I have several doubts, kindly try to help me to understand. 1) What is a partition in VP8 and from MKV file how do I get partition information. 2) I need to put partition size in VP8 header. From where I get the partition size. 3) How do I know partition index, this I need to put in VP8 descriptor. Does VP8 data I read from mkv file contains this. Regards Austin --089e082b5d5cf435910561c22688 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I have a m= kv file, that contains VP8 encoded data.
I need to stream this data to = chrome (through RTP), and need to play it in chrome.

I have parsed the mkv file, and I have VP8 data.
I will be sen= ding this VP8 data in RTP packetc.
Looks I need to follow RFC 774= 1 to put VP8 descriptor / header in RTP packet followed by VP8 data.
<= div>
Now I have several doubts, kindly try to help me to unde= rstand.

1) What is a partition in VP8 and from MKV= file how do I get partition information.
2) I need to put partit= ion size in VP8 header. From where I get the partition size.
3) H= ow do I know partition index, this I need to put in VP8 descriptor.

Does VP8 data I read from mkv file contains this.

Regards
Austin
--089e082b5d5cf435910561c22688-- From nobody Tue Jan 2 06:04:07 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55386126DEE for ; Tue, 2 Jan 2018 06:04:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CZD2MSShdL_y for ; Tue, 2 Jan 2018 06:04:04 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 25E231201F8 for ; Tue, 2 Jan 2018 06:04:04 -0800 (PST) Received: from [146.96.19.240] (port=44646 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eWNAe-000BEe-1G; Tue, 02 Jan 2018 09:04:03 -0500 From: Dave Rice Message-Id: <37C95B2F-0CB3-4FB6-B65C-8B5A16D2A475@dericed.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_D62BE204-2BAA-4EDE-A439-C83655EEB108" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Tue, 2 Jan 2018 09:03:58 -0500 In-Reply-To: Cc: Codec Encoding for LossLess Archiving and Realtime transmission To: Ashley Blewer References: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] status of FFV1 and EBML drafts X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 14:04:06 -0000 --Apple-Mail=_D62BE204-2BAA-4EDE-A439-C83655EEB108 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, In regards to FFV1, I just pushed a pull request at = https://github.com/FFmpeg/FFV1/pull/82 = , mostly grammar and formatting. = Derek=E2=80=99s review at = https://mailarchive.ietf.org/arch/msg/cellar/qItSPq06vc9MaxsZFPJ3GmREsr4 = = has some issues still remaining: consideration of renaming the = JPEG2000-RCT colorspace and some work to be more clear about the = terminology of Frame and Pixel and defining their corresponding = bitstream functions. FFV1 also needs an IANA Considerations section = (some comments from Jerome at that at = https://www.ietf.org/mail-archive/web/cellar/current/msg01168.html = ). I suggest after resolving the current open pull requests than we could = ping for some additional review. At some point we=E2=80=99ll also need = to start the work to separate the Version 0,1,3 informational = specification from the specification drafting of Version 4. In regards, to EBML and Matroska, both have had a lot of active work = over the past few months. I=E2=80=99ll update the document tracker with = new drafts of both specifications within the next day or so and = summarize changes to the list. To the chairs, thus far the Matroska draft is listed as a =E2=80=9CRelated= Internet-Draft=E2=80=9D. At this point should the new draft be = submitted as an =E2=80=9CActive Internet Draft=E2=80=9D? It seems to = meet the criteria. In regards to FLAC, I see that we don=E2=80=99t have a draft in the = document tracker. Andrew, any status? Best Regards, Dave Rice > On Dec 28, 2017, at 12:28 PM, Ashley Blewer = wrote: >=20 > [thread bump]=20 >=20 > Can we figure out how to come to a consensus on these couple of = issues? >=20 > Ashley >=20 > On Sun, Nov 26, 2017 at 4:07 AM, Steve Lhomme > wrote: > Hi, >=20 > I won't comment on FFV1, although I should probably read it to see if > there's anything I don't understand. >=20 > For EBML I think there are still things to be done: > - clarify how to handle bogus data > https://github.com/Matroska-Org/ebml-specification/issues/48 = > - kind of relates to junk in Master Elements > https://github.com/Matroska-Org/ebml-specification/issues/92 = > In the context of long term storage it's probably very important to > solve these issues. >=20 > - DocType 0 as experimental/internal. We need to agree on this and = then word it > https://github.com/Matroska-Org/ebml-specification/issues/157 = >=20 > After that I think we're good to got for EBML. There's currently no > pending pull request. >=20 > 2017-11-19 16:47 GMT+01:00 Dave Rice >: > > Hi cellar, > > > > At the IETF99 meeting in the minutes at = https://datatracker.ietf.org/doc/minutes-99-cellar/ = it discusses = having a WG last call for FFV1 and EBML but also the need for more = reviews of these documents. Is one issue blocking another? Meaning: = should the group announce a last call in order to encourage more reviews = or are more reviews needed in order to have a last call? > > > > Best Regards, > > Dave Rice > > > > _______________________________________________ > > Cellar mailing list > > Cellar@ietf.org > > https://www.ietf.org/mailman/listinfo/cellar = >=20 >=20 >=20 > -- > Steve Lhomme > Matroska association Chairman >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar = >=20 > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --Apple-Mail=_D62BE204-2BAA-4EDE-A439-C83655EEB108 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi,

In regards to FFV1, I just pushed a = pull request at https://github.com/FFmpeg/FFV1/pull/82, mostly grammar = and formatting. Derek=E2=80=99s review at https://mailarchive.ietf.org/arch/msg/cellar/qItSPq06vc9MaxsZFP= J3GmREsr4 has some issues still remaining: consideration of = renaming the JPEG2000-RCT colorspace and some work to be more clear = about the terminology of Frame and Pixel and defining their = corresponding bitstream functions. FFV1 also needs an IANA = Considerations section (some comments from Jerome at that at https://www.ietf.org/mail-archive/web/cellar/current/msg01168.h= tml).

I = suggest after resolving the current open pull requests than we could = ping for some additional review. At some point we=E2=80=99ll also need = to start the work to separate the Version 0,1,3 informational = specification from the specification drafting of Version 4.

In regards, to EBML and = Matroska, both have had a lot of active work over the past few months. = I=E2=80=99ll update the document tracker with new drafts of both = specifications within the next day or so and summarize changes to the = list.

To the = chairs, thus far the Matroska draft is listed as a =E2=80=9CRelated = Internet-Draft=E2=80=9D. At this point should the new draft be submitted = as an =E2=80=9CActive Internet Draft=E2=80=9D? It seems to meet the = criteria.

In = regards to FLAC, I see that we don=E2=80=99t have a draft in the = document tracker. Andrew, any status?

Best Regards,
Dave = Rice

On Dec 28, 2017, at 12:28 PM, Ashley Blewer <ashley.blewer@gmail.com> wrote:

[thread bump]

Can we figure out how to = come to a consensus on these couple of issues?

Ashley
On Sun, Nov 26, 2017 at 4:07 AM, = Steve Lhomme <slhomme@matroska.org> wrote:
Hi,

I won't comment on FFV1, although I should probably read it to see if
there's anything I don't understand.

For EBML I think there are still things to be done:
- clarify how to handle bogus data
https://github.com/Matroska-Org/ebml-specification/issues/48
- kind of relates to junk in Master Elements
https://github.com/Matroska-Org/ebml-specification/issues/92
In the context of long term storage it's probably very important to
solve these issues.

- DocType 0 as experimental/internal. We need to agree on this and then = word it
https://github.com/Matroska-Org/ebml-specification/issues/157

After that I think we're good to got for EBML. There's currently no
pending pull request.

2017-11-19 16:47 GMT+01:00 Dave Rice <dave@dericed.com>:
> Hi cellar,
>
> At the IETF99 meeting in the minutes at https://datatracker.ietf.org/doc/minutes-99-cellar/ it discusses having a WG last call = for FFV1 and EBML but also the need for more reviews of these documents. = Is one issue blocking another? Meaning: should the group announce a last = call in order to encourage more reviews or are more reviews needed in = order to have a last call?
>
> Best Regards,
> Dave Rice
>
> _______________________________________________
> Cellar mailing list
> Cellar@ietf.org
> https://www.ietf.org/mailman/listinfo/cellar



--
Steve Lhomme
Matroska association Chairman

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

_______________________________________________
Cellar = mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

= --Apple-Mail=_D62BE204-2BAA-4EDE-A439-C83655EEB108-- From nobody Tue Jan 2 06:07:37 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BD1012706D for ; Tue, 2 Jan 2018 06:07:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HD6IlVYdtzQb for ; Tue, 2 Jan 2018 06:07:34 -0800 (PST) Received: from 1.mo177.mail-out.ovh.net (1.mo177.mail-out.ovh.net [178.33.107.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 08A7B126DEE for ; Tue, 2 Jan 2018 06:07:34 -0800 (PST) Received: from player688.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo177.mail-out.ovh.net (Postfix) with ESMTP id 204AF9083A for ; Tue, 2 Jan 2018 15:07:31 +0100 (CET) Received: from [192.168.2.120] (p5DDB4B21.dip0.t-ipconnect.de [93.219.75.33]) (Authenticated sender: jerome@mediaarea.net) by player688.ha.ovh.net (Postfix) with ESMTPSA id C0E4F200A2 for ; Tue, 2 Jan 2018 15:07:31 +0100 (CET) To: cellar@ietf.org References: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> <37C95B2F-0CB3-4FB6-B65C-8B5A16D2A475@dericed.com> From: Jerome Martinez Message-ID: <4dba18f3-4ab8-fba6-7b9f-729bfed4f46a@mediaarea.net> Date: Tue, 2 Jan 2018 15:07:32 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <37C95B2F-0CB3-4FB6-B65C-8B5A16D2A475@dericed.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 10530260354432307345 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrjedugdeiudcutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] status of FFV1 and EBML drafts X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2018 14:07:36 -0000 On 02/01/2018 15:03, Dave Rice wrote: > Derek’s review at > https://mailarchive.ietf.org/arch/msg/cellar/qItSPq06vc9MaxsZFPJ3GmREsr4 has > some issues still remaining Sorry for the delay, I hope to have some time for handling the issues pointed by the review this week. Jérôme From nobody Wed Jan 3 04:23:28 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 1EB60126579; Wed, 3 Jan 2018 04:23:27 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.68.2 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <151498220708.30886.10522215943386127741@ietfa.amsl.com> Date: Wed, 03 Jan 2018 04:23:27 -0800 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-ebml-04.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 12:23:27 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : Extensible Binary Meta Language Authors : Steve Lhomme Dave Rice Moritz Bunkus Filename : draft-ietf-cellar-ebml-04.txt Pages : 38 Date : 2018-01-03 Abstract: This document defines the Extensible Binary Meta Language (EBML) format as a generalized file format for any type of data in a hierarchical form. EBML is designed as a binary equivalent to XML and uses a storage-efficient approach to build nested Elements with identifiers, lengths, and values. Similar to how an XML Schema defines the structure and semantics of an XML Document, this document defines how EBML Schemas are created to convey the semantics of an EBML Document. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-ebml-04 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-ebml-04 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-ebml-04 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Wed Jan 3 04:31:16 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B763126FB3 for ; Wed, 3 Jan 2018 04:31:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.859 X-Spam-Level: X-Spam-Status: No, score=-0.859 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTML_OBFUSCATE_05_10=0.26, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kqU7LYX-Ramb for ; Wed, 3 Jan 2018 04:31:13 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 826FC126579 for ; Wed, 3 Jan 2018 04:31:13 -0800 (PST) Received: from cpe-104-162-94-162.nyc.res.rr.com ([104.162.94.162]:42194 helo=[10.0.1.9]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eWiCN-002wa1-Ol for cellar@ietf.org; Wed, 03 Jan 2018 07:31:12 -0500 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_822B553F-21B1-4D53-9237-5F2F39CE4DBF" Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Message-Id: <835D246A-BD86-4434-962E-374D7B7EB1C7@dericed.com> Date: Wed, 3 Jan 2018 07:31:09 -0500 To: Codec Encoding for LossLess Archiving and Realtime transmission X-Mailer: Apple Mail (2.3445.4.7) X-OutGoing-Spam-Status: No, score=-2.6 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] draft-ietf-cellar-ebml-04 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 12:31:15 -0000 --Apple-Mail=_822B553F-21B1-4D53-9237-5F2F39CE4DBF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi cellar, I just uploaded draft-ietf-cellar-ebml-04 to the tracker, see = https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/. This update = reflects the work shown in the commit log at = https://github.com/Matroska-Org/ebml-specification/commits/draft-ietf-cell= ar-ebml-04 from July 2, 2017 through January 3rd, 2018. If there=E2=80=99s consensus for it, I propose moving this document to = =E2=80=9CIn WG Last Call = =E2=80=9D. Happy New Year, Dave Rice= --Apple-Mail=_822B553F-21B1-4D53-9237-5F2F39CE4DBF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi cellar,

I just uploaded draft-ietf-cellar-ebml-04 to the tracker, = see https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/. = This update reflects the work shown in the commit log at https://github.com/Matroska-Org/ebml-specification/commits/draf= t-ietf-cellar-ebml-04 from July 2, 2017 through January 3rd, = 2018.

If there=E2=80=99= s consensus for it, I propose moving this document to =E2=80=9CIn = WG Last Call=E2=80=9D.

Happy New Year,
Dave = Rice
= --Apple-Mail=_822B553F-21B1-4D53-9237-5F2F39CE4DBF-- From nobody Wed Jan 3 04:40:06 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4F94126FDC for ; Wed, 3 Jan 2018 04:40:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.01 X-Spam-Level: X-Spam-Status: No, score=-2.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fk9sAGf6SBvr for ; Wed, 3 Jan 2018 04:40:01 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BCF8D126FB3 for ; Wed, 3 Jan 2018 04:40:01 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:41654) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eWiKl-0004ba-2e for cellar@ietf.org; Wed, 03 Jan 2018 13:39:51 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 981BA65412CF for ; Wed, 3 Jan 2018 13:39:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1514983191; bh=EQMpZahj3WCx//ZlCd5Ej8BYzwz5CeXb7yzduIeA3so=; h=References:From:To:Cc:Subject:In-reply-to:Date:From; b=1HzbX9L2NbgEzKaFwGLpmcrFMEeBPRAN8UzLL7iwRKi/VOzcCV18f7GyHtp1Pio6Q FLKp3YP+Vz437v2EcIEOVepIbWt3kEwhbi3vH0hHalQLyR7NGzawZzsitFK50v9w9T jz1eeeGhU0V2TV9pJhuKF5XauX5HRWHR1YXwvRm9M7ilqYPNPxfXt+amfnVIjA2cEy I8djSD3A5LmLpakSTOjDbHfaAsjnZPSBBb+SMFjPts6XGOhkMbau6dn60fUQjHvDz7 XMOLnnurW+0+vVDl7l8Al6gbTKaK+B1xIfAF0qXumzaWfEd0DaVab8FEGk/QSGpBpT 5P3qnnfbq/Qj1dtXCemOYHSXDIGLqIUGJor/lGt5Ycj+zV2SGuuz5bZqOoClqib9xm fcjX9V83yhV0VUqrQudpKm2sudZAjjP2zXrf/0odEe8m5NIx07ulOBhD39DQbjEfPG utCL0bdXd8Orf343K1BmNL2rqr0qkrIXIH1RQi4n0D0lStnEAkNwAIfppmqsUea4EH m0cz5Gx7IAUL05uXNvHepOmPPGTvI23HRIalTSSUdNNSiWdlmGLZTKRl6daidcqHRF sFYXAg6ClnrN6BQif8yt17HbanSfdI7wwAvogaToDBFxm8746YzKvKOzeSWrOzFAs4 5gJtr+yznL+efVG9damBmqIE= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id 0BF842ABD6BB for ; Wed, 3 Jan 2018 13:39:50 +0100 (CET) References: <835D246A-BD86-4434-962E-374D7B7EB1C7@dericed.com> User-agent: mu4e 0.9.18; emacs 25.3.1 From: Moritz Bunkus To: Codec Encoding for LossLess Archiving and Realtime transmission Cc: In-reply-to: <835D246A-BD86-4434-962E-374D7B7EB1C7@dericed.com> Date: Wed, 03 Jan 2018 13:39:50 +0100 Message-ID: <87608jqfop.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cellar] draft-ietf-cellar-ebml-04 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 12:40:05 -0000 Hey, thanks, Dave! > If there’s consensus for it, I propose moving this document to “In WG > Last Call ”. In favor. Kind regards, mosu From nobody Wed Jan 3 05:55:57 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD76C1241F3 for ; Wed, 3 Jan 2018 05:55:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.801 X-Spam-Level: X-Spam-Status: No, score=-2.801 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0xR06scloggh for ; Wed, 3 Jan 2018 05:55:54 -0800 (PST) Received: from smtp-sh.infomaniak.ch (smtp-sh.infomaniak.ch [128.65.195.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EE778120725 for ; Wed, 3 Jan 2018 05:55:53 -0800 (PST) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w03DtpYW010977 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 3 Jan 2018 14:55:51 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w03Dtoff006660 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 3 Jan 2018 14:55:51 +0100 Date: Wed, 3 Jan 2018 14:55:51 +0100 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 In-Reply-To: <87608jqfop.fsf@bunkus.org> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] draft-ietf-cellar-ebml-04 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 13:55:57 -0000 Moritz Bunkus wrote: >Hey, >> If there=E2=80=99s consensus for it, I propose moving this document to = =E2=80=9CIn WG >> Last Call =E2=80=9D. > >In favor. In favour. Best regards, Reto From nobody Wed Jan 3 06:15:15 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 525D4126B72 for ; Wed, 3 Jan 2018 06:15:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id apufJhiVp_wZ for ; Wed, 3 Jan 2018 06:15:12 -0800 (PST) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 95B761241F3 for ; Wed, 3 Jan 2018 06:15:12 -0800 (PST) Received: by mail-qt0-x22a.google.com with SMTP id u10so2239710qtg.2 for ; Wed, 03 Jan 2018 06:15:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=0Rne7LlhjdkTW59id+4CmkjOLymlyXtmmztTcOUmlNs=; b=pcNHa2Y9uvIuoEd7mqVf2E0IuLJD9Y+WCi+wZIwzipu37SeS3r+b96qzqkWrC6nd+I /7Jfe9GxfIauWt/29jLX2Av4pM1KFQktRXm0sZ02s9oDEUPQRXtHSgiGjrJMo+gNDQxs xKaqpAIY+2vtQ46fazFvCvfHUTNc32hjI1gaKxJ7wXkYT396hyY9fLKVLvu9F2iG7YyR PF8SXM/jnlE6eUofHcri7o+lt0VKAV6NdcLCBumwd7cnyLKEtjFFEx6WiQjVz+Io3e1L vZB5JQPUPCdaJyhZ844kdbzt+9fEJiP1j6VpeQHa45IjdzzCW23k6gs6BjYKjbr3ARYu iLPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=0Rne7LlhjdkTW59id+4CmkjOLymlyXtmmztTcOUmlNs=; b=Mw+AJWXpk4ym0VpRX7qZV9D3Qc4wIrePgsUyBbYBANekiRA/wFFC7jqiM63Yx3rSI3 ladv7x5ZjWB62aoyUQhJQTXbMgM91lpA9cGheJOBMFwxctatQ3dNtKunzdMSgdzqlZTk XIteZTbVogz5cUc4o2mYw+Fx09KcG7lNqDJDnXlfyuJbqo7Y8Rh4ZNGg0vv8zrWtNqki rmM1Pt0QyRK1f4fp40yDWtxnjF2krKxomg1bkEQ9vWQEvTyq/MfQbz4FcNhOmhsHEL7y sYwBpXp2mkqlizSbXlzm6Af8P4OUJ61PP0aonhFb3kw/+mIzbHLSq53pUHVfw2dAiX84 wGMw== X-Gm-Message-State: AKGB3mJuNlU8oHvW/eTwgxdG/vLq+AsLSVQ70nhmPDLYmMx9w47WhFok bUwK6Pki6NWusXqemwWcq2JywcJUieYTavmH0u4rwQ== X-Google-Smtp-Source: ACJfBotCa9lHSQaYLwVE5AqIq9FbolGECc+CUTxes3My5rayBva7IiJib1aOOEMZ2VmSSm0BOV6oz/8QCYXlroPXZtU= X-Received: by 10.237.42.230 with SMTP id t93mr1914980qtd.50.1514988911306; Wed, 03 Jan 2018 06:15:11 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.178 with HTTP; Wed, 3 Jan 2018 06:15:10 -0800 (PST) In-Reply-To: References: <87608jqfop.fsf@bunkus.org> From: Ashley Blewer Date: Wed, 3 Jan 2018 09:15:10 -0500 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="94eb2c122b7ae5ac710561dfd724" Archived-At: Subject: Re: [Cellar] draft-ietf-cellar-ebml-04 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 14:15:14 -0000 --94eb2c122b7ae5ac710561dfd724 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable In favor! Hooray, Ashley On Wed, Jan 3, 2018 at 8:55 AM, Reto Kromer wrote: > Moritz Bunkus wrote: > > >Hey, > >> If there=E2=80=99s consensus for it, I propose moving this document to= =E2=80=9CIn WG > >> Last Call =E2=80=9D= . > > > >In favor. > > In favour. > > Best regards, Reto > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --94eb2c122b7ae5ac710561dfd724 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
In favor!

Hooray,

Ashl= ey

On We= d, Jan 3, 2018 at 8:55 AM, Reto Kromer <lists@reto.ch> wrote:
Moritz Bunkus wrote:

>Hey,
>> If there=E2=80=99s consensus for it, I propose mo= ving this document to =E2=80=9CIn WG
>> Last Call <https://tools.ietf.org/html/= rfc6174#section-4.2.7>=E2=80=9D.
>
>In favor.

In favour.

Best regards, Reto

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--94eb2c122b7ae5ac710561dfd724-- From nobody Wed Jan 3 08:34:54 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0727124D68 for ; Wed, 3 Jan 2018 08:34:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YHhYzWqYcVcd for ; Wed, 3 Jan 2018 08:34:50 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BDF8C12AF6E for ; Wed, 3 Jan 2018 08:34:50 -0800 (PST) Received: from [146.96.19.240] (port=16004 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eWm08-001oq1-Ff for cellar@ietf.org; Wed, 03 Jan 2018 11:34:50 -0500 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_A49D2A35-B68F-4CE2-9CE7-5B0840CAD29E" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Message-Id: <006AF262-FA38-4297-8377-121DD76A672F@dericed.com> Date: Wed, 3 Jan 2018 11:34:47 -0500 To: Codec Encoding for LossLess Archiving and Realtime transmission X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] Updates to Matroska documents X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 16:34:53 -0000 --Apple-Mail=_A49D2A35-B68F-4CE2-9CE7-5B0840CAD29E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi cellar, I=E2=80=99ve updated the Matroska specification in the tracker to = version 04. This includes all commits from July 2, 2017 up to now, see = https://github.com/Matroska-Org/matroska-specification/commits/draft-lhomm= e-cellar-matroska-04 = for those commits. Notably the new document = versions split the specification into three documents as the metadata = tag definitions and the codec mappings are now separated from the main = Matroska document (as discussed at the IETF99 meeting). See: https://datatracker.ietf.org/doc/draft-lhomme-cellar-matroska/ = https://datatracker.ietf.org/doc/draft-lhomme-cellar-codec/ = https://datatracker.ietf.org/doc/draft-lhomme-cellar-tags/ = These documents have the status of =E2=80=9CActive Internet Drafts=E2=80=9D= . I propose assessing consensus to change these to =E2=80=9CWG = Document=E2=80=9D per https://tools.ietf.org/html/rfc6174#section-4.2.4 = . Kind Regards, Dave Rice= --Apple-Mail=_A49D2A35-B68F-4CE2-9CE7-5B0840CAD29E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi cellar,

I=E2=80=99ve updated the Matroska specification in the = tracker to version 04. This includes all commits from July 2, 2017 up to = now, see https://github.com/Matroska-Org/matroska-specification/commits/= draft-lhomme-cellar-matroska-04 for those commits. Notably the = new document versions split the specification into three documents as = the metadata tag definitions and the codec mappings are now separated = from the main Matroska document (as discussed at the IETF99 meeting). = See:

https://datatracker.ietf.org/doc/draft-lhomme-cellar-matroska/<= /a>
https://datatracker.ietf.org/doc/draft-lhomme-cellar-codec/=
https://datatracker.ietf.org/doc/draft-lhomme-cellar-tags/<= /div>

These = documents have the status of =E2=80=9CActive Internet Drafts=E2=80=9D. I = propose assessing consensus to change these to =E2=80=9CWG Document=E2=80=9D= per https://tools.ietf.org/html/rfc6174#section-4.2.4.

Kind = Regards,
Dave Rice
= --Apple-Mail=_A49D2A35-B68F-4CE2-9CE7-5B0840CAD29E-- From nobody Thu Jan 4 00:14:00 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88B4012DA22 for ; Thu, 4 Jan 2018 00:13:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iLwsPnwWXlAa for ; Thu, 4 Jan 2018 00:13:57 -0800 (PST) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B23A112D850 for ; Thu, 4 Jan 2018 00:13:56 -0800 (PST) Received: by mail-wm0-x229.google.com with SMTP id n138so1856243wmg.2 for ; Thu, 04 Jan 2018 00:13:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=R8jBsLSh1HX2OQRTxjFH2ZiFDGJLctkAytpwhZFLPZ4=; b=Dv6bvjNVRQKaLzdHrGHReB59ZV5Ru3rnzECu4dQrjy/2AAknJmqIrAHfKz1XOYOXTB EpALFcfz6zHupY7+rq78llRn5OX3IPllzmoEFwYt0iFZwseWsu++VaUk1uKyxGt3+GL+ nzPLorL/YowObcZjg3DNClR/sIxdiweNfPSCeWIte3K75gHx3KTE/BeqKacApEZZEzHw PBP+ImEmEGTzDWA7vrOHmiwF+001rhDl+mk4JjDFFAoO7vZXfRTlxY4//xMRrwlubZsC A4QVf52EnCxMaWgrLBMO4PGKMrYyWcnr+YpFftWi3BMWZaA72yiqoixZQAlGHQZULHBD xz/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=R8jBsLSh1HX2OQRTxjFH2ZiFDGJLctkAytpwhZFLPZ4=; b=L7C9FGHrc3IUikTanFQ2hxqwmlI1nh+TlLhNE95duA7VNsM+3/Vfwf3mmcedAV4vM2 +F2N2wTxStQdcrrN0l+FpFx8DxekmKmKz7cu0fs+E8ZMDgod7Wxio8DWMxtvWcqmjcpn vsCVS7CZUx1czg97OSKq7dMZcT+3NIxGHHC/EYcI4ONckM7xXz9vg/Mx87G0q1GLJoiB DOuDBBHXNVlbcnC+rPHF+rgbCRE5X8GzNSYwdS88AypMM6xJAwSKqQRFmrZlr9ai1W1p Ma7wS6DzHMroxHyl2Fto8XafwyYFJSogXygtaZ1+xX4J+zqDzF5tf1LBUo0ASg+6t7aW 3GqQ== X-Gm-Message-State: AKGB3mI9eoxGzUPw6lCipLhTFb4GClLIoLti+NwWI9PmTt2QmlkjrxHc DpUVImAliYXqZDNnYRYST6D8qT46FdOxn6oiW90= X-Google-Smtp-Source: ACJfBosRpz6WVkKexUaE2F1EMalKc2fFsP8jR4mYV2/EvyTKczxlHae/+saMe6VG+wnVNFfXzvaqFt2own4WCwD17OA= X-Received: by 10.80.174.230 with SMTP id f35mr6169646edd.25.1515053635030; Thu, 04 Jan 2018 00:13:55 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.177.233 with HTTP; Thu, 4 Jan 2018 00:13:54 -0800 (PST) In-Reply-To: References: From: Austin Einter Date: Thu, 4 Jan 2018 13:43:54 +0530 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary="f403045c2154bb51fd0561eee9c8" Archived-At: Subject: [Cellar] Fwd: Understanding VP8 data in a MKV file X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 08:13:58 -0000 --f403045c2154bb51fd0561eee9c8 Content-Type: text/plain; charset="UTF-8" ---------- Forwarded message ---------- From: Austin Einter Date: Sun, Dec 31, 2017 at 10:32 PM Subject: Understanding VP8 data in a MKV file To: cellar@ietf.org I have a mkv file, that contains VP8 encoded data. I need to stream this data to chrome (through RTP), and need to play it in chrome. I have parsed the mkv file, and I have VP8 data. I will be sending this VP8 data in RTP packetc. Looks I need to follow RFC 7741 to put VP8 descriptor / header in RTP packet followed by VP8 data. Now I have several doubts, kindly try to help me to understand. 1) What is a partition in VP8 and from MKV file how do I get partition information. 2) I need to put partition size in VP8 header. From where I get the partition size. 3) How do I know partition index, this I need to put in VP8 descriptor. Does VP8 data I read from mkv file contains this. Regards Austin --f403045c2154bb51fd0561eee9c8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

---------- Forwarded messag= e ----------
From: Austin Einter <austin.einter@g= mail.com>
Date: Sun, Dec 31, 2017 at 10:32 PM
Subject: = Understanding VP8 data in a MKV file
To: cellar@ietf.org


I have a mkv file, that= contains VP8 encoded data.
I need to stream this data to chrome (throu= gh RTP), and need to play it in chrome.

I have par= sed the mkv file, and I have VP8 data.
I will be sending this VP8= data in RTP packetc.
Looks I need to follow RFC 7741 to put VP8 = descriptor / header in RTP packet followed by VP8 data.

Now I have several doubts, kindly try to help me to understand.
=

1) What is a partition in VP8 and from MKV file how do = I get partition information.
2) I need to put partition size in V= P8 header. From where I get the partition size.
3) How do I know = partition index, this I need to put in VP8 descriptor.

=
Does VP8 data I read from mkv file contains this.

=
Regards
Austi= n


--f403045c2154bb51fd0561eee9c8-- From nobody Thu Jan 4 03:55:54 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 610C5124B18 for ; Thu, 4 Jan 2018 03:55:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nvpVtaA4hwLd for ; Thu, 4 Jan 2018 03:55:48 -0800 (PST) Received: from 4.mo177.mail-out.ovh.net (4.mo177.mail-out.ovh.net [46.105.37.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D1F8F1200C1 for ; Thu, 4 Jan 2018 03:55:47 -0800 (PST) Received: from player688.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo177.mail-out.ovh.net (Postfix) with ESMTP id A14D594F14 for ; Thu, 4 Jan 2018 12:55:45 +0100 (CET) Received: from [192.168.2.120] (p5DDB4B21.dip0.t-ipconnect.de [93.219.75.33]) (Authenticated sender: jerome@mediaarea.net) by player688.ha.ovh.net (Postfix) with ESMTPSA id 35C4C200AD for ; Thu, 4 Jan 2018 12:55:45 +0100 (CET) From: Jerome Martinez References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> To: Codec Encoding for LossLess Archiving and Realtime transmission Message-ID: Date: Thu, 4 Jan 2018 12:55:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Ovh-Tracer-Id: 1603562945629130897 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrjeehgdefhecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 11:55:52 -0000 I added some Pull Requests on https://github.com/FFmpeg/FFV1 in order to fix some of the issues raised in this very useful post, comments inline: On 28/07/2017 16:20, Derek Buitenhuis wrote: > Hi all, > > Dave Rice pointed me at draft-ietf-cellar-ffv1-00 and draft-ietf-cellar-ebml-03, > and these are my thoughts upon a cursory look at the former. > > I'm new to this list / IETF stuff in general, so some of my comments may be moot > or a bit fuzzy. > > Comments follow inline below. > >> 2.2.9.1. remaining_bits_in_bitstream >> >> "remaining_bits_in_bitstream( )" means the count of remaining bits >> after the current position in that bitstream component. It is >> computed from the NumBytes value multiplied by 8 minus the count of >> bits of that component already read by the bitstream parser. > A bit confusing that this is named 'remaining_bits_in_bitstream', but in > reality it is 'remaining_bits_in_bitstream_component', not the whole > bitstream. I prefer "block" instead of "bitstream_component" (too long IMO) PR: https://github.com/FFmpeg/FFV1/pull/87 >> o one column of samples to the left of the coded slice is assumed as >> identical to the samples of the leftmost column of the coded slice >> shifted down by one row > This doesn't specify what the top sample value should be, once the > column has been shifted down by one. Based on the diagram below, it > should be 0. PR: https://github.com/FFmpeg/FFV1/pull/88 >> left16s = l >= 32768 ? ( l - 65536 ) : l >> top16s = t >= 32768 ? ( t - 65536 ) : t >> diag16s = tl >= 32768 ? ( tl - 65536 ) : tl > Does the IETF specs generally define when arithmetic is signed/unsigned, or > do they defer to the C spec on integer promotion, etc.? We use the word "integer" everywhere, IMO it is meaning https://en.wikipedia.org/wiki/Integer and not related to C. Not sure it is worth it to describe integers as "Z". >> 3.7.2. JPEG2000-RCT > It's kind of a bit odd that this is titled like this, considering > section 3.7 is titled 'Color space' and JPEG2000-RCT is not a > color space, but a transform used to store RGB(A). I modified 'Color space' to "pixel transformation", with more details about ho RGB content in encoded in FFV1. PR: https://github.com/FFmpeg/FFV1/pull/89 >> 3.8.2. Huffman coding mode >> >> This coding mode uses Golomb Rice codes. The VLC code is split into > Redundant 'code'. Variable Length Code code. I also change more things in order to use "Golomb Rice" everywhere. PR: https://github.com/FFmpeg/FFV1/pull/90 >> if (run_count == 0 && run_mode == 1) { | >> if (get_bits1()) { | >> run_count = 1 << log2_run[run_index]; | >> if (x + run_count <= w) | >> run_index++; | >> } else { | >> if (log2_run[run_index]) | >> run_count = get_bits(log2_run[run_index]); | > get_bits1() and get_bits() first appear here, with no definition. > Should they not be defined in '2.2.9. Bitstream functions'? this is a part I did not yet work on, anyway let's add the definition now so we don't forget. PR: https://github.com/FFmpeg/FFV1/pull/91 >> "frame_pixel_width" is defined as frame width in pixels. >> >> "frame_pixel_height" is defined as frame height in pixels. > The document, up until this point, and after this point, always refers to > samples. Why is it suddenly pixels? Looking carefully at pixel vs sample in the doc, there are sometimes inversion between the meaning of each word. I changed them and added a definition. PR: https://github.com/FFmpeg/FFV1/pull/92 >> pseudo-code | type >> --------------------------------------------------------------|----- >> ConfigurationRecord( NumBytes ) { | >> ConfigurationRecordIsPresent = 1 | >> Parameters( ) | >> while( remaining_bits_in_bitstream( NumBytes ) > 32 ) | >> reserved_for_future_use | u(1) >> configuration_record_crc_parity | u(32) >> } > Style changes completely half way through the spec, and also adds > types to the types column. It's pretty obvious the earlier pseudo-code > was pasted directly from libavcodec, but this code probably isn't > written by Michael at all. Very different stuff; e.g. the previous > code has variable types in the psuedo-code, but this does not. > Is this specific to bitstream descriptions? As you say, one is a copy/paste, and I did not yet touch to the copy/paste. I plan to make it less dependent of C (i.e. without variable types, as it is expected to b C-like code but not with limitations of C about types) when I work on these parts. > >> 4.1.3.1. In AVI File Format > Poor English. Perhaps remove the "In". Ditto for others. Dave did a PR for it (already merged). >> The Configuration Record extends the sample description box ("moov", >> "trak", "mdia", "minf", "stbl", "stsd") with a "glbl" box which >> contains the ConfigurationRecord bitstream. See [ISO.14496-12.2015] >> for more information about boxes. > Where is 'glbl' defined? If it's only this spec, then it conflicts with > existing custom boxes for e.g.: > > https://wiki.xiph.org/Oggless#Example_and_Discussion_of_the_MOV_container > > Not sure if this ever actually existed in practice or not, but it's > something. I am not sure the 'glbl' atom should be more defined here, as it is generic. Part of another discussion in the thread, I let it as is for the moment. > >> FFV1 SHOULD use "V_FFV1" as the Matroska "Codec ID". For FFV1 >> versions 2 or less, the Matroska "CodecPrivate" Element SHOULD NOT be >> used. For FFV1 versions 3 or greater, the Matroska "CodecPrivate" >> Element MUST contain the FFV1 Configuration Record structure and no >> other data. See [Matroska] for more information about elements. > Why only SHOULD? From IETF definitions: "SHOULD   This word, or the adjective "RECOMMENDED", mean that there    may exist valid reasons in particular circumstances to ignore a    particular item, but the full implications must be understood and    carefully weighed before choosing a different course." There may be good reasons for using another identifier (e.g "V_MS/VFW/FOURCC", as this one is the only supported by old version of FFmpeg, so for compatibility reasons). To be debated, but I am not in favor of MUST. > >> +-------+----------------------------+ >> | value | slice coding mode | >> +-------+----------------------------+ >> | 0 | normal Range Coding or VLC | >> | 1 | raw PCM | >> | Other | reserved for future use | >> +-------+----------------------------+ > This is the first and only time 'PCM' appears in this document. Should > it be defined, at least, or have a reference? this is from an early draft of v4 bitstream, not part of the current work. I suggest to completely remove this part for the moment. PR: https://github.com/FFmpeg/FFV1/pull/93 > >> pseudo-code | type >> --------------------------------------------------------------|----- >> Line( p, y ) { | >> if (colorspace_type == 0) { | >> for( x = 0; x < plane_pixel_width[ p ]; x++ ) | >> Pixel( p, y, x ) | >> } else if (colorspace_type == 1) { | >> for( x = 0; x < slice_pixel_width; x++ ) | >> Pixel( p, y, x ) | >> } | >> } | > This is the first and only time Pixel() appears in the document, and > without explanation. Work in progress of the document. > >> 4.8.5. colorspace_type >> >> "colorspace_type" specifies the color space. >> >> +-------+-------------------------+ >> | value | color space used | >> +-------+-------------------------+ >> | 0 | YCbCr | >> | 1 | JPEG2000-RCT | >> | Other | reserved for future use | >> +-------+-------------------------+ > Same comment as before: JPEG2000-RCT is not a color space. Same answer as before :). > >> 4.8.8. h_chroma_subsample >> >> "h_chroma_subsample" indicates the subsample factor between luma and >> chroma width ("chroma_width = 2^(-log2_h_chroma_subsample) * >> luma_width"). >> >> 4.8.9. v_chroma_subsample >> >> "v_chroma_subsample" indicates the subsample factor between luma and >> chroma height ("chroma_height=2^(-log2_v_chroma_subsample) * >> luma_height"). > I get that the spec is trying to be mathematically correct, but why > can't this contain the method that literally every single implementation > will ever use: bitshifting by {v,h}_chroma_subsample. It's even a > defined operator in "2.2.1. Arithmetic operators", and is used in > e.g. the JPEG2000-RCT equations. > > Further, the name 'log2_h_chroma_subsample', in the definition, is > incorrect. Was in the former draft, and in code if I remember well. Right, I see some issue with having both {v,h}_chroma_subsample and log2_{v,h}_chroma_subsample in the spec. We are not bind to any external code, so we could change that. I switched all to log2_{v,h}_chroma_subsample. PR: https://github.com/FFmpeg/FFV1/pull/94 *** Lot of Pull Requests in a batch, don't hesitate to comment. Jérôme From nobody Thu Jan 4 04:14:27 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C94B5126DC2 for ; Thu, 4 Jan 2018 04:14:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UhfU3t2uJRmG for ; Thu, 4 Jan 2018 04:14:23 -0800 (PST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:c:538::194]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 51AC0124319 for ; Thu, 4 Jan 2018 04:14:23 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id A8D89C5A6F for ; Thu, 4 Jan 2018 13:14:20 +0100 (CET) Date: Thu, 4 Jan 2018 13:14:20 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180104121420.GO4938@michaelspb> References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nljfjKcp9HDtPSOP" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 12:14:26 -0000 --nljfjKcp9HDtPSOP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 04, 2018 at 12:55:45PM +0100, Jerome Martinez wrote: > I added some Pull Requests on https://github.com/FFmpeg/FFV1 in order to = fix > some of the issues raised in this very useful post, comments inline: >=20 > On 28/07/2017 16:20, Derek Buitenhuis wrote: > >Hi all, > > > >Dave Rice pointed me at draft-ietf-cellar-ffv1-00 and draft-ietf-cellar-= ebml-03, > >and these are my thoughts upon a cursory look at the former. > > > >I'm new to this list / IETF stuff in general, so some of my comments may= be moot > >or a bit fuzzy. > > > >Comments follow inline below. > > > >>2.2.9.1. remaining_bits_in_bitstream > >> > >> "remaining_bits_in_bitstream( )" means the count of remaining bits > >> after the current position in that bitstream component. It is > >> computed from the NumBytes value multiplied by 8 minus the count of > >> bits of that component already read by the bitstream parser. > >A bit confusing that this is named 'remaining_bits_in_bitstream', but in > >reality it is 'remaining_bits_in_bitstream_component', not the whole > >bitstream. >=20 > I prefer "block" instead of "bitstream_component" (too long IMO) > PR: https://github.com/FFmpeg/FFV1/pull/87 Block in many video and image codecs generally refers to a=20 rectgangular spatial area of pixels. Macro block would refer to a larger square made of blocks. We should not redefine commonly used terms. That could confuse people who have worked with other video codecs [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Modern terrorism, a quick summary: Need oil, start war with country that has oil, kill hundread thousand in war. Let country fall into chaos, be surprised about raise of fundamantalists. Drop more bombs, kill more people, be surprised about them taking revenge and drop even more bombs and strip your own citizens of their rights and freedoms. to be continued --nljfjKcp9HDtPSOP Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpOGpwACgkQYR7HhwQLD6tbvwCfb4IGkZ+sx838MjddQiQ7PzBg 3TwAnjaqJbLqveOOUDQ3T6mfEaq/3Pma =vG+t -----END PGP SIGNATURE----- --nljfjKcp9HDtPSOP-- From nobody Thu Jan 4 04:53:10 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A393212702E for ; Thu, 4 Jan 2018 04:53:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id emfPYuz0g8ED for ; Thu, 4 Jan 2018 04:53:07 -0800 (PST) Received: from 8.mo177.mail-out.ovh.net (8.mo177.mail-out.ovh.net [46.105.61.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8B5A51200FC for ; Thu, 4 Jan 2018 04:53:07 -0800 (PST) Received: from player688.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo177.mail-out.ovh.net (Postfix) with ESMTP id 31F6F94DD6 for ; Thu, 4 Jan 2018 13:53:05 +0100 (CET) Received: from [192.168.2.120] (p5DDB4B21.dip0.t-ipconnect.de [93.219.75.33]) (Authenticated sender: jerome@mediaarea.net) by player688.ha.ovh.net (Postfix) with ESMTPSA id DEA31200A2 for ; Thu, 4 Jan 2018 13:53:04 +0100 (CET) To: cellar@ietf.org References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> From: Jerome Martinez Message-ID: <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> Date: Thu, 4 Jan 2018 13:53:04 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20180104121420.GO4938@michaelspb> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 2571836864281251985 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrjeehgdegiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 12:53:09 -0000 On 04/01/2018 13:14, Michael Niedermayer wrote: > Block in many video and image codecs generally refers to a > rectgangular spatial area of pixels. > Macro block would refer to a larger square made of blocks. > > We should not redefine commonly used terms. That could confuse people who > have worked with other video codecs I reused the "Block" term from Matroska ( https://www.matroska.org/technical/specs/index.html#block_structure ) in order not to reinvent the wheel and this word is not used in FFV1, but I fully understand that it may be misleading in the video coding world and I have no opinion about which word to use, I just need a different word because "bitstream" is misleading as reported. Please suggest an alternative. Jrme From nobody Thu Jan 4 07:34:59 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 33ADB1267BB for ; Thu, 4 Jan 2018 07:34:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fEkOT8LpvStY for ; Thu, 4 Jan 2018 07:34:56 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8D0C7128D2E for ; Thu, 4 Jan 2018 07:34:56 -0800 (PST) Received: from cpe-104-162-94-162.nyc.res.rr.com ([104.162.94.162]:39831 helo=[10.0.1.9]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eX7Xh-0022ym-ER; Thu, 04 Jan 2018 10:34:54 -0500 From: Dave Rice Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_9952B4B0-0A18-45AE-83BB-AE0872771E00" Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Date: Thu, 4 Jan 2018 10:34:51 -0500 In-Reply-To: <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> Cc: cellar@ietf.org To: Jerome Martinez References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> X-Mailer: Apple Mail (2.3445.4.7) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 15:34:58 -0000 --Apple-Mail=_9952B4B0-0A18-45AE-83BB-AE0872771E00 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Jan 4, 2018, at 7:53 AM, Jerome Martinez = wrote: >=20 > On 04/01/2018 13:14, Michael Niedermayer wrote: >> Block in many video and image codecs generally refers to a >> rectgangular spatial area of pixels. >> Macro block would refer to a larger square made of blocks. >>=20 >> We should not redefine commonly used terms. That could confuse people = who >> have worked with other video codecs >=20 > I reused the "Block" term from Matroska ( = https://www.matroska.org/technical/specs/index.html#block_structure ) in = order not to reinvent the wheel and this word is not used in FFV1, but I = fully understand that it may be misleading in the video coding world and = I have no opinion about which word to use, I just need a different word = because "bitstream" is misleading as reported. Please suggest an = alternative. I added several reviews within the active pull requests at = https://github.com/FFmpeg/FFV1/pulls = . Overall these changes help with = the document a lot. Thanks to Derek and Jerome. I agree with Michael that the term Block may not be best. The NumBytes = definition already uses `FFV1 components`. Could also consider packet, = segment, or IMHO `bitstream component` was fine though long. Kind Regards, Dave Rice --Apple-Mail=_9952B4B0-0A18-45AE-83BB-AE0872771E00 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
On = Jan 4, 2018, at 7:53 AM, Jerome Martinez <jerome@mediaarea.net> wrote:

On = 04/01/2018 13:14, Michael Niedermayer wrote:
Block in many video and image codecs generally = refers to a
rectgangular spatial area of pixels.
Macro block would refer to a larger square made of blocks.

We should not redefine commonly used terms. = That could confuse people who
have worked with other video = codecs

I reused the "Block" = term from Matroska ( https://www.matroska.org/technical/specs/index.html#block_struc= ture ) in order not to reinvent the wheel and this word is not used = in FFV1, but I fully understand that it may be misleading in the video = coding world and I have no opinion about which word to use, I just need = a different word because "bitstream" is misleading as reported. Please = suggest an alternative.

I added several reviews within the active = pull requests at https://github.com/FFmpeg/FFV1/pulls. Overall these = changes help with the document a lot. Thanks to Derek and = Jerome.

I = agree with Michael that the term Block may not be best. The NumBytes = definition already uses `FFV1 components`. Could also consider packet, = segment, or IMHO `bitstream component` was fine though long.

Kind Regards,
Dave Rice

= --Apple-Mail=_9952B4B0-0A18-45AE-83BB-AE0872771E00-- From nobody Thu Jan 4 12:23:20 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 678B512D885 for ; Thu, 4 Jan 2018 12:23:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JGd_Md-a-4oB for ; Thu, 4 Jan 2018 12:23:16 -0800 (PST) Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DC39312D87A for ; Thu, 4 Jan 2018 12:23:15 -0800 (PST) Received: by mail-qt0-x22f.google.com with SMTP id f2so3350350qtj.4 for ; Thu, 04 Jan 2018 12:23:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6AUGcwzrttwmf6xAcnRHG6rfFbXmR09qS8khiu5UGK4=; b=ONRYhPHIVHbNAJpk9eqrPKhdlWk708TFTE7NFs4SKJcZcdapDJ8m7tAae8gVjAI7GC HhXpllz5jUX9Nqo4bESdVjUHZoGlQuM0SzdgchIKY5pMP4oodHKgfQT0yoXVI5I9q0u5 nZzxptuPvJkcG6fra8saENbDbGR/3aCUNSGgT5c+tW0BKkVMkmPTf6a+WKpnrlzU9+8S i3Geebb1E/dkB/kodbXmTGdNEAL2B7Q/Qyp9wCMtOAk4Z4eHz0xfnCT9W4XyyDj+wlIv HQ64EEijVuEXS/KW+mnAD1HSRfMqWHeoCjd91DEIIRARvj18+iF4nERGo36bF5/cmNby WK/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6AUGcwzrttwmf6xAcnRHG6rfFbXmR09qS8khiu5UGK4=; b=eQ+8+2l8/xn05OV7SbzmfUb3NXXkGx8qbgA7RBkmYiHzycT45uQYHY16FbG0OH9Ws8 m7L2dqE8eKDkDe3IUuLnZ0r+N/zI9GaSOEsvICf+yPVObA3HEY8fvAJ68/X0lFNMC1J5 7s75lv8LmbVUWqqGrGWR49aEeQYL0BytJkNu/q5RkBBLeeuau2EgSIdC43PSxSI7+nh2 jPvBXTfHeiX86vcJ97uNxeBHYc2s0VGyXHlY0b8uGGku/wgC+mTU8PJYDiorkzz6M2Up ypUEJnYe3zLu+9MnVPWsK/gVSjPax+SeK0lJW8LuotIlKiqiQpKyHja/9a6efdW1TpXh brRg== X-Gm-Message-State: AKwxytepWmph4FgrahC0xtC8fBiHV1F6/0GYDKAHCTtHrwpx/cc0ZJXJ 4zTu8WE2t1H9BZKeTPhoDsYp9VDhGEDnPzGGl4tXlQ== X-Google-Smtp-Source: ACJfBosRzE+sSBh6hayN1fikyNjp52M33cKLNy/K/t5Jfh0ukr0unZ4DzvCNKxmSzhsINz+BbQBHnMvfeYVK/PIyuSo= X-Received: by 10.200.46.196 with SMTP id i4mr1160371qta.179.1515097394911; Thu, 04 Jan 2018 12:23:14 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.178 with HTTP; Thu, 4 Jan 2018 12:23:14 -0800 (PST) In-Reply-To: <006AF262-FA38-4297-8377-121DD76A672F@dericed.com> References: <006AF262-FA38-4297-8377-121DD76A672F@dericed.com> From: Ashley Blewer Date: Thu, 4 Jan 2018 15:23:14 -0500 Message-ID: To: Dave Rice Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a11413dd8061c7f0561f91adb" Archived-At: Subject: Re: [Cellar] Updates to Matroska documents X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 20:23:18 -0000 --001a11413dd8061c7f0561f91adb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I am in favor of this motion to move the Active Internet Drafts to WG Documents. "+1" Ashley On Wed, Jan 3, 2018 at 11:34 AM, Dave Rice wrote: > Hi cellar, > > I=E2=80=99ve updated the Matroska specification in the tracker to version= 04. This > includes all commits from July 2, 2017 up to now, see https://github.com/ > Matroska-Org/matroska-specification/commits/draft- > lhomme-cellar-matroska-04 for those commits. Notably the new document > versions split the specification into three documents as the metadata tag > definitions and the codec mappings are now separated from the main Matros= ka > document (as discussed at the IETF99 meeting). See: > > https://datatracker.ietf.org/doc/draft-lhomme-cellar-matroska/ > https://datatracker.ietf.org/doc/draft-lhomme-cellar-codec/ > https://datatracker.ietf.org/doc/draft-lhomme-cellar-tags/ > > These documents have the status of =E2=80=9CActive Internet Drafts=E2=80= =9D. I propose > assessing consensus to change these to =E2=80=9CWG Document=E2=80=9D per > https://tools.ietf.org/html/rfc6174#section-4.2.4. > > Kind Regards, > Dave Rice > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a11413dd8061c7f0561f91adb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I am in favor of this motion to move the Active Inter= net Drafts to WG Documents.

"+1"

Ashley

On Wed, Jan 3= , 2018 at 11:34 AM, Dave Rice <dave@dericed.com> wrote:
Hi cellar,=

I=E2=80=99ve updated the Matroska specification in the = tracker to version 04. This includes all commits from July 2, 2017 up to no= w, see=C2=A0https://githu= b.com/Matroska-Org/matroska-specification/commits/draft-lhom= me-cellar-matroska-04=C2=A0for those commits. Notably the new document = versions split the specification into three documents as the metadata tag d= efinitions and the codec mappings are now separated from the main Matroska = document (as discussed at the IETF99 meeting). See:

https://datatracker.ietf.org/doc/draft-lhomme-cell= ar-matroska/

These documents have the status of =E2=80=9CActive Internet Drafts= =E2=80=9D. I propose assessing consensus to change these to =E2=80=9CWG Doc= ument=E2=80=9D per=C2=A0https://tools.ietf.org/html/rfc6174#secti= on-4.2.4.

Kind Regards,
Dave Rice

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a11413dd8061c7f0561f91adb-- From nobody Thu Jan 4 12:44:53 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8F9421241F5 for ; Thu, 4 Jan 2018 12:44:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V_Ry1hpXPJ5O for ; Thu, 4 Jan 2018 12:44:49 -0800 (PST) Received: from mail-qt0-x22e.google.com (mail-qt0-x22e.google.com [IPv6:2607:f8b0:400d:c0d::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 23FD31200B9 for ; Thu, 4 Jan 2018 12:44:49 -0800 (PST) Received: by mail-qt0-x22e.google.com with SMTP id m59so3410965qte.11 for ; Thu, 04 Jan 2018 12:44:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=k5OaHO+dOMXCCMTTx/53PY9BUXWMA4w9eFSggafdTuM=; b=dUNymeeOgB8WS8xLcBT59M657AULkwvaJyIipOh2Rq583jp0DPYTDke+xC33EUbSNO WVAvEB0+LKroLRwPWXf6rLS203n2UBmVLD4IvAJvSg49weP2uSZK4EFj4ZfwqYM03iBz kMxXIjqlkjUrX2flTg632cd2xAyzdl8bNbGZTreVFR7o7rjyV1aUguuVU8iKE9OSWs53 wIaFVZsbt+e63SepmhKZVtreh8kjjvWOjwTx0dTHMYwQ1+Zuq5z5mJET0k4G1hxxt44M nLHTKJbRzW8e/IOBrJMl3NLDVAGGWawP1/okXEZiHiOLjFmTAAjcDIAyPLbz6bCQZGcF sPpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=k5OaHO+dOMXCCMTTx/53PY9BUXWMA4w9eFSggafdTuM=; b=T66pUaM1AVG4FwTRjE20Xbpdu72H8aeDNG2PVvhR6w/Sce5HQYRzC+GkF2Ou1IVM0M 8RbqdQQxZKOr9hSUg4Jszn/ZZSVsps6+rtKC7USe8deiqcZlfwdQsKb4e+o2JNgcWjj6 dGX/GOJAVJoTcMlYOTKjTXzFY0CmjCkI8VDfS6DmdsuDD7TclV2T26a/axeLLwwCU6Pz 5WJRpL0GImaYTw59VZxO0cfIv04LoKc15GaJIP8ORLfh9chA4Q85yhDAba4kwODSHl/L vcJucrR2D+3gsuRIges3O+MRDAFrm2mOAp48MlWjjsvJvFDgHwtWXXPCT0jJIwhPIrhL 6zSw== X-Gm-Message-State: AKwxytfwAJ+FUoP6+uDYXM4IdVoNYuk6mTVlrf46g+B6SOGCOpvi8DHJ YNO7naDoWvz0jnyvtxjuSkOcYXeAsYSvavkE0vKcnMSU X-Google-Smtp-Source: ACJfBotLk9NPkiWGIjqR+1njDlQSJQjjzo1KWggwIVIQtnOTjkjeiJ4GvZ7phOjFJJE8stNZHX4IdVWzOAzp5XK2kwk= X-Received: by 10.200.57.129 with SMTP id v1mr1233126qte.50.1515098688267; Thu, 04 Jan 2018 12:44:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.178 with HTTP; Thu, 4 Jan 2018 12:44:47 -0800 (PST) In-Reply-To: References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> From: Ashley Blewer Date: Thu, 4 Jan 2018 15:44:47 -0500 Message-ID: To: Dave Rice Cc: Jerome Martinez , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="94eb2c0ca6001d2c020561f9671e" Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 20:44:51 -0000 --94eb2c0ca6001d2c020561f9671e Content-Type: text/plain; charset="UTF-8" Thanks Jerome! I brought up a related issue in https://github.com/FFmpeg/FFV1/pull/92 about use of luma vs luminance and am noting it here, as it is a separate issue from the PR at hand, which I will now approve (but still think changes should be made). Ashley On Thu, Jan 4, 2018 at 10:34 AM, Dave Rice wrote: > > On Jan 4, 2018, at 7:53 AM, Jerome Martinez wrote: > > On 04/01/2018 13:14, Michael Niedermayer wrote: > > Block in many video and image codecs generally refers to a > rectgangular spatial area of pixels. > Macro block would refer to a larger square made of blocks. > > We should not redefine commonly used terms. That could confuse people who > have worked with other video codecs > > > I reused the "Block" term from Matroska ( https://www.matroska.org/ > technical/specs/index.html#block_structure ) in order not to reinvent the > wheel and this word is not used in FFV1, but I fully understand that it may > be misleading in the video coding world and I have no opinion about which > word to use, I just need a different word because "bitstream" is misleading > as reported. Please suggest an alternative. > > > I added several reviews within the active pull requests at > https://github.com/FFmpeg/FFV1/pulls. Overall these changes help with the > document a lot. Thanks to Derek and Jerome. > > I agree with Michael that the term Block may not be best. The NumBytes > definition already uses `FFV1 components`. Could also consider packet, > segment, or IMHO `bitstream component` was fine though long. > > Kind Regards, > Dave Rice > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --94eb2c0ca6001d2c020561f9671e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Jerome!

I brought up a relat= ed issue in https://gith= ub.com/FFmpeg/FFV1/pull/92 about use of luma vs luminance and am noting= it here, as it is a separate issue from the PR at hand, which I will now a= pprove (but still think changes should be made).

Ashley

On Thu, Jan 4,= 2018 at 10:34 AM, Dave Rice <dave@dericed.com> wrote:

= On Jan 4, 2018, at 7:53 AM, Jerome Martinez <jerome@mediaarea.net> wrote:
On 04/= 01/2018 13:14, Michael Niedermayer wrote:
Bloc= k in many video and image codecs generally refers to a
rectgangular spat= ial area of pixels.
Macro block would refer to a larger square made of b= locks.

We should not redefine commonly used terms. That could confus= e people who
have worked with other video codecs

I r= eused the "Block" term from Matroska ( htt= ps://www.matroska.org/technical/specs/index.html#block_structure<= /a> ) in order not to reinvent the wheel and this word is not used in FFV1,= but I fully understand that it may be misleading in the video coding world= and I have no opinion about which word to use, I just need a different wor= d because "bitstream" is misleading as reported. Please suggest a= n alternative.

I added se= veral reviews within the active pull requests at=C2=A0https://github.com/FFmpeg/FFV1/pulls. Overall these changes help with the document a lot. Thank= s to Derek and Jerome.

I agree with Michael that t= he term Block may not be best. The NumBytes definition already uses `FFV1 c= omponents`. Could also consider packet, segment, or IMHO `bitstream compone= nt` was fine though long.

Kind Regards,
= Dave Rice


______________________________= _________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--94eb2c0ca6001d2c020561f9671e-- From nobody Thu Jan 4 12:59:08 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9928612773A for ; Thu, 4 Jan 2018 12:59:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mPUF108HaKDM for ; Thu, 4 Jan 2018 12:59:03 -0800 (PST) Received: from mail-wr0-x230.google.com (mail-wr0-x230.google.com [IPv6:2a00:1450:400c:c0c::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4C59D1200B9 for ; Thu, 4 Jan 2018 12:59:03 -0800 (PST) Received: by mail-wr0-x230.google.com with SMTP id o15so2654765wrf.12 for ; Thu, 04 Jan 2018 12:59:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dWgJG2eWbdVEaA3ETsbV2AE4AIzNiLaasIT0dnqM2y8=; b=fDUiEk3/e7oHEjSpsLC6xsBrXt+W8yKdGOv0Xw6crnv4AJPD/hRjN5YRRIVYxYR4uF OhBsguP21C8sOrs34KfBSWUO8yLElzR39JDsJVA0e1Oe8YsCXDd4jDqbHJJdIcb8NPbg Kr18X7uQgvkbIGgMNOziCGXn9a2xRXI/OLZarV676hPAcDCJXFxTfZvY89x+7m/CR6rc GZWnhDvtm8/6MU4uF9EldEon+qWvwatqgEjRSRpa/3GMa9AEVbzZPE/62o0VQsJXhXiV sIcQJrlWx7E3WUUCI01NjZbRkR44jfCZmOYVmtWcpTo7FIrwOCWRhMjZa2xzVDqQ7a9w g0AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dWgJG2eWbdVEaA3ETsbV2AE4AIzNiLaasIT0dnqM2y8=; b=mf4DJhC0w+tHhkhu71sH4/veETug8q1Ouhmhn9Y7/45a2kk5mkjUUqpOnhKw2AJQwk 5usLuae7mEauVFpuPeJ1tttyE5G9kpCVPb4AsWJGVgS1eLiDrUFPNHSKWzfjyftobu4B JsyPCj7muwsP647/POw45xwg/eaM6mVthr14I9rjHNfS32u5NVcCrbQeyFXKsIqLlPHx 1XFzEHoBkfSwd3+HwAbGV1fpRyekF4vyjRkm4P40mTG2f+RIZbMyZ7gv7xVvaWEFwRLD dtTb6qa29TadeJ5AIDkIRbZU5AK7ysYAIT7ZpkQHaNwkZOMW5V1vbsiQyT3zL++f5I+a AAiA== X-Gm-Message-State: AKGB3mLHegkZLBoDN6d7Z+1yYMHrnYAY0k1wbJWdXR+1ZcpdoPgBKSnj 2fyKBmEynFO8kjIZuM7Z4FtY4/Brz4AWkDUVlts= X-Google-Smtp-Source: ACJfBov0MpWZMHQ8u9hagMEeRC2x87PBaXsBoUTmTct4/fcYvyjT8WWFxC92EN94phyeVZ5plcqk6PUPO0Q8jQ56OPc= X-Received: by 10.223.136.210 with SMTP id g18mr707553wrg.223.1515099541637; Thu, 04 Jan 2018 12:59:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.151.37 with HTTP; Thu, 4 Jan 2018 12:59:00 -0800 (PST) In-Reply-To: References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> From: Katherine Frances Date: Fri, 5 Jan 2018 09:59:00 +1300 Message-ID: To: Ashley Blewer Cc: Dave Rice , Jerome Martinez , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a114605fcfa90080561f999e5" Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 20:59:06 -0000 --001a114605fcfa90080561f999e5 Content-Type: text/plain; charset="UTF-8" Luma v. luminance - good catch, Ash! Charles Poynton wrote a great article about that. On Fri, Jan 5, 2018 at 9:44 AM, Ashley Blewer wrote: > Thanks Jerome! > > I brought up a related issue in https://github.com/FFmpeg/FFV1/pull/92 > about use of luma vs luminance and am noting it here, as it is a separate > issue from the PR at hand, which I will now approve (but still think > changes should be made). > > Ashley > > On Thu, Jan 4, 2018 at 10:34 AM, Dave Rice wrote: > >> >> On Jan 4, 2018, at 7:53 AM, Jerome Martinez wrote: >> >> On 04/01/2018 13:14, Michael Niedermayer wrote: >> >> Block in many video and image codecs generally refers to a >> rectgangular spatial area of pixels. >> Macro block would refer to a larger square made of blocks. >> >> We should not redefine commonly used terms. That could confuse people who >> have worked with other video codecs >> >> >> I reused the "Block" term from Matroska ( https://www.matroska.org/techn >> ical/specs/index.html#block_structure ) in order not to reinvent the >> wheel and this word is not used in FFV1, but I fully understand that it may >> be misleading in the video coding world and I have no opinion about which >> word to use, I just need a different word because "bitstream" is misleading >> as reported. Please suggest an alternative. >> >> >> I added several reviews within the active pull requests at >> https://github.com/FFmpeg/FFV1/pulls. Overall these changes help with >> the document a lot. Thanks to Derek and Jerome. >> >> I agree with Michael that the term Block may not be best. The NumBytes >> definition already uses `FFV1 components`. Could also consider packet, >> segment, or IMHO `bitstream component` was fine though long. >> >> Kind Regards, >> Dave Rice >> >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> >> > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a114605fcfa90080561f999e5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Luma v. luminance - good catch, Ash! Charles Poynton wrote= a great article about that.

On Fri, Jan 5, 2018 at 9:44 AM, Ashley Blewer <ashl= ey.blewer@gmail.com> wrote:
Thanks Jerome!

I brought up a rela= ted issue in https://github.com/FFmpeg/FFV1/pull/92 about use of luma vs= luminance and am noting it here, as it is a separate issue from the PR at = hand, which I will now approve (but still think changes should be made).

Ashley
<= div class=3D"gmail_extra">
On Thu, Jan 4, 2018 at 10:34 AM, Dave Rice <dave@dericed.com> wrote:

On Jan 4, 2018, at 7:53 AM, Jer= ome Martinez <= jerome@mediaarea.net> wrote:

On 04/01/2018 13= :14, Michael Niedermayer wrote:
Block in many = video and image codecs generally refers to a
rectgangular spatial area o= f pixels.
Macro block would refer to a larger square made of blocks.
=
We should not redefine commonly used terms. That could confuse people w= ho
have worked with other video codecs

I reused the = "Block" term from Matroska ( https://www.m= atroska.org/technical/specs/index.html#block_structure ) in o= rder not to reinvent the wheel and this word is not used in FFV1, but I ful= ly understand that it may be misleading in the video coding world and I hav= e no opinion about which word to use, I just need a different word because = "bitstream" is misleading as reported. Please suggest an alternat= ive.

I added several revi= ews within the active pull requests at=C2=A0https://github.com/FFmpeg/FFV1/pul= ls. Overall these changes help with the document a lot. Thanks to Derek= and Jerome.

I agree with Michael that the term Bl= ock may not be best. The NumBytes definition already uses `FFV1 components`= . Could also consider packet, segment, or IMHO `bitstream component` was fi= ne though long.

Kind Regards,
Dave Rice<= /div>


________________= _______________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar


_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a114605fcfa90080561f999e5-- From nobody Thu Jan 4 12:59:25 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 58BEA1200B9 for ; Thu, 4 Jan 2018 12:59:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vwtJe2S0xgx6 for ; Thu, 4 Jan 2018 12:59:21 -0800 (PST) Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 32A8312741D for ; Thu, 4 Jan 2018 12:59:21 -0800 (PST) Received: by mail-qt0-x232.google.com with SMTP id r39so3450887qtr.13 for ; Thu, 04 Jan 2018 12:59:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=26iTWnQjylkbXfm+9b+s28m8qVCKF4TV5PO0CDxMqOM=; b=QPcnrDjc6GSsND5nulLcUs6+MBnZmvjGSBrCwuTv8EWzQxvKkCB2mPXwYq9FrGtTZ4 GMInKm+2+vq4ocoYUHcxUieHxzDEC+th+dTrx5XP4vzBqmEUssWh0TNtsS570dGcFlWb I8QAM/YEWyxIwSO0WQEKUHnCnOaiTE0nv6uVNhK1sRiw3DT0mWuAmOuVSZ7nXmF/lcfs P7ljsMtDC0w6yfyTsTCSpXgVZuhbSXkMQTlAPfdDBhAHujUFm4yYZEvUyTsTKFA67Hzi d2ZkjyQdzLHeaFGt1IM7uT/jH90AxmHws8qbC4fwAzj46p8f3vyAYeRPHZZ4QH5/Uaxj jZvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=26iTWnQjylkbXfm+9b+s28m8qVCKF4TV5PO0CDxMqOM=; b=WZ+UA+ZDCV/FdOnTlLtF0/MhZFoWwspM1JcYBkia2sb/sCMBKS5mHiQzYEGysSB5JQ 70rfEzfphQ2pZKtNw52Uz2RkSGCdgleU4VYjoYcFpPP02e+C1srFfA5czLSm6sAGFNxy M9hiaTxYZa9Mg7/BMN+xuUBGkECwHrhkWef2yl+R9pebOxQ+z3sN82+4Zn0/dhUPikx/ D3s7Bh7fpgLJnsWYbx20HHoQ5mCNne06ZOJ9ycH0vLGiJe7AnDDZDnUUuVNgRfDAyi7L AWiwXBIKk5ZaRcNTmyncZN4mdwI5MyLGzzedSjxgeXTZJZe/5rZT/17vwL/PKl09L6XJ N+hw== X-Gm-Message-State: AKwxyteDNAw+RjVO5Fp45rRmoJLQAKJfvKAFUOD69utTyZIILpBGmghn evBDFJsJv0ddQeBMrhE9C1EYNMf4qZ64BWOTqq350Q== X-Google-Smtp-Source: ACJfBovC67mL09Etw0OAZzZX5gXLP3rvN7jWsyaheHDBWeNaepFzWiru3wP+0XL447HkDvLSQEBrBZGDDO8QjMS6Q1M= X-Received: by 10.200.46.196 with SMTP id i4mr1285182qta.179.1515099560213; Thu, 04 Jan 2018 12:59:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.178 with HTTP; Thu, 4 Jan 2018 12:59:19 -0800 (PST) In-Reply-To: References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> From: Ashley Blewer Date: Thu, 4 Jan 2018 15:59:19 -0500 Message-ID: To: Dave Rice Cc: Jerome Martinez , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a11413dd81600b50561f99be4" Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 20:59:23 -0000 --001a11413dd81600b50561f99be4 Content-Type: text/plain; charset="UTF-8" Patch for aforementioned issue in #92 is now open here for review: https://github.com/FFmpeg/FFV1/pull/97 On Thu, Jan 4, 2018 at 3:44 PM, Ashley Blewer wrote: > Thanks Jerome! > > I brought up a related issue in https://github.com/FFmpeg/FFV1/pull/92 > about use of luma vs luminance and am noting it here, as it is a separate > issue from the PR at hand, which I will now approve (but still think > changes should be made). > > Ashley > > On Thu, Jan 4, 2018 at 10:34 AM, Dave Rice wrote: > >> >> On Jan 4, 2018, at 7:53 AM, Jerome Martinez wrote: >> >> On 04/01/2018 13:14, Michael Niedermayer wrote: >> >> Block in many video and image codecs generally refers to a >> rectgangular spatial area of pixels. >> Macro block would refer to a larger square made of blocks. >> >> We should not redefine commonly used terms. That could confuse people who >> have worked with other video codecs >> >> >> I reused the "Block" term from Matroska ( https://www.matroska.org/techn >> ical/specs/index.html#block_structure ) in order not to reinvent the >> wheel and this word is not used in FFV1, but I fully understand that it may >> be misleading in the video coding world and I have no opinion about which >> word to use, I just need a different word because "bitstream" is misleading >> as reported. Please suggest an alternative. >> >> >> I added several reviews within the active pull requests at >> https://github.com/FFmpeg/FFV1/pulls. Overall these changes help with >> the document a lot. Thanks to Derek and Jerome. >> >> I agree with Michael that the term Block may not be best. The NumBytes >> definition already uses `FFV1 components`. Could also consider packet, >> segment, or IMHO `bitstream component` was fine though long. >> >> Kind Regards, >> Dave Rice >> >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> >> > --001a11413dd81600b50561f99be4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Patch for aforementioned issue in #92 is now open here for= review: https://github.com/FFmpeg/FFV1/pull/97

On Thu, Jan 4, 2018 at 3:44 PM, = Ashley Blewer <ashley.blewer@gmail.com> wrote:
Thanks Jerome!
I brought up a related issue in https://github.com/FFmpeg/FFV1/pull/9= 2 about use of luma vs luminance and am noting it here, as it is a sepa= rate issue from the PR at hand, which I will now approve (but still think c= hanges should be made).
<= br>
Ashle= y

On Thu, Jan 4, 2018 at 10:34 AM, Dave Rice <= span dir=3D"ltr"><= dave@dericed.com> wrote:

On Jan= 4, 2018, at 7:53 AM, Jerome Martinez <jerome@mediaarea.net> wrote:

<= div>
On 04/01/2018 13:14, Michael Niedermayer wrote:
Block in many video and image codecs generally refers to a
re= ctgangular spatial area of pixels.
Macro block would refer to a larger s= quare made of blocks.

We should not redefine commonly used terms. Th= at could confuse people who
have worked with other video codecs

I reused the "Block" term from Matroska ( https://www.matroska.org/technical/specs/index.html#block_= structure ) in order not to reinvent the wheel and this word is no= t used in FFV1, but I fully understand that it may be misleading in the vid= eo coding world and I have no opinion about which word to use, I just need = a different word because "bitstream" is misleading as reported. P= lease suggest an alternative.

=
I added several reviews within the active pull requests at=C2=A0https://github= .com/FFmpeg/FFV1/pulls. Overall these changes help with the docume= nt a lot. Thanks to Derek and Jerome.

I agree with= Michael that the term Block may not be best. The NumBytes definition alrea= dy uses `FFV1 components`. Could also consider packet, segment, or IMHO `bi= tstream component` was fine though long.

Kind Rega= rds,
Dave Rice


_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar


--001a11413dd81600b50561f99be4-- From nobody Thu Jan 4 13:03:31 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 825ED12D88A for ; Thu, 4 Jan 2018 13:03:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jq0A9tqoDXyi for ; Thu, 4 Jan 2018 13:03:25 -0800 (PST) Received: from mail-wr0-x232.google.com (mail-wr0-x232.google.com [IPv6:2a00:1450:400c:c0c::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B3FBE12D871 for ; Thu, 4 Jan 2018 13:03:23 -0800 (PST) Received: by mail-wr0-x232.google.com with SMTP id o15so2663569wrf.12 for ; Thu, 04 Jan 2018 13:03:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=z07Xr9JJF0sX5UHHFLkTfYBofySI6tF43b3yUFovUJE=; b=aJT9iS1H+vnXWKcWPCk0n36EFTpmLWmosAa9CoLP08u05sjv0ubzrQdBhFcsYjw6O3 F5ZexpuLlGZBAG8VKmBc7ysY5XMGCXNM5gjZoVQcpeq4bOo3uQxoi4ORC0CoeLX4gM5o A5ew+ADzBc/cEekKOYyYt2ZJKm2tWQNHi9Zoan4BSWdFTOJZJpecyUUYLWkdwuMlXEuA s7hHdrGwBTAIinrW2joal674yG3pOa+7+EQv5Pwf/4eVISYEFWtff0dPISU2Emup1FFr lLA8L75FLoMMdRU02i06aH94rgWoXGu2nmEWTd5OohQQoLMgPIGvNMFkmTyFLQEODZhy p3zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=z07Xr9JJF0sX5UHHFLkTfYBofySI6tF43b3yUFovUJE=; b=a61j4u4ReLPqWZ8MV7et0hqOGCnJfz9ZWEeJWWuFVD/JVeIpnJf0oKWKrseVIVibh6 lkJKhF+nAwUAS1FqnfnCxgh+kEXxNjdinB1CmLdz/zJgkMR9scgb2FvYoKXHGxV//Um4 708Q/89790/NWAe+XGFjuKDVX5LO2gjuYlzWXGXWInohuv3MoH6cRMBZrp9LtGYdJ4lt T7cJnKI8RpWafhjPaUlf7emUVu1DvAhLee5OyweqTqK+0LJn5VoyDTLMo9L1UI8ZfVwn znfKEzEZNGA3z9LmCumYxk8FWaXwgItjErBJ+3O/UCPvqMOn4VgKxFoXe/TX8qcerDF5 ukPA== X-Gm-Message-State: AKGB3mJSSl2O80Or41dxfxaVzxRy7ueB0Splt+UBfruXmdSlvtizQknn XhWTEd5uyhD+VTHNhwFO2L4fDw+Kvm1n4l9Pxm4= X-Google-Smtp-Source: ACJfBoslWCA429BN9l4zahdgFrDHHMyehH8SBS3Mi7pqyhZ1jd2/kfX1GK5wTF1eMoOw61ox4ShUylyC/WL8ZekPJQ8= X-Received: by 10.223.162.141 with SMTP id s13mr724851wra.132.1515099802256; Thu, 04 Jan 2018 13:03:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.138.187 with HTTP; Thu, 4 Jan 2018 13:03:21 -0800 (PST) In-Reply-To: References: <517340df-9bc8-c4e5-1085-011dcbaa54ce@gmail.com> <20180104121420.GO4938@michaelspb> <05a00068-bfd3-2ae1-c35d-5dae0310b723@mediaarea.net> From: Ashley Blewer Date: Thu, 4 Jan 2018 16:03:21 -0500 Message-ID: To: Katherine Frances Cc: Dave Rice , Jerome Martinez , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="f403045e9fb483491e0561f9a9e6" Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 21:03:29 -0000 --f403045e9fb483491e0561f9a9e6 Content-Type: text/plain; charset="UTF-8" On Thu, Jan 4, 2018 at 3:59 PM, Katherine Frances wrote: > Luma v. luminance - good catch, Ash! Charles Poynton wrote a great article > about that. > > Haha, yes, and it has been very fresh in my mind lately! I linked to it in the github issue if anyone is interested. --f403045e9fb483491e0561f9a9e6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Jan 4, 2018 at 3:59 PM, Katherine Frances <= ;knfrances@gmail.c= om> wrote:
Luma v. luminance - good catch, Ash! Charles Poynton wrote a great article= about that.


Haha, yes,= and it has been very fresh in my mind lately! I linked to it in the github= issue if anyone is interested.

--f403045e9fb483491e0561f9a9e6-- From nobody Thu Jan 4 13:03:45 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B040912D88B for ; Thu, 4 Jan 2018 13:03:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.011 X-Spam-Level: X-Spam-Status: No, score=-2.011 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H9E1UKCGT90N for ; Thu, 4 Jan 2018 13:03:40 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 99E1F12D889 for ; Thu, 4 Jan 2018 13:03:37 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:34520) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eXCfY-0002Tr-3A for cellar@ietf.org; Thu, 04 Jan 2018 22:03:21 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id CE13665417DC for ; Thu, 4 Jan 2018 22:03:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1515099800; bh=CiLAJ0rP1hIHaEOVM5yIdZWXF53FCCTGxKz5xUX4+P0=; h=References:From:To:Subject:In-reply-to:Date:From; b=Pn2J73pFQs2zYgrh6dfs47BJx9U2Pd/yrhdyvRBA1Y6i+FEat1ET7pHFPl/stzPqs c3IXABvcfBYbYeFY4q73DEsUyUFZB+LpfXaactPwRTUy43arkY2QezxgZTIsWLRVXW mmQ+8LQ+usWrVE5k2ABsiH52TCn97pjKWmG0GjiviwPrvtpb127+ikTDNDpe3SAeLR ut41V8OfLZ/sXosnPCy76FhNTS9wEu8GgFCi3jtj7ccmawck/dJgck0LH+63Xfbin9 U5LCmJ4dfAstHtTpEQoDlVvDs9SusX8j+x2BxRHVcsU5ir/cn27mOdxMlUuKEOPIfA f/Oi5bBE60vvUehVVLgX7WzgOh6IIDtfUxUkopXDuTSHfItiMbZXK134SnXN625k6g b2HTBqaFLxQm4pBlgh8JCxUufhzB/5FICizF9YvQ2EYPsPTCj9nPOCeNMFrAxav/6W Ooioe4BRwUcALm+TXrAUoBn9/WlkG3/13BKKZM4C4M4/XujmQT1V5qh/7/Aj6MmaRP ZssmVcljHIyK9W/FRFzEGfemCpxpIvpdmMIMIFiPXlQXy2Kq6ef3Q+72M58GbI50lK qMFEoEcJiMW+ajAHbZ4sp9x09+ispAyI4oy4/iztEDfvCu3OLLOUWO5wwftz9DRtn0 /8BNcRKaspvtdIlB/Ee9B8vA= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id C2E222ACD7BD for ; Thu, 4 Jan 2018 22:03:19 +0100 (CET) References: <006AF262-FA38-4297-8377-121DD76A672F@dericed.com> User-agent: mu4e 0.9.18; emacs 25.3.1 From: Moritz Bunkus To: cellar@ietf.org In-reply-to: <006AF262-FA38-4297-8377-121DD76A672F@dericed.com> Date: Thu, 04 Jan 2018 22:03:19 +0100 Message-ID: <87mv1tfiaw.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cellar] Updates to Matroska documents X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2018 21:03:45 -0000 Hey, > I propose assessing consensus to change these to “WG Document” per > https://tools.ietf.org/html/rfc6174#section-4.2.4 In favor! Kind regards, mosu From nobody Fri Jan 5 01:14:27 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D4D7126579 for ; Fri, 5 Jan 2018 01:14:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.501 X-Spam-Level: X-Spam-Status: No, score=-1.501 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vhWor-tGCP-A for ; Fri, 5 Jan 2018 01:14:24 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3B4D0120724 for ; Fri, 5 Jan 2018 01:14:23 -0800 (PST) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w059EL0E024367 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 5 Jan 2018 10:14:21 +0100 Received: from castor.home (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:5018:da0:e454:3afe:8c22:c9e2] (may be forged)) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w059EKPY047556 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Fri, 5 Jan 2018 10:14:21 +0100 Date: Fri, 5 Jan 2018 10:14:22 +0100 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2018 09:14:27 -0000 Ashley Blewer wrote: >I brought up a related issue in >https://github.com/FFmpeg/FFV1/pull/92 >about use of luma vs luminance and am noting it here, as it is >a separate issue from the PR at hand, which I will now approve >(but still think changes should be made). Just a question: Have you checked chroma vs. chrominance as well, as I suggested in my volunteering comment on GitHub? Best regards, Reto From nobody Sun Jan 7 09:39:43 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A348B126DFB for ; Sun, 7 Jan 2018 09:39:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i3enzCIyB35R for ; Sun, 7 Jan 2018 09:39:38 -0800 (PST) Received: from 17.mo1.mail-out.ovh.net (17.mo1.mail-out.ovh.net [87.98.179.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5A5A212025C for ; Sun, 7 Jan 2018 09:39:38 -0800 (PST) Received: from player691.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo1.mail-out.ovh.net (Postfix) with ESMTP id ECCC4BBB12 for ; Sun, 7 Jan 2018 18:39:35 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: zen-lists@mediaarea.net) by player691.ha.ovh.net (Postfix) with ESMTPSA id 61BC026008E; Sun, 7 Jan 2018 18:39:33 +0100 (CET) To: ffmpeg-devel@ffmpeg.org References: <7e618ae2-2e57-9871-c62d-a98ca2df9bfa@noa-archive.com> <1c54159a-1309-2d12-ec89-e6f5d35c141e@mediaarea.net> <20180106010531.GA4926@michaelspb> <1e32a18b-ca31-72b4-7847-e7d75c10c8da@mediaarea.net> <20180106222132.GE4926@michaelspb> Cc: Codec Encoding for LossLess Archiving and Realtime transmission From: Jerome Martinez Message-ID: Date: Sun, 7 Jan 2018 18:39:34 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180106222132.GE4926@michaelspb> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 6581729383775277201 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrkedugddutdeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] [FFmpeg-devel] [PATCH] FFV1: make RGB48 support as non-experimental X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 17:39:42 -0000 On 06/01/2018 23:21, Michael Niedermayer wrote: > On Sat, Jan 06, 2018 at 04:54:18PM +0100, Jerome Martinez wrote: >> On 06/01/2018 02:05, Michael Niedermayer wrote: >>>> ffv1enc.c | 4 ---- >>>> 1 file changed, 4 deletions(-) >>>> acfc60c913b311b148f2eeef2d2d6ea9e37afcf7 0001-avcodec-ffv1enc-mark-RGB48-support-as-non-experiment.patch >>>> From 303f63fb7e6172fdb7de66da1f8a4006b79a535f Mon Sep 17 00:00:00 2001 >>>> From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Martinez?= >>>> Date: Fri, 5 Jan 2018 11:09:01 +0100 >>>> Subject: [PATCH] avcodec/ffv1enc: mark RGB48 support as non-experimental >>>> >>>> Resulting bitstream was tested with a conformance checker >>>> using the last draft of FFV1 specifications. >>> But has the way this is stored been optimized ? >>> >>> Once its marked as non exerimental all future decoders must support the exact >>> way. >> Although this is considered experimental in the encoder, the implementation >> adheres to the description in the specification. The bitstream specification >> does not provide a bitdepth limit so with the current draft of the >> specification, another FFV1 encoder could already encode 16-bit (and more) >> content. The work on the specification has been careful to not break >> compatibility with former drafts and at this point the FFV1 bitstream >> specification for versions 0, 1, and 3 should be considered already >> non-experimental for all bit depths. All current decoders should already >> support the exact way it is currently specified. >> >> To make a change in the specification at this point would have cascading >> consequences as we’d have to retract the part of the specification which >> states that micro_version 4 of version 3 is the first stable variant. Worse, >> it is impossible to indicate a bitstream change in FFV1 version 1, which >> permits RGB 16-bit content too, so it would be difficult for a decoder to >> detect a bitstream not conforming to the bitstream created by the current >> version of FFmpeg encoder. > Are you not making this look alot more complex than it is ? > Or maybe we talk about slightly different things here > > with the next version we can introduce any method of storing 16bit or 9-15 bit > and we certainly do not support in the implementation all possible bit depths > > From what i remember I had always wanted to improve the way that > more than 8bit is handled, not just 16. Although 16 is a bit special > > Consider this: > If we improve get_context() in the next version for >8bit > we still have to support 9-15 bit with the old definition > if we now declare 16bit non experimental then we also must support that with > an old get_context() in the decoder. > the 16bit path is seperate from the lower bit depth. So this is an Additional > codepath that we have to carry in the future > > isnt it smarter now that if we want to improve get_context() that we > dont now extend what can be generated with the current get_context ? > > or are such current get_context() style files already widespread ? > if so then its probably best to accept it and keep supporting it I am lost. Looks like we talk about 2 different subjects: FFV1 bitstream specifications and FFmpeg implementation. Let separate subjects: FFV1 bitstream specifications: Since at least 2012 [1] get_context (in the bitstream) is defined and flagged as stable for **all** bit depths for versions 0, 1, 3. It is still the case today [2]. There was a consensus on discussing about FFV1 bitstream on CELLAR mailing list There was a consensus for not changing the bitstream for version 0, 1, 3 so we can standardize it ASAP without breaking current implementations (reason we document bugs in the main encoder, but the idea is not to accept more changes) If the FFV1 bitstream for version 0, 1, or 3 must be changed in current stable version, it would be a major break in the consensus and it should be discussed on CELLAR list (in copy as they are concerned), but I doubt the consensus would be for breaking the bitstream compatibility as the discussion from day 0 on CELLAR is to document current bitstream without changing it for versions 0, 1, 3. The fact that there was no (publicly available) RGB48 encoder in the wild is not an excuse for breaking the compatibility with current draft of specifications. Same if someone decides to do an encoder for e.g. RGB3000. It is possible to change the bitstream with version 4 (experimental version), and it is a good place for changing get_context for 16-bit content (whatever is the color space, BTW). FFmpeg implementation: FFV1 YCbCr 16-bit is already flagged as non experimental, so there is already some non experimental 16-bit support in FFmpeg FFV1 encoder. 16-bit is not new and for RGB48 the actual encoding after RGB to YCbCr transformation is just 1 bit more (17-bit). FFmpeg experimental flag is for the status of the encoder, not for the status of the bitstream (the bitstream for versions 0, 1, 3 is already considered stable for RGB48 and others) FFV1 RGB48 support in FFmpeg is conform to FFV1 bitstream specifications. Optimizing FFmpeg for better compression of FFV1 RGB48 as specified in spec is not blocked after this change. The only reason for keeping the experimental flag for RGB48 support is if the resulting file does not comply to the FFV1 specification, and this is not the case (tested with a conformance checker complying to FFV1 specifications). As a result, the comment about get_contet for RGB48 is blocking for removing experimental status of version 4, but the suggested patch does not touch on version 4. [1] https://github.com/FFmpeg/FFV1/blob/cbb560873e54bfe2d2042e898a210fe2abd079e0/ffv1.lyx#L659 [2] https://github.com/FFmpeg/FFV1/blob/fdc65b73282633cc8867c185e83a6d21e5c65f3f/ffv1.md#context [3] https://github.com/FFmpeg/FFV1/blob/fdc65b73282633cc8867c185e83a6d21e5c65f3f/ffv1.md#micro_version From nobody Sun Jan 7 11:08:13 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD267126BFD for ; Sun, 7 Jan 2018 11:08:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3AHhH20xOmZa for ; Sun, 7 Jan 2018 11:08:10 -0800 (PST) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:c:538::195]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 106381241FC for ; Sun, 7 Jan 2018 11:08:09 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id F37E8A80C7; Sun, 7 Jan 2018 20:08:06 +0100 (CET) Date: Sun, 7 Jan 2018 20:08:06 +0100 From: Michael Niedermayer To: cellar@ietf.org, ffmpeg-devel@ffmpeg.org Message-ID: <20180107190806.GS4938@michaelspb> References: <7e618ae2-2e57-9871-c62d-a98ca2df9bfa@noa-archive.com> <1c54159a-1309-2d12-ec89-e6f5d35c141e@mediaarea.net> <20180106010531.GA4926@michaelspb> <1e32a18b-ca31-72b4-7847-e7d75c10c8da@mediaarea.net> <20180106222132.GE4926@michaelspb> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H88uUF932U8Oj0a6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] [FFmpeg-devel] [PATCH] FFV1: make RGB48 support as non-experimental X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 19:08:13 -0000 --H88uUF932U8Oj0a6 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi On Sun, Jan 07, 2018 at 06:39:34PM +0100, Jerome Martinez wrote: > On 06/01/2018 23:21, Michael Niedermayer wrote: > >On Sat, Jan 06, 2018 at 04:54:18PM +0100, Jerome Martinez wrote: > >>On 06/01/2018 02:05, Michael Niedermayer wrote: > >>>> ffv1enc.c | 4 ---- > >>>> 1 file changed, 4 deletions(-) > >>>>acfc60c913b311b148f2eeef2d2d6ea9e37afcf7 0001-avcodec-ffv1enc-mark-R= GB48-support-as-non-experiment.patch > >>>> From 303f63fb7e6172fdb7de66da1f8a4006b79a535f Mon Sep 17 00:00:00 20= 01 > >>>>From: =3D?UTF-8?q?J=3DC3=3DA9r=3DC3=3DB4me=3D20Martinez?=3D > >>>>Date: Fri, 5 Jan 2018 11:09:01 +0100 > >>>>Subject: [PATCH] avcodec/ffv1enc: mark RGB48 support as non-experimen= tal > >>>> > >>>>Resulting bitstream was tested with a conformance checker > >>>>using the last draft of FFV1 specifications. > >>>But has the way this is stored been optimized ? > >>> > >>>Once its marked as non exerimental all future decoders must support th= e exact > >>>way. > >>Although this is considered experimental in the encoder, the implementa= tion > >>adheres to the description in the specification. The bitstream specific= ation > >>does not provide a bitdepth limit so with the current draft of the > >>specification, another FFV1 encoder could already encode 16-bit (and mo= re) > >>content. The work on the specification has been careful to not break > >>compatibility with former drafts and at this point the FFV1 bitstream > >>specification for versions 0, 1, and 3 should be considered already > >>non-experimental for all bit depths. All current decoders should already > >>support the exact way it is currently specified. > >> > >>To make a change in the specification at this point would have cascading > >>consequences as we=E2=80=99d have to retract the part of the specificat= ion which > >>states that micro_version 4 of version 3 is the first stable variant. W= orse, > >>it is impossible to indicate a bitstream change in FFV1 version 1, which > >>permits RGB 16-bit content too, so it would be difficult for a decoder = to > >>detect a bitstream not conforming to the bitstream created by the curre= nt > >>version of FFmpeg encoder. > >Are you not making this look alot more complex than it is ? > >Or maybe we talk about slightly different things here > > > >with the next version we can introduce any method of storing 16bit or 9-= 15 bit > >and we certainly do not support in the implementation all possible bit d= epths > > > > From what i remember I had always wanted to improve the way that > >more than 8bit is handled, not just 16. Although 16 is a bit special > > > >Consider this: > >If we improve get_context() in the next version for >8bit > >we still have to support 9-15 bit with the old definition > >if we now declare 16bit non experimental then we also must support that = with > >an old get_context() in the decoder. > >the 16bit path is seperate from the lower bit depth. So this is an Addit= ional > >codepath that we have to carry in the future > > > >isnt it smarter now that if we want to improve get_context() that we > >dont now extend what can be generated with the current get_context ? > > > >or are such current get_context() style files already widespread ? > >if so then its probably best to accept it and keep supporting it >=20 > I am lost. > Looks like we talk about 2 different subjects: FFV1 bitstream specificati= ons > and FFmpeg implementation. > Let separate subjects: >=20 [...] > FFmpeg implementation: > FFV1 YCbCr 16-bit is already flagged as non experimental, so there is > already some non experimental 16-bit support in FFmpeg FFV1 encoder. 16-b= it > is not new and for RGB48 the actual encoding after RGB to YCbCr > transformation is just 1 bit more (17-bit). This is correct but i think misleading a bit the 17bit case this is about uses a seperate codepath. So if its enabled we generate new files that have never been generated before in some sense AFAIK If we change the bitstream in a future version, which i belive we=20 should consider. Then we have an additional "old" version of the 17bit path that needs to be supported in implementations. > FFmpeg experimental flag is for the status of the encoder, not for the > status of the bitstream (the bitstream for versions 0, 1, 3 is already > considered stable for RGB48 and others) I think i added the flag check. The reason behind the check was so that the bitstream syntax can be changed without the need to support every iteration of the bitstream. I had hoped someone would fund research and testing of further improvments= =20 to the various fine details of higher bit depth encoding beyond 8bit. IIRC No further funding was provided, Noone worked on it as a volunteer, No changes where proposed ... But it wasnt intended as a "bitstream is in stone, encoder might mismatch the spec" at the time. We have a draft spec now that does not limit bitdepth in any way, this may have been a mistakke but i dont see this as a big problem honestly. I do not propose to change this. But i would not oppose it if people want to change = it If someone creates a 19bit per sample file based on the draft spec it also will not decode with most real world implementations. There is no question that with a unlimited bitdepth, real implementations will never support some depths I just dont want to create a new variant of files _IF_ we plan to work on improving the "bitstream syntax" in the next version. Of course if these files are already out in the wild then we must support this in the implementation. And then most of this discussion is meaningless [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you fake or manipulate statistics in a paper in physics you will never get a job again. If you fake or manipulate statistics in a paper in medicin you will get a job for life at the pharma industry. --H88uUF932U8Oj0a6 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpScBYACgkQYR7HhwQLD6ugeQCggUfgOoXtOOWiUMDBTwNWVMOF xWkAn2Jpq7hgQYr8AlA0+rVLZmgzS+/s =uRbC -----END PGP SIGNATURE----- --H88uUF932U8Oj0a6-- From nobody Sun Jan 7 12:20:23 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F023E126C2F for ; Sun, 7 Jan 2018 12:20:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gcOKOMkRTdve for ; Sun, 7 Jan 2018 12:20:19 -0800 (PST) Received: from 1.mo1.mail-out.ovh.net (1.mo1.mail-out.ovh.net [178.32.127.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F1CCB1241FC for ; Sun, 7 Jan 2018 12:20:18 -0800 (PST) Received: from player691.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo1.mail-out.ovh.net (Postfix) with ESMTP id AA51DBD65A for ; Sun, 7 Jan 2018 21:20:16 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: zen-lists@mediaarea.net) by player691.ha.ovh.net (Postfix) with ESMTPSA id 1B239260082; Sun, 7 Jan 2018 21:20:14 +0100 (CET) To: ffmpeg-devel@ffmpeg.org References: <7e618ae2-2e57-9871-c62d-a98ca2df9bfa@noa-archive.com> <1c54159a-1309-2d12-ec89-e6f5d35c141e@mediaarea.net> <20180106010531.GA4926@michaelspb> <1e32a18b-ca31-72b4-7847-e7d75c10c8da@mediaarea.net> <20180106222132.GE4926@michaelspb> <20180107190806.GS4938@michaelspb> Cc: Codec Encoding for LossLess Archiving and Realtime transmission From: Jerome Martinez Message-ID: Date: Sun, 7 Jan 2018 21:20:15 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180107190806.GS4938@michaelspb> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 9295429632063967377 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrkedugddufeelucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] [FFmpeg-devel] [PATCH] FFV1: make RGB48 support as non-experimental X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 20:20:22 -0000 On 07/01/2018 20:08, Michael Niedermayer wrote: > [...] > This is correct but i think misleading a bit > the 17bit case this is about uses a seperate codepath. For your (FFmpeg) encoder and decoder. Not mine (same code path is used for 8/10/12/16/... bit RGB and YUV with my encoder and decoder). Again, we mix up bitstream specification and FFmpeg implementation. In the bitstream specifications, there is currently only one path (except for handling files in the wild which are not conform to the unique path, we take care of them for historical reasons). > So if its enabled > we generate new files that have never been generated before in some sense > AFAIK - MediaConch conformance checker can check files up to 30 bits per component, and already implements the way it is written in spec for all supported bit depths, because the spec was flagged as stable whatever is the bit depth for a long time, without having anyone saying that 16 or 17-bit for RGB (reminder: it is already set as "stable" for YCbCr 16-bit) may have a different paths later for versions 0, 1, 3. If you change the bitstream of RGB48 or any stream with bitdepth <=30, you break it despite the fact it is validating correctly and would be a major issue for this tool (breaking trust about the tool or FFV1, as there would be 2 variants of FFV1 RGB48). - I have an early (not yet public, for testing the spec only for the moment) version of an encoder conforming to the spec for all bit depths up to 30-bit per component. - I have heard about other FFV1 encoders able to encode RGB48 > > If we change the bitstream in a future version, which i belive we > should consider. Then we have an additional "old" version of the 17bit path > that needs to be supported in implementations. question: How would you detect old version, from all encoders (not only FFmpeg)? there is nothing permitting that in the bitstream AFAIK (for v1: no micro_version; for v3: micro version >4 must be compatible with v4 as v4 is flagged stable). So it is "bitstream is in stone" now, for at least versions 0, 1, 3. Reason I speak about R&D with version 4 bitstream (I don't speak about FFmpeg), which is still flagged as experimental for all files. > > >> FFmpeg experimental flag is for the status of the encoder, not for the >> status of the bitstream (the bitstream for versions 0, 1, 3 is already >> considered stable for RGB48 and others) > I think i added the flag check. The reason behind the check was so that the > bitstream syntax can be changed without the need to support every iteration > of the bitstream. > I had hoped someone would fund research and testing of further improvments > to the various fine details of higher bit depth encoding beyond 8bit. > IIRC No further funding was provided, Noone worked on it as a volunteer, > No changes where proposed ... This is planned on my side after FFV1 versions 0, 1, 3 are finalized. With version 4 (the experimental version). but it is possible only if we all are in sync and don't do something against the consensus, and as far I as I understand the consensus is that versions 0, 1, 3 are "bitstream is in stone" now (and actually from the beginning of CELLAR) based on the draft of FFV1 specifications, which is detailed before going to IETF but not changed. All the communication, at IETF and conferences, around FFV1 versions 0, 1, 3 is that the bitstream specification is stable, and discussing about breaking this consensus may harm FFV1 spread. > > But it wasnt intended as a "bitstream is in stone, encoder might mismatch > the spec" at the time. I disagree: the idea behind CELLAR is that all encoders **must** match the spec, or there is misunderstanding to clarify. Else everyone does his own version of FFV1, and we can not work together on an unique lossless compression format. The spec is expected to be the reference (for versions 0, 1, 3 for the moment) and encoders are expected to match the spec. > > We have a draft spec now that does not limit bitdepth in any way, this may > have been a mistakke but i dont see this as a big problem honestly. I do not > propose to change this. But i would not oppose it if people want to change it > > If someone creates a 19bit per sample file based on the draft spec it also > will not decode with most real world implementations. It will with my conformance checker. You can not know what is "real world", as a couple of encoders may be in house only. we still mix up bitstream specification and one implementation. The idea behind CELLAR is to have FFV1 a standardized video format, so we can not decide for 1 encoder to change the bitstream. We need to find a consensus on CELLAR mailing-list first. The consensus is what is written in the draft and there was no post on CELLAR mailing-list for limiting to some bit depths. > There is no question that with a unlimited bitdepth, real implementations > will never support some depths Whatever is the support from the encoders you know, you can not know what was created by others based on the bitstream specification. so the bitstream is specified (for versions 0, 1, 3) for all bit depths and we can not change that now, as it is written for a long time in spec that the spec is flagged stable for these versions. The question is not what is the support from decoder x, the question is that if there is a plan to break the compatibility between current version of FFV1 draft and future versions of this document. > > I just dont want to create a new variant of files _IF_ we plan to work on > improving the "bitstream syntax" in the next version. I don't understand: yes, we plan to improve the bitstream, and this is the reason we plan a version 4, 5, 6... But it does not mean that versions 0, 1, 3 are not used by FFmpeg or any other encoder for all bit depths. But again, here (ffmpeg-devel), my goal was to speak about FFmpeg encoder, it was not intended to speak about FFV1 specification, and it is weird for me to have to speak about FFV1 specification on ffmpeg-devel (it should be on CELLAR only). Being sure that FFmpeg is creating a bitstream conforming to FFV1 specifications (so versions 0, 1, 3) should be the only criteria. > Of course if these files are already out in the wild then we must support > this in the implementation. And then most of this discussion is meaningless I confirm that RGB48 files are already in the wild, as it is needed by some archives, and I confirm that they rely on MediaConch for testing the conformance compared to the specification. I still don't see a reason to block 1 encoder to create RGB48 with version 1 or 3, as the spec is clear for a long time about how the bitstream is for such bit depth. Note that the FFV1 RGB48 decoder is not flagged as experimental, so any old version of FFmpeg should be able to decode new files you want to create (I don't see how they could do that as they don't know the get_context you want to implement) if you don't want to have complain that an old stable version of FFmpeg is not able to decode a FFV1 RGB48 file you just created with the new FFmpeg implementation. Again, the experimental flag is for FFmpeg encoder, not the FFV1 bitstream, the FFV1 bitstream is flagged as stable in FFV1 specification, including RGB48. Additionally, FFmpeg FFV1 decoder is currently able to decode RGB48 as specified in FFV1 specification and created by encoders conforming to the spec (FFmpeg "experimental" included) without having to set "-strict experimental" on the command line. For these reasons, I still argue to remove the experimental flag for this encoder about RGB48 as the output is conform to FFV1 specifications and current and older versions of FFmpeg would not be able to decode FFV1 RGB48 if get_context is changed. Jérôme From nobody Sun Jan 7 15:42:05 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D18E124B17 for ; Sun, 7 Jan 2018 15:42:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5kXj1e8TyQ_5 for ; Sun, 7 Jan 2018 15:42:01 -0800 (PST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8AF78120227 for ; Sun, 7 Jan 2018 15:42:01 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 16AA7FB87D; Mon, 8 Jan 2018 00:41:58 +0100 (CET) Date: Mon, 8 Jan 2018 00:41:58 +0100 From: Michael Niedermayer To: FFmpeg development discussions and patches , Codec Encoding for LossLess Archiving and Realtime transmission Message-ID: <20180107234158.GL4926@michaelspb> References: <7e618ae2-2e57-9871-c62d-a98ca2df9bfa@noa-archive.com> <1c54159a-1309-2d12-ec89-e6f5d35c141e@mediaarea.net> <20180106010531.GA4926@michaelspb> <1e32a18b-ca31-72b4-7847-e7d75c10c8da@mediaarea.net> <20180106222132.GE4926@michaelspb> <20180107190806.GS4938@michaelspb> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bxMwjKH1PZzBx/Wb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] [FFmpeg-devel] [PATCH] FFV1: make RGB48 support as non-experimental X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 23:42:04 -0000 --bxMwjKH1PZzBx/Wb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jan 07, 2018 at 09:20:15PM +0100, Jerome Martinez wrote: > On 07/01/2018 20:08, Michael Niedermayer wrote: > >[...] > >This is correct but i think misleading a bit > >the 17bit case this is about uses a seperate codepath. >=20 > For your (FFmpeg) encoder and decoder. Not mine (same code path is used f= or > 8/10/12/16/... bit RGB and YUV with my encoder and decoder). > Again, we mix up bitstream specification and FFmpeg implementation. The seperation of 16bit and >16bit results out of the data type size fitting or not fitting in "int16". A space/cache efficient implementation, possibly one using SIMD would use seperate codepaths for 16 and > 16 on a normal architecture thats based= on 8bit bytes. Its part of spec design what "an efficient implementation" would/could do [...] > > > >If we change the bitstream in a future version, which i belive we > >should consider. Then we have an additional "old" version of the 17bit p= ath > >that needs to be supported in implementations. >=20 > question: How would you detect old version, from all encoders (not only > FFmpeg)? there is nothing permitting that in the bitstream AFAIK (for v1:= no > micro_version; for v3: micro version >4 must be compatible with v4 as v4 = is > flagged stable). > So it is "bitstream is in stone" now, for at least versions 0, 1, 3. Reas= on > I speak about R&D with version 4 bitstream (I don't speak about FFmpeg), > which is still flagged as experimental for all files. very simple, the lowest version that we can make that change in without causing any pain to people using ffv1 and without breaking the spec. [...] > > > >But it wasnt intended as a "bitstream is in stone, encoder might mismatch > >the spec" at the time. >=20 > I disagree: the idea behind CELLAR is that all encoders **must** match the you disagree what i mean in a years old commit in ffmpeg i authored ? > spec, or there is misunderstanding to clarify. Else everyone does his own > version of FFV1, and we can not work together on an unique lossless > compression format. The spec is expected to be the reference (for versions > 0, 1, 3 for the moment) and encoders are expected to match the spec. you make no sense This code was intended to be used for developing the next version, that did= nt happen. If it did happen the code in the implementation would have changed and once it was looking optimal a suggestion with test results would have been posted to CELLAR to update the draft/spec. If that update did happen then once its "stable" the check would have been removed from the encoder implementation so people could use it. [...] >=20 > >There is no question that with a unlimited bitdepth, real implementations > >will never support some depths >=20 > Whatever is the support from the encoders you know, you can not know what > was created by others based on the bitstream specification. so the bitstr= eam > is specified (for versions 0, 1, 3) for all bit depths and we can not cha= nge > that now, as it is written for a long time in spec that the spec is flagg= ed > stable for these versions. I can know what is possible with real implementations build by others becau= se they are constrained by the same physical laws and space and matter availab= le in the observable universe. >=20 > The question is not what is the support from decoder x, the question is t= hat > if there is a plan to break the compatibility between current version of > FFV1 draft and future versions of this document. I think its more like a circular reference The draft/spec intended to describe the existing status quo and while in the implementation something was marked experimental this=20 detail was lost in the spec=20 [...] > I confirm that RGB48 files are already in the wild, as it is needed by so= me > archives, and I confirm that they rely on MediaConch for testing the > conformance compared to the specification. Thanks, then its probably best if we support this format in en and decoder implementations. As its out there already [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety -- Benjamin Franklin --bxMwjKH1PZzBx/Wb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpSsEYACgkQYR7HhwQLD6sLZQCfbmr82bPsE+5kluu3VoFyGSta ClUAn23x7QVl+euVsbvhQISf91baIW0h =snJf -----END PGP SIGNATURE----- --bxMwjKH1PZzBx/Wb-- From nobody Mon Jan 8 09:38:23 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C646129C6C for ; Mon, 8 Jan 2018 09:38:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4n1Y5W1vX7PY for ; Mon, 8 Jan 2018 09:38:20 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 25E01127275 for ; Mon, 8 Jan 2018 09:38:19 -0800 (PST) Received: from smtp8.infomaniak.ch (smtp8.infomaniak.ch [83.166.132.38]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w08HcHwQ026258 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 8 Jan 2018 18:38:17 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp8.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w08HcGH0058257 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 8 Jan 2018 18:38:17 +0100 Date: Mon, 8 Jan 2018 18:38:16 +0100 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 In-Reply-To: <20180107193931.a2b2bdf45b21eda967353007@mi.rr.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] [FFmpeg-devel] [PATCH] FFV1: make RGB48 support as non-experimental X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jan 2018 17:38:22 -0000 Compn wrote: >i am curious what are the names of the other ffv1 encoders?=20 Why is the *name* of interest to you? We have an implementation which benefits of multiple graphic cards and pretends to be... part of the FFmpeg suite. >please upload or link to one of these RGB48 samples and ffmpeg >will support it. They already work fine with FFmpeg! In my understanding, the point of this thread is the experimental flag, in order to allow audio-visual archives to use any regular FFmpeg release to encode their RGB content with FFV1. Best regards, Reto From nobody Tue Jan 9 04:46:42 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EA2212D832 for ; Tue, 9 Jan 2018 04:46:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.698 X-Spam-Level: X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iyWNCGpZI7Gj for ; Tue, 9 Jan 2018 04:46:39 -0800 (PST) Received: from mail-qk0-x232.google.com (mail-qk0-x232.google.com [IPv6:2607:f8b0:400d:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 417BC12420B for ; Tue, 9 Jan 2018 04:46:39 -0800 (PST) Received: by mail-qk0-x232.google.com with SMTP id o126so18299306qke.12 for ; Tue, 09 Jan 2018 04:46:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Hzuj71Rc1+2LK3/fj0pYHMMxBWUX3OOecmVvR5IMkb8=; b=dLNBhm951VeLK7EHVBsItsbvxp5XGtJ6cWaJ9sGqad8JlD8sfLioLDqyEMzHO7xBfx 4DTSQQ0Qth36MTppTEVyf6kAfOeBYWp+t3Fpj0ToypnS1HzZuslLqW+ECsrbD7FLBd53 BdXShilJyisKzPK7ACeyKUUR3jFVgoAet9e9sfK8eBR8yuM79zSP+M6bnVJxmPAQviYH soHPEPqZK4XxHfFT1JizIdhccqRfqbdrZ8kAK7XIp7Q+aHlj5+JGQ6lxk8WPfsbqr7SH 6VUtQLOv0I9cZXoPdRDVSlyRkIPBxr39dBnlVmfJX9kZnNZd0hZ/DNE0Iu+q85QU3cte 2DKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Hzuj71Rc1+2LK3/fj0pYHMMxBWUX3OOecmVvR5IMkb8=; b=jt79pmVX9qPd7jj/p+eYEC8LFp/JBHmEsgojRvlh6yxii7gYf7pEjK0AS2QRWzSHT4 qleri4exaY367R96XXQ6GW4TLudx0bozZhwRRDYMKMgx05oRjwTHmO/+cfcPZYlrAVoa B57WsiNYl51vTwCLHYctpESm3bZ8CN+4RT0kYgzHP9h60LSWsMImRiiJhHYFQJPGsphu hp87q7fqofr99fMq0SoChwlByCwICljj5nxtdYAwgCuEXPqvA+HuSo/w9fWrHWv2UA1X 9mz9CsW7sATNexmfpzSXbpcmo4gdshNmc9aHO2xQCYZBs+LuDpG59Zjg+pIJ15biQOE2 6GUQ== X-Gm-Message-State: AKwxytc4C1c5gUKyuvmhCtXBA+f/162COW++RcZlBFNQ2+pz+9rs1bm+ noh2Xxf1DP7AZA2oS2xT6HgQVlM15cyEmC5hwafiMA== X-Google-Smtp-Source: ACJfBosvq7g+XPf1r6WQV1/pYi6EGAMe+tKHiJwqpd1nVUJp3s975TH5SKaWcutjxX04GQppGjxMR0O4sa9vCjtJFcI= X-Received: by 10.55.212.69 with SMTP id l66mr10294223qki.252.1515501998216; Tue, 09 Jan 2018 04:46:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.178 with HTTP; Tue, 9 Jan 2018 04:46:37 -0800 (PST) In-Reply-To: References: From: Ashley Blewer Date: Tue, 9 Jan 2018 07:46:37 -0500 Message-ID: To: Reto Kromer Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a1149cc82429b240562574edc" Archived-At: Subject: Re: [Cellar] Quick Review of draft-ietf-cellar-ffv1-00 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jan 2018 12:46:41 -0000 --001a1149cc82429b240562574edc Content-Type: text/plain; charset="UTF-8" Chroma is used throughout and in segment names and the only usage of Chrominance is in the definition of YCbCr at the top. Although is it not still correct? For clarity and consistency, should it, too, be changed? On Fri, Jan 5, 2018 at 4:14 AM, Reto Kromer wrote: > Ashley Blewer wrote: > > >I brought up a related issue in > >https://github.com/FFmpeg/FFV1/pull/92 > >about use of luma vs luminance and am noting it here, as it is > >a separate issue from the PR at hand, which I will now approve > >(but still think changes should be made). > > Just a question: Have you checked chroma vs. chrominance as > well, as I suggested in my volunteering comment on GitHub? > > Best regards, Reto > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --001a1149cc82429b240562574edc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Chroma is used throughout and in segment names and the onl= y usage of Chrominance is in the definition of YCbCr at the top. Although i= s it not still correct? For clarity and consistency, should it, too, be cha= nged?

On= Fri, Jan 5, 2018 at 4:14 AM, Reto Kromer <lists@reto.ch> wrote:=
Ashley Blewer wrote:
>I brought up a related issue in
>https://github.com/FFmpeg/FFV1/pull/92
>about use of luma vs luminance and am noting it here, as it is
>a separate issue from the PR at hand, which I will now approve
>(but still think changes should be made).

Just a question: Have you checked chroma vs. chrominance as
well, as I suggested in my volunteering comment on GitHub?

Best regards, Reto

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a1149cc82429b240562574edc-- From nobody Fri Jan 12 08:22:45 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3441312D943 for ; Fri, 12 Jan 2018 08:22:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.2 X-Spam-Level: X-Spam-Status: No, score=-1.2 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3pedKPhs2jPs for ; Fri, 12 Jan 2018 08:22:41 -0800 (PST) Received: from vie01a-dmta-pe04-2.mx.upcmail.net (vie01a-dmta-pe04-2.mx.upcmail.net [62.179.121.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7E37312E89F for ; Fri, 12 Jan 2018 08:22:41 -0800 (PST) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe04.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1ea26J-0005qg-2S for cellar@ietf.org; Fri, 12 Jan 2018 17:22:39 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id xUNX1w01P0S5wYM01UNYm6; Fri, 12 Jan 2018 17:22:33 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: Codec Encoding for LossLess Archiving and Realtime transmission Date: Fri, 12 Jan 2018 17:22:31 +0100 Message-Id: <20180112162231.6181-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.1 Archived-At: Subject: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 16:22:44 -0000 Idea-from: https://github.com/FFmpeg/FFV1/issues/47 Signed-off-by: Michael Niedermayer --- ffv1.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/ffv1.md b/ffv1.md index e57ab3e..c6a8433 100644 --- a/ffv1.md +++ b/ffv1.md @@ -836,7 +836,7 @@ Inferred to be 1 if not present. ### quant_table_set_index_count -`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`. +`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 ) + hamming_weight(extra_plane_mask)`. ### quant_table_set_index @@ -904,7 +904,7 @@ SliceContent( ) { | ### primary_color_count -`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 ). +`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 ) + hamming_weight(extra_plane_mask). ### plane_pixel_height @@ -1036,6 +1036,9 @@ Parameters( ) { | ec | ur intra | ur } | + if (version >= 4) { | + extra_plane_mask | ur + } | } | ``` @@ -1147,6 +1150,21 @@ RFC:`log2_v_chroma_subsample` indicates the subsample factor, stored in powers t | 0 | transparency plane is not present| | 1 | transparency plane is present | +### extra_plane_mask + +`extra_plane_mask` is a bitmask indicating additional planes beyond RGBA / YCbCrA. +Each 1 bit indicates one additional plane. + +|bit | color space used | +|-------|:---------------------------------| +| 0 | Near infra red | +| 1 | Mid infra red | +| 2 | Far infra red | +| 3 | Near ultra violet | +| 4 | Mid ultra violet | +| 5 | Far ultra violet | +| 24-31 | User defined planes | + ### num_h_slices `num_h_slices` indicates the number of horizontal elements of the slice raster. -- 2.15.1 From nobody Fri Jan 12 08:34:30 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3677D12E8AC for ; Fri, 12 Jan 2018 08:34:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TBBh6cYlIB-1 for ; Fri, 12 Jan 2018 08:34:27 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1D16712420B for ; Fri, 12 Jan 2018 08:34:27 -0800 (PST) Received: from [146.96.19.240] (port=34320 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1ea2HY-003B07-Gd; Fri, 12 Jan 2018 11:34:20 -0500 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Dave Rice In-Reply-To: <20180112162231.6181-1-michael@niedermayer.cc> Date: Fri, 12 Jan 2018 11:34:10 -0500 Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Transfer-Encoding: quoted-printable Message-Id: References: <20180112162231.6181-1-michael@niedermayer.cc> To: Michael Niedermayer X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 16:34:29 -0000 Hi Michael, > On Jan 12, 2018, at 11:22 AM, Michael Niedermayer = wrote: >=20 > Idea-from: https://github.com/FFmpeg/FFV1/issues/47 >=20 > Signed-off-by: Michael Niedermayer > --- > ffv1.md | 22 ++++++++++++++++++++-- > 1 file changed, 20 insertions(+), 2 deletions(-) >=20 > diff --git a/ffv1.md b/ffv1.md > index e57ab3e..c6a8433 100644 > --- a/ffv1.md > +++ b/ffv1.md > @@ -836,7 +836,7 @@ Inferred to be 1 if not present. >=20 > ### quant_table_set_index_count >=20 > -`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || = version \<=3D 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`. > +`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || = version \<=3D 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 ) + = hamming_weight(extra_plane_mask)`. The hamming_weight function needs to be defined. > ### quant_table_set_index >=20 > @@ -904,7 +904,7 @@ SliceContent( ) { = | >=20 > ### primary_color_count >=20 > -`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( = alpha_plane ? 1 : 0 ). > +`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( = alpha_plane ? 1 : 0 ) + hamming_weight(extra_plane_mask). nit: equation should be backtick-quoted > ### plane_pixel_height >=20 > @@ -1036,6 +1036,9 @@ Parameters( ) { = | > ec | ur > intra | ur > } | > + if (version >=3D 4) { | > + extra_plane_mask | ur > + } | > } | > ``` >=20 > @@ -1147,6 +1150,21 @@ RFC:`log2_v_chroma_subsample` indicates the = subsample factor, stored in powers t > | 0 | transparency plane is not present| > | 1 | transparency plane is present | >=20 > +### extra_plane_mask > + > +`extra_plane_mask` is a bitmask indicating additional planes beyond = RGBA / YCbCrA. The color space section may need to be updated to acknowledge the = possibility an extra planes. For instance "An FFV1 Frame using YCbCr = MUST use one of the following arrangements=E2=80=9D or perhaps in color = space there should be an acknowledge that FFV1 permits storage an extra = planes that are not part of the defined transformations. > +Each 1 bit indicates one additional plane. > + > +|bit | color space used | > +|-------|:---------------------------------| > +| 0 | Near infra red | > +| 1 | Mid infra red | > +| 2 | Far infra red | > +| 3 | Near ultra violet | > +| 4 | Mid ultra violet | > +| 5 | Far ultra violet | Is this list based on any source? > +| 24-31 | User defined planes | Perhaps some reserved values rather than letting all remaining values to = user definition. > + > ### num_h_slices >=20 > `num_h_slices` indicates the number of horizontal elements of the = slice raster. =20 > --=20 > 2.15.1 Thanks, Dave Rice From nobody Fri Jan 12 08:54:59 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A0DBF12762F for ; Fri, 12 Jan 2018 08:54:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MvY-1II8KWo8 for ; Fri, 12 Jan 2018 08:54:55 -0800 (PST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:c:538::198]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2CED712420B for ; Fri, 12 Jan 2018 08:54:55 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 58153FB87D for ; Fri, 12 Jan 2018 17:54:52 +0100 (CET) Date: Fri, 12 Jan 2018 17:54:51 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180112165451.GA17711@michaelspb> References: <20180112162231.6181-1-michael@niedermayer.cc> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WYTEVAkct0FjGQmd" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 16:54:58 -0000 --WYTEVAkct0FjGQmd Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi On Fri, Jan 12, 2018 at 11:34:10AM -0500, Dave Rice wrote: > Hi Michael, >=20 > > On Jan 12, 2018, at 11:22 AM, Michael Niedermayer wrote: > >=20 > > Idea-from: https://github.com/FFmpeg/FFV1/issues/47 > >=20 > > Signed-off-by: Michael Niedermayer > > --- > > ffv1.md | 22 ++++++++++++++++++++-- > > 1 file changed, 20 insertions(+), 2 deletions(-) > >=20 > > diff --git a/ffv1.md b/ffv1.md > > index e57ab3e..c6a8433 100644 > > --- a/ffv1.md > > +++ b/ffv1.md > > @@ -836,7 +836,7 @@ Inferred to be 1 if not present. > >=20 > > ### quant_table_set_index_count > >=20 > > -`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || = version \<=3D 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`. > > +`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || = version \<=3D 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 ) + hamming_weight(extr= a_plane_mask)`. >=20 > The hamming_weight function needs to be defined. >=20 > > ### quant_table_set_index > >=20 > > @@ -904,7 +904,7 @@ SliceContent( ) { = | > >=20 > > ### primary_color_count > >=20 > > -`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( = alpha_plane ? 1 : 0 ). > > +`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( = alpha_plane ? 1 : 0 ) + hamming_weight(extra_plane_mask). >=20 > nit: equation should be backtick-quoted will fix >=20 > > ### plane_pixel_height > >=20 > > @@ -1036,6 +1036,9 @@ Parameters( ) { = | > > ec | ur > > intra | ur > > } | > > + if (version >=3D 4) { | > > + extra_plane_mask | ur > > + } | > > } | > > ``` > >=20 > > @@ -1147,6 +1150,21 @@ RFC:`log2_v_chroma_subsample` indicates the subs= ample factor, stored in powers t > > | 0 | transparency plane is not present| > > | 1 | transparency plane is present | > >=20 > > +### extra_plane_mask > > + > > +`extra_plane_mask` is a bitmask indicating additional planes beyond RG= BA / YCbCrA. >=20 > The color space section may need to be updated to acknowledge the possibi= lity an extra planes. For instance "An FFV1 Frame using YCbCr MUST use one = of the following arrangements=E2=80=9D or perhaps in color space there shou= ld be an acknowledge that FFV1 permits storage an extra planes that are not= part of the defined transformations. will update >=20 > > +Each 1 bit indicates one additional plane. > > + > > +|bit | color space used | > > +|-------|:---------------------------------| > > +| 0 | Near infra red | > > +| 1 | Mid infra red | > > +| 2 | Far infra red | > > +| 3 | Near ultra violet | > > +| 4 | Mid ultra violet | > > +| 5 | Far ultra violet | >=20 > Is this list based on any source? no, it was just a random idea to start some discussion. I actually will post an alternative idea in a moment [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Let us carefully observe those good qualities wherein our enemies excel us and endeavor to excel them, by avoiding what is faulty, and imitating what is excellent in them. -- Plutarch --WYTEVAkct0FjGQmd Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpY6FsACgkQYR7HhwQLD6tRQwCglhT5X1zIg+4eQyclta3gsYzK h0oAmgLNhrlinJe5ipO+I8lPmajG7Jrz =Vodc -----END PGP SIGNATURE----- --WYTEVAkct0FjGQmd-- From nobody Fri Jan 12 08:58:19 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A80512E89C for ; Fri, 12 Jan 2018 08:58:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ssHuXh3BzJPC for ; Fri, 12 Jan 2018 08:58:04 -0800 (PST) Received: from vie01a-dmta-pe05-2.mx.upcmail.net (vie01a-dmta-pe05-2.mx.upcmail.net [84.116.36.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DF61412E89D for ; Fri, 12 Jan 2018 08:57:58 -0800 (PST) Received: from [172.31.216.43] (helo=vie01a-pemc-psmtp-pe01) by vie01a-dmta-pe05.mx.upcmail.net with esmtp (Exim 4.88) (envelope-from ) id 1ea2eS-00041y-Ou for cellar@ietf.org; Fri, 12 Jan 2018 17:57:56 +0100 Received: from localhost ([213.47.41.20]) by vie01a-pemc-psmtp-pe01 with SMTP @ mailcloud.upcmail.net id xUxo1w00t0S5wYM01UxpgL; Fri, 12 Jan 2018 17:57:49 +0100 X-SourceIP: 213.47.41.20 From: Michael Niedermayer To: Codec Encoding for LossLess Archiving and Realtime transmission Date: Fri, 12 Jan 2018 17:57:48 +0100 Message-Id: <20180112165748.6294-1-michael@niedermayer.cc> X-Mailer: git-send-email 2.15.1 Archived-At: Subject: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 16:58:19 -0000 Idea-from: https://github.com/FFmpeg/FFV1/issues/47 Signed-off-by: Michael Niedermayer --- ffv1.md | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/ffv1.md b/ffv1.md index e57ab3e..e8c2f22 100644 --- a/ffv1.md +++ b/ffv1.md @@ -297,7 +297,7 @@ In YCbCr color space, the Cb and Cr planes are optional, but if used then MUST b - Y, Cb, Cr - Y, Cb, Cr, Alpha -When FFV1 uses the YCbCr color space, the Y plane MUST be coded first. If the Cb and Cr planes are used then they MUST be coded after the Y plane. If an Alpha (transparency) plane is used, then it MUST be coded last. +When FFV1 uses the YCbCr color space, the Y plane MUST be coded first. If the Cb and Cr planes are used then they MUST be coded after the Y plane. If an Alpha (transparency) plane is used, then it MUST be coded after them. Any additional planes are coded after that. ### JPEG2000-RCT @@ -374,6 +374,8 @@ An FFV1 `Frame` using JPEG2000-RCT MUST use one of the following arrangements: - Y, Cb, Cr - Y, Cb, Cr, Alpha +After these additional components can be encoded. + When FFV1 uses the JPEG2000-RCT color space, the horizontal lines are interleaved to improve caching efficiency since it is most likely that the RCT will immediately be converted to RGB during decoding. The interleaved coding order is also Y, then Cb, then Cr, and then if used Alpha. As an example, a `Frame` that is two pixels wide and two pixels high, could be comprised of the following structure: @@ -836,7 +838,7 @@ Inferred to be 1 if not present. ### quant_table_set_index_count -`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`. +`quant_table_set_index_count` is defined as `primary_color_count - ( chroma_planes ? 2 : 0 ) + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 )`. ### quant_table_set_index @@ -904,7 +906,9 @@ SliceContent( ) { | ### primary_color_count -`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 ). +`primary_color_count` is either stored or if not, it is defined as +`1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 )`. +If it is stored it must be larger or equal to the default. ### plane_pixel_height @@ -1036,6 +1040,14 @@ Parameters( ) { | ec | ur intra | ur } | + if (version >= 4) { | + primary_color_count | ur + for( i = 0; i < primary_color_count; i++ ) { | + frequency[i] | ur + bandwidth[i] | ur + gamma[i] | ur + } | + } | } | ``` @@ -1147,6 +1159,17 @@ RFC:`log2_v_chroma_subsample` indicates the subsample factor, stored in powers t | 0 | transparency plane is not present| | 1 | transparency plane is present | +### frequency +`frequency` is the middle frequency of the given component in herz. +It is 0 if not applicable, for example for an alpha component or if it is unknown. +For Cb the blue channel shall be used for frequency and bandwidth. +For Cr the red channel shall be used for frequency and bandwidth. +For Y in YCbCr the green channel shall be used for frequency and bandwidth. + +### bandwidth +`bandwidth` is the bandwidth of of the given component in herz. +It is 0 if not applicable, for example for an alpha plane. + ### num_h_slices `num_h_slices` indicates the number of horizontal elements of the slice raster. -- 2.15.1 From nobody Fri Jan 12 12:02:44 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A0F91242F5 for ; Fri, 12 Jan 2018 12:02:42 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.921 X-Spam-Level: X-Spam-Status: No, score=-1.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iBMeTetEXpfr for ; Fri, 12 Jan 2018 12:02:39 -0800 (PST) Received: from 1.mo1.mail-out.ovh.net (1.mo1.mail-out.ovh.net [178.32.127.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F2C801241F8 for ; Fri, 12 Jan 2018 12:02:38 -0800 (PST) Received: from player691.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo1.mail-out.ovh.net (Postfix) with ESMTP id 18772CB4B0 for ; Fri, 12 Jan 2018 21:02:36 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: jerome@mediaarea.net) by player691.ha.ovh.net (Postfix) with ESMTPSA id 7344F26008C for ; Fri, 12 Jan 2018 21:02:36 +0100 (CET) To: cellar@ietf.org References: <20180112165748.6294-1-michael@niedermayer.cc> From: Jerome Martinez Message-ID: <119e17a1-dfec-726b-f887-0f36e94434fc@mediaarea.net> Date: Fri, 12 Jan 2018 21:02:36 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180112165748.6294-1-michael@niedermayer.cc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 1467047579942260881 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrledvgddufedtucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] [PATCH] ffv1.md: Add support for more planes beyond RGBA / YCbCrA X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 20:02:42 -0000 On 12/01/2018 17:57, Michael Niedermayer wrote: > [...] > > ### quant_table_set_index_count > > -`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )`. > +`quant_table_set_index_count` is defined as `primary_color_count - ( chroma_planes ? 2 : 0 ) + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 )`. As we can break everything thanks to version bump, I suggest to simplify it and separate completely data from each plane, so: +`quant_table_set_index_count` is defined as `1 + ( ( chroma_planes || version \<= 3 ) ? 1 : 0 ) + ( alpha_plane ? 1 : 0 )` if versions <= 3, else as ` primary_color_count` > > ### quant_table_set_index > > @@ -904,7 +906,9 @@ SliceContent( ) { | > > ### primary_color_count > > -`primary_color_count` is defined as 1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 ). > +`primary_color_count` is either stored or if not, it is defined as > +`1 + ( chroma_planes ? 2 : 0 ) + ( alpha_plane ? 1 : 0 )`. > +If it is stored it must be larger or equal to the default. > > ### plane_pixel_height > > @@ -1036,6 +1040,14 @@ Parameters( ) { | > ec | ur > intra | ur > } | > + if (version >= 4) { | > + primary_color_count | ur > + for( i = 0; i < primary_color_count; i++ ) { | > + frequency[i] | ur > + bandwidth[i] | ur > + gamma[i] | ur > + } | > + } | > } | > ``` As we can break everything thanks to version bump, I suggest to simplify it and separate completely data from each plane, and put a "if (version >= 4) sooner, e.g. after colorspace_type. It would also the advantage to avoid conflicting content (e.g. chroma_plane not present and alpha plane present but frequency is about blue channel, what to prioritize?) I had in mind, after v3 finalized, to suggest that (modified in order to comply with your patch new fields): just after colorspace and if version can, we could: - trash unique bits_per_raw_sample/chroma_plane/log2_h_chroma_subsample/log2_v_chroma_subsample/alpha_plane, - add primary_color_count, and for each primary color add bits_per_raw_sample/x_subsample/y_subsample/frequency/bandwidth/gamma Background: we have some files with different bit depth per component. A public file is https://github.com/openexr/openexr-images/blob/master/v2/LeftView/Balls.exr (16-bit for 4 first channels and 32-bit for 5th channel) but I see in H.265 that there is the possibility to have a different bit depth per component so I prefer to be future proof even if we don't use all possibilities now. Note that I am fully aware that they are corner cases but I think we could have a spec for such corner case even if the most used encoder/decoder does not support them (and is not requested to support them). > > @@ -1147,6 +1159,17 @@ RFC:`log2_v_chroma_subsample` indicates the subsample factor, stored in powers t > | 0 | transparency plane is not present| > | 1 | transparency plane is present | > > +### frequency > +`frequency` is the middle frequency of the given component in herz. > +It is 0 if not applicable, for example for an alpha component or if it is unknown. > +For Cb the blue channel shall be used for frequency and bandwidth. > +For Cr the red channel shall be used for frequency and bandwidth. > +For Y in YCbCr the green channel shall be used for frequency and bandwidth. > > > + > +### bandwidth > +`bandwidth` is the bandwidth of of the given component in herz. > +It is 0 if not applicable, for example for an alpha plane. > + I practice, what are the expected values for legacy RGBA or YCbCrA? I don't get how to do the difference between Alpha and something else not capable to be represented as frequency/bandwidth/gamma, if we have both in same file. For instance, I need (in a not so late future, in order to compress in a lossless manner some uncommon files) to store the following "colors": AR, AG, AB: Red,  green  and  blue  alpha/opacity,  for  colored mattes  (required  to  composite images of objects like colored glass correctly). Z: Distance of the front of a sample from the viewer. Z Back: Distance of the back of a sample from the viewer. Source: http://www.openexr.com/TechnicalIntroduction.pdf Note: the source document talks also about numerical identifier or nested components e.g. “light1.R”, “light1.G”, “light1.B”, “light2.R”, “light2.G”, “light2.B” but I never saw such file (and it would become hard to catch for applying JPEG2000-RCT) Maybe a choice with a "standard form" like what you suggest + a "non standard" form with free text? Jérôme From nobody Fri Jan 12 13:11:46 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EF55126C19 for ; Fri, 12 Jan 2018 13:11:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jyKr3BnkCkSl for ; Fri, 12 Jan 2018 13:11:41 -0800 (PST) Received: from 9.mo1.mail-out.ovh.net (9.mo1.mail-out.ovh.net [178.32.108.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E9FFB1200FC for ; Fri, 12 Jan 2018 13:11:40 -0800 (PST) Received: from player691.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo1.mail-out.ovh.net (Postfix) with ESMTP id E3EFBCC27F for ; Fri, 12 Jan 2018 22:11:38 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: jerome@mediaarea.net) by player691.ha.ovh.net (Postfix) with ESMTPSA id B5FE026008C for ; Fri, 12 Jan 2018 22:11:37 +0100 (CET) To: Codec Encoding for LossLess Archiving and Realtime transmission From: Jerome Martinez Message-ID: <96b8ef11-ab4d-99fd-b260-ebec6b563a6c@mediaarea.net> Date: Fri, 12 Jan 2018 22:11:38 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 2632635460638740625 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrledvgddugeegucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: [Cellar] [FFV1] Adding Bayer support X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 21:11:44 -0000 I added a pull request on GitHub: https://github.com/FFmpeg/FFV1/pull/100 Suggesting to use DgCoCg-R transformation before transformation for Bayer "color space". This is a new colorspace_type value so we could put it in FFV1 v0 to v3 without conflict with decoders in the wild. I did not send the patch sooner, waiting for another patch to be accepted and waiting for finishing my corresponding encoder (so it is currently theory), but as we are talking about adding more planes and the corresponding metadata, I would like to be sure we handle this case in FFV1 v4. Jérôme From nobody Sat Jan 13 03:17:41 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0931E1267BB for ; Sat, 13 Jan 2018 03:17:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cTVw-UaxMu1V for ; Sat, 13 Jan 2018 03:17:37 -0800 (PST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:c:538::198]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 94041124D6C for ; Sat, 13 Jan 2018 03:17:37 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id CF451FB882 for ; Sat, 13 Jan 2018 12:17:35 +0100 (CET) Date: Sat, 13 Jan 2018 12:17:34 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180113111734.GB17711@michaelspb> References: <96b8ef11-ab4d-99fd-b260-ebec6b563a6c@mediaarea.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="O3RTKUHj+75w1tg5" Content-Disposition: inline In-Reply-To: <96b8ef11-ab4d-99fd-b260-ebec6b563a6c@mediaarea.net> User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] [FFV1] Adding Bayer support X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jan 2018 11:17:40 -0000 --O3RTKUHj+75w1tg5 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 12, 2018 at 10:11:38PM +0100, Jerome Martinez wrote: > I added a pull request on GitHub: > https://github.com/FFmpeg/FFV1/pull/100 > Suggesting to use DgCoCg-R transformation before transformation for Bayer > "color space". > This is a new colorspace_type value so we could put it in FFV1 v0 to v3 > without conflict with decoders in the wild. >=20 > I did not send the patch sooner, waiting for another patch to be accepted > and waiting for finishing my corresponding encoder (so it is currently > theory), but as we are talking about adding more planes and the > corresponding metadata, I would like to be sure we handle this case in FF= V1 > v4. Is there some existing comparissions of YDgCoCg-R I found only "Custom Lossless Compression and High-Quality Lossy Compression of White Blood Cell Microscopy Images for Display and Machine Learning Applications" http://lup.lub.lu.se/luur/download?func=3DdownloadFile&recordOId=3D3810913&= fileOId=3D3810914 which in figure 4.1 compares 8 lossless compression strategies YDgCoCg in that is at the 3rd worst place, only 2 are worse Now looking at YDgCoCg specifically, and asking, are there obvious changes worth trying ? i see 2 immedeatly first the green channel has twice as many samples as the others, YDgCoCg wo= rks around this by spliting it into 2 channels using a very simple diagonal=20 transform. This should theoretically not be needed and would preserve more= =20 spatial correlation if its not done. To show what i mean, please look at the green array G G G G G G G G G G G G G G G G If you rotate the axis by 45=B0 you will see that there are no more missing= samples and you have with the exception of the border a full 2d array to compress. The boarder samples are comparably few. I guess and thats a pure guess that this would compress better than spliting this in 2 arrays and compressing them independant. The 2nd obvious improvment to try is that the tranform transforms samples that are spatially shifted relative to each other. I suspect if this is=20 compensated for my some means the seperation would work better. =20 with these suggestions you could still have a sepearte transform to do befo= re using the existing compression algorithm How can one try other transforms than YDgCoCg ? do you have some patch for the ffmpeg ffv1 implementation that one could experiment with to try different strategies ? [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In fact, the RIAA has been known to suggest that students drop out of college or go to community college in order to be able to afford settlements. -- The RIAA --O3RTKUHj+75w1tg5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpZ6s4ACgkQYR7HhwQLD6s+3wCgjy8SHF3XleGcpfwXevsHVGuf VGkAn0lYKR2AeoXMZ1pZvz4HUjwDMN6v =4TOx -----END PGP SIGNATURE----- --O3RTKUHj+75w1tg5-- From nobody Sat Jan 13 04:02:04 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6FD9712426E for ; Sat, 13 Jan 2018 04:02:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.92 X-Spam-Level: X-Spam-Status: No, score=-1.92 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zE77kwgXgSJn for ; Sat, 13 Jan 2018 04:01:59 -0800 (PST) Received: from 1.mo1.mail-out.ovh.net (1.mo1.mail-out.ovh.net [178.32.127.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8E85C12D835 for ; Sat, 13 Jan 2018 04:01:59 -0800 (PST) Received: from player691.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo1.mail-out.ovh.net (Postfix) with ESMTP id BFFE3CC05F for ; Sat, 13 Jan 2018 13:01:57 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: jerome@mediaarea.net) by player691.ha.ovh.net (Postfix) with ESMTPSA id 52F94260082 for ; Sat, 13 Jan 2018 13:01:57 +0100 (CET) To: cellar@ietf.org References: <96b8ef11-ab4d-99fd-b260-ebec6b563a6c@mediaarea.net> <20180113111734.GB17711@michaelspb> From: Jerome Martinez Message-ID: <9a28a229-0016-a7cb-e656-7ffcd0e690bc@mediaarea.net> Date: Sat, 13 Jan 2018 13:01:57 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <20180113111734.GB17711@michaelspb> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 17669028716949278865 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtuddrleeggdefjecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] [FFV1] Adding Bayer support X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jan 2018 12:02:02 -0000 On 13/01/2018 12:17, Michael Niedermayer wrote: > On Fri, Jan 12, 2018 at 10:11:38PM +0100, Jerome Martinez wrote: >> I added a pull request on GitHub: >> https://github.com/FFmpeg/FFV1/pull/100 >> Suggesting to use DgCoCg-R transformation before transformation for Bayer >> "color space". >> This is a new colorspace_type value so we could put it in FFV1 v0 to v3 >> without conflict with decoders in the wild. >> >> I did not send the patch sooner, waiting for another patch to be accepted >> and waiting for finishing my corresponding encoder (so it is currently >> theory), but as we are talking about adding more planes and the >> corresponding metadata, I would like to be sure we handle this case in FFV1 >> v4. > Is there some existing comparissions of YDgCoCg-R > > I found only "Custom Lossless Compression and High-Quality > Lossy Compression of White Blood Cell Microscopy > Images for Display and Machine Learning > Applications" > http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=3810913&fileOId=3810914 > > which in figure 4.1 compares 8 lossless compression strategies > YDgCoCg in that is at the 3rd worst place, only 2 are worse If I understand well the report after a quick review (I'll read it more a bit later), all but 1 have a similar compression ratio (+/- 1%), to be compared to affinity with current FFV1 bitstream (just changing JPEG2000-RCT to the suggested RCT). So there would be only 1 interesting method to check (9% better), and this is a completely different one (wavelet) with some risk about patents. Would it worth it? IMO it is not. > > Now looking at YDgCoCg specifically, and asking, are there obvious changes > worth trying ? > i see 2 immedeatly > first the green channel has twice as many samples as the others, YDgCoCg works > around this by spliting it into 2 channels using a very simple diagonal > transform. This should theoretically not be needed and would preserve more > spatial correlation if its not done. > To show what i mean, please look at the green array > G G G G > G G G G > G G G G > G G G G > > If you rotate the axis by 45 you will see that there are no more missing samples > and you have with the exception of the border a full 2d array to compress. > The boarder samples are comparably few. > I guess and thats a pure guess that this would compress better than spliting > this in 2 arrays and compressing them independant. I don't get "compressing them independent": the suggested RCT is focused on having a green difference, and I don't see how to do without that, as we would have a Green plane 2x bigger than the others so JPEG2000-RCT is not applicable. > > The 2nd obvious improvment to try is that the tranform transforms samples > that are spatially shifted relative to each other. I suspect if this is > compensated for my some means the seperation would work better. > > with these suggestions you could still have a sepearte transform to do before > using the existing compression algorithm > > How can one try other transforms than YDgCoCg ? > do you have some patch for the ffmpeg ffv1 implementation that one could > experiment with to try different strategies ? It is difficult for me to understand the suggested other methods. I suggest the following steps: - I try to find enough real world sample streams I can publicly share, and set up a test platform, for easy comparisons. - I implement YDgCoCg-R support and I run tests on it - Anyone having an idea for better compression implements it and runs it against the same sample streams - Based on the comparison (compression performance / speed performance / complexity), we pick one of the suggested methods. Jrme From nobody Sat Jan 13 04:09:39 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CC5912D835 for ; Sat, 13 Jan 2018 04:09:38 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qXl2itDmvKiw for ; Sat, 13 Jan 2018 04:09:36 -0800 (PST) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:c:538::194]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 823B712426E for ; Sat, 13 Jan 2018 04:09:36 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id D142FC5A4E for ; Sat, 13 Jan 2018 13:09:34 +0100 (CET) Date: Sat, 13 Jan 2018 13:09:33 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180113120933.GC17711@michaelspb> References: <96b8ef11-ab4d-99fd-b260-ebec6b563a6c@mediaarea.net> <20180113111734.GB17711@michaelspb> <9a28a229-0016-a7cb-e656-7ffcd0e690bc@mediaarea.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u65IjBhB3TIa72Vp" Content-Disposition: inline In-Reply-To: <9a28a229-0016-a7cb-e656-7ffcd0e690bc@mediaarea.net> User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] [FFV1] Adding Bayer support X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jan 2018 12:09:38 -0000 --u65IjBhB3TIa72Vp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 13, 2018 at 01:01:57PM +0100, Jerome Martinez wrote: > On 13/01/2018 12:17, Michael Niedermayer wrote: > >On Fri, Jan 12, 2018 at 10:11:38PM +0100, Jerome Martinez wrote: [...] > >How can one try other transforms than YDgCoCg ? > >do you have some patch for the ffmpeg ffv1 implementation that one could > >experiment with to try different strategies ? >=20 > It is difficult for me to understand the suggested other methods. >=20 > I suggest the following steps: > - I try to find enough real world sample streams I can publicly share, and > set up a test platform, for easy comparisons. > - I implement YDgCoCg-R support and I run tests on it > - Anyone having an idea for better compression implements it and runs it > against the same sample streams > - Based on the comparison (compression performance / speed performance / > complexity), we pick one of the suggested methods. thats very close to what i had in mind, i think this is a great plan about samples, please include both very sharp and more blurry (at pixel lev= el) samples. These 2 should behave quite different compression wise with sparse sampling of colors. Also high noise and low noise content should be included high ISO low light would produce very noisy content, low ISO bright light should produce low noise. [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB What does censorship reveal? It reveals fear. -- Julian Assange --u65IjBhB3TIa72Vp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpZ9v0ACgkQYR7HhwQLD6sqrACdFj3e8UMnZWmZQUPpXEV195er 1wMAnRfCKuyVKdkUhpyNETYRVmJZiH4N =9ykF -----END PGP SIGNATURE----- --u65IjBhB3TIa72Vp-- From nobody Sat Jan 13 07:26:06 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C92612D85F for ; Sat, 13 Jan 2018 07:26:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.654 X-Spam-Level: X-Spam-Status: No, score=0.654 tagged_above=-999 required=5 tests=[HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.652, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lzhIM7Qt_8JX for ; Sat, 13 Jan 2018 07:26:04 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1970112D848 for ; Sat, 13 Jan 2018 07:26:04 -0800 (PST) Received: from cpe-104-162-94-162.nyc.res.rr.com ([104.162.94.162]:48009 helo=[10.0.1.9]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eaNh4-002lsB-3K for cellar@ietf.org; Sat, 13 Jan 2018 10:26:03 -0500 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_C6425A68-4EBB-47E2-82F9-A8B0CCAB4EB0" Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Message-Id: Date: Sat, 13 Jan 2018 10:25:49 -0500 To: Codec Encoding for LossLess Archiving and Realtime transmission X-Mailer: Apple Mail (2.3445.4.7) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] [Matroska Tags] restructure tags lists into xml X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jan 2018 15:26:05 -0000 --Apple-Mail=_C6425A68-4EBB-47E2-82F9-A8B0CCAB4EB0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi all, The name, types, class, and descriptions of Matroska=E2=80=99s official = tags are currently in formatted text within a Markdown document, such as = in = https://github.com/Matroska-Org/matroska-specification/blob/c1f7802bd258e0= 847b4892f8a9499c2ea786d3de/tagging.md#nesting-information = . I propose = to move these tags into a structured xml document. The contents of the = xml can then be formatted into a RFC friendly form during the make = process. I think the advantages of this could be that the tags are = easier to maintain in xml and it may make it easier for developers to = integrate these vocabularies. Currently I have a draft of that xml at = https://github.com/Matroska-Org/matroska-specification/pull/220/files = = for comment (either on the xml itself or the general idea). I=E2=80=99ll = later add in the make process to add the same data into the Matroska = Tags RFC draft from the xml source rather than the current markdown = source. Kind Regards, Dave Rice --Apple-Mail=_C6425A68-4EBB-47E2-82F9-A8B0CCAB4EB0 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi = all,

The name, types, = class, and descriptions of Matroska=E2=80=99s official tags are = currently in formatted text within a Markdown document, such as = in https://github.com/Matroska-Org/matroska-specification/blob/c1f= 7802bd258e0847b4892f8a9499c2ea786d3de/tagging.md#nesting-information. = I propose to move these tags into a structured xml document. The = contents of the xml can then be formatted into a RFC friendly form = during the make process. I think the advantages of this could be that = the tags are easier to maintain in xml and it may make it easier for = developers to integrate these vocabularies. Currently I have a draft of = that xml at https://github.com/Matroska-Org/matroska-specification/pull/220= /files for comment (either on the xml itself or the general = idea). I=E2=80=99ll later add in the make process to add the same data = into the Matroska Tags RFC draft from the xml source rather than the = current markdown source.

Kind Regards,
Dave Rice

= --Apple-Mail=_C6425A68-4EBB-47E2-82F9-A8B0CCAB4EB0-- From nobody Sun Jan 14 07:33:19 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4CFC712D7EB for ; Sun, 14 Jan 2018 07:33:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.779 X-Spam-Level: X-Spam-Status: No, score=0.779 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VszOTkGQYHMn for ; Sun, 14 Jan 2018 07:33:16 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8FE02126BFD for ; Sun, 14 Jan 2018 07:33:16 -0800 (PST) Received: from cpe-104-162-94-162.nyc.res.rr.com ([104.162.94.162]:35146 helo=[10.0.1.9]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eakHS-0030zD-Fh for cellar@ietf.org; Sun, 14 Jan 2018 10:33:10 -0500 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_80410B60-E665-4D3D-9FDC-12C86F1B6801" Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Date: Sun, 14 Jan 2018 10:33:03 -0500 References: To: Codec Encoding for LossLess Archiving and Realtime transmission In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3445.4.7) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] [Matroska Tags] restructure tags lists into xml X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jan 2018 15:33:18 -0000 --Apple-Mail=_80410B60-E665-4D3D-9FDC-12C86F1B6801 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 13, 2018, at 10:25 AM, Dave Rice wrote: >=20 > Hi all, >=20 > The name, types, class, and descriptions of Matroska=E2=80=99s = official tags are currently in formatted text within a Markdown = document, such as in = https://github.com/Matroska-Org/matroska-specification/blob/c1f7802bd258e0= 847b4892f8a9499c2ea786d3de/tagging.md#nesting-information = . I propose = to move these tags into a structured xml document. The contents of the = xml can then be formatted into a RFC friendly form during the make = process. I think the advantages of this could be that the tags are = easier to maintain in xml and it may make it easier for developers to = integrate these vocabularies. Currently I have a draft of that xml at = https://github.com/Matroska-Org/matroska-specification/pull/220/files = = for comment (either on the xml itself or the general idea). I=E2=80=99ll = later add in the make process to add the same data into the Matroska = Tags RFC draft from the xml source rather than the current markdown = source. https://github.com/Matroska-Org/matroska-specification/pull/220 = is = updated with a revised XML representing the Matroska tagging vocabulary = and definitions. Also the Makefile process is updated so that the = tagging registry is removed from tagging.md but will not be built into = RFC outputs using a mix of the tagging.md files and the new xml. The RFC = txt output is identical before and after this pull request, but I think = these changes will help make the tagging registry more useful, allow = easier internationalization, and better allow us to extend the = definition of Matroska tags. Best Regards, Dave Rice --Apple-Mail=_80410B60-E665-4D3D-9FDC-12C86F1B6801 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On = Jan 13, 2018, at 10:25 AM, Dave Rice <dave@dericed.com> = wrote:

Hi all,

The name, types, class, and = descriptions of Matroska=E2=80=99s official tags are currently in = formatted text within a Markdown document, such as in https://github.com/Matroska-Org/matroska-specification/blob/c1f= 7802bd258e0847b4892f8a9499c2ea786d3de/tagging.md#nesting-information. = I propose to move these tags into a structured xml document. The = contents of the xml can then be formatted into a RFC friendly form = during the make process. I think the advantages of this could be that = the tags are easier to maintain in xml and it may make it easier for = developers to integrate these vocabularies. Currently I have a draft of = that xml at https://github.com/Matroska-Org/matroska-specification/pull/220= /files for comment (either on the xml itself or the general = idea). I=E2=80=99ll later add in the make process to add the same data = into the Matroska Tags RFC draft from the xml source rather than the = current markdown source.

https://github.com/Matroska-Org/matroska-specification/pull/220=  is updated with a revised XML representing the Matroska = tagging vocabulary and definitions. Also the Makefile process is updated = so that the tagging registry is removed from tagging.md but will not be = built into RFC outputs using a mix of the tagging.md files and the new = xml. The RFC txt output is identical before and after this pull request, = but I think these changes will help make the tagging registry more = useful, allow easier internationalization, and better allow us to extend = the definition of Matroska tags.

Best Regards,
Dave = Rice


= --Apple-Mail=_80410B60-E665-4D3D-9FDC-12C86F1B6801-- From nobody Mon Jan 15 11:21:07 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 824BB12EB8E for ; Mon, 15 Jan 2018 11:21:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.01 X-Spam-Level: X-Spam-Status: No, score=-2.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jFS2_chjc07u for ; Mon, 15 Jan 2018 11:21:03 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5EAEA12EB7E for ; Mon, 15 Jan 2018 11:21:02 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:37016) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1ebAJT-0002dC-1u; Mon, 15 Jan 2018 20:20:55 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 988846540E89; Mon, 15 Jan 2018 20:20:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1516044054; bh=U8KQzlH/H3B0JtQtAFfusGGwkKcF4Vsj2PjDkeT/whI=; h=From:To:Subject:Date:From; b=v3UEsbGsCWCNIT2TB/GjhbnWah+MSchSuCX92nKwdySMvWWrgwT24vmDRki1i61Go +jkDxSZFzAAOelEmiS6fVXOCXhZnE1XCUikkYVr2sKCDR8dy5LrOtcHMafBeh3+bjx sZCGJxQxhxO1TDu4iNbwU3seBZu3PQ/F2io2pthkItfrG4AJGk1ho6iHllwhCK9bul vdZlsQzm+iYHY0BFaobUA3EZYf6JQxH2XZawA/BCqcN/YQQgc7e+6zDXlYtnO+0EOn ++wXymZluWDQttOhC91nFnpRfFCB2Jo55gz4HhgwxDJ9Br8/INTAVwa76KlSYwF2NE OQfOGqIGYnU0EdAg4ckHWbpgh2lksoL+SxtsU+aSEmqxFyJsU5s19+s+nlxAz6j0tW FlFIFI09j4vfYXVpBvOIjnoJr5UbPl9UmDJ0sfCRZaY2fAIpJT6rKqcdM0yoTphbim I/iGebn/cJpWgR+61+TRVyUIPCuo8Lbhc+B7RA8H3Tr3JSWfSM4dmpbEXDjlnHyfF4 C3Q+cSHU7HMI63x6CE2hgnEx8ZaKeDo4XKqq61vNSWrMVvBpGD9+No+uEQF8GS+orG duUYZavkzi0Rw+r35c6Xjk5zoQ6G/z5d32LCTMUkWAe1ahxVgC2n/gIm+vXUMfC3ga YbKfLUgCJ8Gjm58WDnBYMpUM= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTP id E05682BC60C6; Mon, 15 Jan 2018 20:20:53 +0100 (CET) User-agent: mu4e 0.9.18; emacs 25.3.1 From: Moritz Bunkus To: Cellar list , help Questions Date: Mon, 15 Jan 2018 20:20:53 +0100 Message-ID: <87h8rnapyi.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: [Cellar] MKVToolNix v20.0.0 released X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2018 19:21:06 -0000 Well, hello everyone! The holidays are always a nice time for coding as the world seems to turn just a tad slower than usual and people are that tiny bit more relaxed. I took the opportunity to work on my backlog and fix quite a number of bugs in mkvmerge. I've also removed those options and features that I had deprecated a year ago. Another huge change was the near complete rewrite of mkvinfo's internals. The goal is to include its GUI into MKVToolNix GUI in the next release. It won't be a 1:1 copy; instead, the new GUI will have more features. A result is that the output generated by the command-line utility has changed in several ways. See the corresponding NEWS entry below for more details. An important change for package maintainers is the new requirement for the "cmark" library. Fedora/CentOS/openSUSE already contain the necessary package ("cmark-devel"), whereas Debian/Ubuntu don't just yet (there's the "cmark" package, but that only contains the binary, whereas MKVToolNix needs the library & header files). Here are the usual links: …to the source code: https://mkvtoolnix.download/source.html …to the binaries: https://mkvtoolnix.download/downloads.html The Windows and macOS binaries are available already. The Linux binaries are still being built and will be available of the course of the next couple of hours. Here are the NEWS since the previous release: ------------------------------------------------------------ # Version 20.0.0 "I Am The Sun" 2018-01-15 ## Important notes * Feature removal: several deprecated features have been removed: * mkvmerge: the deprecated options `--identify-verbose` (and its counterpart `-I`), `--identify-for-gui`, `--identify-for-mmg` and `--identification-format verbose-text` * all command line tools: support for the deprecated, old, proprietary format used for option files * all command line tools: support for passing command line options via the deprecated environment variables `MKVTOOLNIX_OPTIONS`, `MKVEXTRACT_OPTIONS`, `MKVINFO_OPTIONS`, `MKVMERGE_OPTIONS` and `MKVPROPEDIT_OPTIONS` * mkvinfo: most of its code was re-written in order to lay the groundwork for including its functionality in MKVToolNix GUI but with more features than the existing mkvinfo GUI. The result is that a lot of its output has been changed slightly while keeping the basic layout. Changes include but aren't limited to: * Several element names are a bit clearer (e.g. `Maximum cache` instead of `MaxCache`). * All timestamps and durations are now output as nanoseconds in formatted form (e.g. `01:23:45.67890123`). All additional formats (e.g. floating point numbers output in seconds or milliseconds) were removed. * Element names for chapters and tags are now translated if a translation is available. * Elements located in wrong positions within the Matroska document are handled better. While mkvinfo's output is mostly kept very stable, it is not designed to be parsed by other utilities. Even though I've tried hard to cram all changes and cleanups into this version, additional changes may be made in the next couple of releases depending on user feedback and bug reports. ## New features and enhancements * mkvmerge: AVC/h.264 packetizer (framed): access unit delimiter NALUs will now be removed. Implements #2173. ## Bug fixes * mkvmerge: AVC/h.264 parser: when fixing the bitstream timing information mkvmerge will now use exact representations of the desired field duration if possible. For example, when indicating 50 fields/second `num_units_in_tick` is set to 1 and `time_scale` to 50 instead of 5368709 and 268435456. Part of the fix for #1673. * mkvmerge: AVC/h.264 parser: mkvmerge no longer assumes that encountering sequence parameter set or picture parameter set NALUs signal the start of a new frame. Fixes #2179. * mkvmerge: AVC/h.264 packetizer (framed): when mkvmerge is told to fix the bitstream timing information, it will now update all SPS NALUs, not just the ones in the AVCC. Part of the fix for #1673. * mkvmerge: MPEG TS reader: TS packet payloads will only be treated as PES packets if the payload actually starts with a PES start code. The prior behavior led to wrong timestamps and potentially broken frame data. Fixes #2193. * mkvmerge: MPEG TS reader: mkvmerge will now drop incomplete PES packets as soon as an error is detected in the transport stream instead of passing the incomplete frame to the packetizer. An error is assumed either if the `transport_error_indicator` flag is set or if the value of the `continuity_counter` header field doesn't match the expected value. Fixes #2181. * mkvmerge: Opus: when re-muxing Opus from Matroska mkvmerge will now write "block duration" elements for all block groups where a "discard padding" is set, too. Fixes #2188. * mkvmerge: SRT reader: mkvmerge can now handle SRT files with timestamps without decimal places (e.g. `00:01:15` instead of `00:01:15.000`). * mkvmerge: read buffer I/O class: the class could get out of sync regarding the file position of the underlying file I/O class causing wrong data to be returned on subsequent read operations. One result was that trying to identifying MPLS files that refer to very short M2TS files caused mkvmerge to segfault. * mkvmerge: multiplexer core: if there's a gap in audio timestamps, a new block group/lace will be started for the first frame after each gap. Before the fix the frame after the gap was often stored in the previous block group causing the gap to be in the wrong place: at the end of that block group. Fixes #1700. * mkvextract: AVC/h.264: if two consecutive IDR frames with the same `idr_pic_id` parameter and no access unit delimiters are found between them, mkvextract will insert an access unit delimiter in order to signal the start of a new access unit. Fixes #1704. * MKVToolNix GUI: update check dialog: Markdown links will now be converted to clickable links. Fixes #2176. * build system: fixed a race condition when creating new directories if `rake` is run with `-jN` in newer versions of Ruby/`rake`. Fixes #2194. ## Build system changes * [cmark](https://github.com/commonmark/cmark), the CommonMark parsing and rendering library in C, is now required when building the GUIs. ------------------------------------------------------------ Have fun :) mosu From nobody Mon Jan 15 17:32:12 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79CEF12F251 for ; Mon, 15 Jan 2018 17:32:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vqa1gSOPGVfH for ; Mon, 15 Jan 2018 17:32:09 -0800 (PST) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFB5912F257 for ; Mon, 15 Jan 2018 17:32:08 -0800 (PST) Received: by mail-wm0-x234.google.com with SMTP id 143so5101848wma.5 for ; Mon, 15 Jan 2018 17:32:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=6a0HlZZZ3DF3++LeVNOgtCSWC1xTfY79FIiSDUMD+kk=; b=RqETqET5X6AcCzRazuQ/307+LbwPU7t1hsOnYnomXtsL9j1VMo2gatMBaBEUbIJ3jQ CCKwxYNxOpB28iwIv2e3sCaUI55RTKbftscIQRhs6G7AVqNa4bVRvROwYHo+o7na6Om9 L2V/vVzRyghGAoC6qR4oa5SwKfdOM4EggJ76fqJaj0S8Ts/DXh42TufPYexT+gZtlGFL 2Fjf7UJsfqYZhvAJl2ntrTzMHZJSwlY9dev8f+Yrwzn7YE+JczMFwzf3lpc/VaolHn1E rtYTlxFrppgY1sSHIS7E6dtOiZu25OOB0Z4cH55dVWEww5JNqgOAc17w++G6zacRkn4E 1v3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6a0HlZZZ3DF3++LeVNOgtCSWC1xTfY79FIiSDUMD+kk=; b=qxd80PnOat4EIS77ewaLbpRiG4lGAP1BHvzYKhD0UFAf2ilApYm4v6eRc8+6cJUB/1 53qs5g5NNwteG1L1Gj8u9HvaKJjxiNhNHQYTy7hWhsVok2a/bRKM8Zn51gQ2htz8jMBH 6RRPNdnthUZTOn77scMSePH3Vc/44cxMuKeawusVtOzEfN7LAv72wN9odYB2hQf0x8fS MIxp97ozJrtax5HcyIMMMOTi9z76ikHCLNvXIJBTOBfiZ5qhP1jpU6eyqdYOEtJPXgMA CfAeg8/nF8TorqHQ+TmPjdO4K3K3OpwGLe8mF/7kipL8gco02oqtDjn9mXM4QEV6VCde 8ELQ== X-Gm-Message-State: AKwxytdGfBhwhXwsmiDR06ysCIOrOhjI3U12nn/2PMt4qblP1EfKUAPg Kw54E8iAvVzhDK7tjTj81tH7+qsOI8e1DXSDD5I= X-Google-Smtp-Source: ACJfBovjDrkNK8w9oHiHx5Df7j/Gt6RK2NEQx0Z+oEU851wq9A3W+DW7k3jStIGkDjntXW5WsrXnJQ0Qb/jGhPJI6qQ= X-Received: by 10.80.194.89 with SMTP id t25mr23283813edf.233.1516066327276; Mon, 15 Jan 2018 17:32:07 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.212.33 with HTTP; Mon, 15 Jan 2018 17:32:06 -0800 (PST) From: Austin Einter Date: Tue, 16 Jan 2018 07:02:06 +0530 Message-ID: To: cellar@ietf.org Content-Type: multipart/alternative; boundary="94eb2c1cc990e4b1580562dab2e3" Archived-At: Subject: [Cellar] How to use libmatroska? X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 01:32:10 -0000 --94eb2c1cc990e4b1580562dab2e3 Content-Type: text/plain; charset="UTF-8" Say I have a MKV file. How do I parse it? How do I write a new MKV file? I hope libmatroska can do this. Is there any sample code / example that I can refer to read / write matroska files. Thanks Austin --94eb2c1cc990e4b1580562dab2e3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Say I have a MKV file.

How do I parse i= t?
How do I write a new MKV file?

I hope= libmatroska can do this.
Is there any sample code / example that I can = refer to read / write matroska files.

Thanks
Au= stin
--94eb2c1cc990e4b1580562dab2e3-- From nobody Tue Jan 16 07:53:58 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53254131524 for ; Tue, 16 Jan 2018 07:53:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.12 X-Spam-Level: X-Spam-Status: No, score=-1.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EoWX3VPRFsTI for ; Tue, 16 Jan 2018 07:53:53 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0C82E12D833 for ; Tue, 16 Jan 2018 07:53:53 -0800 (PST) Received: from [146.96.19.240] (port=65379 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1ebTYT-004B34-95; Tue, 16 Jan 2018 10:53:42 -0500 From: Dave Rice Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_993AA199-A126-4244-A6C8-125CEBF9FEEA" Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Date: Tue, 16 Jan 2018 10:50:00 -0500 In-Reply-To: Cc: cellar@ietf.org To: Austin Einter References: X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] How to use libmatroska? X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 15:53:55 -0000 --Apple-Mail=_993AA199-A126-4244-A6C8-125CEBF9FEEA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Austin, > On Jan 15, 2018, at 8:32 PM, Austin Einter = wrote: >=20 > Say I have a MKV file. >=20 > How do I parse it? For parsing an MKV file I suggest reviewing the EBML specification at = https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ = , particularly = the principles of the EBML Element and its Element ID, Element Data = Size, and Element Data. Understanding how to read EBML Elements will = help decipher the structure of a Matroska file (or any other EBML = document). Then the EBML Schema of that file type is needed in order to = understand the semantics of each element. For Matroska that EBML Schema = is available at = https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_ma= troska.xml = . > How do I write a new MKV file? >=20 > I hope libmatroska can do this. > Is there any sample code / example that I can refer to read / write = matroska files. >=20 > Thanks > Austin > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --Apple-Mail=_993AA199-A126-4244-A6C8-125CEBF9FEEA Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi Austin,

On Jan 15, 2018, at 8:32 PM, = Austin Einter <austin.einter@gmail.com> wrote:

Say I have a MKV file.

How do I parse it?

For parsing an MKV file I suggest reviewing the = EBML specification at https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/, = particularly the principles of the EBML Element and its Element ID, = Element Data Size, and Element Data. Understanding how to read EBML = Elements will help decipher the structure of a Matroska file (or any = other EBML document). Then the EBML Schema of that file type is needed = in order to understand the semantics of each element. For Matroska that = EBML Schema is available at https://github.com/Matroska-Org/matroska-specification/blob/mas= ter/ebml_matroska.xml.

How= do I write a new MKV file?

I hope libmatroska can do this.
Is there any = sample code / example that I can refer to read / write matroska = files.

ThanksAustin
_______________________________________________
Cellar = mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

= --Apple-Mail=_993AA199-A126-4244-A6C8-125CEBF9FEEA-- From nobody Wed Jan 17 12:34:00 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8C6B112E898 for ; Wed, 17 Jan 2018 12:33:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.71 X-Spam-Level: X-Spam-Status: No, score=-2.71 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pixsrCyXOyqQ for ; Wed, 17 Jan 2018 12:33:55 -0800 (PST) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC46C127275 for ; Wed, 17 Jan 2018 12:33:39 -0800 (PST) Received: by mail-it0-x22a.google.com with SMTP id b5so10780417itc.3 for ; Wed, 17 Jan 2018 12:33:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=y6/2XpVJ+iicGPNIIxNrTjQYRYiUzJkkKtrmQFdiMtQ=; b=XScdZb1giAlX25ljK/a5WNzoUZTLL+OoutSOBTMVVNd1GB9+jb88uNeNUvDXG0ErfT uJGD3hiU/1slkZNO9VA5gKyfGppsvXeVUvOKKvL9OZWUd/TLJ/25/+A4DZ90znJD0bh7 yHmU7Av7NIdXz29uk00I6UlkJTSJNG3T8AjUtmA0cUbf8ed+WImdVNoYtHA67xNZYTJ9 ddJifydgoeAu93prrOFp06+SuLlqclG8uQ5u890of4wjudUcd69T3Qeyxmdntr/Aldqb 2FeWv/BYMn5pWdiCHK2bbG0/ShU0yjSTSDJnrjafa59c2u1DpEPtfLwFUmwE8z/zqhri LUPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=y6/2XpVJ+iicGPNIIxNrTjQYRYiUzJkkKtrmQFdiMtQ=; b=qaQ3Qouuh9IXRtgu0nCc5QH34c+ENor8nbRpmo3PMrAg/qiOXiR31ypZ2Az08uCy/k s2Ix+Ei78+vmqUnDGQFJU8Mg6EHQzYlp/Ex+0ZMVQWOD2xaficiuqXeJxE68F6ewqJ7e 3knNwO3CrEK+9hM9LgPadR1fuG6S9QCtvs1E3mRe7nrn/XsJtJ1vkg/SBV2hz8o6w7iE 4RdmqOcsW/OeSKCfrUlA4eJxr3MGZxUZdhFLUQsI5ptfXprNZhVY2+tM2kxvnUINK8ss RsmVBxv+r5Y6HFbsUC2RZq5R5NyFd+DhkKBCEMZGfByBqiP7nEwR30tZ9faG+fKCvO7+ OuUA== X-Gm-Message-State: AKwxytc+qpnwBB5I2bCiGV7K2lDjA7kJfZsPlf87r6YEptlDV75yme9z RpNt17JhHSVnA/q4XeHZjGpsM6l8CbZDRBHJGYiC+Zl+ X-Google-Smtp-Source: ACJfBovA8CsIwboy7JwJiCbYma7Va0ywkVdUFiUD5ImIg6chmKH0ek66fueL7yVv7ulJkRXSBvnwqxWn4Az682la00c= X-Received: by 10.36.111.4 with SMTP id x4mr12457964itb.133.1516221218483; Wed, 17 Jan 2018 12:33:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.36.75 with HTTP; Wed, 17 Jan 2018 12:33:17 -0800 (PST) From: Michael Bradshaw Date: Wed, 17 Jan 2018 12:33:17 -0800 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a1144965a21d35b0562fec36f" Archived-At: Subject: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 20:33:59 -0000 --001a1144965a21d35b0562fec36f Content-Type: text/plain; charset="UTF-8" The EBML and Matroska specs currently don't mention the possibility of a stack overflow due to deeply nested recursive elements. Currently, there's no limit on the recursion depth (unless I've overlooked it somewhere). I think it would be worth adding to the security section of EBML that one type of attack on an EBML Reader could include deep element recursion. Additionally, I would like to see what people think about potentially adding/suggesting an upper limit on recursion (either as a MUST or a MAY). This could also include a lower limit too. For example, something like "a parser SHOULD handle recursion up to X levels deep, and MAY abort the parse if it reaches Y levels deep". Thoughts from others? --001a1144965a21d35b0562fec36f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The EBML and Matroska specs currently don't mention th= e possibility of a stack overflow due to deeply nested recursive elements. = Currently, there's no limit on the recursion depth (unless I've ove= rlooked it somewhere).

I think it would be worth adding = to the security section of EBML that one type of attack on an EBML Reader c= ould include deep element recursion.

Additionally,= I would like to see what people think about potentially adding/suggesting = an upper limit on recursion (either as a MUST or a MAY). This could also in= clude a lower limit too. For example, something like "a parser SHOULD = handle recursion up to X levels deep, and MAY abort the parse if it reaches= Y levels deep".

Thoughts from others?
<= /div> --001a1144965a21d35b0562fec36f-- From nobody Wed Jan 17 12:38:57 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 45F0812E896 for ; Wed, 17 Jan 2018 12:38:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IeHtjDEK0SIQ for ; Wed, 17 Jan 2018 12:38:53 -0800 (PST) Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 65F1C12E89D for ; Wed, 17 Jan 2018 12:38:53 -0800 (PST) Received: by mail-qt0-x232.google.com with SMTP id c2so24912733qtn.9 for ; Wed, 17 Jan 2018 12:38:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=pXwDXK+7Z+d8J2TcjiEcro1eIFpFx54MhKXby6dqOss=; b=iDx1RElPFQbygaOpPwdQXlhCGJXy4+SIjdv4Rlq4kQVOpwLI2l7jPBzmmCXM2B5Pey KPiSrpHQn1Px445Xrre5uYsRjqiNK3kWKPHs1hpk/WGKrhLvf/0ovdfJkdMS4pSftZD0 jwnaVQm8jUCkNSAUCyUEU/pVU4k5fv8EqKab7KgCHmnfT0WV9rbr0US1GVCZUI2jfix9 FcnzRD66ZFaagq6Em1MeFkrM9JbUL1kbfssATjVMdeFOld7CsJFkXbQ9/82ApNQB6KOD TKKFe6wMsT0xgbS/8IPslFAH3jLkNvTznbFJzCNDdvKL2H3esd7DxXhbwEVXb4vHhbG7 AS0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=pXwDXK+7Z+d8J2TcjiEcro1eIFpFx54MhKXby6dqOss=; b=fsUg+09AFghsBDDia7Y0z6DK34ufiPPQwXGrgr+C7oXeytPIQR/+LTKiHbJMCsPvj5 +uLi1Cn8drRystMmyjFsnYCPT5UjZ76jnbqnFhOulwXHwnNgizzj9cBZryWVSWYYTuiE 6FsfZgJq1U2iU2G6JjCAP8w4u7+7uuDnS6vo6XCWFBuyoqdgop8mr18lpOzOoVZCs15z d3vzDxU6iadqLHpAsztIqfCwxAV6BqkDkArj1sy+A3rv8FmU5i9W5LMY/VCxGRN9W3E8 iO7sIOKYtoRqKK1BzwPxMK55xTj/Gjlv/U/iJR1A27G4Rvi3YJ0IPs7NzoZlbLUJsggN eLKA== X-Gm-Message-State: AKwxytfkHMh8ynTOOggK/xQpRxVcCCcdGvV/93DHfoaDevtjbLet4IrG Habrw/IoonR2blXzsFARbsKrga4/Umnvo6yvoiA= X-Google-Smtp-Source: ACJfBovFMAEaw2hxjT0onw7sazEuHBRE8hyPviFNy9CLzTEQnK+/DnAby23kMjgnImH5DxEeeO777nKo9+GCF25UXHs= X-Received: by 10.55.212.69 with SMTP id l66mr50522727qki.252.1516221532485; Wed, 17 Jan 2018 12:38:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.163 with HTTP; Wed, 17 Jan 2018 12:38:51 -0800 (PST) In-Reply-To: References: From: Ashley Blewer Date: Wed, 17 Jan 2018 15:38:51 -0500 Message-ID: To: Michael Bradshaw Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a1149cc82d83dd20562fed525" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 20:38:55 -0000 --001a1149cc82d83dd20562fed525 Content-Type: text/plain; charset="UTF-8" I agree with this. +1 On Wed, Jan 17, 2018 at 3:33 PM, Michael Bradshaw wrote: > The EBML and Matroska specs currently don't mention the possibility of a > stack overflow due to deeply nested recursive elements. Currently, there's > no limit on the recursion depth (unless I've overlooked it somewhere). > > I think it would be worth adding to the security section of EBML that one > type of attack on an EBML Reader could include deep element recursion. > > Additionally, I would like to see what people think about potentially > adding/suggesting an upper limit on recursion (either as a MUST or a MAY). > This could also include a lower limit too. For example, something like "a > parser SHOULD handle recursion up to X levels deep, and MAY abort the parse > if it reaches Y levels deep". > > Thoughts from others? > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a1149cc82d83dd20562fed525 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I agree with this. +1
=
On Wed, Jan 17, 2018 at 3:33 PM, Michael Bra= dshaw <mjbshaw@google.com> wrote:
The EBML and Matroska specs currently don't m= ention the possibility of a stack overflow due to deeply nested recursive e= lements. Currently, there's no limit on the recursion depth (unless I&#= 39;ve overlooked it somewhere).

I think it would be wort= h adding to the security section of EBML that one type of attack on an EBML= Reader could include deep element recursion.

Addi= tionally, I would like to see what people think about potentially adding/su= ggesting an upper limit on recursion (either as a MUST or a MAY). This coul= d also include a lower limit too. For example, something like "a parse= r SHOULD handle recursion up to X levels deep, and MAY abort the parse if i= t reaches Y levels deep".

Thoughts from other= s?

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a1149cc82d83dd20562fed525-- From nobody Wed Jan 17 12:56:08 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70FB612D850 for ; Wed, 17 Jan 2018 12:56:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DBFN3OB0uFm5 for ; Wed, 17 Jan 2018 12:56:05 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BFB7812E8B1 for ; Wed, 17 Jan 2018 12:55:52 -0800 (PST) Received: from smtp8.infomaniak.ch (smtp8.infomaniak.ch [83.166.132.38]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0HKtomv029311 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 17 Jan 2018 21:55:50 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp8.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0HKtoXX125077 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 17 Jan 2018 21:55:50 +0100 Date: Wed, 17 Jan 2018 21:55:50 +0100 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 20:56:07 -0000 Michael Bradshaw wrote: >Additionally, I would like to see what people think about >potentially adding/suggesting an upper limit on recursion >(either as a MUST or a MAY). I don't think that a MUST is appropriate here. Best regards, Reto From nobody Wed Jan 17 13:01:47 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 98BC312E9A1 for ; Wed, 17 Jan 2018 13:01:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.92 X-Spam-Level: X-Spam-Status: No, score=-1.92 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7iCcCnxE9gpu for ; Wed, 17 Jan 2018 13:01:44 -0800 (PST) Received: from 10.mo5.mail-out.ovh.net (10.mo5.mail-out.ovh.net [46.105.52.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A74A12E8B1 for ; Wed, 17 Jan 2018 13:01:44 -0800 (PST) Received: from player786.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 4C32917C6CA for ; Wed, 17 Jan 2018 22:01:41 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: jerome@mediaarea.net) by player786.ha.ovh.net (Postfix) with ESMTPSA id 5B7A08008A for ; Wed, 17 Jan 2018 22:01:41 +0100 (CET) To: cellar@ietf.org References: From: Jerome Martinez Message-ID: Date: Wed, 17 Jan 2018 22:01:41 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-Ovh-Tracer-Id: 13381601869916606609 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 50 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtvddrtddvgdduvdduucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenucgoteefjeefqddtgeculdehtddm Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 21:01:46 -0000 On 17/01/2018 21:33, Michael Bradshaw wrote: > [...] > > For example, something like "a parser SHOULD handle recursion up to X > levels deep, and MAY abort the parse if it reaches Y levels deep". > > Thoughts from others? an EBML parser can not dig in nested element without the corresponding dictionary (from DocType). So IMO it is not like XML or JSON, because XML or JSON parser tries to read the nested elements, but n EBML parser does not try to read any element not in the dictionary (so the limit is the dictionary, usually can not be provided by the attacker. It is lie JSON or XML only if the attacker can provide the dictionary. And the JSON spec specifies no limit: https://tools.ietf.org/html/rfc7159#section-9 "An implementation may set limits on the maximum depth of nesting" There may be legitimate reason to have thousands of nesting level, and there is already a lot of debates about that with JSON (last tests I saw about that is ~100 levels for Ruby JSON default parser and ~1000 levels for Python JSON parser) So the sentence should split security issues, from the input (the file itself) or the dictionary (e.g. matroska specs): - a generic parser (reading the dictionary from input) MAY set limits on the maximum depth of nesting. An implementation MAY set limits on the length and contents. - a specialized parser (e.g. Matroska parser) SHOULD handle recursion up to the maximum nesting level provided by the supported dictionary of the document, or 2 nesting levels, whichever is smaller (a specialized format could have only 1 nesting level but EBML needs at least 2, for DocType etc...). An alternative is to say to rely on the supported format security paragraph. I am not in favor of writing a number, because there is no good number to provide, it depends too much of the content, I am in favor of doing like JSON sentences (without any number, but explicitly saying that limits are expected). I hesitate in writing such kind of text in a "parser" section instead of security section, like the JSON RFC does. From nobody Wed Jan 17 13:09:30 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93DFA12D84D for ; Wed, 17 Jan 2018 13:09:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f47K_NKobDnJ for ; Wed, 17 Jan 2018 13:09:27 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2EC0512D82E for ; Wed, 17 Jan 2018 13:09:27 -0800 (PST) Received: from smtp7.infomaniak.ch (smtp7.infomaniak.ch [83.166.132.30]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0HL9PQn012703 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 17 Jan 2018 22:09:25 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp7.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0HL9O4t057972 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Wed, 17 Jan 2018 22:09:25 +0100 Date: Wed, 17 Jan 2018 22:09:25 +0100 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 21:09:29 -0000 Jerome Martinez wrote: >I am not in favor of writing a number, because there is no good >number to provide, Me neither. >I hesitate in writing such kind of text in a "parser" section >instead of security section, like the JSON RFC does. I would suggest in the parser section. Best regards, Reto From nobody Wed Jan 17 14:32:02 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1678F12EA93 for ; Wed, 17 Jan 2018 14:32:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wUfAefjfZB-X for ; Wed, 17 Jan 2018 14:31:59 -0800 (PST) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34BE712EA8C for ; Wed, 17 Jan 2018 14:31:59 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id A4C7E172094 for ; Wed, 17 Jan 2018 23:31:57 +0100 (CET) Date: Wed, 17 Jan 2018 23:31:56 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20180117223156.GD3493@michaelspb> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UoPmpPX/dBe4BELn" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 22:32:01 -0000 --UoPmpPX/dBe4BELn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jan 17, 2018 at 12:33:17PM -0800, Michael Bradshaw wrote: > The EBML and Matroska specs currently don't mention the possibility of a > stack overflow due to deeply nested recursive elements. Currently, there's > no limit on the recursion depth (unless I've overlooked it somewhere). >=20 > I think it would be worth adding to the security section of EBML that one > type of attack on an EBML Reader could include deep element recursion. >=20 > Additionally, I would like to see what people think about potentially > adding/suggesting an upper limit on recursion (either as a MUST or a MAY). > This could also include a lower limit too. For example, something like "a > parser SHOULD handle recursion up to X levels deep, and MAY abort the par= se > if it reaches Y levels deep". If a limit is specified then theres a limit that every compliant parser must support and which a compliant muxer can use. If no such limit is specified then it is possible that in a system where all parts are specification compliant and otherwise compatible it doesnt wo= rk. So I think its more a question about what/when compatibility is guranteed / what compatibility we want the specification to gurantee =66rom that it follows if we need a limit in the specification/draft or not [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If a bugfix only changes things apparently unrelated to the bug with no further explanation, that is a good sign that the bugfix is wrong. --UoPmpPX/dBe4BELn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlpfztwACgkQYR7HhwQLD6sJ8ACfT6xlUX8gPwOCoszqqBgcfLco Xy8AoJDdSA9KaLzZACeMLhFYCJgo5QVE =c1r+ -----END PGP SIGNATURE----- --UoPmpPX/dBe4BELn-- From nobody Wed Jan 17 14:48:01 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 82E1412D886 for ; Wed, 17 Jan 2018 14:47:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.709 X-Spam-Level: X-Spam-Status: No, score=-2.709 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jy6js4Gvej2i for ; Wed, 17 Jan 2018 14:47:57 -0800 (PST) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B53FB12D875 for ; Wed, 17 Jan 2018 14:47:57 -0800 (PST) Received: by mail-it0-x233.google.com with SMTP id q8so11246646itb.2 for ; Wed, 17 Jan 2018 14:47:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1Eaayh+JBVio0z7yc2g5F+9SVL8g3kK0FE1wBKrVHds=; b=cX8hLArWxD3y5hOgwZa0eDDDa7Y9Q22eoXjx6HYnZsWeU2DD1xIQVXUTdEv3mE6crj Dey4mL5eyRDv+iT3GUqlWQ8a2W5qqZqeYZYtg4JjpMqaDXa10PJIpv2/kj+Wd9+5i8fQ z2GYITTrV6stDbKGeGeuKJX3sbBzw1vydr3/IRNQ5NOWpv1VM57ELpO9/nFTP3mlv5dh /FhgBv7h/tHmao+C9+fAKDfrL2mxh5psswHg7d5tfVD02y+p2t4F+gQuwohgMHwbvnwy 3LC4BtQJScz8dM5hH6Bq4uTeDANIvTXR50c7SaxQ9ypQJo/BePM0c1/+OvRIyQiOMeoo RyTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1Eaayh+JBVio0z7yc2g5F+9SVL8g3kK0FE1wBKrVHds=; b=g9DZFC5fvf1aQp8PE9WL+OUwn77/Q+22+JAYS5k+XJq5H0BIf7qrMZ/qH312P0J6F3 OFgnY05PGaAmGYBYYlbyXw2gNqE724G5m4AOFWyBq9C4KKMct/Z79GuQhhXptXlkNGMY nlxei44MYQO0mLfrMnzydnCplAqW6XvY3mYYL790OU05yFPBwFtbCpYzjX6AIjoYWOOg ddzsfTtwvk2XpZyzLcK+fu6DFZ47qWMVQ/+1iWdgjCHuprUZ7LBi2YVfKIfa10mO9WEH fj4s98CSjbtx+9J6hJjBx8gRA0q7WFZSg8M3WyDdsBB8ZrDRyAnSTYaw3WWbmBD8HlmX fpZQ== X-Gm-Message-State: AKwxytdIojGzmP7h3ZCwIS5VZmIkRC7KRw0yysOt7BdM3yAJBhVOUXxj kliTFx3h+emUbIf25HGb6fE2nkbE/d5fhbqFaeVsCy71WF4= X-Google-Smtp-Source: ACJfBosfjlfLCnddidzpGVznIH84yXBVVapdO6Yeitve/MJ4KaO9YGQ9minZty+Bwsc2xxsK92tYpVSyI2cHIyudMVk= X-Received: by 10.36.20.145 with SMTP id 139mr16090619itg.15.1516229276543; Wed, 17 Jan 2018 14:47:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.36.75 with HTTP; Wed, 17 Jan 2018 14:47:35 -0800 (PST) In-Reply-To: <20180117223156.GD3493@michaelspb> References: <20180117223156.GD3493@michaelspb> From: Michael Bradshaw Date: Wed, 17 Jan 2018 14:47:35 -0800 Message-ID: To: Michael Niedermayer Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a1143e54c6da1c4056300a3ee" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 22:47:59 -0000 --001a1143e54c6da1c4056300a3ee Content-Type: text/plain; charset="UTF-8" Thanks for the responses, all! I agree that setting minimum/maximum recursion limits doesn't belong in the EBML spec. I do think the EBML spec should have a note in the Security Considerations that EBML Readers need to be cognizant of how they handle recursive elements and should be resilient to deeply nested recursion attacks (without giving particular recommendations for how that resilience is achieved). But EBML itself shouldn't attempt to set any specific limits on recursion depth. The reason I suggested setting some kind of minimum/maximum recursion depth (for Matroska specifically, not for EBML in general) is because of the points Michael Niedermayer raises. Setting a minimum allows muxers to write files with confidence that readers will be able to use/play the data being written (so long as the minimum isn't exceeded). Setting a maximum let's muxers know that if they exceed it, they run the risk of making a file that some readers can't (fully) play. Specifying recursion limits using the example language in my first email ("a parser SHOULD handle recursion up to X levels deep, and MAY abort the parse if it reaches Y levels deep") for ChapterAtom and SimpleTag elements would allow a muxing program to emit a warning to the user if the recursion level exceeded Y (it's not a fatal error, but it would be nice to let the user know their file has abnormally deep recursion that might not be compatible with various parsers). On Wed, Jan 17, 2018 at 2:31 PM, Michael Niedermayer wrote: > On Wed, Jan 17, 2018 at 12:33:17PM -0800, Michael Bradshaw wrote: > > The EBML and Matroska specs currently don't mention the possibility of a > > stack overflow due to deeply nested recursive elements. Currently, > there's > > no limit on the recursion depth (unless I've overlooked it somewhere). > > > > I think it would be worth adding to the security section of EBML that one > > type of attack on an EBML Reader could include deep element recursion. > > > > > Additionally, I would like to see what people think about potentially > > adding/suggesting an upper limit on recursion (either as a MUST or a > MAY). > > This could also include a lower limit too. For example, something like "a > > parser SHOULD handle recursion up to X levels deep, and MAY abort the > parse > > if it reaches Y levels deep". > > If a limit is specified then theres a limit that every compliant parser > must support and which a compliant muxer can use. > > If no such limit is specified then it is possible that in a system where > all parts are specification compliant and otherwise compatible it doesnt > work. > > So I think its more a question about what/when compatibility is guranteed > / what compatibility we want the specification to gurantee > from that it follows if we need a limit in the specification/draft or not > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > If a bugfix only changes things apparently unrelated to the bug with no > further explanation, that is a good sign that the bugfix is wrong. > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > --001a1143e54c6da1c4056300a3ee Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the responses, all!

I agree that setting minimum/maximum recursion limits doesn't belong= in the EBML spec. I do think the EBML spec should have a note in the Secur= ity Considerations that EBML Readers need to be cognizant of how they handl= e recursive elements and should be resilient to deeply nested recursion att= acks (without giving particular recommendations for how that resilience is = achieved). But EBML itself shouldn't attempt to set any specific limits= on recursion depth.

The reason I suggested settin= g some kind of minimum/maximum recursion depth (for Matroska specifically, = not for EBML in general) is because of the points=C2=A0Michael Niedermayer = raises. Setting a minimum allows muxers to write files with confidence that= readers will be able to use/play the data being written (so long as the mi= nimum isn't exceeded). Setting a maximum let's muxers know that if = they exceed it, they run the risk of making a file that some readers can= 9;t (fully) play.

Specifying recursion limits usin= g the example language in my first email ("a parser SHOULD handle recu= rsion up to X levels deep, and MAY abort the parse if it reaches Y levels d= eep") for ChapterAtom and SimpleTag elements would allow a muxing prog= ram to emit a warning to the user if the recursion level exceeded Y (it'= ;s not a fatal error, but it would be nice to let the user know their file = has abnormally deep recursion that might not be compatible with various par= sers).

On Wed, Jan 17, 2018 at 2:31 PM, Michael Niedermayer <= ;michael@nieder= mayer.cc> wrote:
On Wed, Jan 17, 2018 at 12:33:17PM -0800, Michael Bradshaw wrote:<= br> > The EBML and Matroska specs currently don't mention the possibilit= y of a
> stack overflow due to deeply nested recursive elements. Currently, the= re's
> no limit on the recursion depth (unless I've overlooked it somewhe= re).
>
> I think it would be worth adding to the security section of EBML that = one
> type of attack on an EBML Reader could include deep element recursion.=
>

> Additionally, I would like to see what people think about potentially<= br> > adding/suggesting an upper limit on recursion (either as a MUST or a M= AY).
> This could also include a lower limit too. For example, something like= "a
> parser SHOULD handle recursion up to X levels deep, and MAY abort the = parse
> if it reaches Y levels deep".

If a limit is specified then theres a limit that every compliant par= ser
must support and which a compliant muxer can use.

If no such limit is specified then it is possible that in a system where all parts are specification compliant and otherwise compatible it doesnt wo= rk.

So I think its more a question about what/when compatibility is guranteed / what compatibility we want the specification to gurantee
from that it follows if we need a limit in the specification/draft or not
[...]

--
Michael=C2=A0 =C2=A0 =C2=A0GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC= 787040B0FAB

If a bugfix only changes things apparently unrelated to the bug with no
further explanation, that is a good sign that the bugfix is wrong.

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar

--001a1143e54c6da1c4056300a3ee-- From nobody Wed Jan 17 23:42:48 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FC8512D944 for ; Wed, 17 Jan 2018 23:42:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7j9LVCYjRKN4 for ; Wed, 17 Jan 2018 23:42:45 -0800 (PST) Received: from 8.mo5.mail-out.ovh.net (8.mo5.mail-out.ovh.net [178.32.116.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 771FD12D832 for ; Wed, 17 Jan 2018 23:42:45 -0800 (PST) Received: from player786.ha.ovh.net (b7.ovh.net [213.186.33.57]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 2D2BB17C6BD for ; Thu, 18 Jan 2018 08:42:42 +0100 (CET) Received: from [192.168.2.120] (p5DDB56EF.dip0.t-ipconnect.de [93.219.86.239]) (Authenticated sender: jerome@mediaarea.net) by player786.ha.ovh.net (Postfix) with ESMTPSA id 903E9800A6 for ; Thu, 18 Jan 2018 08:42:41 +0100 (CET) To: cellar@ietf.org References: <20180117223156.GD3493@michaelspb> From: Jerome Martinez Message-ID: <082fb94e-75ed-bb3f-462d-c56a347af693@mediaarea.net> Date: Thu, 18 Jan 2018 08:42:42 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 5760666874554224785 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedtvddrtdefgdduudegucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 07:42:48 -0000 On 17/01/2018 23:47, Michael Bradshaw wrote: > Thanks for the responses, all! > > I agree that setting minimum/maximum recursion limits doesn't belong > in the EBML spec. I do think the EBML spec should have a note in the > Security Considerations that EBML Readers need to be cognizant of how > they handle recursive elements I missed the case of recursive elements, specified in EBML. for a security section, I don't think that talking about a parser is right, reading https://tools.ietf.org/html/rfc3552 it seems to more about usage of the correct behavior of a protocol rather than badly implemented parser, I am more in favor of  something like "it is possible to have unlimited recursion, potential source of denial of service attacks" in security section and "An implementation may set limits on the maximum depth of nesting" in a parser section (similar to JSON RFC). > and should be resilient to deeply nested recursion attacks (without > giving particular recommendations for how that resilience is > achieved). But EBML itself shouldn't attempt to set any specific > limits on recursion depth. > > The reason I suggested setting some kind of minimum/maximum recursion > depth (for Matroska specifically, not for EBML in general) is because > of the points Michael Niedermayer raises. Setting a minimum allows > muxers to write files with confidence that readers will be able to > use/play the data being written (so long as the minimum isn't exceeded). We don't talk anymore about security, and confidence depends on the underlying document e.g. for my EBML document x, I may need only 1 level beside the EBML header and it is fine with that, no need of more. So: - A minimum should not be in Security section (it is not about security) - A minimum depends on the goal of the parser, e.g. a parser just checking top level element size and CRC for checking a file transfer is fine does not need more than 2 levels. Not all tools are video players. > Setting a maximum let's muxers know that if they exceed it, they run > the risk of making a file that some readers can't (fully) play. As far as I know, there is nothing mandatory: tags are not mandatory, a specific stream format is not mandatory, it would be a full new part if we say what a reader should play, with lot of debates (chapters mandatory? tags display mandatory? FFV1 mandatory? H.265 and its royalties mandatory? NTSC only or 8K? Subtitles support mandatory? color description mandatory? maximum length of a text in a tag? can we have a gap in streams?). IMO we talk about a file format and not about what a reader expects to fully play. > > Specifying recursion limits using the example language in my first > email ("a parser SHOULD handle recursion up to X levels deep, and MAY > abort the parse if it reaches Y levels deep") for ChapterAtom and > SimpleTag elements would allow a muxing program to emit a warning to > the user if the recursion level exceeded Y (it's not a fatal error, > but it would be nice to let the user know their file has abnormally > deep recursion that might not be compatible with various parsers). But a player may just ignore any chapter and tag elements and still play fine the file. IMO expected supported elements of Matroska is a different document (e.g. a document about a reader capability? I usually see that with a "format compliance program", having lot of sample files), not in format description which is more general and does not say what element is mandatory, just how to store it. I don't say that it is a bad idea, just that I don't think that saying minimal requirements or warnings about potentially unsupported nesting level should be in a bitstream specification permitting lot of optional parts. Jérôme From nobody Thu Jan 18 00:22:37 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8BDD4124D68 for ; Thu, 18 Jan 2018 00:22:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.221 X-Spam-Level: X-Spam-Status: No, score=-4.221 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iRyKdWgwnZ-t for ; Thu, 18 Jan 2018 00:22:34 -0800 (PST) Received: from smtp-sh.infomaniak.ch (smtp-sh.infomaniak.ch [128.65.195.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 426001205F0 for ; Thu, 18 Jan 2018 00:22:34 -0800 (PST) Received: from smtp7.infomaniak.ch (smtp7.infomaniak.ch [83.166.132.30]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0I8MVSN005548 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 18 Jan 2018 09:22:32 +0100 Received: from castor.home (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:5018:da0:3d51:6963:5ea8:dd81] (may be forged)) (authenticated bits=0) by smtp7.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0I8MVoC119139 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 18 Jan 2018 09:22:31 +0100 Date: Thu, 18 Jan 2018 09:22:31 +0100 From: Reto Kromer To: Codec Encoding for LossLess Archiving and Realtime transmission X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 08:22:37 -0000 Michael Bradshaw wrote: >Setting a maximum >let's muxers know that if they exceed it, they run the risk of >making a file that some readers can't (fully) play. Sorry, I do not support the idea of fixing a maximum value here, because, in my opinion, it's an useless limitation. Best regards, Reto From nobody Thu Jan 18 00:24:36 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C3EC12EB13 for ; Thu, 18 Jan 2018 00:24:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.202 X-Spam-Level: X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g7P_LuIAhEUd for ; Thu, 18 Jan 2018 00:24:32 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6F9871205F0 for ; Thu, 18 Jan 2018 00:24:32 -0800 (PST) Received: from smtp6.infomaniak.ch (smtp6.infomaniak.ch [83.166.132.19]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0I8OUxN012353 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 18 Jan 2018 09:24:30 +0100 Received: from castor.home (dynamic.wline.6rd.res.cust.swisscom.ch [IPv6:2a02:1205:5018:da0:3d51:6963:5ea8:dd81] (may be forged)) (authenticated bits=0) by smtp6.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0I8OT4R040642 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Thu, 18 Jan 2018 09:24:30 +0100 Date: Thu, 18 Jan 2018 09:24:30 +0100 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: <082fb94e-75ed-bb3f-462d-c56a347af693@mediaarea.net> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 08:24:34 -0000 Jerome Martinez wrote: >"An implementation may set limits on the maximum depth of >nesting" in a parser section (similar to JSON RFC). I agree, this is up to the single implementations. >- A minimum depends on the goal of the parser,=20 I strongly agree! Best regards, Reto From nobody Thu Jan 18 06:27:07 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B81F0126E3A for ; Thu, 18 Jan 2018 06:27:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zsRgah0YV6UY for ; Thu, 18 Jan 2018 06:27:04 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5C5071205D3 for ; Thu, 18 Jan 2018 06:27:04 -0800 (PST) Received: from [146.96.19.240] (port=21438 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1ecB9f-0004T4-Cq; Thu, 18 Jan 2018 09:27:03 -0500 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Dave Rice X-Priority: 3 In-Reply-To: Date: Thu, 18 Jan 2018 09:26:57 -0500 Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Transfer-Encoding: quoted-printable Message-Id: <8F278A6D-2EBC-4A8E-9C55-22E2E1230768@dericed.com> References: To: Reto Kromer X-Mailer: Apple Mail (2.3273) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 14:27:06 -0000 > On Jan 18, 2018, at 3:22 AM, Reto Kromer wrote: >=20 > Michael Bradshaw wrote: >=20 >> Setting a maximum >> let's muxers know that if they exceed it, they run the risk of >> making a file that some readers can't (fully) play. >=20 > Sorry, I do not support the idea of fixing a maximum value here, > because, in my opinion, it's an useless limitation. Since Matroska uses an EBMLMaxSizeLength of 8, then after reserving = space for the other required elements, then it=E2=80=99s feasible to = have a valid Matroska file with a recursive Chapter Atom Element to the = depth of 36,028,797,018,963,944. But Chapter Atom recursion to = 36,028,797,018,963,945 is not possible in Matroska (maybe try another = container for this requirement). Dave Rice= From nobody Thu Jan 18 07:41:27 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AA08A124217 for ; Thu, 18 Jan 2018 07:41:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZT4yOd09OWGq for ; Thu, 18 Jan 2018 07:41:25 -0800 (PST) Received: from zucker2.schokokeks.org (zucker2.schokokeks.org [178.63.68.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E92341204DA for ; Thu, 18 Jan 2018 07:41:24 -0800 (PST) Received: from [10.0.0.11] (1360030002.d-dsl.at [::ffff:81.16.105.50]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Thu, 18 Jan 2018 16:41:23 +0100 id 000000000000006D.000000005A60C023.000060AE To: cellar@ietf.org References: From: "Peter B." Message-ID: <1f52a329-febb-eb2e-6610-6509990037d3@das-werkstatt.com> Date: Thu, 18 Jan 2018 16:41:21 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 15:41:26 -0000 On 2018-01-18 09:24, Reto Kromer wrote: > Jerome Martinez wrote: > >> "An implementation may set limits on the maximum depth of >> nesting" in a parser section (similar to JSON RFC). > I agree, this is up to the single implementations. Could it be useful to suggest (not define) a "reasonable" value for max nesting depth there, instead of just saying it "could be limited by an implementation"? This might reduce implementation-specific interoperability fluctuations, I guess. Just an idea... Pb From nobody Thu Jan 18 09:53:41 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FF6612D0C3 for ; Thu, 18 Jan 2018 09:53:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.71 X-Spam-Level: X-Spam-Status: No, score=-2.71 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l58q6BIVc0zA for ; Thu, 18 Jan 2018 09:53:34 -0800 (PST) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F27BD129C56 for ; Thu, 18 Jan 2018 09:53:33 -0800 (PST) Received: by mail-io0-x231.google.com with SMTP id m11so16339171iob.2 for ; Thu, 18 Jan 2018 09:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Q1ilj7FAo5HAgbxGakMUKyQD6+n6bacbpLBNDQwT7AM=; b=mKuR3qOMui1vwnT153D/JioYXCQK3Nw4MDr0vR/DOhvxvzHtK4+vXzc0WfCSPTNOGq 05uo8bpvjfmrJNFYgLMJ2CKcZFNDLn3Q6Pl4ap7tadN5dO+AJcO39Vpz8S9ufdJDzTtZ jPuAPD0QNsmWeYwqjIcM4gQjJIT4Cc6EP6Hf7Z3zhyC7Heu+9sTZ0qHkB+9085Lq7Oqk fauQCA1hU1Al2O9nEZG7k6Uj6/eq7ZWJAspkqcGUxPjhHwvFtaT6uKPLMKZBaBlRkz+E z98xNekSHdctOFHU+zDWYPxezebsXZe8ToWFaZSaW6wBn2HIgd0P7r+BrFw8u3wA7Pca flcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Q1ilj7FAo5HAgbxGakMUKyQD6+n6bacbpLBNDQwT7AM=; b=FyebTPY+n+mNdmwp5/RSu1EpO2+1gAmsPB0SKMPWtIpywK1ajB7HqXyJFWHYSX5hsm kP61QzR0HMlg65pJUyVkXD3rDbWqYKzjZva09Z7vV+DH+Qr0M3jRE/D/GF6tUM9UazXa Icy8YiymDbms/vNhRq/RMBhfFvDWLDezViptyW3sNW1IPJD4G7tVvsrpV+Mzme6Q8kFm Ot6UWNHzrgU08aJOKUMQJvJLULOl78RvXzPdKf4fDyR5ez4IvvdSk51nE0gTDTxpHXEK MDalZCI69J+Lru5XXs4ZchtRj1H8aNWKGi8cSx9RswP8A7MurCRJd50uIcRDoRFRwMkH 3m+g== X-Gm-Message-State: AKwxyte3dQa+bWWLC5KtVcvu6cbk4RM6u0eL0HytUnBF4ZZlW5kGQusY YJkFASKPbradmvf5TUKghL8xhl1xbQo2tNAsIZoHn08T8w0= X-Google-Smtp-Source: ACJfBou8P5XumZRaRjAK95xHIyzkKBsPGLwpaIW3g9EnT5u7r7eeXT6MUwsGew/2nvXmJwGakq9gtbVucT7P+rQqrzc= X-Received: by 10.107.33.65 with SMTP id h62mr16905369ioh.104.1516298012848; Thu, 18 Jan 2018 09:53:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.36.75 with HTTP; Thu, 18 Jan 2018 09:53:12 -0800 (PST) In-Reply-To: <082fb94e-75ed-bb3f-462d-c56a347af693@mediaarea.net> References: <20180117223156.GD3493@michaelspb> <082fb94e-75ed-bb3f-462d-c56a347af693@mediaarea.net> From: Michael Bradshaw Date: Thu, 18 Jan 2018 09:53:12 -0800 Message-ID: To: Jerome Martinez Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a1140ffe66e8708056310a4b5" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 17:53:36 -0000 --001a1140ffe66e8708056310a4b5 Content-Type: text/plain; charset="UTF-8" On Wed, Jan 17, 2018 at 11:42 PM, Jerome Martinez wrote: > > I missed the case of recursive elements, specified in EBML. > for a security section, I don't think that talking about a parser is right The current EBML specification's Security Consideration section contains a bulleted list for "Attacks on an EBML Reader" (where an "EBML Reader" is explicitly defined as a "data parser[...]"). So if we shouldn't be talking about security considerations for parsers, then that whole bulleted list should be removed (which I'm opposed to doing). Perhaps I've misunderstood your point here, though. I am more in favor of something like "it is possible to have unlimited > recursion, potential source of denial of service attacks" in security > section and > "An implementation may set limits on the maximum depth of nesting" in a > parser section (similar to JSON RFC). My apologies if my previous communications have been unclear, but this is actually what I'm trying to advocate for, so I think we might agree more than we realize. While I have been suggesting including minimum/maximum levels, I think that using something like you've written here (and completely omitting minimum/maximums) would still be a great improvement. I do think that also including some kind of minimum/maximum levels could be useful. I also think that if we do have them, they should use SHOULD/MAY language instead of MUST. Using SHOULD/MAY would give guidance to implementors on how they can maximize interoperability while still allowing them total freedom (the implementation would still be conforming to the spec even if it went against a SHOULD/MAY guidance). We don't talk anymore about security, and confidence depends on the > underlying document e.g. for my EBML document x, I may need only 1 level > beside the EBML header and it is fine with that, no need of more. > So: > - A minimum should not be in Security section (it is not about security) > I agree that minimum or even maximum limits should not be placed in the security section. They belong elsewhere in the Matroska specification (exactly which section, I don't know). Sorry for not stating this more clearly in my original email. - A minimum depends on the goal of the parser, e.g. a parser just checking > top level element size and CRC for checking a file transfer is fine does > not need more than 2 levels. Not all tools are video players. I'm not advocating for HOW the parsers should handle deep recursion (I probably shouldn't have used the language "abort the parse" in my original email; it was just meant to be a starting point for the conversation). Handling it by skipping it is fine. If a particular element is not needed for the goals of the application, the parser should certainly feel at liberty to skip over it. But a player may just ignore any chapter and tag elements and still play > fine the file. Yes, the encoded media data can still be parsed/decoded and the media can still be played. But part of playing a video is showing to the user the chapter and tag information to the video. If the player ignores the chapter or tag elements, then the playback experience is incomplete. Not all players utilize the chapter and tag elements, but those that do will likely have some kind of upper limit on the level of recursion they can meaningfully handle. --001a1140ffe66e8708056310a4b5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On W= ed, Jan 17, 2018 at 11:42 PM, Jerome Martinez <jerome@mediaarea.net= > wrote:
I missed the case of recursive elements, specified in EBML.
for a security section, I don't think that talking about a parser is ri= ght

The current EBML specification's Se= curity Consideration section contains a bulleted list for "Attacks on = an EBML Reader" (where an "EBML Reader" is explicitly define= d as a "data parser[...]"). So if we shouldn't be talking abo= ut security considerations for parsers, then that whole bulleted list shoul= d be removed (which I'm opposed to doing). Perhaps I've misundersto= od your point here, though.

I am more in favor of=C2=A0 something like "it = is possible to have unlimited recursion, potential source of denial of serv= ice attacks" in security section and
"An implementation may set limits on the maximum depth of nesting"= ; in a parser section (similar to JSON RFC).

My apologies if my previous communications have been unclear, but this is= actually what I'm trying to advocate for, so I think we might agree mo= re than we realize. While I have been suggesting including minimum/maximum = levels, I think that using something like you've written here (and comp= letely omitting minimum/maximums) would still be a great improvement.
=

I do think that also including some kind of minimum/max= imum levels could be useful. I also think that if we do have them, they sho= uld use SHOULD/MAY language instead of MUST. Using SHOULD/MAY would give gu= idance to implementors on how they can maximize interoperability while stil= l allowing them total freedom (the implementation would still be conforming= to the spec even if it went against a SHOULD/MAY guidance).

=
We don't talk anymore about security, and confidence depends on the und= erlying document e.g. for my EBML document x, I may need only 1 level besid= e the EBML header and it is fine with that, no need of more.
So:
- A minimum should not be in Security section (it is not about security)

I agree that minimum or even maximum limi= ts should not be placed in the security section. They belong elsewhere in t= he Matroska specification (exactly which section, I don't know). Sorry = for not stating this more clearly in my original email.

- A minimum depends on the goal of the parser, e.g. a parser just checking = top level element size and CRC for checking a file transfer is fine does no= t need more than 2 levels. Not all tools are video players.

I'm not advocating for HOW the parsers should handle d= eep recursion (I probably shouldn't have used the language "abort = the parse" in my original email; it was just meant to be a starting po= int for the conversation). Handling it by skipping it is fine. If a particu= lar element is not needed for the goals of the application, the parser shou= ld certainly feel at liberty to skip over it.

But a player may just ignore any chapter and tag elements and still play fi= ne the file.

Yes, the encoded media data ca= n still be parsed/decoded and the media can still be played. But part of pl= aying a video is showing to the user the chapter and tag information to the= video. If the player ignores the chapter or tag elements, then the playbac= k experience is incomplete. Not all players utilize the chapter and tag ele= ments, but those that do will likely have some kind of upper limit on the l= evel of recursion they can meaningfully handle.
--001a1140ffe66e8708056310a4b5-- From nobody Sun Jan 21 08:28:57 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B165128896 for ; Sun, 21 Jan 2018 08:28:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mm_7CDM65PE8 for ; Sun, 21 Jan 2018 08:28:54 -0800 (PST) Received: from mail-pg0-x229.google.com (mail-pg0-x229.google.com [IPv6:2607:f8b0:400e:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 47CCF1270AB for ; Sun, 21 Jan 2018 08:28:54 -0800 (PST) Received: by mail-pg0-x229.google.com with SMTP id n17so5164756pgf.10 for ; Sun, 21 Jan 2018 08:28:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1Rja5eYV8f1u7mTDSDj0mjfF3LCEybcAOk2yw2yIvzM=; b=uLoflOlH3ZL0tN4IGGn53vAF9BOhsosp4c99MUxpb2ItPDFKDdRTCFlX/OO+3Wzc9o HcUu8jCbeiKwUVp1gaw0KBwXUe3qZsYjz15DvlyfrC9MGODR/pd//xyDl6lYztal/I1z 0Igk6HemdgmvasJ4nuGf75xeKwk5fUcYFRi/oZ542pt71HeER9hERHoKlKKrxdH7goEb c1TFuK1etjvJaHF4Q0YqoYGpQ5kjCNCT3wyGFeXXRzidox52m41PCho3fnLIMqpLrSSz Vrutf/53yS2cKtcJtCgxmfIPJ+aXLrlBw7fQi8bXq+WhAD1nXq3wsbdkUpd7BGRV38qr 9dJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1Rja5eYV8f1u7mTDSDj0mjfF3LCEybcAOk2yw2yIvzM=; b=o6rT2cDKJ+ih/5JOV0S+LwgDIbG04a469K3/5EUSs9L8zKFI9usAwJtnnDQmKkm88G N+Wm/hGECvcslT9TKyIPPQmAozPCYWtXtU+66v7JdMkuPs9/X16MwC8NXstZUw7zfElc KxHpwje2DFZDH52+bhc07VeLeSEduUnLpngiIFUWjKmztshTngIzXeyC3UM0jMFz6XAU VyPgYMmWG0/jM81En0+yJuyK4lXoyTuyMrkJYy2TN9d6B/xBqWVCD/dmy56sHXKNgwuo iOZyLENGKKqfFhov7ywwCeoqHWy34aEPbOBD2iq+1IuTnu2pbGxQ+WPpJ0m48B32SABY febg== X-Gm-Message-State: AKwxytc+Y5Y3gysdOsiroOuFUjmxgANJyCV6iEdD6obv8+2/u5hkIeVV tYvY2g1R7c70FABB4mp3QdOyICdY424AaLBUJYTErQ== X-Google-Smtp-Source: AH8x225wOQWNq4Bz/i48a7ZxqofwyUhwjZE8S7937OPW992uyFcjRvidYJurg9jie+rN24rV6aJVTSGgAMplOhnu6S4= X-Received: by 10.98.147.154 with SMTP id r26mr5610243pfk.207.1516552133575; Sun, 21 Jan 2018 08:28:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 08:28:53 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 21 Jan 2018 17:28:53 +0100 Message-ID: To: Dave Rice Cc: Austin Einter , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] How to use libmatroska? X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 16:28:56 -0000 Hi, I think this question should go in matroska-devel rather than the CELLAR mailing list. That being said, libmatroska doesn't come with good example code nor good documentation. it's a rather low level API to the format that doesn't do a lot of things for you (it even allows writing incorrect files if you want to do that on purpose). Also since Matroska is an interleave audio/video(/other) format, writing means you have to handle the interleaving yourself. There are higher level APIs that can do that for you like libavformat (from ffmpeg/libav). 2018-01-16 16:50 GMT+01:00 Dave Rice : > Hi Austin, > > On Jan 15, 2018, at 8:32 PM, Austin Einter wrote: > > Say I have a MKV file. > > How do I parse it? > > > For parsing an MKV file I suggest reviewing the EBML specification at > https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/, particularly the > principles of the EBML Element and its Element ID, Element Data Size, and > Element Data. Understanding how to read EBML Elements will help decipher the > structure of a Matroska file (or any other EBML document). Then the EBML > Schema of that file type is needed in order to understand the semantics of > each element. For Matroska that EBML Schema is available at > https://github.com/Matroska-Org/matroska-specification/blob/master/ebml_matroska.xml. > > How do I write a new MKV file? > > I hope libmatroska can do this. > Is there any sample code / example that I can refer to read / write matroska > files. > > Thanks > Austin > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 08:49:05 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD816126D73 for ; Sun, 21 Jan 2018 08:49:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GU3hLDgr3fiv for ; Sun, 21 Jan 2018 08:49:01 -0800 (PST) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A16E41205F0 for ; Sun, 21 Jan 2018 08:49:01 -0800 (PST) Received: by mail-pg0-x22a.google.com with SMTP id u1so5208417pgr.0 for ; Sun, 21 Jan 2018 08:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=SpnTznBpc+koj2X881Yeyv6wf+P67tIOXb9paZcuZdk=; b=iW05xUNWGYIB/fBseucSsOpd3/jR2Ram4uA7fFHHeQhC2RhYxtGRT+ITyCZGRrdVgi nrRflbroqNU7YGuaeVRsTN2FpvRR9bUoJTfzuSE/kp+04E3aOuNVbURL2WhGThs4yimn sQWet8iBmndlVXv3v9ksD2N46AO6uBG0nwvdXcDivq5tnbxq+g5OUIVNcLMnCSU6hqcF w03I3BwAYtPb/jRP4FE0nC8TBUMEmHEfLvXNHEHLFYcBBMs7kUkzI8MpPnABds1/9Fps 7cVsOFpECEwEKAP7tF4rSYZilBwmQ54EqQFjqMk+RpQfoeW1FXg+6DY3Ye1BR0g7E1IN Po/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=SpnTznBpc+koj2X881Yeyv6wf+P67tIOXb9paZcuZdk=; b=OB0k0oOOp573liV0Ua+Q4XTT9KyUCWNuB7xKu3jCotDiHFcaXnUpCnLe9K97q1BlOk rMB1wpKFhdipCfQCyyOSJJTarFdmNTSVnd012bs+r0r23Pme4CtRAbk2nc+WOG793GLu PaJgObkBuXADBd/8ZGvqD59PsmN3CzChnUVrVAmObKqcsPOgY6K2F0KuAeGlJqs1A5so iwgTXQsBGz2KxYbNEKNxz0FdxUeT0n6srTvqWrA2V+dwwI744kIu8gSRHo9M20JD5Tg/ tZvkqVuImqtYzhklMPNCY5Mzk/NegRz8aE1esG2LWR/gQafMrX+VfG6Zuyn6r5t7ylLg /vDw== X-Gm-Message-State: AKwxytd6klmEbOF/LpNn5P2tudvSuiljI+LqwSPydBNzRe/enjaTgBF0 vjy86DfByiLys+omMWXc0g0t7j+FX+umNXFxGtB3LQ== X-Google-Smtp-Source: AH8x226f49JzWOcdbFs5lm+vnf42aC1doyl/l+RMnrR3Ecx1+zWRpOa/FdvgYT5fhgqMoSeKOq55vLhD6dfGVdLA0dU= X-Received: by 10.98.27.5 with SMTP id b5mr5735221pfb.103.1516553341169; Sun, 21 Jan 2018 08:49:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 08:49:00 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 21 Jan 2018 17:49:00 +0100 Message-ID: To: Dave Rice Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] [Matroska Tags] restructure tags lists into xml X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 16:49:04 -0000 That sounds like a good move. Thanks. 2018-01-14 16:33 GMT+01:00 Dave Rice : > > On Jan 13, 2018, at 10:25 AM, Dave Rice wrote: > > Hi all, > > The name, types, class, and descriptions of Matroska=E2=80=99s official t= ags are > currently in formatted text within a Markdown document, such as in > https://github.com/Matroska-Org/matroska-specification/blob/c1f7802bd258e= 0847b4892f8a9499c2ea786d3de/tagging.md#nesting-information. > I propose to move these tags into a structured xml document. The contents= of > the xml can then be formatted into a RFC friendly form during the make > process. I think the advantages of this could be that the tags are easier= to > maintain in xml and it may make it easier for developers to integrate the= se > vocabularies. Currently I have a draft of that xml at > https://github.com/Matroska-Org/matroska-specification/pull/220/files for > comment (either on the xml itself or the general idea). I=E2=80=99ll late= r add in > the make process to add the same data into the Matroska Tags RFC draft fr= om > the xml source rather than the current markdown source. > > > https://github.com/Matroska-Org/matroska-specification/pull/220 is update= d > with a revised XML representing the Matroska tagging vocabulary and > definitions. Also the Makefile process is updated so that the tagging > registry is removed from tagging.md but will not be built into RFC output= s > using a mix of the tagging.md files and the new xml. The RFC txt output i= s > identical before and after this pull request, but I think these changes w= ill > help make the tagging registry more useful, allow easier > internationalization, and better allow us to extend the definition of > Matroska tags. > > Best Regards, > Dave Rice > > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --=20 Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 08:51:46 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4AB721205F0 for ; Sun, 21 Jan 2018 08:51:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m9BM1ZdD4JAf for ; Sun, 21 Jan 2018 08:51:42 -0800 (PST) Received: from mail-pf0-x22d.google.com (mail-pf0-x22d.google.com [IPv6:2607:f8b0:400e:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 69D5E126CE8 for ; Sun, 21 Jan 2018 08:51:42 -0800 (PST) Received: by mail-pf0-x22d.google.com with SMTP id i66so5211855pfd.7 for ; Sun, 21 Jan 2018 08:51:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=XhrRm0skvu5OUy7q7Clbfuj3biuQlFm1YTxYc/XA+vE=; b=HKHftgV5N2lrDsKnoKxbWWyZSqcSvAANSLVDshV8+Rs80TqTwbIXGktAdlhfKEV/sz ylQ35UrOfAqAtDTfMf7+305VRzwNXWtMiMsZrvwQnhaTIXNfU3/2EZ0K6W2PUhnYUqn7 RsKvbtgZHBPciVPv3S72JTRX3GS1drWK0bMkJV4i8I+show1yFFSCdVZ7rwYE3W5/eFO hm3KsFk+S9ZOypwhrdDAf2cwWNHit+EqkMrFzbF8Mxc0o2y3OgyVWlrvTE7JqUmM7nGe j3Cl1iv/RIiVqX4Ci21j0Ud+eLGzWj0smLQjY6DFy9WYcwclvlQN2ThJyeprlcJHpEnN Ce8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=XhrRm0skvu5OUy7q7Clbfuj3biuQlFm1YTxYc/XA+vE=; b=lcoLOzPxrc7P03pLMdbbsw1414gaHmpN/dIzmAMWAIypPsCuS3woL6q90J+842Eain SkbSzYBySuNP9wu33kOASywHn6HyefaYiO4u6K6uGiAKxBBKPiNLwhQRjpGY2jZdQPCh AubLB8PTSWFQVJxW8l1owF2twXThVRlk/tqqy9Vgg77pbMt/YYWV6Mv7akRx0b6G03oo VvBLA0YsJYYYmn6CJAxp2pJSBaikbwZP31ZrdyizIiOqhMrpEuQ+R+HAor+4PBi6LQqD v0/sDruvHVW/kx0ARdEjx70gIxEQCcgJta4d/aeLjN9SGwP59sXTv8CBTmYNsBrlh/fn Q/WA== X-Gm-Message-State: AKwxytdTIQoUt9j0h3trWmuA6ujyoKKV2udNoFoLASmDH/hD5XSzSqAz 7vywH3ApYBzWpG6T9jJ0lbSFp5Z+gDgM9czj0aPIVw== X-Google-Smtp-Source: AH8x227KdgxTKjNVOQUzw0tBhlaPmoQj8P1XE7JX5shg5lJlSWBJ5HaesUZQchiUcse80DAHBfZvQ3z+JqdvvZ66/Ho= X-Received: by 10.101.64.196 with SMTP id u4mr4830575pgp.336.1516553501827; Sun, 21 Jan 2018 08:51:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 08:51:41 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 21 Jan 2018 17:51:41 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Fwd: Understanding VP8 data in a MKV file X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 16:51:44 -0000 2018-01-04 9:13 GMT+01:00 Austin Einter : > > ---------- Forwarded message ---------- > From: Austin Einter > Date: Sun, Dec 31, 2017 at 10:32 PM > Subject: Understanding VP8 data in a MKV file > To: cellar@ietf.org > > > I have a mkv file, that contains VP8 encoded data. > I need to stream this data to chrome (through RTP), and need to play it in > chrome. > > I have parsed the mkv file, and I have VP8 data. > I will be sending this VP8 data in RTP packetc. > Looks I need to follow RFC 7741 to put VP8 descriptor / header in RTP packet > followed by VP8 data. > > Now I have several doubts, kindly try to help me to understand. > > 1) What is a partition in VP8 and from MKV file how do I get partition > information. > 2) I need to put partition size in VP8 header. From where I get the > partition size. > 3) How do I know partition index, this I need to put in VP8 descriptor. > > Does VP8 data I read from mkv file contains this. Sorry, that sounds like a term internal to the VP8 documentation. Maybe Moritz know about such things but that's out of the scope of CELLAR. > Regards > Austin > > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 09:11:04 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C94F126D73 for ; Sun, 21 Jan 2018 09:11:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UulfYyje1NeP for ; Sun, 21 Jan 2018 09:11:00 -0800 (PST) Received: from mail-pg0-x22c.google.com (mail-pg0-x22c.google.com [IPv6:2607:f8b0:400e:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A2612126CE8 for ; Sun, 21 Jan 2018 09:11:00 -0800 (PST) Received: by mail-pg0-x22c.google.com with SMTP id g16so5219756pgn.7 for ; Sun, 21 Jan 2018 09:11:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=i7gh5/iD+f4RVQwvb57I6E4tFzi+HXWhYrKV4750eGY=; b=CBr5aT0CF8ojHJMTwqoyKX7YuubsKY2ktrCBv2G+Gpx1UHCX9pp9XMsTa//yxb5dCC gnOEVeAq0X1DKmDXA0RP3k6PM0qQQhnYjJqh/tgl9JDr/Weqe8V3yvXtw1yAvEDcdibA OfxNBPrZtK8B3zv9KiOur8Bc0PV4wvth8XwYhuHMyu+cru6nVZ4JDUASsQMjXRaT7zb0 BgYVH9LAjsgziMhGoPdiLQeCM/Lk+kGW/+XHdhK0kKB1C83J3+wP4U7CsOoKO9Pt2l8n /EEgsMKV0z2A9V1NglDo/m2R6PR6Qk4cp2ghGPXxpw9OvxDez51ieF9QTpV1lk86XOci B9ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=i7gh5/iD+f4RVQwvb57I6E4tFzi+HXWhYrKV4750eGY=; b=o07hOMpfQM5BrWyaxVPogwles7EEz255auGMoD+VMCigIdo+/YE5NWVrq2txJfE4mB Pf7UeMS4okQUmwzLVEGkivud09cTJqlaMM/RqkqH8KDtU7MUOg4YhlID42lkHSCadNPz 7NwemhdpMQrkP6lC9Oh5vNG2Ylo4WRTtyET4su/Ck3fVvKdYlmO7YomKqDDrgS1GVsic Js1pLlVRUp7vB+N6eqDudqzvGvJjQYHeouOdXupsmUghar8Pb/X01UKCQiVk7uKcPRPs fKCvuF7qVUiA/NQcf/PmM4Zx4gAIqASoFSUIsqsBcBiamKXB3zyejdSCdA94+lMHjKxc awoQ== X-Gm-Message-State: AKwxyteJuEa2cPMaFtZ2bTrvdRuxNRYIARR7vRb8V9UrOccREGvJFiWQ TrgTwdIu7UY+TVdfDl+1g117PTeNT4fJLnDCfju4tNcU X-Google-Smtp-Source: AH8x227vDZe5XBDeF7OqCM1FlTZBdBNkIZwjuD9rT/+mCJnhjQhVgdpEj1uZOQ49uzkC9HpQQdaTd5QqwyWz/+bONAw= X-Received: by 10.99.147.21 with SMTP id b21mr4844894pge.318.1516554660027; Sun, 21 Jan 2018 09:11:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 09:10:59 -0800 (PST) In-Reply-To: References: <20180117223156.GD3493@michaelspb> <082fb94e-75ed-bb3f-462d-c56a347af693@mediaarea.net> From: Steve Lhomme Date: Sun, 21 Jan 2018 18:10:59 +0100 Message-ID: To: Michael Bradshaw Cc: Jerome Martinez , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 17:11:02 -0000 (starting to reply from the bottom of the thread) 2018-01-18 18:53 GMT+01:00 Michael Bradshaw : > On Wed, Jan 17, 2018 at 11:42 PM, Jerome Martinez > wrote: >> >> I missed the case of recursive elements, specified in EBML. >> for a security section, I don't think that talking about a parser is right > > > The current EBML specification's Security Consideration section contains a > bulleted list for "Attacks on an EBML Reader" (where an "EBML Reader" is > explicitly defined as a "data parser[...]"). So if we shouldn't be talking > about security considerations for parsers, then that whole bulleted list > should be removed (which I'm opposed to doing). Perhaps I've misunderstood > your point here, though. I agree with Michael, this is a legitimate attack and it should be mentioned. Putting limits is a different topic that would probably go in a different section of the specs. >> I am more in favor of something like "it is possible to have unlimited >> recursion, potential source of denial of service attacks" in security >> section and >> "An implementation may set limits on the maximum depth of nesting" in a >> parser section (similar to JSON RFC). > > > My apologies if my previous communications have been unclear, but this is > actually what I'm trying to advocate for, so I think we might agree more > than we realize. While I have been suggesting including minimum/maximum > levels, I think that using something like you've written here (and > completely omitting minimum/maximums) would still be a great improvement. > > I do think that also including some kind of minimum/maximum levels could be > useful. I also think that if we do have them, they should use SHOULD/MAY > language instead of MUST. Using SHOULD/MAY would give guidance to > implementors on how they can maximize interoperability while still allowing > them total freedom (the implementation would still be conforming to the spec > even if it went against a SHOULD/MAY guidance). I agree with that too. EBMLMaxSizeLength was mentioned in the thread and it would make sense to make a recursion limit per file in the EBML header. That would let parsers know if they can handle the file or not in advance and/or detect security attacks. We could either have a default value for that element or none which means there's no de-facto maximum. It also not be an element but something more complex, since the amount of recursion allowed may differ between elements (see nested tags vs nested chapters). But it may be easier for the reader and the writer to have only one limit for all. In any case a default maximum valu would belong in the Matroska specifications, not the EBML ones. >> We don't talk anymore about security, and confidence depends on the >> underlying document e.g. for my EBML document x, I may need only 1 level >> beside the EBML header and it is fine with that, no need of more. >> So: >> - A minimum should not be in Security section (it is not about security) > > > I agree that minimum or even maximum limits should not be placed in the > security section. They belong elsewhere in the Matroska specification > (exactly which section, I don't know). Sorry for not stating this more > clearly in my original email. We all agree. >> - A minimum depends on the goal of the parser, e.g. a parser just checking >> top level element size and CRC for checking a file transfer is fine does not >> need more than 2 levels. Not all tools are video players. > > > I'm not advocating for HOW the parsers should handle deep recursion (I > probably shouldn't have used the language "abort the parse" in my original > email; it was just meant to be a starting point for the conversation). > Handling it by skipping it is fine. If a particular element is not needed > for the goals of the application, the parser should certainly feel at > liberty to skip over it. > >> But a player may just ignore any chapter and tag elements and still play >> fine the file. Not if the ordered chapters are truncated by enforcing a bogus maximum. > Yes, the encoded media data can still be parsed/decoded and the media can > still be played. But part of playing a video is showing to the user the > chapter and tag information to the video. If the player ignores the chapter > or tag elements, then the playback experience is incomplete. Not all players > utilize the chapter and tag elements, but those that do will likely have > some kind of upper limit on the level of recursion they can meaningfully > handle. > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 09:15:42 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC63D129C6B for ; Sun, 21 Jan 2018 09:15:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XLWn_VjdIOcV for ; Sun, 21 Jan 2018 09:15:37 -0800 (PST) Received: from mail-pg0-x229.google.com (mail-pg0-x229.google.com [IPv6:2607:f8b0:400e:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C5A4E12773A for ; Sun, 21 Jan 2018 09:15:37 -0800 (PST) Received: by mail-pg0-x229.google.com with SMTP id s9so5219177pgq.13 for ; Sun, 21 Jan 2018 09:15:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vRlnJ57y+k2OHLW77mFssx5pX0os7rDIrk+fBmu/8Oc=; b=yy/FFIehazGycj/Sndo54ku0EUezF+86h5k7G5/sH2kJ3GuhOkAbdafwzdK0kGD2SO erxk9XGPATc9Z1LcWlgPj9OyQLOwHpWhNAVYziGoHPTMrrNOlwKblI1Usyigj1YeDzav c+AQRUa0M0cKpg6df4hvJuOoPt7XVjxpiFBN2IwiVZ2P8/2WATpKRWfeE02XVcQfnFSA GnHG1cqmqaCFZprk+MGrwyF7g3Cld/zvTmNfHsaMZeCcFr+Mu1hySfeSoB2lOkgVClPR /QIpQoeeYJ3BWudPtRdXU/MM9tIu/J5cmztP0Bb0YLHdGMQQMqfOfyXTLmb6Al8pXr2Y Gq1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vRlnJ57y+k2OHLW77mFssx5pX0os7rDIrk+fBmu/8Oc=; b=Ydiv5kgQS8MPRbqKodzH6OxnZrCG2AeYu3erbWVmtgsfLxTCGXGr7bWdMk1Ysl8ETa a9GKVRxTSE6RmJLXqALQmS5zGzEa1MLpRFD2POLPRbN6THpL77+xY0WtwEhrGVuye63M 5a9SbT52k8UCZB/kz/7Aum1FG/6GuTrEdH9cv3MgLAgtu2K1gJmMLDgG+iji514iNJ55 Yk6IqCgQ4aCDIHIx13ixGToi5ViM1h5iAngNCXfWeZyowG8b4t0AGy8PVScOYw0S30jt UrzjdNqqhHjLsudC1/05JW6Wcxzx8AH1kWIe1lktjuMGuiq7h39LmdpKT1bnaOAVuXqp RA0Q== X-Gm-Message-State: AKwxytfqySPmm7zY0xNQXlKPt6N29ObeAKccLYhQMKArIjV7ttQwvm4O S/YyYo+nNv2DayhlW+SDyAKmn4032TnTEdX9kHNhvg== X-Google-Smtp-Source: AH8x225lNjqmAFmaPjuu9uk8BCzWGhqnqUpwmk3VaGbifpoDf60k3oD+LOLzJxobLnChZjUToxbzmtBaMVcMNQoZ4AE= X-Received: by 10.98.102.4 with SMTP id a4mr5750573pfc.210.1516554937354; Sun, 21 Jan 2018 09:15:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 09:15:36 -0800 (PST) In-Reply-To: References: <20180117223156.GD3493@michaelspb> From: Steve Lhomme Date: Sun, 21 Jan 2018 18:15:36 +0100 Message-ID: To: Michael Bradshaw Cc: Michael Niedermayer , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 17:15:40 -0000 2018-01-17 23:47 GMT+01:00 Michael Bradshaw : > Thanks for the responses, all! > > I agree that setting minimum/maximum recursion limits doesn't belong in the > EBML spec. I do think the EBML spec should have a note in the Security > Considerations that EBML Readers need to be cognizant of how they handle > recursive elements and should be resilient to deeply nested recursion > attacks (without giving particular recommendations for how that resilience > is achieved). But EBML itself shouldn't attempt to set any specific limits > on recursion depth. > > The reason I suggested setting some kind of minimum/maximum recursion depth > (for Matroska specifically, not for EBML in general) is because of the > points Michael Niedermayer raises. Setting a minimum allows muxers to write > files with confidence that readers will be able to use/play the data being > written (so long as the minimum isn't exceeded). Setting a maximum let's > muxers know that if they exceed it, they run the risk of making a file that > some readers can't (fully) play. On the writer's side it's easier to know the maximum depth it reached when writing a particular file and write the exact maximum value at the end of the muxing. Security-wise it's stricter than a loose default maximum. > Specifying recursion limits using the example language in my first email ("a > parser SHOULD handle recursion up to X levels deep, and MAY abort the parse > if it reaches Y levels deep") for ChapterAtom and SimpleTag elements would > allow a muxing program to emit a warning to the user if the recursion level > exceeded Y (it's not a fatal error, but it would be nice to let the user > know their file has abnormally deep recursion that might not be compatible > with various parsers). > > On Wed, Jan 17, 2018 at 2:31 PM, Michael Niedermayer > wrote: >> >> On Wed, Jan 17, 2018 at 12:33:17PM -0800, Michael Bradshaw wrote: >> > The EBML and Matroska specs currently don't mention the possibility of a >> > stack overflow due to deeply nested recursive elements. Currently, >> > there's >> > no limit on the recursion depth (unless I've overlooked it somewhere). >> > >> > I think it would be worth adding to the security section of EBML that >> > one >> > type of attack on an EBML Reader could include deep element recursion. >> > >> >> > Additionally, I would like to see what people think about potentially >> > adding/suggesting an upper limit on recursion (either as a MUST or a >> > MAY). >> > This could also include a lower limit too. For example, something like >> > "a >> > parser SHOULD handle recursion up to X levels deep, and MAY abort the >> > parse >> > if it reaches Y levels deep". >> >> If a limit is specified then theres a limit that every compliant parser >> must support and which a compliant muxer can use. >> >> If no such limit is specified then it is possible that in a system where >> all parts are specification compliant and otherwise compatible it doesnt >> work. >> >> So I think its more a question about what/when compatibility is guranteed >> / what compatibility we want the specification to gurantee >> from that it follows if we need a limit in the specification/draft or not >> >> [...] >> >> -- >> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB >> >> If a bugfix only changes things apparently unrelated to the bug with no >> further explanation, that is a good sign that the bugfix is wrong. >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 09:19:24 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 189B31270AB for ; Sun, 21 Jan 2018 09:19:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AZ9WQbs3PqYW for ; Sun, 21 Jan 2018 09:19:21 -0800 (PST) Received: from mail-pg0-x235.google.com (mail-pg0-x235.google.com [IPv6:2607:f8b0:400e:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E5E53127023 for ; Sun, 21 Jan 2018 09:19:20 -0800 (PST) Received: by mail-pg0-x235.google.com with SMTP id k68so5240334pga.3 for ; Sun, 21 Jan 2018 09:19:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0PkdHs/6dbECV6mwoNKhzUEL15eRc9sl7YM5m/fWQN4=; b=jYz7XpBZIeCZOMBcDvg4tgNkI43Lex25movVAkM/r+/oDEO9h41iHM6cYQG7Yuq3lN rznqauk0DIWwvvhnf8IO9sWDLwumxmOBYJSpoQChjqGhtuAQQQiNKVzbB+umNrAl5IVg DFSUtaX0LPUI16U9FGgVJWkum1wNIX99f81C5y6h144FMgHQyNQzcgDvsNOhXBBW0qLC +ppp3Qba0FIifK68CokMXS5bCBP++6Cvuj9q1dUTePGfn/z2GejBPr4gxXacFkHKyO5p 2cbUSArC4HEGxf8/bwmtXmJ8BHeudVdNFtGoKYCRVmlomGVGPOF5xK9sm5/kpLA5aBQ7 jidg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0PkdHs/6dbECV6mwoNKhzUEL15eRc9sl7YM5m/fWQN4=; b=ChCI8ZKCtV2Dyh9pFaPAFCf3MiDxv2MUJRsoxSGufEAgy/ZGG+oSnlwMIwszV+flW4 lH5NbcAIWBOU8CS7g4z3jThBENjnUnx3zUncc05T5I9PG0EnGith0Pjcib+6VhubD3YH 4XEB2aKHeGAuPPcaVI6upAhx1frWW7Px2Eey5ji6Bq4KO7w0O+v38GV/i8jRI8oSqBn9 K3OHnVDlDLnCEcEUqy8xkTHDno5colcgy7yE3bBYb/0F9Y5wCnGh6gpTRek8yXCeKAYN lcLBX8fqiZm9f++fiXfq2DRr3J2aGsiG+Fq5hviXxAXFskysr4NxRgtftS1LgrB8WSIy YQ+Q== X-Gm-Message-State: AKwxytf0kTKMzXeHgokugf8Dp1jzHsFheXzP8mb9aeX++D6WCL9iw6g6 hFgRHeCc/cQMW9WjkouZBeNGor8FHJnI5FQO+wxdEg== X-Google-Smtp-Source: AH8x226E71E0jtp2L+8J7C4258O9I1xpNxmJcSiiP96eUOQ2I3waOlEwvCiSUGnWMC9BNfNkdqPatiqoyFCKMq/9XYw= X-Received: by 10.99.147.21 with SMTP id b21mr4860603pge.318.1516555160417; Sun, 21 Jan 2018 09:19:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 09:19:20 -0800 (PST) In-Reply-To: <20180117223156.GD3493@michaelspb> References: <20180117223156.GD3493@michaelspb> From: Steve Lhomme Date: Sun, 21 Jan 2018 18:19:20 +0100 Message-ID: To: Michael Niedermayer Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 17:19:23 -0000 2018-01-17 23:31 GMT+01:00 Michael Niedermayer : > On Wed, Jan 17, 2018 at 12:33:17PM -0800, Michael Bradshaw wrote: >> The EBML and Matroska specs currently don't mention the possibility of a >> stack overflow due to deeply nested recursive elements. Currently, there's >> no limit on the recursion depth (unless I've overlooked it somewhere). >> >> I think it would be worth adding to the security section of EBML that one >> type of attack on an EBML Reader could include deep element recursion. >> > >> Additionally, I would like to see what people think about potentially >> adding/suggesting an upper limit on recursion (either as a MUST or a MAY). >> This could also include a lower limit too. For example, something like "a >> parser SHOULD handle recursion up to X levels deep, and MAY abort the parse >> if it reaches Y levels deep". > > If a limit is specified then theres a limit that every compliant parser > must support and which a compliant muxer can use. IMO it should be per-file and not mandatory (ie no default max). > If no such limit is specified then it is possible that in a system where > all parts are specification compliant and otherwise compatible it doesnt work. > > So I think its more a question about what/when compatibility is guranteed > / what compatibility we want the specification to gurantee > from that it follows if we need a limit in the specification/draft or not IMO it should be left to the parser to know what to do. The thing it knows is that there's something wrong with the depth of the file and may decide not to go deeper than the limit or issue a warning to the user or ask if the extra parsing should be done. All the specs have to say is that the maximum depth indicate the EBML depth the file should not go further from. Emphasis on the SHOULD. > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > If a bugfix only changes things apparently unrelated to the bug with no > further explanation, that is a good sign that the bugfix is wrong. > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 09:22:40 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3FA8D1270AB for ; Sun, 21 Jan 2018 09:22:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MvXG3IgXqBDy for ; Sun, 21 Jan 2018 09:22:36 -0800 (PST) Received: from mail-pg0-x22d.google.com (mail-pg0-x22d.google.com [IPv6:2607:f8b0:400e:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B98D2127023 for ; Sun, 21 Jan 2018 09:22:36 -0800 (PST) Received: by mail-pg0-x22d.google.com with SMTP id 136so5234593pgd.8 for ; Sun, 21 Jan 2018 09:22:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=zvIOLQQsNGhYvuKj+CfwJlSMRfvYd6KxNYMVYp8kKTo=; b=1voogSz5eSUpadzk/ZQ3hUTI6rcardNHEziCzdgBdl1eyQVF4mPJYh29K/FUdJcUcR f960lKxxkwQy216s+Ex8jTiqDMaxk4eG1CCzMGeiSr/p3H268yxZu7uYfdmImywa1OwV CJ1JL8k8sYZ9cJuosT7Ej5kc3CE7E62ADsXAKUtnlXfFmGMU2z8DOS36rES92If6HuXQ i9PxhK2WiEal2hp5oP3HjeUVGyVBk99j2QzDYHmo/gbYCecZvHjcfpaTB4pQRCkg0wmw vxjSmNJFzY4YDYDlsl10uABTo94z5KrdfnpFHaiB0n9/BOC3/lsY8nyF7AcfUV3QL3eN yx1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=zvIOLQQsNGhYvuKj+CfwJlSMRfvYd6KxNYMVYp8kKTo=; b=kgy8fHCJDEX0alwSov49iHh1zX4NaAz2uWmsh0aBrBgg7kWnqrqzTUOCe3mPWRhooi b/Dyv13+4iQXDPt9M/DmUHbhbATAJx+N9gy1agB1QsOKVNXxkSev0aXWuqKaO8yzToOI H4d0Yk7OU6fqyncve4taxd+hLYj4Wyd734EkdR2vJLlwrmVPmbSSBy+hMMyCPdR4t5lW 3SvnnSnyMLcPjfiGiyt+UcuhQFNWk1+YWGm5e1ueopkR33MrDWX4kujW7WcRI8r6dN4q WHYotGKKLKfcycG6v/RP1iIfHEKOOcK7SKbcpD+HLVRkRanK3zKW1+h0ezozPYQOifF4 ZL9g== X-Gm-Message-State: AKwxyteIAZvProD3sKsDqk7ZmjdGLsfix1W4ItQAePtshIay427WKOrG lC9R26MrVesgqGNZ8lXt106EgKzHWj3pzwwGwU6eAQ== X-Google-Smtp-Source: AH8x224D9d6ug+HF/mOmv2+Z/vPgVB81J1RXKKZcOkGzqBVYsk9g/747e75wT9KbMF2m25BczBHA6243KLlYcJxbxHs= X-Received: by 2002:a17:902:20c8:: with SMTP id v8-v6mr2214228plg.226.1516555356313; Sun, 21 Jan 2018 09:22:36 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.189.150 with HTTP; Sun, 21 Jan 2018 09:22:35 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 21 Jan 2018 18:22:35 +0100 Message-ID: To: Jerome Martinez Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 17:22:39 -0000 2018-01-17 22:01 GMT+01:00 Jerome Martinez : > On 17/01/2018 21:33, Michael Bradshaw wrote: >> >> [...] >> >> For example, something like "a parser SHOULD handle recursion up to X >> levels deep, and MAY abort the parse if it reaches Y levels deep". >> >> Thoughts from others? > > > an EBML parser can not dig in nested element without the corresponding > dictionary (from DocType). > So IMO it is not like XML or JSON, because XML or JSON parser tries to read > the nested elements, but n EBML parser does not try to read any element not > in the dictionary (so the limit is the dictionary, usually can not be > provided by the attacker. > It is lie JSON or XML only if the attacker can provide the dictionary. > > And the JSON spec specifies no limit: > https://tools.ietf.org/html/rfc7159#section-9 > "An implementation may set limits on the maximum depth of nesting" > > There may be legitimate reason to have thousands of nesting level, and there > is already a lot of debates about that with JSON (last tests I saw about > that is ~100 levels for Ruby JSON default parser and ~1000 levels for Python > JSON parser) > > So the sentence should split security issues, from the input (the file > itself) or the dictionary (e.g. matroska specs): > - a generic parser (reading the dictionary from input) MAY set limits on the > maximum depth of nesting. An implementation MAY set limits on the length and > contents. Yes, we need an element to specify that. > - a specialized parser (e.g. Matroska parser) SHOULD handle recursion up to > the maximum nesting level provided by the supported dictionary of the Yes, and going further is left to implementation. Parsers will need to handle the case where the element is not defined too. So they can fallback to the rule in case the maximum is reached. > document, or 2 nesting levels, whichever is smaller (a specialized format > could have only 1 nesting level but EBML needs at least 2, for DocType > etc...). An alternative is to say to rely on the supported format security > paragraph. > > I am not in favor of writing a number, because there is no good number to > provide, it depends too much of the content, I am in favor of doing like > JSON sentences (without any number, but explicitly saying that limits are > expected). > I hesitate in writing such kind of text in a "parser" section instead of > security section, like the JSON RFC does. > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Sun Jan 21 09:40:45 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 91D17126D73 for ; Sun, 21 Jan 2018 09:40:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.9 X-Spam-Level: X-Spam-Status: No, score=-6.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0801ULSBECVy for ; Sun, 21 Jan 2018 09:40:28 -0800 (PST) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2842D124D37 for ; Sun, 21 Jan 2018 09:40:27 -0800 (PST) Received: from eris.local (unknown [85.163.9.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lu_zero) by smtp.gentoo.org (Postfix) with ESMTPSA id C894C335C51 for ; Sun, 21 Jan 2018 17:40:26 +0000 (UTC) To: cellar@ietf.org References: <20180117223156.GD3493@michaelspb> From: Luca Barbato Message-ID: <648259fc-5cd3-99eb-d536-18138308c3d4@libav.org> Date: Sun, 21 Jan 2018 18:40:21 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:58.0) Gecko/20100101 Thunderbird/58.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2018 17:40:44 -0000 On 21/01/2018 18:19, Steve Lhomme wrote: > All the specs have to say is that the maximum depth indicate the EBML > depth the file should not go further from. Emphasis on the SHOULD. +1 From nobody Fri Jan 26 15:58:29 2018 Return-Path: X-Original-To: cellar@ietf.org Delivered-To: cellar@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BE1FD1270FC; Fri, 26 Jan 2018 15:58:22 -0800 (PST) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: Cc: cellar@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 6.70.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <151701110273.23497.10332019715099200596@ietfa.amsl.com> Date: Fri, 26 Jan 2018 15:58:22 -0800 Archived-At: Subject: [Cellar] I-D Action: draft-ietf-cellar-ffv1-01.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jan 2018 23:58:23 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Codec Encoding for LossLess Archiving and Realtime transmission WG of the IETF. Title : FF Video Codec 1 Authors : Michael Niedermayer Dave Rice Jerome Martinez Filename : draft-ietf-cellar-ffv1-01.txt Pages : 40 Date : 2018-01-26 Abstract: This document defines FFV1, a lossless intra-frame video encoding format. FFV1 is designed to efficiently compress video data in a variety of pixel formats. Compared to uncompressed video, FFV1 offers storage compression, frame fixity, and self-description, which makes FFV1 useful as a preservation or intermediate video format. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-cellar-ffv1/ There are also htmlized versions available at: https://tools.ietf.org/html/draft-ietf-cellar-ffv1-01 https://datatracker.ietf.org/doc/html/draft-ietf-cellar-ffv1-01 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-cellar-ffv1-01 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Sat Jan 27 00:28:50 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED4B912D93E for ; Sat, 27 Jan 2018 00:28:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.221 X-Spam-Level: X-Spam-Status: No, score=-4.221 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NH1GnO-q-PDA for ; Sat, 27 Jan 2018 00:28:46 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F76312422F for ; Sat, 27 Jan 2018 00:28:46 -0800 (PST) Received: from smtp8.infomaniak.ch (smtp8.infomaniak.ch [83.166.132.38]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0R8ShsM015446 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Sat, 27 Jan 2018 09:28:43 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp8.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0R8Sgd4187968 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Sat, 27 Jan 2018 09:28:43 +0100 Date: Sat, 27 Jan 2018 09:28:43 +0100 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: [Cellar] EBML: add handling of other encodings than binary X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jan 2018 08:28:49 -0000 Add a vector in the EBML specification, which allows to define other data encodings than binary, like the Gray codes. It works very fine on my end since almost four years now. From nobody Sat Jan 27 00:44:10 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E02EF12D880 for ; Sat, 27 Jan 2018 00:44:09 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.22 X-Spam-Level: X-Spam-Status: No, score=-4.22 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FWfuabLzX9yZ for ; Sat, 27 Jan 2018 00:44:08 -0800 (PST) Received: from smtp-sh2.infomaniak.ch (smtp-sh2.infomaniak.ch [128.65.195.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E2B4E120713 for ; Sat, 27 Jan 2018 00:44:07 -0800 (PST) Received: from smtp7.infomaniak.ch (smtp7.infomaniak.ch [83.166.132.30]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0R8i6la002989 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Sat, 27 Jan 2018 09:44:06 +0100 Received: from Castor.local (84-73-238-96.dclient.hispeed.ch [84.73.238.96]) (authenticated bits=0) by smtp7.infomaniak.ch (8.14.5/8.14.5) with ESMTP id w0R8i5cN167352 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Sat, 27 Jan 2018 09:44:05 +0100 Date: Sat, 27 Jan 2018 09:44:06 +0100 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: <151701110273.23497.10332019715099200596@ietfa.amsl.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] I-D Action: draft-ietf-cellar-ffv1-01.txt X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jan 2018 08:44:10 -0000 internet-drafts@ietf.org wrote: >A New Internet-Draft is available from the on-line Internet- >Drafts directories. >This draft is a work item of the Codec Encoding for LossLess >Archiving and Realtime transmission WG of the IETF. > > Title : FF Video Codec 1 > Authors : Michael Niedermayer > Dave Rice > Jerome Martinez > Filename : draft-ietf-cellar-ffv1-01.txt > Pages : 40 > Date : 2018-01-26 Thank you for publishing this! Could the next version possibly have a better presentation? In the plain text parts, end of lines like: and "plane_pixel_height[ 1 + ( chroma_planes ? 2 : 0 ) ]" or: value is "ceil(slice_pixel_width / (1 << log2_h_chroma_subsample))" would be much easier to read when written as: and "plane_pixel_height[ 1 + ( chroma_planes ? 2 : 0 ) ]" or: value is "ceil(slice_pixel_width / (1 << log2_h_chroma_subsample))" Thank you for consideration! From nobody Wed Jan 31 17:00:28 2018 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AFA2912DA16 for ; Wed, 31 Jan 2018 17:00:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.368 X-Spam-Level: X-Spam-Status: No, score=-2.368 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xz19Bj5OhSyp for ; Wed, 31 Jan 2018 17:00:24 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1B51512EB22 for ; Wed, 31 Jan 2018 16:53:10 -0800 (PST) Received: from [192.168.2.101] ([93.243.138.192]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MGXV6-1eTphM1wLw-00DIzd for ; Thu, 01 Feb 2018 01:53:08 +0100 To: cellar@ietf.org References: <20180117223156.GD3493@michaelspb> From: hubblec4 Message-ID: Date: Thu, 1 Feb 2018 01:52:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------650C2DFFD456EDFB69043CEF" X-Provags-ID: V03:K0:/gJsOeO0GQCEQo0hpst8yz0bN89ldvOYKXyi44W1tmu0taKSCB+ ote0lx9AYqFxNIOUUNMXXjhojdJjOWouMauUSwqITmp8s0CznTd69MGpDbwMLknv0a/wPCY JSHbcrownZULzWXJhpJEVOC0Eq/kA2YAAqv3TWq+Cml6zyomITjnKsFmcy51yPhNFJIy+hr Yg0RszcO/NJR3dgHV3/6Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:6VQnOnABLNs=:X/0cyNXwZhhzhQho5ezVus vSbjHXZ+O/S0m9inpyvoY3arClGOv49U3wjwzpByg4YwAOwLlTzRPlXxtLg9sQNj4u8YK55JL oy+Cdoz8iEfoMGizzbzdeDM0oFuZXdu8+ZE/KTmkD0tn5UQAifaAtRCXOBFnNcZSAwQyBP7BK 1+38qdzPV48roPW76SOQQRgO8I28wjbJzscCZV7apT0INOjnmxW2CvkuFnV2bgRI2C9AfszvX 4SaUXlr2+wHUlQmy3ohfhZQgjEx+O9L3zsnCltorG+skGwbQBLPOx6RV3OMkmHRHTnOkvwu/T xrAH72zAD6nC2P1MEiMT0xqz8nMxheCNS8ouUnhgvwkyJPVNM8eDnnLvE97G8vKQXTue/hhHP yU7d95NiGz2QWBn/ko8xVZxM2aSlQyPaqfTSIQap23NA8VIphoOKf69RVh5cHtswLaMGKSEYP xI1LSn7qXDXXtYcKxcEGVl+g7WC4PY48oruRSpM/zKLIc76IBqHfYhKcMIott5NrPrTWEA1Fb hqRzBlK7VlwxhunZN2h2q7yfrOUhDsJ0eME9NeWIuXnm6XcQ5v9M9UbbdHRO9Vgvgn3vyIlGE Bnb/65QpwaroTZ239l9V5rp3z7xzIsIEX1vMAfZ1Lugg6OLgM2vc0GeEE7X3oMW8iGX14AA+l bpu9nl9dUhA7QhjWD8AkkMXHSn64JSgK3AVIO2ih8K8lJ6R6+YRNT4lcN0DfuRvQXfOjgylSZ hJBJ2b+i3hoJhhoSvwscWmBSLdVQ1zoW2XX6gbDMfDxvDWsWB5yae2Yu88UWb/S9UVayeV/Fh F5flC0wtOyhG10Q36R/EGBOmM6JSwgoaD5QWQsELSJVY9NWT3Y= Archived-At: Subject: Re: [Cellar] Security considerations: recursive elements X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Feb 2018 01:00:27 -0000 This is a multi-part message in MIME format. --------------650C2DFFD456EDFB69043CEF Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi all Late, but I also say yes, we need such an element for the maximum depth of nesting. Am 21.01.2018 um 18:10 schrieb Steve Lhomme: > (starting to reply from the bottom of the thread) > > 2018-01-18 18:53 GMT+01:00 Michael Bradshaw : >> On Wed, Jan 17, 2018 at 11:42 PM, Jerome Martinez >> wrote: >>> I missed the case of recursive elements, specified in EBML. >>> for a security section, I don't think that talking about a parser is >>> right >> >> The current EBML specification's Security Consideration section >> contains a >> bulleted list for "Attacks on an EBML Reader" (where an "EBML Reader" is >> explicitly defined as a "data parser[...]"). So if we shouldn't be >> talking >> about security considerations for parsers, then that whole bulleted list >> should be removed (which I'm opposed to doing). Perhaps I've >> misunderstood >> your point here, though. > I agree with Michael, this is a legitimate attack and it should be > mentioned. Putting limits is a different topic that would probably go > in a different section of the specs. > >>> I am more in favor of  something like "it is possible to have unlimited >>> recursion, potential source of denial of service attacks" in security >>> section and >>> "An implementation may set limits on the maximum depth of nesting" in a >>> parser section (similar to JSON RFC). >> >> My apologies if my previous communications have been unclear, but >> this is >> actually what I'm trying to advocate for, so I think we might agree more >> than we realize. While I have been suggesting including minimum/maximum >> levels, I think that using something like you've written here (and >> completely omitting minimum/maximums) would still be a great >> improvement. >> >> I do think that also including some kind of minimum/maximum levels >> could be >> useful. I also think that if we do have them, they should use SHOULD/MAY >> language instead of MUST. Using SHOULD/MAY would give guidance to >> implementors on how they can maximize interoperability while still >> allowing >> them total freedom (the implementation would still be conforming to >> the spec >> even if it went against a SHOULD/MAY guidance). > I agree with that too. EBMLMaxSizeLength was mentioned in the thread > and it would make sense to make a recursion limit per file in the EBML > header. That would let parsers know if they can handle the file or not > in advance and/or detect security attacks. > > We could either have a default value for that element or none which > means there's no de-facto maximum. > > It also not be an element but something more complex, since the amount > of recursion allowed may differ between elements (see nested tags vs > nested chapters). But it may be easier for the reader and the writer > to have only one limit for all. Yes only one element is enough. We have only 2 nested elements and in pratice almost the top-level is used. In the specs are nested Tags as an example present, but I have never seen an mkv in practice. Nested chapters with a first lower level are used in practice. The Matroska DVD-Menu system uses nested chapters and maximum of levels is 7. (Each level of a chapter corresponds to a logical level in the DVD system) I prefer an element in the Matroska header which is mandatory WITH an default value. Why mandatory? While I write my own Matroska parser it is every easy to have a mandatory element. And with a default value the element can be omitted but in my program I have every time a valid "init"-value. The default vaule can be 7. All existing Matroska files without this new element could be handled safely. > > In any case a default maximum valu would belong in the Matroska > specifications, not the EBML ones. > >>> We don't talk anymore about security, and confidence depends on the >>> underlying document e.g. for my EBML document x, I may need only 1 >>> level >>> beside the EBML header and it is fine with that, no need of more. >>> So: >>> - A minimum should not be in Security section (it is not about >>> security) >> >> I agree that minimum or even maximum limits should not be placed in the >> security section. They belong elsewhere in the Matroska specification >> (exactly which section, I don't know). Sorry for not stating this more >> clearly in my original email. > We all agree. > >>> - A minimum depends on the goal of the parser, e.g. a parser just >>> checking >>> top level element size and CRC for checking a file transfer is fine >>> does not >>> need more than 2 levels. Not all tools are video players. >> >> I'm not advocating for HOW the parsers should handle deep recursion (I >> probably shouldn't have used the language "abort the parse" in my >> original >> email; it was just meant to be a starting point for the conversation). >> Handling it by skipping it is fine. If a particular element is not >> needed >> for the goals of the application, the parser should certainly feel at >> liberty to skip over it. >> >>> But a player may just ignore any chapter and tag elements and still >>> play >>> fine the file. > Not if the ordered chapters are truncated by enforcing a bogus maximum. Ordered chapters are used in top-level chapters only (expect VLC, with Matroska DVD-Menu system), so the file should play fine. > >> Yes, the encoded media data can still be parsed/decoded and the media >> can >> still be played. But part of playing a video is showing to the user the >> chapter and tag information to the video. If the player ignores the >> chapter >> or tag elements, then the playback experience is incomplete. Not all >> players >> utilize the chapter and tag elements, but those that do will likely have >> some kind of upper limit on the level of recursion they can meaningfully >> handle. >> >> _______________________________________________ >> Cellar mailing list >> Cellar@ietf.org >> https://www.ietf.org/mailman/listinfo/cellar >> > > --------------650C2DFFD456EDFB69043CEF Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Hi all

Late, but I also say yes, we need such an element for the maximum depth of nesting.

Am 21.01.2018 um 18:10 schrieb Steve Lhomme:
(starting to reply from the bottom of the thread)

2018-01-18 18:53 GMT+01:00 Michael Bradshaw <mjbshaw@google.com>:
On Wed, Jan 17, 2018 at 11:42 PM, Jerome Martinez <jerome@mediaarea.net>
wrote:
I missed the case of recursive elements, specified in EBML.
for a security section, I don't think that talking about a parser is right

The current EBML specification's Security Consideration section contains a
bulleted list for "Attacks on an EBML Reader" (where an "EBML Reader" is
explicitly defined as a "data parser[...]"). So if we shouldn't be talking
about security considerations for parsers, then that whole bulleted list
should be removed (which I'm opposed to doing). Perhaps I've misunderstood
your point here, though.
I agree with Michael, this is a legitimate attack and it should be
mentioned. Putting limits is a different topic that would probably go
in a different section of the specs.

I am more in favor of  something like "it is possible to have unlimited
recursion, potential source of denial of service attacks" in security
section and
"An implementation may set limits on the maximum depth of nesting" in a
parser section (similar to JSON RFC).

My apologies if my previous communications have been unclear, but this is
actually what I'm trying to advocate for, so I think we might agree more
than we realize. While I have been suggesting including minimum/maximum
levels, I think that using something like you've written here (and
completely omitting minimum/maximums) would still be a great improvement.

I do think that also including some kind of minimum/maximum levels could be
useful. I also think that if we do have them, they should use SHOULD/MAY
language instead of MUST. Using SHOULD/MAY would give guidance to
implementors on how they can maximize interoperability while still allowing
them total freedom (the implementation would still be conforming to the spec
even if it went against a SHOULD/MAY guidance).
I agree with that too. EBMLMaxSizeLength was mentioned in the thread
and it would make sense to make a recursion limit per file in the EBML
header. That would let parsers know if they can handle the file or not
in advance and/or detect security attacks.

We could either have a default value for that element or none which
means there's no de-facto maximum.

It also not be an element but something more complex, since the amount
of recursion allowed may differ between elements (see nested tags vs
nested chapters). But it may be easier for the reader and the writer
to have only one limit for all.
Yes only one element is enough. We have only 2 nested elements and in pratice almost the top-level is used.
In the specs are nested Tags as an example present, but I have never seen an mkv in practice.
Nested chapters with a first lower level are used in practice.
The Matroska DVD-Menu system uses nested chapters and maximum of levels is 7.
(Each level of a chapter corresponds to a logical level in the DVD system)

I prefer an element in the Matroska header which is mandatory WITH an default value.
Why mandatory?
While I write my own Matroska parser it is every easy to have a mandatory element.
And with a default value the element can be omitted but in my program I have every time a valid "init"-value.
The default vaule can be 7.
All existing Matroska files without this new element could be handled safely.



In any case a default maximum valu would belong in the Matroska
specifications, not the EBML ones.

We don't talk anymore about security, and confidence depends on the
underlying document e.g. for my EBML document x, I may need only 1 level
beside the EBML header and it is fine with that, no need of more.
So:
- A minimum should not be in Security section (it is not about security)

I agree that minimum or even maximum limits should not be placed in the
security section. They belong elsewhere in the Matroska specification
(exactly which section, I don't know). Sorry for not stating this more
clearly in my original email.
We all agree.

- A minimum depends on the goal of the parser, e.g. a parser just checking
top level element size and CRC for checking a file transfer is fine does not
need more than 2 levels. Not all tools are video players.

I'm not advocating for HOW the parsers should handle deep recursion (I
probably shouldn't have used the language "abort the parse" in my original
email; it was just meant to be a starting point for the conversation).
Handling it by skipping it is fine. If a particular element is not needed
for the goals of the application, the parser should certainly feel at
liberty to skip over it.

But a player may just ignore any chapter and tag elements and still play
fine the file.
Not if the ordered chapters are truncated by enforcing a bogus maximum.
Ordered chapters are used in top-level chapters only (expect VLC, with Matroska DVD-Menu system), so the file should play fine.




Yes, the encoded media data can still be parsed/decoded and the media can
still be played. But part of playing a video is showing to the user the
chapter and tag information to the video. If the player ignores the chapter
or tag elements, then the playback experience is incomplete. Not all players
utilize the chapter and tag elements, but those that do will likely have
some kind of upper limit on the level of recursion they can meaningfully
handle.

_______________________________________________
Cellar mailing list
Cellar@ietf.org
https://www.ietf.org/mailman/listinfo/cellar




--------------650C2DFFD456EDFB69043CEF--