From nobody Sun Nov 5 10:18:52 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 609BD13FAEB for ; Sun, 5 Nov 2017 10:18:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JqG6adF9k6Ua for ; Sun, 5 Nov 2017 10:18:49 -0800 (PST) Received: from mail-pg0-x22b.google.com (mail-pg0-x22b.google.com [IPv6:2607:f8b0:400e:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 913F213FBAF for ; Sun, 5 Nov 2017 10:18:49 -0800 (PST) Received: by mail-pg0-x22b.google.com with SMTP id b192so6478512pga.2 for ; Sun, 05 Nov 2017 10:18:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=n/K9m1nHZRyaQRkWmktzPGbJE6Cx0AiYXUdUQiOMCcg=; b=PAeRHQmgnRgdND/a886PYuJFZKW0rsnVb6xZPnOFVlNqXrJmAsSGEOUc5jbaeA2mbS HJw1wxgINohVX3APmayS1G5MaWj2rqGXnh8xC/dYYUyOS+UZvsa6IVPxa+qCJ52M2cbX GXOvlpCnPD776o/7Hm0cyUKLAoj7q8aoh895/kt/qOrfB8Mf3OtcmQlsfcdlVFnVgCcn 7e6RN9swf/FAiAerLcOBUAsY/HcIP4sP1HjDQrOcMWWNejAbJYZWsQ3RScqCPwhGFlJ8 0rovmZ91+76JJ053BkOEJLe4dFWHISQaCdOCznk6KL+J/R8DMBMEL2D5jD8nqRm78yuD 0R7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=n/K9m1nHZRyaQRkWmktzPGbJE6Cx0AiYXUdUQiOMCcg=; b=RW5juub1hSyNtmUfuDGX/NAixTk9R0Cv0FA7VOMUA9lFuV8gtaqDfP/PaQEvQZSY6U 7kMwed3OZ0cTmquiuWRMsp9nVmbx9UqVnf16swfNIOuRH2rXd3ZafokrzOT0C/w6R2/w mgVDSnzzQI6RqP30+UeKXjV5Le0GvwXyMbjnJIia+rXLm9iIUTdXor5ZKMydUWzw7gWy SMDlFMk3g5FxBUSBW+fCD5Ug08V+zW80auJ/7+YtMAeqpgQqGIoef7l5B+cIxwNvjsp+ prR3etzq1/zpT9j07z61ZPuI2Dl+W76JRWxKeD1ahAlglX2Ys9TUsXmlO5ggB49uyoAh IWrQ== X-Gm-Message-State: AMCzsaXGpV3FKWm0/l3qnuT/+qJENs3VFtkzrkaw0yKAdN+RfUcI/coI l8sw0jLD9PwF5Owt8ytXYIHz3tXIkEmPcptOIQ2mvkJ3 X-Google-Smtp-Source: ABhQp+TSOWPafImiAVypOUODYyt92uWx8HnxyQPtJrfl7amhIetvK8RUqBgyqnQb7iVEpXZFvBjqHEarNE15NZzjSUY= X-Received: by 10.84.135.101 with SMTP id 92mr12743634pli.180.1509905928678; Sun, 05 Nov 2017 10:18:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.185.73 with HTTP; Sun, 5 Nov 2017 10:18:48 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 5 Nov 2017 19:18:48 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Lacing for video frames X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Nov 2017 18:18:51 -0000 Hi, I see the subject is not very popular but I still want Matroska to be the best, so I had a deeper quick look. As I don't do any adaptive streaming encoding for the web I tried to do it with ffmpeg but I failed miserably. Then I found these samples from Apple [1]. So I took the lowest bitrate video [2] which is 530kbps according to the manifest and 369 kbps according to MediaInfo. Then I remuxed it with mkvmerge. Then go through mkclean and here are the results: - 27 672 619 original fMP4 with H264 - 27 449 794 mkvmerge with default options (we win already) - 27 447 068 mkclean with default options - 27 439 197 mkclean with --live - 27 357 090 mkclean with --remux and --optimize - 27 349 220 mkclean with --remux and --optimize and --live Same thing with the advanced HEVC sample that has 165 kbps variants for HEVC and H264: - 10 663 861 original fMP4 with H264 - 10 558 115 mkvmerge with default options (we win already) - 10 554 002 mkclean with default options - 10 498 886 mkclean with --remux and --optimize - 11 492 052 original fMP4 with HEVC - 11 410 786 mkvmerge with default options (we win already) - 11 407 257 mkclean with default options - 11 371 266 mkclean with --remux and --optimize So in short, even with the 4KB padding that mkvmerge uses, the Matroska variants are still smaller. With boundaries set at every keyframe, that should do proper segments for adaptive streaming just like with fMP4. Except with have a proper seeking cue, checksums on vital parts and the possible extreme low latency described in my earlier post. And all that without using lacing at all. At 1.54% smaller in the second H264 sample, that's quite a difference with no data lost at all. [1]: https://developer.apple.com/streaming/examples/ [2]: https://devstreaming-cdn.apple.com/videos/streaming/examples/img_bipbop_adv_example_fmp4/v2/main.mp4 2017-10-29 16:53 GMT+01:00 Steve Lhomme : > As some of you may know I was in San Fancisco a few weeks ago for 2 > conferences (FOMS and Demuxed) about video, mostly on the web. I did a > blog about it here > http://robux4.blogspot.fr/2017/10/foms-and-demuxed.html > > Among other interresting things I found out about a new video > container called CMAF. It seems to be a set of guidelines around > fragmented MP4 which I didn't know existed either. It's targeted > mainly at web video distribution with adaptive streaming in mind. CMAF > seems even more specified in distribution of encrypted (DRM) content. > > WebM is used the same way and can achieve the same level of features > (and even more with Matroska). But while I dug a little more, I > noticed that fragmented MP4 tend to be much smaller than the same > content in Matroska/WebM (see blog post for some details). It's > because with fragmented MP4 (or the files I analyzed) the video frames > are packed contiguously, the boundaries and timestamp being stored in > a separate atom (equivalent of an EBML Master) which itself doesn't > contain more info that that. > > To compare a fragmented MP4 frame with Matroska it would be something like this: > fMP4 = [encoded frame] + [size of encoded frame] + [key frame flag] > MKV = [SimpleBlock ID] + [SimpleBlock Length] + [SimpleBlock Header] + > [encodec frame] > > I think we could reduce our overhead to reach something more in line > with fMP4. It would be mean allowing lacing for video frames > explicitly. Right now there's nothing that prevent it from being used. > It may lack some descriptions though on how to interpret the > timestamps and the SimpleBlock flags. I think for video it should > required a DefautlDuration being present for the track and also that > all flags are the same in the (Simple)Block, especially the keyframe. > I'm also not sure if mkvmerge blocks it for video or not. > > Because the same flag has to be present and because we store frames in > coding order, there might be cases where this doesn't work. When B > frames are present you cannot tell the frame timestamp within a lace. > So that may limit the use with codecs like H264 or HEVC. > > -- > Steve Lhomme > Matroska association Chairman -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 5 10:34:05 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4EC5213FCA5 for ; Sun, 5 Nov 2017 10:34:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.102 X-Spam-Level: X-Spam-Status: No, score=-0.102 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ty4_3Z3s6CND for ; Sun, 5 Nov 2017 10:34:02 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F10813FC9E for ; Sun, 5 Nov 2017 10:34:02 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:49282) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eBPk4-0001ib-2y for cellar@ietf.org; Sun, 05 Nov 2017 19:33:56 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id ABC7F654001A for ; Sun, 5 Nov 2017 19:33:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1509906836; bh=Ne3p6W1SGt0m4GIIhS95qkZ7+YaywTYkNmn37MspLYo=; h=References:From:To:Cc:Subject:In-reply-to:Date:From; b=BPRemop4cyNJkl/+acacTEWT17Mbh2JSEzObdm0FovdlEP8mxovqTa3RkP473eJH0 bd/1IzibGzRj/c5T3O7q0e+dBdJ2ekA5vwmDv33N9K2jx0QBH7fTd/n9hKMdnJJ+e+ wIPeIEn00iwSZK6vcmiEzWITVlGw22ngST8lMfTO5AS+XVBNZnopbmENQgpScEXKrh NDu4P1gknGXkhDOyAmJse3q+Q02/XK2Cw4TlciWPBar072joKvRomQEPmSXRBzKcAf z5JeEgNqKTDZ2iINsPXRCDsFb9NEmgaoJQzz1fNdColmd+IcbR8+V4/mhVuO25tMhk nf1+v9FfkHse3eeJITWzQGS0Be/9fyBTPqi6gl+3lwylFWkAKpo92h4vw+sHnNN++t wx9qNODQHnI3D7HYFdbNbK1M9P2tvtMgJGO8QciOR+jCp8o7/Gqs9hFUNzw/2cFNTT DSKn7Cw3fuuwdUCv4smDjVAZ6nzPoMlu0f2g87Z75Dma3qh2odj0kdFjQth/yKg14L L7CL7ud9cg+8NyuMDXb3zFygJUGaS17W+42hg0fFvgKuf/IEU26G+EekOGx5QVjnoM QFC2qb7gGuoVUQw6QA+Tuqw1DvfPCmkDoX8d6kceHjqoIMKkFnFzXs6VFX2p9XXRw0 UwYOLGcgEY2KnbE8yIKnTQtA= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTPS id EFB8B253B43B for ; Sun, 5 Nov 2017 19:33:55 +0100 (CET) References: User-agent: mu4e 1.0-alpha0; emacs 25.3.1 From: Moritz Bunkus To: Codec Encoding for LossLess Archiving and Realtime transmission Cc: In-reply-to: Date: Sun, 05 Nov 2017 19:33:55 +0100 Message-ID: <874lq8bmng.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain Archived-At: Subject: Re: [Cellar] Lacing for video frames X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Nov 2017 18:34:03 -0000 Hey, I did read your mail, but to be honest I don't care all that much about having the smallest possible files in all cases. In all comparisons I did in the past Matroska came out looking very competitive (only Nut was consistently smaller). On top of that my gut feeling was that lacing video frames wouldn't give that much of an advantage to warrant a large investment of time and effort. Your measurements seem corroborate my gut feeling. Kind regards, mosu From nobody Sun Nov 5 10:37:53 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 492C713FCA8 for ; Sun, 5 Nov 2017 10:37:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gB2dDFQKuIw2 for ; Sun, 5 Nov 2017 10:37:51 -0800 (PST) Received: from mail-pg0-x233.google.com (mail-pg0-x233.google.com [IPv6:2607:f8b0:400e:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F116413FCA6 for ; Sun, 5 Nov 2017 10:37:50 -0800 (PST) Received: by mail-pg0-x233.google.com with SMTP id k7so6493848pga.3 for ; Sun, 05 Nov 2017 10:37:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=nBRJ0VlqJTWvj5F2DedMpZuRiOQlER+TADG/FmtzLtE=; b=aD+nmBR0v+shK5xFt5lYuf+p+pOmeVkeQEeCFrr1s2CtLPhUw9p1Sr21cqXy+ODhVD sqjuneKD5fHrIqammCBpqui70j+YCQQCnw/aP3JdGLr42DIH68/2EedtUyA9G/KG8R9J eLqqCsLr6ZhySnl88yEXLFYn6T+NZkUtP81NjBqaFepVBrXCK4jlNNxpnA0v2TKnn3pf 1vtHYudUD/duTifTvSy9ifqiwwQlTnd+1p1qCo+jKFTuvTucjUbYozrYMsbedKLKtPXN uK398xZtrbVqcZQybFBGjV5bp4oBDulNIWKocTqwR1GtmuUkhZX214acDAg8WKvxr1ya ZArA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=nBRJ0VlqJTWvj5F2DedMpZuRiOQlER+TADG/FmtzLtE=; b=NANSXIP7UdSsaJDM5MxDUnw7M+MKFr0MbgE3sdVHRLNuR6t597GWg2r/Nl5ML3pH0F tvAsdfyYYvyzcEf5UOheCvtYQnS1NEYBtDvfTwAcm1mKW0rpPcXIdbEsVFnFX2rOiRDg gjtBopTqewM7pz4VKQ4sTIG5wEyjR9BNETYMy7f50Gb3WCbZXYKFC55mVj7/v5FDMh5h XZTFNgOr/rvovrXMPXARWlr/dP83S8ef7A9eG+SpfhcbPVsvgQOC39mYCQY8M+wLvB85 vW7Ft4F1ctmRY93MCA5mRzID5VHjMchYlSHBMNUm6z8d4gD5X3hg+Hjx7EjIY+XpBNFY U/AA== X-Gm-Message-State: AMCzsaVbpZymDqHRVqBsuPXwCqNTlXa7d6lixrtJM5JhO6OptAQSmc0W 1LOs5cdyHBmVhckoXgr0jFth44cA0s/Fwz+SFij3gQ== X-Google-Smtp-Source: ABhQp+RwXWDsHBTfT+tC48w4/lOWmfuQepRoN49l/1T4zdAS2di0EL8lLmBKlXHJ5VxG0tCe6Jt24DMQEX21XZnLVTw= X-Received: by 10.98.202.74 with SMTP id n71mr14292841pfg.202.1509907070538; Sun, 05 Nov 2017 10:37:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.185.73 with HTTP; Sun, 5 Nov 2017 10:37:50 -0800 (PST) In-Reply-To: <874lq8bmng.fsf@bunkus.org> References: <874lq8bmng.fsf@bunkus.org> From: Steve Lhomme Date: Sun, 5 Nov 2017 19:37:50 +0100 Message-ID: To: Moritz Bunkus Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Lacing for video frames X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Nov 2017 18:37:52 -0000 I agree. The only case where lacing is useful is when it actually saves bytes. But lacing itself is not space efficient. So it's only worth for very small frames and possibly similar frame sizes. That's a rare/edge case for video. On the other hand, this overhead thing is one of the key feature of Matroska. It's why EBML and default values and lacing exist. That was one of the main design goal and it would be side to have that competitive feature lost. 2017-11-05 19:33 GMT+01:00 Moritz Bunkus : > Hey, > > I did read your mail, but to be honest I don't care all that much about > having the smallest possible files in all cases. In all comparisons I did > in the past Matroska came out looking very competitive (only Nut was > consistently smaller). On top of that my gut feeling was that lacing video > frames wouldn't give that much of an advantage to warrant a large investment > of time and effort. > > Your measurements seem corroborate my gut feeling. > > Kind regards, > mosu > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Mon Nov 6 00:41:46 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3026A13FB51 for ; Mon, 6 Nov 2017 00:41:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.201 X-Spam-Level: X-Spam-Status: No, score=-1.201 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1dX7wVyvmsZW for ; Mon, 6 Nov 2017 00:41:44 -0800 (PST) Received: from mx01.mail.netstorage.at (mx01.mail.netstorage.at [89.207.144.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3BCD813F963 for ; Mon, 6 Nov 2017 00:41:43 -0800 (PST) Received: from p1002.netstorage.at (p1002.netstorage.at [89.207.146.186]) by mx01.mail.netstorage.at (Postfix) with ESMTPS id 2332AA002A for ; Mon, 6 Nov 2017 09:41:29 +0100 (CET) Received: from mailix (noaport.de [46.237.252.213]) by p1002.netstorage.at (Postfix) with ESMTPA id C517581B51 for ; Mon, 6 Nov 2017 09:41:28 +0100 (CET) Received: from [127.0.0.1] (HSI-KBW-46-237-252-214.hsi.kabel-badenwuerttemberg.de [46.237.252.214]) by mailix with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128) ; Mon, 6 Nov 2017 09:41:28 +0100 To: cellar@ietf.org References: <874lq8bmng.fsf@bunkus.org> From: Tobias Rapp Organization: NOA GmbH Message-ID: <632329e8-195c-a6a6-597e-48296633d2f7@noa-archive.com> Date: Mon, 6 Nov 2017 09:41:28 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <874lq8bmng.fsf@bunkus.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-PPP-Message-ID: <20171106084129.7306.97310@p1002.netstorage.at> X-PPP-Vhost: noa-archive.com X-NetStorage-MailScanner-Information: Please contact the ISP for more information X-NetStorage-MailScanner-ID: 2332AA002A.A46B5 X-NetStorage-MailScanner: Found to be clean X-NetStorage-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (nicht zwischen gespeichert, Wertung=-0.5, benoetigt 6, autolearn=not spam, BAYES_00 -0.50) X-NetStorage-MailScanner-From: t.rapp@noa-archive.com Archived-At: Subject: Re: [Cellar] Lacing for video frames X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Nov 2017 08:41:46 -0000 On 05.11.2017 19:33, Moritz Bunkus wrote: > I did read your mail, but to be honest I don't care all that much about > having the smallest possible files in all cases. In all comparisons I did > in the past Matroska came out looking very competitive (only Nut was > consistently smaller). On top of that my gut feeling was that lacing video > frames wouldn't give that much of an advantage to warrant a large investment > of time and effort. Similar experience on my side. I looked at MP4 some time ago and found that this way of putting all index data into one box and all encoded data into another box has a negative impact on playback initialization time: A player has to load all the index data to be able to start playing. When file duration is some hours and network throughput is slow (web) the effect really hurts much more than what is gained by the possibly lower container overhead. Fragmented MP4 compensates for the playback initialization time and enables DASH. As fast player initialization and DASH already seem to be supported by Matroska/WebM I see no big gain in further file size optimization -- the current numbers look comparable enough. Just my personal thoughts. Tobias From nobody Sat Nov 11 11:59:15 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 619E61204DA for ; Sat, 11 Nov 2017 11:59:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_LOW=-0.7, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j2Q8UNZnOKqq for ; Sat, 11 Nov 2017 11:59:11 -0800 (PST) Received: from zucker2.schokokeks.org (zucker2.schokokeks.org [178.63.68.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 881FA12008A for ; Sat, 11 Nov 2017 11:59:11 -0800 (PST) Received: from [10.0.0.11] (1360030002.d-dsl.at [::ffff:81.16.105.50]) (AUTH: PLAIN bubestinger@schokokeks.org, TLS: TLSv1/SSLv3, 128bits, ECDHE-RSA-AES128-GCM-SHA256) by zucker.schokokeks.org with ESMTPSA; Sat, 11 Nov 2017 20:59:08 +0100 id 0000000000000044.000000005A07568C.00004673 To: "cellar@ietf.org" References: <887DF3D8-04E6-49CD-8F28-C3C06D07CCC9@northwestern.edu> From: "Peter B." Message-ID: Date: Sat, 11 Nov 2017 20:59:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <887DF3D8-04E6-49CD-8F28-C3C06D07CCC9@northwestern.edu> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US Archived-At: Subject: Re: [Cellar] FFV1 for Film X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Nov 2017 19:59:13 -0000 On 2017-10-30 17:55, Joshua Michael Yocum wrote: > And here’s a direct link to the Google Doc in that folder as well: > https://drive.google.com/open?id=1-nZj53eCuY_81qYVkVDdZRa_pkHicVzETaTAxkCcYD8 Thanks for the link to the document :) btw: The Mediateka of the National Broadcaster of Slovenia (RTV: http://www.rtvslo.si/) are also very interested in FFV1 for film (DPX source). MediaArea is working on a project for converting DPX to FFV1/MKV, called "RAWcooked" Also will have the ability to regenerate the original DPX files bit-exact: https://mediaarea.net/Events/2017-11-10_NTTW_RAWcooked Still looking for sponsors, so if you are interested please contact them! Kind regards, Peter B. From nobody Mon Nov 13 06:10:54 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9545512943E for ; Mon, 13 Nov 2017 06:10:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.119 X-Spam-Level: X-Spam-Status: No, score=-1.119 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nTSA4B5vVlRy for ; Mon, 13 Nov 2017 06:10:45 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 22A6F129438 for ; Mon, 13 Nov 2017 06:10:36 -0800 (PST) Received: from [146.96.19.240] (port=51396 helo=[10.10.201.39]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eEFRb-000ufr-Jy for cellar@ietf.org; Mon, 13 Nov 2017 09:10:36 -0500 From: Dave Rice Content-Type: multipart/alternative; boundary="Apple-Mail=_F8130192-56C3-4027-BB7F-3274B5929E50" Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Message-Id: <45088E7F-1B88-48E0-8914-955153598573@dericed.com> References: <151056093229.5515.2754485162745879017.idtracker@ietfa.amsl.com> To: Codec Encoding for LossLess Archiving and Realtime transmission Date: Mon, 13 Nov 2017 09:10:34 -0500 X-Mailer: Apple Mail (2.3226) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] Fwd: I-D was expired X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 14:10:46 -0000 --Apple-Mail=_F8130192-56C3-4027-BB7F-3274B5929E50 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi all, > Begin forwarded message: >=20 > From: I-D Expiring System > Subject: I-D was expired > Date: November 13, 2017 at 3:15:32 AM EST > To: > Cc: ben@nostrum.com, cellar-chairs@ietf.org > Resent-From: > Resent-To: michael@niedermayer.cc, dave@dericed.com, = jerome@mediaarea.net >=20 > was just expired. > This draft is in the state "I-D Exists" in the Datatracker. Just a note for those seeing these notices. = draft-niedermayer-cellar-ffv1-02 has expired but last July this document = was superseded by draft-ietf-cellar-ffv1-00 which remains active. Dave Rice= --Apple-Mail=_F8130192-56C3-4027-BB7F-3274B5929E50 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi all,

Begin forwarded = message:

From: = I-D Expiring System <ietf-secretariat-reply@ietf.org>
Subject: = I-D was expired = <draft-niedermayer-cellar-ffv1-02.txt>
Date: = November 13, 2017 at 3:15:32 AM = EST
To: = <draft-niedermayer-cellar-ffv1@ietf.org>
Cc: = ben@nostrum.com, cellar-chairs@ietf.org
Resent-From: = <alias-bounces@ietf.org>
Resent-To: = michael@niedermayer.cc, dave@dericed.com, jerome@mediaarea.net

<draft-niedermayer-cellar-ffv1-02.txt> was just = expired.
This draft is in the state "I-D Exists" in the = Datatracker.

Just a note for those seeing these notices. = draft-niedermayer-cellar-ffv1-02 has expired but last July this document = was superseded by draft-ietf-cellar-ffv1-00 which remains = active.

Dave Rice
= --Apple-Mail=_F8130192-56C3-4027-BB7F-3274B5929E50-- From nobody Tue Nov 14 10:11:16 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 762EB1286D6 for ; Tue, 14 Nov 2017 10:11:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.722 X-Spam-Level: X-Spam-Status: No, score=-0.722 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nx3DKjuET-4E for ; Tue, 14 Nov 2017 10:11:13 -0800 (PST) Received: from 14.mo3.mail-out.ovh.net (14.mo3.mail-out.ovh.net [188.165.43.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D6AA3127342 for ; Tue, 14 Nov 2017 10:11:12 -0800 (PST) Received: from player758.ha.ovh.net (b6.ovh.net [213.186.33.56]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id 4DEEB16F91C for ; Tue, 14 Nov 2017 19:11:10 +0100 (CET) Received: from [192.168.2.120] (p5DDB4690.dip0.t-ipconnect.de [93.219.70.144]) (Authenticated sender: jerome@mediaarea.net) by player758.ha.ovh.net (Postfix) with ESMTPSA id D66DB2C008E for ; Tue, 14 Nov 2017 19:11:09 +0100 (CET) To: Codec Encoding for LossLess Archiving and Realtime transmission From: Jerome Martinez Message-ID: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> Date: Tue, 14 Nov 2017 19:11:09 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 3584302356011356305 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrjedugdduuddvucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Nov 2017 18:11:15 -0000 Hello, from http://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219642.html there is a questioning (by myself) about the purpose of the following text in Matroska specs, about BlocckDuration: "When set to 0 that means the frame is not a keyframe" I don't understand this part. What is the relationship between being a key frame and the duration of the block? I don't find an use case for setting 0 here, and what would be the expected result for a player. Jérôme From nobody Wed Nov 15 23:49:26 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1625A1287A0 for ; Wed, 15 Nov 2017 23:49:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z1B_qTpckYfg for ; Wed, 15 Nov 2017 23:49:23 -0800 (PST) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3644A120724 for ; Wed, 15 Nov 2017 23:49:23 -0800 (PST) Received: by mail-pg0-x22a.google.com with SMTP id 4so11119001pge.1 for ; Wed, 15 Nov 2017 23:49:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=QmkuWzYCktV8fOBWhN1uHUO4t0tPgW7CZZo3Re8PYeY=; b=NkpGxmr0eoBn80G8GuZbsyQsbTEZyh1+N1tTdBJ18f00rlANwhS9ZEkZnjlS0D7WVS GOZFjpoND9j4ccMk67XsY8zOESQUSxXzB5oNgAT/8eiGsCOv5L8HNLIgKje6iInn5EKQ welmneSeOE5tfU7KvqXIVaJ0q8hB9wuw4bpa1wYcWbDUT2FjTOcIoCMgW6E8s6XpzOTX rpMunt+NooC2G2MsshvD602b7pxX/VbRgX/Voyz5IePwiv23heJmbdjuWk0Tx9T67NiB ZKS2nvXjbmSPqMDrq6et+rRqKPPTc65gkyf7J2mHYaTboxc0q2YY/+w+ie76B1nzICxY D5pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=QmkuWzYCktV8fOBWhN1uHUO4t0tPgW7CZZo3Re8PYeY=; b=AtoT3VowWD717rqWyKyPhu8ZWNURu62oRWE6YGPkR7VT4/AuPXacdIpWK1mTMnCrSc 2ZHSN8b/TNkrPG9MLqutz0RcRD5RvX5Zj2vllc4gDvfN4wdIuBWXGt/RH2XdS6A88zUc wgILd9IfFujkSVKh1SMJexhdYWGpD+CnmAbZm7HIVjzrYKB4bTmqkZ6S1/qQrKieX2dC igFfQOE/3o9gyEspsUu5o2Aa0+dWO5heMBkpeR2bYQWqiNV1M3hcXUxnoKeB/NeuvMCe pZBZJhrbitlnwQfna1W3cmC5ArtsoOTJRDutk2Mnb8Hv+B3JaulMahJs4nwtM5mhKcoy DxIw== X-Gm-Message-State: AJaThX7FO7Nsl193leKPa66SYZOL0qEbPE64y22HoOUanTbQy1pGngZf MaMOu5Hgy77xzUf+O4kjSaA1cMMhMv4QCV/cOdFqQA== X-Google-Smtp-Source: AGs4zMYi6zxnsAD9sQKpBMdWSC3rL9ixgJOsAU3+RTxU6mSNkgVFaqLKSAOOZ5wjuev+uBhXTDtCr+0pZqDuWxGMRxQ= X-Received: by 10.99.8.194 with SMTP id 185mr847896pgi.202.1510818562584; Wed, 15 Nov 2017 23:49:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.161.7 with HTTP; Wed, 15 Nov 2017 23:49:22 -0800 (PST) In-Reply-To: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> From: Steve Lhomme Date: Thu, 16 Nov 2017 08:49:22 +0100 Message-ID: To: Jerome Martinez Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 07:49:25 -0000 2017-11-14 19:11 GMT+01:00 Jerome Martinez : > Hello, > > from http://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219642.html > > there is a questioning (by myself) about the purpose of the following tex= t > in Matroska specs, about BlocckDuration: > "When set to 0 that means the frame is not a keyframe" > > I don't understand this part. What is the relationship between being a ke= y > frame and the duration of the block? > I don't find an use case for setting 0 here, and what would be the expect= ed > result for a player. I think this is a mistake. Indeed it doesn't make sense. So it should be removed. Even a duration of 0 in itself doesn't seem to make much sense. After all Blocks in Matroska don't really have a duration unless you explicitly set it. Which BTW should be something that should be clarified. Especially in the context of preservation, ie you're supposed to display the frame as long as there is another usable frame, unless there's a duration. In which case what happens is undefined behaviour. Same thing for audio. Should it revert to black/silence ? > J=C3=A9r=C3=B4me > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Thu Nov 16 00:01:54 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9649712947C for ; Thu, 16 Nov 2017 00:01:47 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IxA0dWNHfbNR for ; Thu, 16 Nov 2017 00:01:44 -0800 (PST) Received: from 7.mo3.mail-out.ovh.net (7.mo3.mail-out.ovh.net [46.105.57.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E9741120724 for ; Thu, 16 Nov 2017 00:01:43 -0800 (PST) Received: from player758.ha.ovh.net (b6.ovh.net [213.186.33.56]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id F3BB5170849 for ; Thu, 16 Nov 2017 09:01:41 +0100 (CET) Received: from [192.168.2.120] (p5DDB4690.dip0.t-ipconnect.de [93.219.70.144]) (Authenticated sender: jerome@mediaarea.net) by player758.ha.ovh.net (Postfix) with ESMTPSA id 98F8C2C009C for ; Thu, 16 Nov 2017 09:01:41 +0100 (CET) To: Codec Encoding for LossLess Archiving and Realtime transmission References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> From: Jerome Martinez Message-ID: Date: Thu, 16 Nov 2017 09:01:42 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Ovh-Tracer-Id: 5036431759663829137 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrjeeggdduudejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 08:01:47 -0000 On 16/11/2017 08:49, Steve Lhomme wrote: > 2017-11-14 19:11 GMT+01:00 Jerome Martinez : >> Hello, >> >> from http://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219642.html >> >> there is a questioning (by myself) about the purpose of the following text >> in Matroska specs, about BlocckDuration: >> "When set to 0 that means the frame is not a keyframe" >> >> I don't understand this part. What is the relationship between being a key >> frame and the duration of the block? >> I don't find an use case for setting 0 here, and what would be the expected >> result for a player. > I think this is a mistake. Indeed it doesn't make sense. So it should > be removed. OK. I'll send a patch. Edit: in the meanwhile, I see the PR, you were quicker than me :). > Even a duration of 0 in itself doesn't seem to make much > sense. After all Blocks in Matroska don't really have a duration > unless you explicitly set it. The test case was reported https://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219945.html It makes sense, as you can not know in advance how long is the subtitle (it is the case for PGS in the example, it would be same for e.g. 608/708 closed captions real time transcoding to UTF-8 Plain Text), so in the case of streaming you can not wait up to know the duration of the block before writing/sending the subtitle block. Is there something else in Matroska for saying "do not display anymore the previous content"? > > Which BTW should be something that should be clarified. Especially in > the context of preservation, ie you're supposed to display the frame > as long as there is another usable frame, unless there's a duration. > In which case what happens is undefined behaviour. Same thing for > audio. Should it revert to black/silence ? IMO it is the role of the player to decide what to show for "empty" stuff, and it is normal that the format does not define how to display no video (!= black, e.g. on a transparent screen it would be 100% transparent instead, and for any future technology you don't know what will be the way to display "no video") and no audio (!= silence). Jérôme From nobody Thu Nov 16 01:53:44 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F3BF126CD6 for ; Thu, 16 Nov 2017 01:53:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MU7mfBr1Hd3A for ; Thu, 16 Nov 2017 01:53:36 -0800 (PST) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 62D1E1292AE for ; Thu, 16 Nov 2017 01:52:46 -0800 (PST) Received: by mail-pg0-x22a.google.com with SMTP id z184so14813005pgd.13 for ; Thu, 16 Nov 2017 01:52:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=pZqjRNh0GHbO8Yko/bfSr5NsG43nnwGrGqTPzlxDGLs=; b=XJe8dfkK4w6nx0FmI1X0hEkIXRZ5YptAPF2nT2UXxF0jVRR6BqHstQ9QyN1wc55lky IKXaCOhlhvolvGEZ17nvryXYWQzDZ66TCg2EnkVIUOpGassXLbEtm4DkH5wtz2fgj8yz oIzax2cIsoUdGNvly9NVSEY0toCYLsmYTGV/I3K8cXlu4JgdxIFeCholW/1JqxtOOnhG QzWR3vgzO0KOO5B/ZN75xhIzXVUoo/ALm8xJhLifcC1jGKL0y8fA1OdLxZLUNJyZkVMX 92/0e2lSKYHulkLUnaILKmOsZBHex38E/AB/UnipHs9rYYruW6xIPlg3ll14JOzzON14 RgBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=pZqjRNh0GHbO8Yko/bfSr5NsG43nnwGrGqTPzlxDGLs=; b=qGZ0Io0PX1df5mnE/IVjYK2Fd//5xFqvZAZ0y+lISJoVd7unEPG/ZM13mkoqnABMrQ 6HkaU8fkVtcw7hwgRMDUDIRunfBvioG77Et1R1DqXLW0v65UWoT28AcwXpTuP4HFHDPZ IKkxEWj+Itp0SgfzW7OMhylntlkgCNYEZ8EWUsPpiDq9OwIOHpblSJLfNNADhu3VL+HZ XXNAdUHCwrRGW552lwzzzrXGpf/zpx8s0NyyEOEinR9s/aGnKyGz4KrNI9JGPmuBPa26 dQw2lH0/q6jMyKmT1VEPxFe4QtNhXmzL1mGeuXgwK7ISwR7Z8Ft9Wwhvtrd0FrELXQMp jtTw== X-Gm-Message-State: AJaThX7RdiJmz9/97qQhOZL+LNBOc446xvqevxNIY56PChjOtdmhVudh jelaC7G9SGDqfsYmCNdqi7FyhZMLE40t4FxKrV1ZFg== X-Google-Smtp-Source: AGs4zMa0aYnTcSCe/CtjIHZ+8QD9AaE7vs1IO4k4eNviCtH1IKhfDqq2pYRNH0jKlD7mIBCh0tLrBKz0EqmujHl1N30= X-Received: by 10.101.69.2 with SMTP id n2mr1113628pgq.79.1510825965479; Thu, 16 Nov 2017 01:52:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.161.7 with HTTP; Thu, 16 Nov 2017 01:52:45 -0800 (PST) In-Reply-To: References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> From: Steve Lhomme Date: Thu, 16 Nov 2017 10:52:45 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:53:39 -0000 2017-11-16 9:01 GMT+01:00 Jerome Martinez : > On 16/11/2017 08:49, Steve Lhomme wrote: >> >> 2017-11-14 19:11 GMT+01:00 Jerome Martinez : >>> >>> Hello, >>> >>> from http://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219642.html >>> >>> there is a questioning (by myself) about the purpose of the following >>> text >>> in Matroska specs, about BlocckDuration: >>> "When set to 0 that means the frame is not a keyframe" >>> >>> I don't understand this part. What is the relationship between being a >>> key >>> frame and the duration of the block? >>> I don't find an use case for setting 0 here, and what would be the >>> expected >>> result for a player. >> >> I think this is a mistake. Indeed it doesn't make sense. So it should >> be removed. > > > OK. I'll send a patch. > Edit: in the meanwhile, I see the PR, you were quicker than me :). > >> Even a duration of 0 in itself doesn't seem to make much >> sense. After all Blocks in Matroska don't really have a duration >> unless you explicitly set it. > > > The test case was reported > https://ffmpeg.org/pipermail/ffmpeg-devel/2017-November/219945.html Not sure what needs to be fixed in mkvalidator. A 0 BlockDuration doesn't seem right. In fact I wonder if we should just set in the specs that the field MUST NOT be 0. It has a default value (of DefaultDuration) which cannot be 0 too. Is there a case where it would make sense to set 0 ? > It makes sense, as you can not know in advance how long is the subtitle (= it > is the case for PGS in the example, it would be same for e.g. 608/708 clo= sed > captions real time transcoding to UTF-8 Plain Text), so in the case of > streaming you can not wait up to know the duration of the block before > writing/sending the subtitle block. If it's streaming then the issue already exists in the original stream, you can't tell the duration until you receive the next one, if I understand correctly. It seems you don't need to set the duration in Matroska for such a case. The next "frame" will erase the previous one. > Is there something else in Matroska for saying "do not display anymore th= e > previous content"? Not for the moment. Does this exist anywhere else ? That sounds like something that may be internal to the codec. >> >> Which BTW should be something that should be clarified. Especially in >> the context of preservation, ie you're supposed to display the frame >> as long as there is another usable frame, unless there's a duration. >> In which case what happens is undefined behaviour. Same thing for >> audio. Should it revert to black/silence ? > > > IMO it is the role of the player to decide what to show for "empty" stuff= , > and it is normal that the format does not define how to display no video = (!=3D > black, e.g. on a transparent screen it would be 100% transparent instead, > and for any future technology you don't know what will be the way to disp= lay > "no video") and no audio (!=3D silence). > > > J=C3=A9r=C3=B4me > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Thu Nov 16 02:23:53 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46A751200CF for ; Thu, 16 Nov 2017 02:23:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6FYaPVcxHZIV for ; Thu, 16 Nov 2017 02:23:48 -0800 (PST) Received: from 2.mo3.mail-out.ovh.net (2.mo3.mail-out.ovh.net [46.105.75.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1CE4C127909 for ; Thu, 16 Nov 2017 02:23:47 -0800 (PST) Received: from player758.ha.ovh.net (b6.ovh.net [213.186.33.56]) by mo3.mail-out.ovh.net (Postfix) with ESMTP id CC07116E9A8 for ; Thu, 16 Nov 2017 11:23:45 +0100 (CET) Received: from [192.168.2.120] (p5DDB4690.dip0.t-ipconnect.de [93.219.70.144]) (Authenticated sender: jerome@mediaarea.net) by player758.ha.ovh.net (Postfix) with ESMTPSA id 652302C00D1 for ; Thu, 16 Nov 2017 11:23:45 +0100 (CET) To: cellar@ietf.org References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> From: Jerome Martinez Message-ID: <59098280-b8b8-b967-5c63-251ed830e5ec@mediaarea.net> Date: Thu, 16 Nov 2017 11:23:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-Ovh-Tracer-Id: 7435724461462851729 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrjeehgddufecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecu Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 10:23:51 -0000 On 16/11/2017 10:52, Steve Lhomme wrote: > Not sure what needs to be fixed in mkvalidator. A 0 BlockDuration > doesn't seem right. In fact I wonder if we should just set in the > specs that the field MUST NOT be 0. It has a default value (of > DefaultDuration) which cannot be 0 too. > > Is there a case where it would make sense to set 0 ? In the case there is nothing to show, e.g. a previous subtitle element which should disappear. > >> It makes sense, as you can not know in advance how long is the subtitle (it >> is the case for PGS in the example, it would be same for e.g. 608/708 closed >> captions real time transcoding to UTF-8 Plain Text), so in the case of >> streaming you can not wait up to know the duration of the block before >> writing/sending the subtitle block. > If it's streaming then the issue already exists in the original > stream, you can't tell the duration until you receive the next one, if > I understand correctly. It seems you don't need to set the duration in > Matroska for such a case. The next "frame" will erase the previous > one. The next frame is too far away. and has content. Here, the issue is that the content is not linear, you have "holes" between frames. One could provide an empty subtitle with no duration, but it is not exactly same as "no data" (e.g. black frame is a frame content, it is not the lack of frame). IIUC, a duration of 0 is for saying "no more content starting now" in the use case. How to say that if mkvalidator says it is invalid? An example: 1/ several frames with BlockDuration having the right value, content is correctly displayed. 2/ Last subtitle is displayed, it is file streaming and we don't know at this moment the duration, so BlockDuration is not filled (="value is assumed to be the difference between the timestamp of this Block and the timestamp of the next Block") 3/ At some point, we know that the subtitle stream should stop to be displayed (before the full duration indicated in the header). End of this stream. an empty subtitle with duration of 0 in order to say "this is the end of the stream", as the expected marker for the duration of the previous block. With file on disk (no streaming), it is easy, BlockDuration in last subtitle block is used for setting the end of the subtitle track (step 2 from above and there is no step 3), easy, but in the case of streaming there is no solution if I understand correctly, and a BlockDuration of 0 could be a method for saying that this is the end of the previous block. In other words, a BlockDuration of 0 is a method for saying "when there is a break in a track like for subtitle tracks" (in description of BlockDuration) when we do streaming without knowing in advance the duration of the previous block. I think it should not be marked as an error in mkvalidator. I guess that a workaround is to have an empty subtitle block and to remove the BlockDuration when the goal is to clear the previous subtitle, just in the case of live streaming the meaning of the block is not really clear between "show black" and "this is a break in the track" as it is for differed time (with BlockDuration of the last frame). Not big impact, just some semantic in the case of live streaming. From nobody Fri Nov 17 00:25:23 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AECEF12940B for ; Fri, 17 Nov 2017 00:25:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4ygMIGO0Z-t3 for ; Fri, 17 Nov 2017 00:25:20 -0800 (PST) Received: from mail-pg0-x22f.google.com (mail-pg0-x22f.google.com [IPv6:2607:f8b0:400e:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 55B7F126B6D for ; Fri, 17 Nov 2017 00:25:20 -0800 (PST) Received: by mail-pg0-x22f.google.com with SMTP id t10so1448752pgo.3 for ; Fri, 17 Nov 2017 00:25:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=5KeDbhoNntozCDyc8J6IpSu29hFosuinDot+uTBQ7Nk=; b=Gz474Uvi18tw2sRiKO1rqKFnCRgjbxsMooWaDigsGdEDyrOv4ZKJ1xRdJovtlMRG5D Jqaq4dDayx7y/CyKTIx3wo7akGS8I/UbWxP2TbBtLE3U+l/j+mifsuA2RXhu5bvZv1li 5AjCYeXxjbT31ejPrZCluWfsaAHh0ImXn088NHJW0waO7NaGiSIH1KC77BolPAxzRc79 BaElMb3+C4nNWy24So4ZMec8VwVIdntuYb+L1PleEA8KgBBDmOOcQUqEHtmAmJOe8UYQ 8AoN6w+RN9XCB1uc++IuOHaH1R/kZ+pBY/SPgUUYCll2xlZqeqZnXsZJGcJhO4FBz0fG qypQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=5KeDbhoNntozCDyc8J6IpSu29hFosuinDot+uTBQ7Nk=; b=SD8QZWrSfM4vlKv4DMgMLderPY7x+kfAaVwviBeAoTAgHxB+kAmmNpx9sigRqtK1xS zRs5K3Y1MvjQ8WZo09h92wbMkUZZl5RZmXeZlaEtJ05KK7wadbrGlV+nRwGtGBXvuPiP rxiULFlTFhvpUgN+tpCezAn8BzmuLUjG9qSsQNZEVPVY6HaWnh1P1kIGcV5mEbQrXpmy gltm89MUuJSgJOdc/TpsCb02jtPmCBehwAhXTHY3FwSFl5RShCqanFDD9bj6a6l01MOp m6PEgxzYoy9agD/8RKTzp5zPeLL0Z1FJNiX0ccY5wj1TbsyGiODlB4sEOn2ZmfV+tSWV Vm4g== X-Gm-Message-State: AJaThX6txiYNlpTPfqqff1m64fRNbRPbacyVdL+n97lfALqkM2rqzTG2 wh69ZfQ+YMTkT+Hc6raK+oeVLT/IXjWvnlOeQkHUpQ== X-Google-Smtp-Source: AGs4zMaNNMABDPTcFGUpbX9T0JcLLaVVx22pm+cXJBVVzRiU/npAN7cYeZhe1vVFUGLhTXBmTaz5IaDNYq4hEyQ5iGA= X-Received: by 10.98.141.150 with SMTP id p22mr1221496pfk.207.1510907119452; Fri, 17 Nov 2017 00:25:19 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.161.7 with HTTP; Fri, 17 Nov 2017 00:25:18 -0800 (PST) In-Reply-To: <59098280-b8b8-b967-5c63-251ed830e5ec@mediaarea.net> References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> <59098280-b8b8-b967-5c63-251ed830e5ec@mediaarea.net> From: Steve Lhomme Date: Fri, 17 Nov 2017 09:25:18 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2017 08:25:23 -0000 2017-11-16 11:23 GMT+01:00 Jerome Martinez : > On 16/11/2017 10:52, Steve Lhomme wrote: >> >> Not sure what needs to be fixed in mkvalidator. A 0 BlockDuration >> doesn't seem right. In fact I wonder if we should just set in the >> specs that the field MUST NOT be 0. It has a default value (of >> DefaultDuration) which cannot be 0 too. >> >> Is there a case where it would make sense to set 0 ? > > > In the case there is nothing to show, e.g. a previous subtitle element which > should disappear. OK but "should disappear" it's too vague. How is it signaled in the original content ? This signaling defines how it should be handled in Matroska. If there's an actual element that says the result rendering of the previous element should stop then we need it in Matroska. If there's a duration, then it needs to be set. >> >>> It makes sense, as you can not know in advance how long is the subtitle >>> (it >>> is the case for PGS in the example, it would be same for e.g. 608/708 >>> closed >>> captions real time transcoding to UTF-8 Plain Text), so in the case of >>> streaming you can not wait up to know the duration of the block before >>> writing/sending the subtitle block. >> >> If it's streaming then the issue already exists in the original >> stream, you can't tell the duration until you receive the next one, if >> I understand correctly. It seems you don't need to set the duration in >> Matroska for such a case. The next "frame" will erase the previous >> one. > > > The next frame is too far away. and has content. > Here, the issue is that the content is not linear, you have "holes" between > frames. But if the original content has "holes" it needs to be preserved that way. It may be intentional, it may be a bug in the source material but how do you decide how it should be handled and where it should stop ? You do not have that information in the first place. > One could provide an empty subtitle with no duration, but it is not exactly > same as "no data" (e.g. black frame is a frame content, it is not the lack > of frame). > > IIUC, a duration of 0 is for saying "no more content starting now" in the > use case. How to say that if mkvalidator says it is invalid? > > An example: > 1/ several frames with BlockDuration having the right value, content is > correctly displayed. > 2/ Last subtitle is displayed, it is file streaming and we don't know at > this moment the duration, so BlockDuration is not filled (="value is assumed > to be the difference between the timestamp of this Block and the timestamp > of the next Block") > 3/ At some point, we know that the subtitle stream should stop to be > displayed (before the full duration indicated in the header). End of this > stream. an empty subtitle with duration of 0 in order to say "this is the > end of the stream", as the expected marker for the duration of the previous > block. I don't think this is a good solution. On the muxer side you'd need to know beforehand this is the last element of the stream. And as you mentioned this may not even be an option when streaming. Also if that element is damaged you'd end up with a stream that has subtitles that never end ? I agree we need to define how end of stream is handled for subtitles. It's already done (maybe not in the specs) for video. The duration should be set on the last video frame. The same thing should be done for subtitles. But as seen above, that rule doesn't work if the last element is damaged or the file truncated. So we should rather define how to interpret data in case the duration is not known. Right now it's a best effort approach. Most player will just use the audio as basis and end playback with audio. > With file on disk (no streaming), it is easy, BlockDuration in last subtitle > block is used for setting the end of the subtitle track (step 2 from above > and there is no step 3), easy, but in the case of streaming there is no > solution if I understand correctly, and a BlockDuration of 0 could be a > method for saying that this is the end of the previous block. > > In other words, a BlockDuration of 0 is a method for saying "when there is a > break in a track like for subtitle tracks" (in description of BlockDuration) > when we do streaming without knowing in advance the duration of the previous > block. I think it should not be marked as an error in mkvalidator. > > I guess that a workaround is to have an empty subtitle block and to remove > the BlockDuration when the goal is to clear the previous subtitle, just in > the case of live streaming the meaning of the block is not really clear > between "show black" and "this is a break in the track" as it is for > differed time (with BlockDuration of the last frame). Not big impact, just > some semantic in the case of live streaming. > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Fri Nov 17 00:42:42 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 310FC1293F9 for ; Fri, 17 Nov 2017 00:42:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.62 X-Spam-Level: X-Spam-Status: No, score=-2.62 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cZEamLs8XmNR for ; Fri, 17 Nov 2017 00:42:38 -0800 (PST) Received: from 16.mo4.mail-out.ovh.net (16.mo4.mail-out.ovh.net [188.165.55.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 438871200CF for ; Fri, 17 Nov 2017 00:42:37 -0800 (PST) Received: from player159.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo4.mail-out.ovh.net (Postfix) with ESMTP id D18C0FBEE9 for ; Fri, 17 Nov 2017 09:42:35 +0100 (CET) Received: from [192.168.2.120] (p5DDB6742.dip0.t-ipconnect.de [93.219.103.66]) (Authenticated sender: jerome@mediaarea.net) by player159.ha.ovh.net (Postfix) with ESMTPSA id 767B348008B for ; Fri, 17 Nov 2017 09:42:35 +0100 (CET) To: cellar@ietf.org References: <52184945-6143-9ced-adb5-2a5a6e07eff8@mediaarea.net> <59098280-b8b8-b967-5c63-251ed830e5ec@mediaarea.net> From: Jerome Martinez Message-ID: <36054d92-1473-c83b-d265-f47c6a42693c@mediaarea.net> Date: Fri, 17 Nov 2017 09:42:36 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Ovh-Tracer-Id: 11599865266056073361 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrjeeigdduvdefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecufedttdenuc Archived-At: Subject: Re: [Cellar] Matroska BlockDuration X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2017 08:42:41 -0000 On 17/11/2017 09:25, Steve Lhomme wrote: > 2017-11-16 11:23 GMT+01:00 Jerome Martinez : >> On 16/11/2017 10:52, Steve Lhomme wrote: >>> Not sure what needs to be fixed in mkvalidator. A 0 BlockDuration >>> doesn't seem right. In fact I wonder if we should just set in the >>> specs that the field MUST NOT be 0. It has a default value (of >>> DefaultDuration) which cannot be 0 too. >>> >>> Is there a case where it would make sense to set 0 ? >> >> In the case there is nothing to show, e.g. a previous subtitle element which >> should disappear. > OK but "should disappear" it's too vague. How is it signaled in the > original content ? For what I know (608/708), it is a "clear content" command if there is a still a stream (bytes are provided), or also the stream stops to be provided (i.e. 608 during content, no 608 bytes at all during ads, then 608 again during content) I understand that it could be considered as "this is a new frame with empty content so just send a black frame in your transcoding with no BlockDuration". What is weird IMO is that you can indicate that the stream stops to be provided when you know it in advance (before the last frame so you can fill BlockDuration) but you can't indicate the exact same meaning when you don't know it in advance. IMO it is incoherency in the specs, with a "feature" available only in some cases (as Matroska is, if I understand well, aslo intended for streaming). "vague" or not is not the topic, as this issue is already there when you set BlockDuration. > [...] > > > I agree we need to define how end of stream is handled for subtitles. > It's already done (maybe not in the specs) for video. The duration > should be set on the last video frame. The same thing should be done > for subtitles. I don't think that there is any difference between video and subtitles (and audio). In all case it is easy if you know in advance (before writing the block) when it ends (you set BlockDuration), else it is not possible. So I don't think that it is already done for video, it is the same issue (but no practical use case) for streaming. *** Back to the practical issue: do you mean that in this case (hole in subtitles during file streaming), in order to have mkvalidator not complaining, the implementation should just put an empty subtitle with no BlockDuration, without doing any semantic difference between empty/black/silence content and hole in the stream? Jérôme From nobody Sat Nov 18 05:00:35 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C3660126DEE for ; Sat, 18 Nov 2017 05:00:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LF1QBdhood03 for ; Sat, 18 Nov 2017 05:00:32 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 594991200E5 for ; Sat, 18 Nov 2017 05:00:32 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:35690) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eG2jL-00061X-1o for cellar@ietf.org; Sat, 18 Nov 2017 14:00:19 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 332256540F49; Sat, 18 Nov 2017 14:00:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1511010018; bh=++yDZcH9rI+5niYQS1mwmtSnkYNeMrWhYKMEYECq5GY=; h=From:To:Subject:Date:From; b=qZf8FhiHK0lnyiPa4k8MVzNU7JUYfgJT5AasMNjWjQKF5YWdSkzMNpaM+nVU2lw1v TEnyRn1d8F3SlLHsIG0FltLUHx0Tzd8qh5znGbyTb/yKr0RfqWGrs7VbEzlGrKORqZ mQlJ7Lt7aGnkNEkimOxvv1d/iahiAJ0s+yO2jAqDqfIun5wvggTgPxcnlJod/UjJCH Xg/ajG+njKBTGt/+5GS0ZGUejRGRrXzMybnH1x/+Wl9y6m8hNkznNxF34GnByg//HO w4DJIYSbZxBCCa7OFnZ6t95PteJwMBbvQEyxrwjNrIL4/9E7Qoj41XPiBNCPTrr81c PJTsqOYu6lwJg33TqRorUkmITp/l0+uxL8U972wjTmrnNsku9AGmqloCQpCbSHanzC 4bRRjLUKARWkGYk29RpPTbLsfKjoQ+I3UjUjroEbf9zSBUtuylaS4pb0wCA2KI6iFP fmpFbPjosdozrxDrYuGI1D/yJeiLDRU/2W4/jsCR0vZPX2QevwOHq0I1YBOhmFxo2M HHBds2OZValzWaaKknkyOTi864RNdw24Mnt43RE91z6LAMgkQVktPp+Cbvg5WktzeI z6gtiGl5oKdBNeqHD6bgmjymO+GIASU9Eevni/XMgwTvzkimXugJYf03Drhv0QdFU3 8hVMMRoF5W+j9Ce8+PAc/ANw= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTPS id B3577261BE51; Sat, 18 Nov 2017 14:00:00 +0100 (CET) User-agent: mu4e 0.9.18; emacs 25.3.1 From: Moritz Bunkus To: Cellar list , help Questions Date: Sat, 18 Nov 2017 13:59:53 +0100 Message-ID: <87vai74u9i.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: [Cellar] MKVToolNix v18.0.0 releasedBcc: dev-zero@gentoo.org, "Tim Harder" , "Lubomor Sedlacik" , "Christian Morales Vega" , "George Vlahavas" X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Nov 2017 13:00:34 -0000 Hey people, Welcome to release v18.0.0 of MKVToolNix. This is just a smallish bug fix release which also contains a couple of performance improvements. There were no changes for package maintainers. Here are the usual links: …to the source code: https://mkvtoolnix.download/source.html …to the binaries: https://mkvtoolnix.download/downloads.html The Windows and macOS binaries are available already. The Linux binaries are stil being built and will be available of the course of the next couple of hours. Here are the NEWS since the previous release: ------------------------------------------------------------ # Version 18.0.0 "Apricity" 2017-11-18 ## New features and enhancements * build system: when building with clang v3.8.0 or newer, `configure` will no longer restrict optimization flags to `-O1` and use `-O3` again (older versions of clang suffered from excessive memory usage with higher optimization levels). * build system: when building with mingw 7.2.0 or newer, `configure` will no longer restrict optimization flags to `-O2` and use `-O3` again (older versions of mingw suffered from bugs such as segmentation faults with higher optimization levels). * build system: stack protection is enabled when building with clang 3.5.0 or newer on all platforms. * mkvmerge: AVC & HEVC ES parsers: performance improvements by copying much less memory around. * mkvmerge: tags: reintroduced a workaround for non-compliant files with tags that do not contain the mandatory `SimpleTag` element. This workaround was removed during code refactoring in release v15.0.0. * GUI: multiplexer: the "AAC is SBR/HE-AAC/AAC+" checkbox in the "audio properties" section will be disabled if the functionality is not implemented for the selected track's codec & container. * GUI: multiplexer: the "reduce to core" checkbox in the "audio properties" section will be disabled if the functionality is not implemented for the selected track's codec. See #2134. ## Bug fixes * mkvmerge: AAC ADTS parser: fixed interpretation of the `channel_configuration` header element for ADTS files that do not contain a program configuration element: value 7 means 7.1 channels. Fixes #2151. * mkvmerge: Matroska identification: the `date_local` and `date_utc` attributes will only be output if the identified Matroska file actually contains the "date" header field. * mkvmerge: WebVTT: mkvmerge did not recognize timestamp lines if the hours components were absent. Fixes #2139. * mkvpropedit, GUI's header editor: the `date` header field won't be added automatically anymore whenever the segment info section is edited and the `date` element is either deleted or not present in the first place. Fixes #2143. ------------------------------------------------------------ Have fun :) mosu From nobody Sat Nov 18 05:07:24 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 270B6126CBF for ; Sat, 18 Nov 2017 05:07:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (4096-bit key) header.d=bunkus.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wSUxW1OGAzZJ for ; Sat, 18 Nov 2017 05:07:22 -0800 (PST) Received: from adara.bunkus.org (adara.bunkus.org [144.76.6.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 378591200E5 for ; Sat, 18 Nov 2017 05:07:22 -0800 (PST) Received: from liselle.bunkus.org ([2a01:4f8:190:8147::105:1]:36584) by adara.bunkus.org with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eG2q2-0006A1-1y; Sat, 18 Nov 2017 14:07:14 +0100 Received: from sweet-chili.local (unknown [192.168.191.4]) by liselle.bunkus.org (Postfix) with ESMTPS id 270D06540F49; Sat, 18 Nov 2017 14:07:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bunkus.org; s=mail2017070101; t=1511010434; bh=tQv4R3dqdNvD7RZ1bBBYsn8+7Pl2AsBYZF16YjsQBNc=; h=References:From:To:Subject:In-reply-to:Date:From; b=XVuQFQI7Uir0Hk8mPa3emad8sYd18gCHlADwnfik8lW+FzQjm3u0VnyY5M0Mk/nCH rWDsE1ESGLE4WJYsNcOv1KBKpZXXETP/5n/SYKQJkPnZ4fZDfCvaWhNBRvD6Ceyj60 h7oMD4rpht53tBj98bV4I09dDoVh5C0rtDi5JFOMOW/mREjFTtmaP0kVfrulq9p5f0 xxKe2SSgNz59I80tKrhNCPse/byDoF/D+IgnFbRCVwagMpFgzCv1+eOwJSezmUlNtC XIoq0lGxrAb2ootEvbW+5PPiEfky8zkFDPt3KH/1r2CmWvZKnyWxJJcBBRMXsySPK6 RbPiXlfgVydUme+5qdl52mYe1b3X9+ZEGbZ7mJ1s4B7tg5KbdfKH4aspx2D+YCovm8 M7gfq+FyE5vnUUu1mF7IwSh9jRpdlPM4btB2p18dsXWdCj2E9O5hHPQw1S3QL+nTvx EGS1Xo5suX6LHKvvtOHLORvcpO0uD60zqDUz7ImwylKroaqABS9GPibzJSZrNC1t26 ZWUs18w6GhS02XzfgRKzwCEG3xkW4xDN0Km4K6Q46hXowJeKoZ6/Sb3cgDYGsGqacC KMI/GxDrejAkS5Fxl//K8M1RH5nSHLodv+emmanicHm+GDFIvhHXyHgtCi5WsPwwxu 1sgmtbtWx75PUye0Fyrxg8+E= Received: from sweet-chili (localhost [IPv6:::1]) by sweet-chili.local (Postfix) with ESMTPS id 29CC4261BF96; Sat, 18 Nov 2017 14:07:13 +0100 (CET) References: <87vai74u9i.fsf@bunkus.org> User-agent: mu4e 0.9.18; emacs 25.3.1 From: Moritz Bunkus To: Cellar list , help Questions In-reply-to: <87vai74u9i.fsf@bunkus.org> Date: Sat, 18 Nov 2017 14:06:41 +0100 Message-ID: <87tvxr4ty6.fsf@bunkus.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [Cellar] MKVToolNix v18.0.0 released X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Nov 2017 13:07:23 -0000 Hey, …and I fail at email, obviously. My profound apology; I sincerely hope never to do that again. Kind regards, mosu From nobody Sun Nov 19 04:44:05 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D1544127077 for ; Sun, 19 Nov 2017 04:44:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MrFfq2n8V9K7 for ; Sun, 19 Nov 2017 04:44:02 -0800 (PST) Received: from mail-pf0-x230.google.com (mail-pf0-x230.google.com [IPv6:2607:f8b0:400e:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 427CE126E01 for ; Sun, 19 Nov 2017 04:44:02 -0800 (PST) Received: by mail-pf0-x230.google.com with SMTP id u70so5276757pfa.7 for ; Sun, 19 Nov 2017 04:44:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=JBUt8aeHeKgQOtO0Tjd8ZBwMqCfZN+dRw8FSK9hsH5M=; b=UV+xDtLqHiUtu+zqse2tNvFkIWdnrdhVtm9tTz5nxHvjvZ9D8YxmxF3F1UseeWQRpQ lLGuhFQ/RNCO9gbfJT+3Kze1TOsk6yJ0TMAChjIMvMZirBy7LTHs5CvvRhd32VEVwMM4 scGVzD8liEFrAaoHRELP+6RiQtA0DfTy0f7XQHkLKCHMiQYk+XJrCM8G5fE2UxZ65cV6 NSbbV2EgmWMAWtsefTRBL1qSvulN9bvYZNrYzQg7ndOdOgM+QZc+xKiSx8M35lGh99kF qJXnJSyv7WWQWytGddntzoLbqJ4S6QsMbRd2g8/VqyFLQN1/1MyfcK9zbXAHZysX9ljg 4BUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=JBUt8aeHeKgQOtO0Tjd8ZBwMqCfZN+dRw8FSK9hsH5M=; b=gtkLHlw+wNEEATpgUnFS+ZmZskOq5b75LQw0B0qvV0TLi2t1PEW32i8+CUCFWdpz14 Gu+4fWRR3bV6MEf5iiyn/gXMttSUiegCEZnCK98mh8AVs/t9+IMaLAzXaXKy0OfVDnPs DJEP6Vcxpbz053LBbiocpmUTI1k01kZL4kb30q4OUCxHOZRE2Fme2oY3eom6HzEd20nc RQSIjV0SZlz/0di/XdQB1w02qIBdynk2ufvSNsSWrql+ZtmrhVFuCGueCoecX8/24fxv hCbSVcm8JRak3LAcnEg+b3wiRR1WJiOd3soaHo4q+kp8RE8PMPbiH5shU1/EfNXYjd1A nZfw== X-Gm-Message-State: AJaThX5LDUVmk9982fww1QJk8divcDCXVvKN2gHpqfh1qhQJPhN2gJ4X QwO/oVv0vLEMfrcawCTlI1iewi1rk9i934XW2wWHSLfD X-Google-Smtp-Source: AGs4zMadTflUIn8RabMNa8aLJHmf6rBbXq3i/+We+UYGvvyILrUL1a5xTc7GoBSira5SF4JJ6YiHkhzE78XtdkeY5nU= X-Received: by 10.101.69.2 with SMTP id n2mr10490496pgq.79.1511095441553; Sun, 19 Nov 2017 04:44:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 19 Nov 2017 04:44:01 -0800 (PST) In-Reply-To: References: <82cca552-117c-fa17-2dae-816763c253ee@mediaarea.net> <87po8f4p3n.fsf@bunkus.org> From: Steve Lhomme Date: Sun, 19 Nov 2017 13:44:01 +0100 Message-ID: To: Discussion about the current and future development of Matroska , Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] [Matroska-devel] Code Of Conduct X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Nov 2017 12:44:04 -0000 All 4 PR's updated to use the same text to tell text has been added. And all PR's merged. So now the Code Of Conduct will be enforced on all our projects. 2017-11-18 15:59 GMT+01:00 Steve Lhomme : > All 4 PR's updated. > > 2017-11-18 15:54 GMT+01:00 Steve Lhomme : >> Yes I will update the PR's. I originally misread them, that if you >> violate some CoC elsewhere you may not participate here. Adding lines >> to the original may "taint" the text and confuse people (or they may >> not notice). So I'll try to make this clear. >> >> 2017-11-18 15:51 GMT+01:00 Moritz Bunkus : >>> Hey, >>> >>> great! >>> >>> I just noticed, though, that you didn't include the extension to the >>> "Scope" section that Jerome talked about. What I mean is the following >>> additional sentence: >>> >>>> In addition, violations of this code outside these spaces may affect a >>>> person's ability to participate within them. >>> >>> I was under the impression you were OK with that? >>> >>> PS: I'm taking this as an opportunity to add the same CoC (including the >>> wider "Scope" section) to MKVToolNix, too. >>> >>> Kind regards, >>> mosu >>> _______________________________________________ >>> Matroska-devel mailing list >>> Matroska-devel@lists.matroska.org >>> https://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel >>> Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel >> >> >> >> -- >> Steve Lhomme >> Matroska association Chairman > > > > -- > Steve Lhomme > Matroska association Chairman -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 19 07:47:45 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 836C21200C5 for ; Sun, 19 Nov 2017 07:47:44 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.121 X-Spam-Level: X-Spam-Status: No, score=-1.121 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m2000E1PEyQ2 for ; Sun, 19 Nov 2017 07:47:43 -0800 (PST) Received: from server172-2.web-hosting.com (server172-2.web-hosting.com [68.65.122.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8CF0E126DEE for ; Sun, 19 Nov 2017 07:47:42 -0800 (PST) Received: from cpe-104-162-94-162.nyc.res.rr.com ([104.162.94.162]:36073 helo=[10.0.1.9]) by server172.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1eGRom-002FNh-1q for cellar@ietf.org; Sun, 19 Nov 2017 10:47:41 -0500 From: Dave Rice Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Message-Id: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> Date: Sun, 19 Nov 2017 10:47:32 -0500 To: Codec Encoding for LossLess Archiving and Realtime transmission X-Mailer: Apple Mail (2.3445.4.7) X-OutGoing-Spam-Status: No, score=-2.9 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server172.web-hosting.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - dericed.com X-Get-Message-Sender-Via: server172.web-hosting.com: authenticated_id: dave@dericed.com X-Authenticated-Sender: server172.web-hosting.com: dave@dericed.com X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched Archived-At: Subject: [Cellar] status of FFV1 and EBML drafts X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Nov 2017 15:47:44 -0000 Hi cellar, At the IETF99 meeting in the minutes at = https://datatracker.ietf.org/doc/minutes-99-cellar/ it discusses having = a WG last call for FFV1 and EBML but also the need for more reviews of = these documents. Is one issue blocking another? Meaning: should the = group announce a last call in order to encourage more reviews or are = more reviews needed in order to have a last call? Best Regards, Dave Rice From nobody Sun Nov 26 01:07:07 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 154E41293D9 for ; Sun, 26 Nov 2017 01:07:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ngs5huEPeffC for ; Sun, 26 Nov 2017 01:07:04 -0800 (PST) Received: from mail-pl0-x22a.google.com (mail-pl0-x22a.google.com [IPv6:2607:f8b0:400e:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B3B64120725 for ; Sun, 26 Nov 2017 01:07:04 -0800 (PST) Received: by mail-pl0-x22a.google.com with SMTP id l16so6702431pli.6 for ; Sun, 26 Nov 2017 01:07:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=THl5U29sAU8MwXDHCWxZV1TZj4EzfbquyJNsBmaXIto=; b=erCCXOd35zBFgrLjs5n/Mr1/a4LtC0de0T51hZJ9nqeOwAprtQMdvtFSNCd4YncCvx nEeiVrCnW7cwfwpJ1E3af6KgOtJcbFd8f73P+mQsIjw7XVvBg0A8Hz+wCLV/N62Ce7C1 CaM9RQDHEswnXzL0PHY0TO+4kYLKl7owOUjybRkTgXMpfK6CxLJhFigPOTzqQB/Ehqq7 2dWrhM5Og8gf0e+SEPqaXwhGnuN5ZOQ9IAWXMkwvnn7IxjBYfzwmIUgFeqHlOdULX7J0 DVlX1r2spsgk9YwGdyT6O1egMM0GxkXJ9g7eE5Tqxe5qyDCLsY+zaspWG0jPOOMjxFc5 ZONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=THl5U29sAU8MwXDHCWxZV1TZj4EzfbquyJNsBmaXIto=; b=IfCj9GcW0i9kJ1/TYJAE37Fgo22N+lZsGcP0+IDMPCq0O4+8rBlkOkf7RRA9gwtAEZ LtLGM+3mn7/VuktPzj1hKuBxgjALL3VlajdXmCIg/Sjf/89xr0AFYfqnWGl6r0EeSVLQ Fobiunv+3J5rCJr3pVYLGVfM0SfQx8xL2ufF7+Axr9ZLEyeCtsrUEpIsnQWa4mk5Dzi7 CLpL1wkZ6C4B+Uz69rubM4LWoPvnZD5mRAcmmsBEg2shGRUFGjgczLccr9mroKbUauby +u5qvfHCIWi0WMpBiXiheoBuMmB1lvgE0vbJocH//sFbNmvg5qhWhDppI1u8j4AQkx6U qp7w== X-Gm-Message-State: AJaThX4FR/07AP4Vh2K4JxOelCcsn1INII4SRIm6bM6qk6zIH/soSK0V nJaWxbaZVJg3IHd4crSDKkCgJelgqjnUFjtb7bUGtR/H X-Google-Smtp-Source: AGs4zMb0whHXj3hDa64zsf+C2uYadOUOD9feL6uYRvOAV2IcpO99gauGaJTS57YqJ1oxXEBcefLIjH30HSpYnjPJ0wg= X-Received: by 10.159.230.13 with SMTP id u13mr15751658plq.226.1511687224004; Sun, 26 Nov 2017 01:07:04 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 26 Nov 2017 01:07:03 -0800 (PST) In-Reply-To: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> References: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> From: Steve Lhomme Date: Sun, 26 Nov 2017 10:07:03 +0100 Message-ID: To: Dave Rice Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Cellar] status of FFV1 and EBML drafts X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 09:07:06 -0000 Hi, I won't comment on FFV1, although I should probably read it to see if there's anything I don't understand. For EBML I think there are still things to be done: - clarify how to handle bogus data https://github.com/Matroska-Org/ebml-specification/issues/48 - kind of relates to junk in Master Elements https://github.com/Matroska-Org/ebml-specification/issues/92 In the context of long term storage it's probably very important to solve these issues. - DocType 0 as experimental/internal. We need to agree on this and then wor= d it https://github.com/Matroska-Org/ebml-specification/issues/157 After that I think we're good to got for EBML. There's currently no pending pull request. 2017-11-19 16:47 GMT+01:00 Dave Rice : > Hi cellar, > > At the IETF99 meeting in the minutes at https://datatracker.ietf.org/doc/= minutes-99-cellar/ it discusses having a WG last call for FFV1 and EBML but= also the need for more reviews of these documents. Is one issue blocking a= nother? Meaning: should the group announce a last call in order to encourag= e more reviews or are more reviews needed in order to have a last call? > > Best Regards, > Dave Rice > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar --=20 Steve Lhomme Matroska association Chairman From nobody Sun Nov 26 01:17:27 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 07D6C120725 for ; Sun, 26 Nov 2017 01:17:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sYnuOa3VWS0P for ; Sun, 26 Nov 2017 01:17:25 -0800 (PST) Received: from mail-pg0-x22b.google.com (mail-pg0-x22b.google.com [IPv6:2607:f8b0:400e:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 98378120721 for ; Sun, 26 Nov 2017 01:17:25 -0800 (PST) Received: by mail-pg0-x22b.google.com with SMTP id q7so4798419pgr.8 for ; Sun, 26 Nov 2017 01:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=hHCRsJfpVMUiKEuyF/Gbng9GFiqeZ7wZmMRirjQpJGU=; b=HNhxcgqupLL85IkSWB4JxRAH+H67xOpoRPPbaoBFzterDwL3LVBtfv+1lFW3i5PGKl P8HbVQjW6SF/9np7JFtzt1DFx4DS38Xzg06PpSgCYJmUGSwxP40nAhMzkcQ2SE+zCXs3 36nNR0pyCACsKQLh7/oD94BmnMQgcXYnYyuB9STdPQGGYjs0vWnuDbWjXLaJrg3pxO+h 8zSAg9duM/uX8ehR25QpQ6n/AELDrXcLYAqehYb7O4uGMBHVif03N3ykAlWWas508Uu0 DeyNruDYue/wURE8wi99W4DS4UFDHqXGmrSBGVM3CqXy4gx7C8TleKw7rSLm8ZFL1vVa +XXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hHCRsJfpVMUiKEuyF/Gbng9GFiqeZ7wZmMRirjQpJGU=; b=t54ycfG9cL5nJmnolTiXXB7lWVyKPVQ59GIaDoJ2rRlL5eVFT5Ym4uYI9PuWOWSazo nWGww+tWAS/hKnPAnutYEvRFbOJD6qkKMSfU9BW+zph+3Qu7m1FvkKPINv816PN2YpjL s3vzmPJRszoF5e5XfKODgxe66fnvn6SetXJ+4ZmhtK76Q3AkmrElGT/2kWD0imfL/gra I8Dw8ECCyATBGXfuSbaylwvq5HW/tXa26c/TquH+Ak8O4kBqO2NL3RsJfqLTvyk7EpD3 6nWQgRF0XK7Sz2rINZ4hnp0TSg2OnnwrSNSLLlFooyGkUN/AYRzl+ky/dtMnGyYZh7ED WLdQ== X-Gm-Message-State: AJaThX5rrEtjyB1I5PSlS8RXYY8dathi1A5fVO5iByEpdfESB+EDTZhH JTuiW/a1T6rtRaJhRLJMdy0fAIZQwo8FfM8rASxpDPI8 X-Google-Smtp-Source: AGs4zMYJstZyB1Di7T7+XhH0XYo3oda7UZIxkFFZP5Uyr2qzlZNTkJkrC6nPqruYzN+j5H1UuA/CI2JnDeXaQyRESXQ= X-Received: by 10.101.69.2 with SMTP id n2mr33182277pgq.79.1511687844993; Sun, 26 Nov 2017 01:17:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 26 Nov 2017 01:17:24 -0800 (PST) From: Steve Lhomme Date: Sun, 26 Nov 2017 10:17:24 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 09:17:27 -0000 Hi, We currently have a pending issue about how to define an experimental version of a format. https://github.com/Matroska-Org/ebml-specification/issues/157 We could either use DocType 0 to mark the version is in experimental phase. Or we could add an element in the EBML Header that states the status of the elements used. I think the latter offers more flexibility. We could define that the format in use in fully compliant with the version(s) set, that it adds elements. In that case extra elements might be listed in the header. That would be helpful to detect errors in the file since you know the element IDs. But it might be complex. Since it would need the Path of the element, its ID and the type. It would just mean we embed a DTD in the EBML header (but just skipping the default/known elements). IMO it is blocking the finalization of the EBML specifications, even though it's an addition, because it will be key on how EBML formats can evolve and still be compliant. If we want to go the short way we might just add an element that we know could be extended to a full DTD later. So a Master and a child flag (standard/experimental/other?) -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 26 07:13:08 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 382F912711A for ; Sun, 26 Nov 2017 07:13:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0C_ApyImliXP for ; Sun, 26 Nov 2017 07:13:04 -0800 (PST) Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D8054126CF6 for ; Sun, 26 Nov 2017 07:13:04 -0800 (PST) Received: by mail-pf0-x232.google.com with SMTP id j28so16261908pfk.8 for ; Sun, 26 Nov 2017 07:13:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=j3vKzFccKKOb8WtSVs23A0b75CDgruQ09+QgftwiVCM=; b=yiI5uJ55bu2eNKoIyzXScvQji9maExMmPGTrKTnw+Ub/7fgiVHvHyKe9f96yWiprtW a1xfp/q9/4TXSItfnOfr319NUnssAMMS0u3R7AUdRIqTl9d3FvOVN1VsLQlDQMxG9q1h eK8j5Cwr6xCJ7zeLpE/5d+Mfrtz14P+dSlS9OkVxjoQaz93vu7iFbqYqatgQMdKskti+ II1iJW9dF9MuNJ439whSHZ2ORAsDjLC5lgO6XSXLEVdlAK6PAYnSXBlgradQ8NIPfLHJ i124bd4MLtUG+XLrCG9kp6dxCE+WOVFeYIceK07k4h53co+Qo6kVivFAMRNF1N4QS3BO xISA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=j3vKzFccKKOb8WtSVs23A0b75CDgruQ09+QgftwiVCM=; b=KmGRlh6SdkQh4qMQTa+nFydvwqcPrHtRZoTTvAxGMEb6123yx6zAQO4/ALBujork4X QP+5ovD8SeZtxmF+sMLp+K4pi7vzI40oHet7iI5RdtHo+eFlpE3gwHsW9bRmrL7m31J6 beyoCfwPJK5npEJ/5CEzCjz6z4f1xDDw73PYri0itT1X1+yO2/h9OKHsE8Wh5GVAvvnz Gd4kc++rUagj+g7j4U9uzq9ifNExt5Opp2fk7UPTLc+nVHc5EIBMmdYXLGmMCok36j9v gnWUcWODZJ6XmD0pf45qdeJi7gRrS055NxTZnVilv37Yqr4Tk/+g1zWtDpWQr0VY00nR Wg9A== X-Gm-Message-State: AJaThX5KrxESBz0LlVNHDCf0ruWlLy7c0IrJ1bVnmDFY1m62CNoTPn7f c5jMK+PwMSokKGUZX1l/R0XvFPa/SyBw1fyWX1fHAHOR X-Google-Smtp-Source: AGs4zMYgmmPkfWeGgxCcK5Or+jkfoliV8XUHVDYYsHOH3/cwTtYCVfgfOzl5nQtigpNODlPC/Q0ShNgUVyFwEn9op4E= X-Received: by 10.98.141.150 with SMTP id p22mr33271473pfk.207.1511709183801; Sun, 26 Nov 2017 07:13:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 26 Nov 2017 07:13:03 -0800 (PST) From: Steve Lhomme Date: Sun, 26 Nov 2017 16:13:03 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: [Cellar] EBML Error Handling X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 15:13:06 -0000 Hi, The last remaining issues to fix before we can issue the last call on the EBML specs is more text to explain how error handling is done. It covers many possible cases. Here are the cases I can think of (some mentioned here [1] and here [2]): - multiple unique elements - CRC element misplaced (it should always be first) - ClusterTimecode misplaced (it should always be first or after the CRC) - CRC mismatch with the content - invalid EBML ID - invalid EBML length - Master element with junk data (most likely unidentified or bogus IDs) - reserved EBML ID values - elements not found anymore or not yet for the given DocType - IDs with the inefficient size - huge EBML Master size that can't hold in memory - Elements with no default value with a size 0 - EBML element not in the right place according to the semantic There are all reading/parsing errors. Based on what the reader is supposed to do will also define the possibilities to create safe files for a writer (see [2]). I will reply to this email with what libebml does and how it was originally designed to work. I think it's better we agree on the rules for each case before doing patches for the specs. [1] https://github.com/Matroska-Org/ebml-specification/issues/48 [2] https://github.com/Matroska-Org/ebml-specification/issues/92 -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 26 07:14:53 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 98E8112711A for ; Sun, 26 Nov 2017 07:14:52 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R9w9hupfLiaN for ; Sun, 26 Nov 2017 07:14:50 -0800 (PST) Received: from smtp.gentoo.org (mail.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B1572126CF6 for ; Sun, 26 Nov 2017 07:14:50 -0800 (PST) Received: from eris.local (dynamic-adsl-84-220-90-213.clienti.tiscali.it [84.220.90.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: lu_zero) by smtp.gentoo.org (Postfix) with ESMTPSA id AF85033BF24 for ; Sun, 26 Nov 2017 15:14:49 +0000 (UTC) To: cellar@ietf.org References: From: Luca Barbato Message-ID: Date: Sun, 26 Nov 2017 16:15:01 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:57.0) Gecko/20100101 Thunderbird/57.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 15:14:52 -0000 On 26/11/2017 10:17, Steve Lhomme wrote: > Hi, > > We currently have a pending issue about how to define an experimental > version of a format. > https://github.com/Matroska-Org/ebml-specification/issues/157 > > We could either use DocType 0 to mark the version is in experimental > phase. Or we could add an element in the EBML Header that states the > status of the elements used. > > I think the latter offers more flexibility. We could define that the > format in use in fully compliant with the version(s) set, that it adds > elements. In that case extra elements might be listed in the header. > That would be helpful to detect errors in the file since you know the > element IDs. But it might be complex. Since it would need the Path of > the element, its ID and the type. It would just mean we embed a DTD in > the EBML header (but just skipping the default/known elements). > > IMO it is blocking the finalization of the EBML specifications, even > though it's an addition, because it will be key on how EBML formats > can evolve and still be compliant. > > If we want to go the short way we might just add an element that we > know could be extended to a full DTD later. So a Master and a child > flag (standard/experimental/other?) > Having the DTD would be nice to build programs to inspect the container or convert to the non-experimental version later. Since experimental files is pretty much an exceptional path and we all want to not have broken files disseminated around the world, probably the additional overhead might help dissuading the usual early adopters. lu From nobody Sun Nov 26 07:49:44 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7566D1272E1 for ; Sun, 26 Nov 2017 07:49:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D8MOnIhkmGlE for ; Sun, 26 Nov 2017 07:49:41 -0800 (PST) Received: from mail-qk0-x22d.google.com (mail-qk0-x22d.google.com [IPv6:2607:f8b0:400d:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 58207126CC4 for ; Sun, 26 Nov 2017 07:49:41 -0800 (PST) Received: by mail-qk0-x22d.google.com with SMTP id b85so29579679qkc.13 for ; Sun, 26 Nov 2017 07:49:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=QLKjYQeqHqTUhxCTrtCKdMw3f+ntTah0YkyHL/RknVg=; b=Ro/5tBBIcKm7VZuINfZg1KX+v7swlFWewpxF7BPfjZE6dvziL6/izD7dvIp67/yQMn kLcLJUmxVh2gs/WrpCfX154nJTgsMsV+anY9m3D0lvYHHPYpjcR12QKj4acIBUAu+Yff gTxUIp5tNdQX5BgoihlJTePjkOeGDS+hUTUWfoEp/eXwDvfkAK0X/xANV2wcP0zkArds kXzPnFJUbTS17sEjMtezv224OT7U6p6baBdh6Fsoiy5GhqF7qr4XEgske5ivEDlhiuEW YYTYAQ6ouvM4iQtRqj8lnh1XblUdcdn9ziSg+SObpivl4PY6Vz8R0oMvoUCBdF/RXmMR I1Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=QLKjYQeqHqTUhxCTrtCKdMw3f+ntTah0YkyHL/RknVg=; b=QkIO9RpH90+V3pS/Cp5SGPpQLK5IBfHlfxbRqjvpzvv/tppQAsaCP0rzJuIgSy7c3M uKMs71qq/zJitWDfDS+ew2D4gK1VcQj1SUmBAJ6NGVVQZc85W/CgsMdl4olukhpVODN6 iQxpGSJYXZpb8mm7EVsiGPGxPC11bM6qCaikc4B+PUQ4cWID8a9i+uJceLwxGTZ0uMEP jC6qUQsamPBwzqk0gyZd/0Y/MNAoto4Am3ZeGhAyAX5ZmzhFcezUnheq9O/bOEGdpsd9 7phDyqiLKwHWfzBJR1sDtisbSAYiqrVrLMIi099vHqVXSpQQBNDofSO04gJmy7iBfwoT CjHQ== X-Gm-Message-State: AJaThX7XnBRdnAjGcICxmu0eVSEmnMahc8airBiT74HXm+Qm//kvtn3Q YJIgny+tuFO4H20jaYByd//4UmBkuI8iKTuobhhr4Q== X-Google-Smtp-Source: AGs4zMZ/JRG87E61s0O9ERNRe4FakLFEUWa5JMiHeIWRQSFZa+A20e82R5EDgZAE3RaFVgGblTg07Nfgs1Hi8S8Djd0= X-Received: by 10.55.72.6 with SMTP id v6mr38541266qka.333.1511711380248; Sun, 26 Nov 2017 07:49:40 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.83.228 with HTTP; Sun, 26 Nov 2017 07:49:39 -0800 (PST) In-Reply-To: References: From: Ashley Blewer Date: Sun, 26 Nov 2017 10:49:39 -0500 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: multipart/alternative; boundary="001a114a9bc0d29af4055ee4bbb0" Archived-At: Subject: Re: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 15:49:43 -0000 --001a114a9bc0d29af4055ee4bbb0 Content-Type: text/plain; charset="UTF-8" I like Dave Rice's suggestion in the above Github Issue, following FFV1's pattern of version subtypes to set and express intention, which also leaves the wording of the element broad. On Sun, Nov 26, 2017 at 10:15 AM, Luca Barbato wrote: > On 26/11/2017 10:17, Steve Lhomme wrote: > >> Hi, >> >> We currently have a pending issue about how to define an experimental >> version of a format. >> https://github.com/Matroska-Org/ebml-specification/issues/157 >> >> We could either use DocType 0 to mark the version is in experimental >> phase. Or we could add an element in the EBML Header that states the >> status of the elements used. >> >> I think the latter offers more flexibility. We could define that the >> format in use in fully compliant with the version(s) set, that it adds >> elements. In that case extra elements might be listed in the header. >> That would be helpful to detect errors in the file since you know the >> element IDs. But it might be complex. Since it would need the Path of >> the element, its ID and the type. It would just mean we embed a DTD in >> the EBML header (but just skipping the default/known elements). >> >> IMO it is blocking the finalization of the EBML specifications, even >> though it's an addition, because it will be key on how EBML formats >> can evolve and still be compliant. >> >> If we want to go the short way we might just add an element that we >> know could be extended to a full DTD later. So a Master and a child >> flag (standard/experimental/other?) >> >> > Having the DTD would be nice to build programs to inspect the container or > convert to the non-experimental version later. > > Since experimental files is pretty much an exceptional path and we all > want to not have broken files disseminated around the world, probably the > additional overhead might help dissuading the usual early adopters. > > lu > > > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar > --001a114a9bc0d29af4055ee4bbb0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I like Dave Rice's suggestion in the above Github Issu= e, following FFV1's pattern of version subtypes to set and express inte= ntion, which also leaves the wording of the element broad.

On Sun, Nov 26, 2017 at= 10:15 AM, Luca Barbato <luca.barbato@libav.org> wrote:=
On 26/11/2017 10:17, St= eve Lhomme wrote:
Hi,

We currently have a pending issue about how to define an experimental
version of a format.
https://github.com/Matroska-Org/eb= ml-specification/issues/157

We could either use DocType 0 to mark the version is in experimental
phase. Or we could add an element in the EBML Header that states the
status of the elements used.

I think the latter offers more flexibility. We could define that the
format in use in fully compliant with the version(s) set, that it adds
elements. In that case extra elements might be listed in the header.
That would be helpful to detect errors in the file since you know the
element IDs. But it might be complex. Since it would need the Path of
the element, its ID and the type. It would just mean we embed a DTD in
the EBML header (but just skipping the default/known elements).

IMO it is blocking the finalization of the EBML specifications, even
though it's an addition, because it will be key on how EBML formats
can evolve and still be compliant.

If we want to go the short way we might just add an element that we
know could be extended to a full DTD later. So a Master and a child
flag (standard/experimental/other?)


 Having the DTD would be nice to build programs to inspect the container or = convert to the non-experimental version later.

Since experimental files is pretty much an exceptional path and we all want= to not have broken files disseminated around the world, probably the addit= ional overhead might help dissuading the usual early adopters.

lu


_______________________________________________
Cellar mailing list
Cellar@ietf.org https://www.ietf.org/mailman/listinfo/cellar

--001a114a9bc0d29af4055ee4bbb0-- From nobody Sun Nov 26 08:03:40 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1908E128854 for ; Sun, 26 Nov 2017 08:03:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.621 X-Spam-Level: X-Spam-Status: No, score=-2.621 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4I06xB_RKTri for ; Sun, 26 Nov 2017 08:03:36 -0800 (PST) Received: from 11.mo5.mail-out.ovh.net (11.mo5.mail-out.ovh.net [46.105.47.167]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 868C3127333 for ; Sun, 26 Nov 2017 08:03:36 -0800 (PST) Received: from player695.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo5.mail-out.ovh.net (Postfix) with ESMTP id 67BBA156889 for ; Sun, 26 Nov 2017 17:03:34 +0100 (CET) Received: from [192.168.2.120] (p5DDB6C07.dip0.t-ipconnect.de [93.219.108.7]) (Authenticated sender: jerome@mediaarea.net) by player695.ha.ovh.net (Postfix) with ESMTPSA id 00507460086 for ; Sun, 26 Nov 2017 17:03:33 +0100 (CET) To: cellar@ietf.org References: From: Jerome Martinez Message-ID: Date: Sun, 26 Nov 2017 17:03:35 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Ovh-Tracer-Id: 16561424680519667857 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 50 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrleeigdekiecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecuogetfeejfedqtdegucdlhedtmd Archived-At: Subject: Re: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 16:03:39 -0000 I am in favor to EBMLDocTypeVersionExperimental item, illegal for files complying to the standard. I am not in favor to add a kind of "DTD" in the file, I don't see how to use it (if it is experimental, people know that the file is maybe not well supported, and the idea is not to spread such file) IMO the idea of having a sub version in stable files as in FFV1 is not relevant here as FFV1 has no way to indicate that the stream is extended without this field, but we have a way to add more elements in EBML without breaking parsers (just by adding a new element value, old parsers just skip it). On 26/11/2017 10:17, Steve Lhomme wrote: > Hi, > > We currently have a pending issue about how to define an experimental > version of a format. > https://github.com/Matroska-Org/ebml-specification/issues/157 > > We could either use DocType 0 to mark the version is in experimental > phase. Or we could add an element in the EBML Header that states the > status of the elements used. > > I think the latter offers more flexibility. We could define that the > format in use in fully compliant with the version(s) set, that it adds > elements. In that case extra elements might be listed in the header. > That would be helpful to detect errors in the file since you know the > element IDs. But it might be complex. Since it would need the Path of > the element, its ID and the type. It would just mean we embed a DTD in > the EBML header (but just skipping the default/known elements). > > IMO it is blocking the finalization of the EBML specifications, even > though it's an addition, because it will be key on how EBML formats > can evolve and still be compliant. > > If we want to go the short way we might just add an element that we > know could be extended to a full DTD later. So a Master and a child > flag (standard/experimental/other?) > From nobody Sun Nov 26 08:28:30 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B33B127444 for ; Sun, 26 Nov 2017 08:28:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ij46JZxXUQNj for ; Sun, 26 Nov 2017 08:28:26 -0800 (PST) Received: from smtp-sh.infomaniak.ch (smtp-sh.infomaniak.ch [128.65.195.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0F07C126D74 for ; Sun, 26 Nov 2017 08:28:25 -0800 (PST) Received: from smtp7.infomaniak.ch (smtp7.infomaniak.ch [83.166.132.30]) by smtp-sh.infomaniak.ch (8.14.5/8.14.5) with ESMTP id vAQGSN6X024160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Sun, 26 Nov 2017 17:28:23 +0100 Received: from Castor.local ([12.131.149.5]) (authenticated bits=0) by smtp7.infomaniak.ch (8.14.5/8.14.5) with ESMTP id vAQGSLS7071031 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Sun, 26 Nov 2017 17:28:22 +0100 Date: Sun, 26 Nov 2017 17:28:22 +0100 From: Reto Kromer To: cellar@ietf.org X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4 (470) X-Antivirus: Dr.Web (R) for Unix mail servers drweb plugin ver.6.0.2.8 X-Antivirus-Code: 0x100000 Archived-At: Subject: Re: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 16:28:28 -0000 Luca Barbato wrote: >Having the DTD would be nice to build programs to inspect the >container or convert to the non-experimental version later. I agree with Luca. Best regards, Reto From nobody Sun Nov 26 08:36:46 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5589612778D for ; Sun, 26 Nov 2017 08:36:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hHtAKjcbg-rq for ; Sun, 26 Nov 2017 08:36:43 -0800 (PST) Received: from mail-pf0-x229.google.com (mail-pf0-x229.google.com [IPv6:2607:f8b0:400e:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A228126D74 for ; Sun, 26 Nov 2017 08:36:43 -0800 (PST) Received: by mail-pf0-x229.google.com with SMTP id j28so16312462pfk.8 for ; Sun, 26 Nov 2017 08:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=NxBqlEtiGOEJQQQ9N3j5hzv3BTPB2Y4gnhNSPRoz9Aw=; b=TLnL8p3dcve8VrNY3CTnr7BQnbAmS1mh6QdQuHTZUtKZUSkjOV3nrnrRM4iupYGJgU vu1oXU9R3SUgo9DcDdYoArY78HMel4Ze7l1yhh3K0VVV4NDXaFFKF/KWlVS2DWMaVlUQ 4sOD6qMpZ0Y/zcNgcfAx48vXsgvLzfa2BuIU6Q90Ou2ApHv+BE7fP6hbj+GMkXhJIM+L WIeCVh+e+AKs9JwnzhPCogNW3OVHA78lND22qXhVG0HiuNz1usXotrl3JQUhShWqVXfl SGCcD//k/01oklEy14PrH8x65YC+v6lEBF7jgSV0+VXLvrbtebDorFCr9gS+den+kenz J5Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=NxBqlEtiGOEJQQQ9N3j5hzv3BTPB2Y4gnhNSPRoz9Aw=; b=YD877Qeihvm4r7gBKdZ5UkKkLjpivEPdyHkpsDz0xHuIvv5iVXTXc24R/Qg/F6Cd6u sx2Y3CAfIKq/rQgiMU6KBGhWtnrgQ7moHqqWDzo8nfWWpj11kG63K/ZqnitP97yuYmro JtOerfVJxpzon17YCuGh1AFexk9VAUyZMlqCKztkfpoKqF1abLG/fQD3U3OCuNK0tpWJ miGbo0kWlA4dYh3KgE/p0F+Q7udvCHMBnl+Hnl2CEP+FxFYlMHujeSphwc1swVg4b/Uu g3ICeN/AbM46lXpNMe9Tc1VxTEvozoK/Db+9c0lHGfCPAbkhQd9mN4q34l9wNzWPkhsQ XM5g== X-Gm-Message-State: AJaThX6v0UfkDMdCMc13m+uwXFkhUE08+GP4NkJX/u9cUPGitwbSKPdR E0BBeloiLO4wvybntnX70I/hxzehQV3Vf9zUJ90E1XAr X-Google-Smtp-Source: AGs4zMbzhH4AQAJXzUqHkUXGzOxlTPhhhYdC8l9wTljeIEBPdxwZ4Bx02hTiyMBcEkQWUY2gSF5f2QcyINJvuk5gLQc= X-Received: by 10.98.56.69 with SMTP id f66mr15947673pfa.38.1511714202104; Sun, 26 Nov 2017 08:36:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 26 Nov 2017 08:36:41 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 26 Nov 2017 17:36:41 +0100 Message-ID: To: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] EBML Error Handling X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 16:36:45 -0000 2017-11-26 16:13 GMT+01:00 Steve Lhomme : > Hi, > > The last remaining issues to fix before we can issue the last call on > the EBML specs is more text to explain how error handling is done. It > covers many possible cases. > > Here are the cases I can think of (some mentioned here [1] and here [2]): Here's what libebml does and how we originally wanted to handle those cases. Let me know if it could be improved or if it's just wrong. > - multiple unique elements libebml doesn't actually for reading unique elements only once. But there is a helper to read the first element of one EBML ID. So by default the first value encountered is the one that readers would use. Different parsers (like the one in VLC) usually work differently, they parse an EBML level and finding a known value that should be unique (for example video width) they set that value. If the element is encountered many time the value is overwritten. So the last value is actually used. I don't have any preference. I'm curious to know what mkvmerge does. We should also check what ffmpeg does. Given the streaming nature of EBML/Matroska it would make sense that the first value encountered is used ASAP. But on the other hand a unique value usually sets a state that must be interpreted when the whole master element is read. So it may not matter that much. But there is the case of the ClusterTimecode. It does matter for streaming, it even has to be early in the Cluster. For this element subsequent values should be ignored (unless they're equal, which makes no difference). That would favor the 'first found sets the value' approach. > - CRC element misplaced (it should always be first) IMO it should be considered as a false positive detection. In other words if the parser reads [BF] as an ID but it's not the first element, the data is probably incorrect and so should be handled as such (see further in the mail). There's no way a correct writer would misplace the CRC element. > - ClusterTimecode misplaced (it should always be first or after the CRC) Should not go in the EBML specs. But that's a case that needs to be considered (see above). > - CRC mismatch with the content libebml doesn't enforce any behavior. It just tells if you the CRC-32 is valid when you ask. In the case of VLC (using libebml and libmatroska), it doesn't care. A player in general will try to play content even if it's partly bogus/damaged. So it's more a concern for data analysis/recovery. So we may not need to define any policy at all. An invalid CRC-32 value just means there's something wrong, but you don't know where. > - invalid EBML ID libebml just skips data until it finds a proper ID. That is an ID that is valid at the semantic level. The library also has a mode that allows reading bogus data that seem to make sense (dummy reading). That's useful for forward compatibility with files. It's the preferred approach in VLC (it's optional and was the other way around in the past). So here we may not define a specific policy either. forward compatibility is something that is very important so it should probably be the default way of handling things. Using dummy elements also allow remuxing a file and keeping the original data even if you don't understand them. It's not foolproof though if dummy data depend on data or metadata (like position) that may be altered during remuxing. libebml has a twist though. Even if it finds a dummy element, it also checks that the length is legit in the context (ie not bigger than the parent). If it doesn't then the dummy ID is considered invalid and it starts parsing the next octets. Meaning there's a "hole" in the EBML data. > - invalid EBML length That's the case described above. libebml doesn't allow elements outside of the parent boundaries. That helps with false positive when encountering unknown element IDs. If such an error is detected, it starts parsing the next octets, resulting in a "hole" in the EBML data. By next octet I mean the first octet after the EBML ID+Length fields. So if you have [AA][BB][CC] octets, it will start parsing the next ID using the [BB] octet and further. The hole is the [AA] octet. Just like with the CRC-32 a player will not care much about those errors. It's more important for data analysis and possibly detecting the position of bogus data. > - Master element with junk data (most likely unidentified or bogus IDs) Again, we live in a world where bogus files will exist, even unintentionally. Players (at least the sane ones) will try to extract as much valid data from the source and not care about the rest. Regular users will not care if something may be wrong somewhere in the file. So the question is more for data analysis/recovery. As seen above, there might be dummy data. That looks totally right but are unknown. And there might be "holes" for data that really didn't make sense the EBML way. Data analysis programs may want to treat them differently. But I'm not sure defining a policy makes much sense. What is more important is that the same bogus data is handled the same way between parsers. As mentioned during No Time To Wait 2, the same bits should produce the same output. So we should probably agree on how these 2 kinds data error are generated rather than what to do with them. > - reserved EBML ID values For libebml it falls in the category of dummy data. The specs currently say "Any Element ID with the VINT_DATA component set as all zero values or all one values MUST be ignored and MUST NOT be considered an error in the EBML Document". It doesn't say what to do with [80] as an ID for example. IMO it should be considered an invalid ID (see above) and so skip to the next octet (EBML "hole"). > - elements not found anymore libebml doesn't do any version check on the semantic. This may not be good. Finding old IDs may be a sign of damaged data rather than bogus writer. On the other mkvalidator does such check and it's common to find mismatches. Especially with an ever evolving format like WebM. I think it's better to read these elements and let the upper level reader deal with the version mismatch. In the case of deprecated value there's a big chance it's element that are supposedly never used, so the data can be handled as valid but unused by the upper layer. > or not yet for the given DocType This is related to the Doctype 0/Experimental field/DTD discussion. libebml currently reads them as valid "dummy" elements (unless you disable dummy mode). As seen above the data could be remuxed as is even when not knowing what they do. I don't think they should be handled as errors. But it may change depending on how we define for experimental IDs. Especially how we handle current existing files, if they are considered as strict (therefore all unknown IDs are junk data, not dummy elements). > - IDs with the inefficient size libebml predates the decision of enforcing the most efficient size for IDs. So it allows that. Now the specs say "The VINT_DATA component of the Element ID MUST be encoded at the shortest valid length". So when a data with the improper size is encountered, it must be considered as junk data. I don't think any files exist with IDs stored as such (or very early Matroska files). So it should be safe to go that way. > - huge EBML Master size that can't hold in memory "640 KB ought to be enough for everyone" is famous for being right at the time and wrong now. So it would be hard to set a hard limit. And if we don't it would violate the "same bit produces the same data rule". Also the amount of memory available really depends on the particular machine the program runs on at a given time. Also Unknown Size elements are not meant to be held in memory anyway but to minimize latency (and backward writing when it's not possible). Not sure what to do here. > - Elements with no default value with a size 0 libebml gives 0 for an integer with no size. Should it be considered a bug in the writing app or bogus data ? Not sure what to do here. If we decide to read it as the element it is (ie not as junk data producing a hole) it is in an unreadable state. It has no value that can be read. And it cannot be used without a value. > - EBML element not in the right place according to the semantic libebml would produce a dummy element or a hole depending on the reading mode. IMO it's junk data. If a writer writes this it cannot expect the data to be handled properly. Depending on the philosophy we use here will define how strict the writer output needs to be. If we say it's OK then it's kinda OK to have such bugs in writers. > There are all reading/parsing errors. Based on what the reader is > supposed to do will also define the possibilities to create safe files > for a writer (see [2]). > > I will reply to this email with what libebml does and how it was > originally designed to work. I think it's better we agree on the rules > for each case before doing patches for the specs. So this is a long reply to a complex topic. We have 2 current outcomes (dummy and hole) in libebml. Maybe you can think of different ways to handle things. > [1] https://github.com/Matroska-Org/ebml-specification/issues/48 > [2] https://github.com/Matroska-Org/ebml-specification/issues/92 > > -- > Steve Lhomme > Matroska association Chairman -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 26 08:43:22 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 17A81127522 for ; Sun, 26 Nov 2017 08:43:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=matroska-org.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id avf0HitFaNCA for ; Sun, 26 Nov 2017 08:43:21 -0800 (PST) Received: from mail-pf0-x231.google.com (mail-pf0-x231.google.com [IPv6:2607:f8b0:400e:c00::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EFB85126D74 for ; Sun, 26 Nov 2017 08:43:20 -0800 (PST) Received: by mail-pf0-x231.google.com with SMTP id i15so16333186pfa.3 for ; Sun, 26 Nov 2017 08:43:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=matroska-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=72fxc9XbHlrf4bHw/r4U8xsGBmoHkXgeK7yhf9GlWYI=; b=DdzLwFak7xhZ56R1nL0HBLIMlS1GfwEaVlS4K3skGzBtmnkXEWRlFqyB7r66fDon0u XrPM49vN9KTPkLJZuf6cYnTg06xJS2NHCQDZztw9JwnCLZh4pQuxukuyFWoZOZSvISlP QVXu0xYC0kA9koBeX/Q6DU2qCKM0APA/Mwn0Tf4LkkK+zUZ763L6KqnZVcdcfQL+ZgFh QF/Bj3+RKjoVAREiyIa6XjCzypN0JeZdDByTGh7Y7ndRFAyXfZX+ASJZhk8CPGVl46Bb mWPKMZcALvgGKitBWxo//VFGD8cq8BtfsNWx2FQbPCrhufb9XONjVB2NEvHKcHplgnW/ jD3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=72fxc9XbHlrf4bHw/r4U8xsGBmoHkXgeK7yhf9GlWYI=; b=ZIbwxHzGNPMvOfNlSrCAOmatfPbi5YikDDzJwVsJJeS4Pcy7nmeOci/pD+tu0oFiEM iAGwdhQZ1YnUkeDC+O28w0h+W/UVNbXHQFwY4nZxwrcOJ/SGrQEJtxDEV/S1MIO1sb+k jt14lAApvBX9ztxH4uVuMRHEzGvqwAaoRWfBj02EbO5CM+KPSwjXurLuPvsxaLta8JvD rjbXLBJw4TpeYn0tmShCrT8yxDzGQKURVeTp6WJ4F8eCOZ3UHvEvaQTQZ2PdJKD91gJy lrapbvYhI02xe25BNBRD09CoS16r5OdQt3+5vJwI8+IsH9yKzUDXDgpWWT7KZaRmXmXZ 7H7A== X-Gm-Message-State: AJaThX6QPByPOPBodo2CQA7UhGWxhRZMfh4Ag8QKy7g9WE0DruqGJ6AO 6aKhix29AvX7uUopV/RUcMgq7eka1RRIxp4x4dFr4Q== X-Google-Smtp-Source: AGs4zMZFWZonBhK0mpE1JjHWuLxbKEZKvKlqC+nkPyI63xLACDBoT7KOxVl+SKhS3vwVpGYS9N2OIpX89JuPAYc/KBQ= X-Received: by 10.98.141.150 with SMTP id p22mr33482289pfk.207.1511714600493; Sun, 26 Nov 2017 08:43:20 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.164.142 with HTTP; Sun, 26 Nov 2017 08:43:20 -0800 (PST) In-Reply-To: References: From: Steve Lhomme Date: Sun, 26 Nov 2017 17:43:20 +0100 Message-ID: To: Reto Kromer Cc: Codec Encoding for LossLess Archiving and Realtime transmission Content-Type: text/plain; charset="UTF-8" Archived-At: Subject: Re: [Cellar] EBML DocType 0 X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 16:43:22 -0000 2017-11-26 17:28 GMT+01:00 Reto Kromer : > Luca Barbato wrote: > >>Having the DTD would be nice to build programs to inspect the >>container or convert to the non-experimental version later. > > I agree with Luca. Best regards, Reto I agree too that it's the cleanest approach. The problem is that it is going be tricky to define such a DTD. Parsers will have to implement an EBML Path parser and integrate that in various libraries. IMO that should be for an EBML 2. We could have a rough idea of how to do it though and just keep the structure that tells a file may contain IDs outside of the known scope. If anything just having an EBMLDoctypeDTD master element present with nothing in it. Signaling that there might be unknown elements in the file. (or rather something like EBMLDoctypeDTD/ExtraElements). > _______________________________________________ > Cellar mailing list > Cellar@ietf.org > https://www.ietf.org/mailman/listinfo/cellar -- Steve Lhomme Matroska association Chairman From nobody Sun Nov 26 09:26:11 2017 Return-Path: X-Original-To: cellar@ietfa.amsl.com Delivered-To: cellar@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C56012025C for ; Sun, 26 Nov 2017 09:26:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0o2qtmFy1r7h for ; Sun, 26 Nov 2017 09:26:08 -0800 (PST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:c:538::198]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 069FB120227 for ; Sun, 26 Nov 2017 09:26:08 -0800 (PST) X-Originating-IP: 213.47.41.20 Received: from localhost (213-47-41-20.cable.dynamic.surfer.at [213.47.41.20]) (Authenticated sender: michael@niedermayer.cc) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 1CA53FB8A4 for ; Sun, 26 Nov 2017 18:26:05 +0100 (CET) Date: Sun, 26 Nov 2017 18:26:01 +0100 From: Michael Niedermayer To: cellar@ietf.org Message-ID: <20171126172601.GF24699@nb4> References: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cpvLTH7QU4gwfq3S" Content-Disposition: inline In-Reply-To: <99534033-A516-425E-A9A5-84F9F0B39183@dericed.com> User-Agent: Mutt/1.5.21 (2010-09-15) Archived-At: Subject: Re: [Cellar] status of FFV1 and EBML drafts X-BeenThere: cellar@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Codec Encoding for LossLess Archiving and Realtime transmission List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 17:26:10 -0000 --cpvLTH7QU4gwfq3S Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Nov 19, 2017 at 10:47:32AM -0500, Dave Rice wrote: > Hi cellar, >=20 > At the IETF99 meeting in the minutes at https://datatracker.ietf.org/doc/= minutes-99-cellar/ it discusses having a WG last call for FFV1 and EBML but= also the need for more reviews of these documents. Is one issue blocking a= nother? Meaning: should the group announce a last call in order to encourag= e more reviews or are more reviews needed in order to have a last call? There are quite a few things in FFV1 i do want to look at. I just keep pushing it away as ive too many other things to do Bayer RGB, floating point samples, palettes, motion compensation, ... Iam not sure how any of it relates to the "last call" ... Also in general i would be in favor of "release early, release often" Thanks [...] --=20 Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Modern terrorism, a quick summary: Need oil, start war with country that has oil, kill hundread thousand in war. Let country fall into chaos, be surprised about raise of fundamantalists. Drop more bombs, kill more people, be surprised about them taking revenge and drop even more bombs and strip your own citizens of their rights and freedoms. to be continued --cpvLTH7QU4gwfq3S Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAloa+SkACgkQYR7HhwQLD6utsgCglkkd2TdIT7dzw8ddvej7h11V KQ4AniqicXM1SfXHc+++WeAa0cdBy6P4 =zdYw -----END PGP SIGNATURE----- --cpvLTH7QU4gwfq3S--