Subject: Re: Update regarding draft-bryan-metalink (intro section)

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 09 Dec 2009 22:06:30 +0900

Hi,

> On Fri, Nov 27, 2009 at 5:11 PM, Peter Pöml <poeml at cmdline.net> wrote:
>>
>> I'm sending an update to the introductory section of the
>> draft-bryan-metalink I-D. The changes are:
>>
>> - mention the multi-protocol (HTTP, FTP, P2P) nature of Metalinks.
>> - describe how verification in Metalinks can solve issues trusting content
>> replicas on mirror servers
>> - distinguish more clearly between human users and user agents
>> - various minor improvements.
> 
> This last week we received some healthy review from Lisa and Eran,
> document shepherd.
> 
> We addressed most issues in an interim version that can be found at
> http://metalinks.svn.sourceforge.net/viewvc/metalinks/internetdraft/
> but await clarifications on a few others.
> 
> The other remaining issues:
> 
> 1) Can we use Creative Commons for machine readable alternative to our
> license element (Sec 4.2.8)?
> 

Sorry, I don't understand. Do you mean adding new element for "Creative
Commons"? What does it look like?

> 2) Do we need to use ABNF for the generator element (4.2.4)?
> 
> We talked about using this from HTTP
> 
>       product         = token ["/" product-version]
>       product-version = token
> 

I think yes, it is clearer than current form which explains in English text.
Or can we borrow it 3.8 Product Tokens from rfc2616 or something similar?

> 3) metaurl element's type attribute value (4.2.10.2). It uses MIME
> type, but "torrent" is defined.
> 

The thing is application/x-bittorrent is used for BitTorrent's mime type
widely but it is not officially registered in IANA, right?
I prefer MIME type than inventing new name.

> 4) signature element's type attribute value (4.2.15.1). pgp is defined.
> 

Same thing as bittorrent case.
I prefer "application/pgp-signature", which sounds like more "official".
Using MIME type will be useful when we will add new signature algorithm.
We don't have to invent new name.

> 5) We allow one or more OS elements. Should we allow one or more
> license and language elements?
> 

According to http://www.ietf.org/id/draft-bryan-metalink-24.txt,
multiple OS elements are allowed under file element.
Some programs are licensed under dual or more licenses, so I think it
would be good to allow multiple license elements.

For language element, we can use several of them for movies or DVD
contents which might be restricted (or partially modified to suite their
laws) by region code. One region contains several countries, so multi
language elements are nice in this case.

> And here is the intro with only a few small changes:
> 
>    Metalink is an XML-based document format that describes a file or
>    list of files to be downloaded from a server.  Metalinks can list a
>    number of files, each with an extensible set of attached metadata.
>    Each listed file can have a description, checksum, and a list of URIs
>    that it is available from.
> 
>    Often, identical copies of a file are accessible in multiple
>    locations on the Internet over a variety of protocols (FTP, HTTP, and
>    Peer-to-Peer).  In some cases, users are shown a list of these
>    multiple download locations (mirror servers) and must manually select
>    one based on geographical location, priority, or bandwidth.  This is
>    done to distribute the load across multiple servers, and to give
>    human users the opportunity to choose a download location that they
>    expect to work best for them.
> 
>    At times, individual servers can be slow, outdated, or unreachable,
>    but this can not be determined until the download has been initiated.
>    This can lead to the user canceling the download and needing to
>    restart it.  During downloads, errors in transmission can corrupt the
>    file.  There are no easy ways to repair these files.  For large
>    downloads this can be especially troublesome.  Any of the number of
>    problems that can occur during a download lead to frustration on the
>    part of users, and bandwith wasted with retransmission.
> 
>    Knowledge about availability of a download on mirror servers can be
>    acquired and maintained by the operators of the origin server, or by
>    third a party.  This knowledge, together with checksums, digital
>    signatures, and more can be stored in a machine-readable Metalink
>    file.  The Metalink file can transfer this knowledge to the user
>    agent, which can peruse it in automatic ways or present the
>    information to a human user.  User agents can fall back to alternate
>    mirrors if the current one has an issue.  Thereby, clients are
>    enabled to work their way to a successful download even under adverse
>    circumstances.  All this can be done transparently to the human user
>    and the download is much more reliable and efficient.  In contrast, a
>    traditional HTTP redirect to one mirror conveys only comparatively
>    minimal information - a referral to a single server, and there is no
>    provision in the HTTP protocol to handle failures.
> 
>    Other features that some clients provide include multi-source
>    downloads, where chunks of a file are downloaded from multiple
>    mirrors (and optionally, Peer-to-Peer) simultaneously, which
>    frequently results in a faster download.  Metalinks can leverage
>    HTTP, FTP and Peer-to-Peer protocols together, because regardless
>    over which protocol the Metalink was obtained, it can make a resource
>    accessible through other protocols.  If the Metalink was obtained
> 
>    from a trusted source, included verification metadata can solve trust
>    issues when downloading files from replica servers operated by third
>    parties.  Metalinks also provide structured information about
>    downloads that can be indexed by search engines.
> 

Looks good for me. I find one typo "bandwith"(Thunderbird found it;) ),
I'll fix it in svn.

Best regards,

Tatsuhiro Tsujikawa

---

From: Peter Pöml <[email-masked]>
Date: Mon, 14 Dec 2009 02:19:09 +0100

Hi,

(comments inline)

Am 09.12.2009 um 14:06 schrieb Tatsuhiro Tsujikawa:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>>

I also don't understand. Maybe I'm missing some context, but I wonder  
if there might be misunderstanding? Creative Commons would just be one  
out of many different licenses that might find their way into the  
license element.

> Or can we borrow it 3.8 Product Tokens from rfc2616 or something  

That was where the concept comes from. I agree, using ABNF to achieve  
more clarity makes sense to me.

> I prefer MIME type than inventing new name.
[...]
> We don't have to invent new name.

That's a very good idea to use a MIME type here - regardless of the  
fact that one of them isn't registered!

> language elements are nice in this case.

I agree it can be useful to allow multiple values here.

Peter

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Mon, 14 Dec 2009 00:10:52 -0700

> >> 1) Can we use Creative Commons for machine readable alternative to
> >> our license element (Sec 4.2.8)?
> >>
> >
> > Sorry, I don't understand. Do you mean adding new element for
> > "Creative Commons"? What does it look like?
> 
> I also don't understand. Maybe I'm missing some context, but I wonder if
> there might be misunderstanding? Creative Commons would just be one out
> of many different licenses that might find their way into the license element.

There seems to be value in supporting a machine-readable copyright element in addition to the human-readable element you have today. It might just be that we are not ready at this point to offer a list of token-identified licenses. But if most metalink implementations today already default to a small set of licenses (MIT, BSD, etc.), is it worth defining such a list?

EHL

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 14 Dec 2009 10:22:38 -0800

Such a list could well be broadly useful, and certainly not limited
just to this application.  In the last 3.5 years I've seen at least
three of our own standards that could have used token-identified
licenses or copyright statements, and there are probably other
non-IETF documents (W3C or other schemas) that could use just the same
tokens.

Lisa

On Sun, Dec 13, 2009 at 11:10 PM, Eran Hammer-Lahav <[email-masked]> wrote:
> _______________________________________________
> Apps-Discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Mon, 14 Dec 2009 12:28:16 -0700

I am happy to draft an I-D for such a registry if there is interest.

EHL

> -----Original Message-----
> From: Lisa Dusseault [mailto:[email-masked]]
> Sent: Monday, December 14, 2009 10:23 AM
> To: Eran Hammer-Lahav
> Cc: Peter Pöml; [email-masked]
> Subject: Re: Update regarding draft-bryan-metalink (intro section)
> 
> Such a list could well be broadly useful, and certainly not limited just to this
> application.  In the last 3.5 years I've seen at least three of our own standards
> that could have used token-identified licenses or copyright statements, and
> there are probably other non-IETF documents (W3C or other schemas) that
> could use just the same tokens.
> 
> Lisa
> 
> >> if there might be misunderstanding? Creative Commons would just be
> >> one out of many different licenses that might find their way into the
> license element.
> >
> > There seems to be value in supporting a machine-readable copyright
> element in addition to the human-readable element you have today. It might
> just be that we are not ready at this point to offer a list of token-identified
> licenses. But if most metalink implementations today already default to a
> small set of licenses (MIT, BSD, etc.), is it worth defining such a list?
> >
> > EHL

---

From: Anthony Bryan <[email-masked]>
Date: Mon, 14 Dec 2009 22:18:12 -0500

> There seems to be value in supporting a machine-readable copyright element in addition to the human-readable element you have today. It might just be that we are not ready at this point to offer a…

Here are some that Metalink Editor lets users select.

Unknown
Commercial
Shareware
Public Domain
GNU GPL 3, http://www.gnu.org/licenses/gpl.html
GNU LGPL 3, http://www.gnu.org/licenses/lgpl.html
GNU FDL 1.3, http://www.gnu.org/licenses/fdl.html
Creative Common Public Domain, http://creativecommons.org/licenses/publicdomain/
Creative Commons Music Sharing,
http://creativecommons.org/licenses/by-nc-nd/2.0/deed-music
Creative Commons Dev Nations (deprecated),
http://creativecommons.org/licenses/devnations/2.0/
Creative Commons by-nc-nd, http://creativecommons.org/licenses/by-nc-nd/2.5/
Creative Commons by-nc-sa, http://creativecommons.org/licenses/by-nc-sa/3.0/
Creative Commons by-nc, http://creativecommons.org/licenses/by-nc/3.0/
Creative Commons by-nd, http://creativecommons.org/licenses/by-nd/2.5/
Creative Commons by-sa, http://creativecommons.org/licenses/by-sa/2.5/
Creative Commons by, http://creativecommons.org/licenses/by/2.5/
Creative Commons Sampling (deprecated),
http://creativecommons.org/licenses/sampling/1.0/
Creative Commons Sampling+, http://creativecommons.org/licenses/sampling+/1.0/
Creative Commons nc-sampling+,
http://creativecommons.org/licenses/nc-sampling+/1.0/
BSD, http://opensource.org/licenses/bsd-license.php
MIT, http://opensource.org/licenses/mit-license.php
MPL 1.0, http://opensource.org/licenses/mozilla1.0.php
MPL 1.1, http://opensource.org/licenses/mozilla1.1.php
AFL 3.0, http://opensource.org/licenses/afl-3.0.php
OSL 3.0, http://opensource.org/licenses/osl-3.0.php
zlib/libpng, http://opensource.org/licenses/zlib-license.php
Artistic License 1.0, http://opensource.org/licenses/artistic-license-1.0.php
Artistic License 2.0, http://opensource.org/licenses/artistic-license-2.0.php

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

---

From: Anthony Bryan <[email-masked]>
Date: Mon, 14 Dec 2009 22:24:36 -0500

On Sun, Dec 13, 2009 at 8:19 PM, Peter Pöml <[email-masked]> wrote:
> there might be misunderstanding? Creative Commons would just be one out of
> many different licenses that might find their way into the license element.

My fault for not explaining. Creative Commons offers machine readable
license info.

>From what I can tell, this was in RDF (
http://xml.coverpages.org/CCommons-RDF-Schema200212.html ) but don't
know if that's active anymore. It looks like they are using RDFa,
embedded in HTML now so this no longer applies to us:
http://wiki.creativecommons.org/RDFa

> That was where the concept comes from. I agree, using ABNF to achieve more
> clarity makes sense to me.

Great, can you add that?

> That's a very good idea to use a MIME type here - regardless of the fact
> that one of them isn't registered!

To answer Tatsuhiro, correct, BitTorrent's MIME type is not officially
registered, so some people are against allowing that.

P2P, in general, seems to move fast & not worry about standardization much.

For digital signatures, we can probably count on them having
registered MIME types - if they use them.

> I agree it can be useful to allow multiple values here.

Ok.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Dec 2009 09:43:57 +0100

Eran Hammer-Lahav wrote:
 > ...

Sounds like a can of worms to me.

How about considering an extension spec for his, and making sure that 
the Metalink extension model makes this possible?

Best regards, Julian

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 15 Dec 2009 23:37:35 +0900

Hi,

> http://wiki.creativecommons.org/RDFa

Thanks Anthony, I understand. I think machine readable license tag is
good, but I think it is too much for Metalink I-D. It would be good it
is defined in separate I-D so that everyone can refer to it. More
license to come in the future, so the speed of license grows and
evolution of Metalink I think differs.
Metalink I-D have extension mechanism so we can adopt them easily.

>> That was where the concept comes from. I agree, using ABNF to achieve
more
>> clarity makes sense to me.

This is my draft:

The "metalink:generator" element's content identifies the
generating agent name and version used to generate a Metalink
Document, for debugging and other purposes.

metalinkGenerator =
   element metalink:generator {
     metalinkTextConstruct
   }

metalink:generator element's content is defined as ABNF below.

agent         = token ["/" agent-version]
agent-version = token

token is of type text. Although any character allowed in text MAY
appear in an agent-version, this token SHOULD only be used for a
version identifier (i.e., successive versions of the same agent
SHOULD only differ in the agent-version portion of the agent
value).

I replaced product with agent, since first paragraph refers "agent".
Since "token" is not mentioned in Metalink I-D, we have to give its
type. I just added "token is of type text", but I'm not confortable with
it, so please suggest better one. The most part of last paragraph is
borrowed from the last paragraph of RFC2616 3.8 Product Tokens.

Best regards,

Tatsuhiro Tsujikawa

---

From: Anthony Bryan <[email-masked]>
Date: Tue, 15 Dec 2009 15:59:00 -0500

On Tue, Dec 15, 2009 at 3:43 AM, Julian Reschke <[email-masked]> wrote:
>> There seems to be value in supporting a machine-readable copyright element
>> in addition to the human-readable element you have today. It might just be
>> that we are not ready at this point to offer a list of token-identified
> How about considering an extension spec for his, and making sure that the
> Metalink extension model makes this possible?

Sounds good.

License element removed from the metalink ID, extension spec later if
there is interest.

---

From: Anthony Bryan <[email-masked]>
Date: Tue, 15 Dec 2009 16:02:22 -0500

> Metalink I-D have extension mechanism so we can adopt them easily.

license element removed. :)

> borrowed from the last paragraph of RFC2616 3.8 Product Tokens.

Thanks. I changed the one line to:

metalink:generator element's content is defined below in ABNF notation
[RFC5234].

And added RFC5234 to the normative references section.