Subject: [apps-discuss] seeking pragmatic guidelines for content-type 'structure': when to go top-level?

From: Dave CROCKER <[email-masked]>
Date: Thu, 10 Nov 2011 10:54:52 +0800

Folks,

The font thread has re-raised the issue of registering a Top-Level Content-Type 
(TLCT).  I haven't seen clear statements or documents of a theory or rationale 
that would guide accepting or rejecting a request for a TLCT, and I think we 
should (finally) remedy this deficiency.

In an earlier note I cited a recollection of resistance to new TLCTs.  But I 
don't remember the reasons, though I seem to recall both substance and vigor to 
the reasons.

We should try to resurrect or re-create a simple, pragmatic set of guidelines, 
and document them, so we don't have to go through the kind of vague exchanges 
happening now, with respect to arguing whether a new TLCT is appropriate.

The purpose of this thread is to press for a brief document that resolves this 
kind of debate and that can be used going forward in designing the "structure" 
of new registration groups.  This ought to be in terms of real pragmatics, 
rather than aesthetics or personal preference.

I can think of only two pragmatic issues in registering names like this:

    1.  Administrative convenience.  A hierarchy permits administrative grouping 
and possibly the convenience of delegating subsets of registration.  Obviously 
the DNS is the prime example.  At very large scale, this is essential.  At small 
scale, extra structure can be a hassle.

        I'm not seeing convenience as an issue for Content-Type, but perhaps 
others see the issue differently.  That is, my sense is that the entire data 
base of content-types is small enough to a) remain centralized, and b) not 
otherwise needing to benefit from sub-setting.

        This calls to question why there are /any/ sub-types, of course,  but 
I'd rather cast the issue in terms of going forward, rather than of history.

    2.  Code dispatching.  Different content-types invoke different software 
segments.  Parameters are input to the code but do invoke different "programs".

        I am not seeing how a TLCT vs. sub-type distinction affects this issue. 
  That is, it seems to me that

           C-T: major/minor

has equal software utility as

           C-T:application/major-minor

Any useful difference seems to be to fall under #1, above, rather than under a 
software behavior distinction.  What am I missing?

The 'model' document talks about a benefit of grouping sub-types, but I do not 
understand what the software benefit is, in terms of MIME Content-type.  That 
is, the benefit seems to be a local benefit within the model world, rather than 
more generally within the MIME world.  Again, what am I missing?

To take the 'font' discussion as an immediate, real-world exemplar, I think the 
above says that a logical hierarchy should have the representation engine be 
superior to the typeface or font specification, since it affects software 
dispatch.

Also, I don't see what the driving need for a TLCT is for font (or face...) 
It's aesthetically appealing, but I don't see the technical or administrative 
benefit.  Hence, here's one last:  what am I missing?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 10 Nov 2011 13:29:04 +0900

Hello Dave,

On 2011/11/10 11:54, Dave CROCKER wrote:
> Folks,
>
> The font thread has re-raised the issue of registering a Top-Level
> Content-Type (TLCT). I haven't seen clear statements or documents of a
> theory or rationale that would guide accepting or rejecting a request
> for a TLCT, and I think we should (finally) remedy this deficiency.

If somebody has some free time to work on this, why not. But both in the 
IETF and elsewhere, my experience is that it's difficult to create a 
general theory with very few examples. And this is a case with extremely 
few examples. So I'd rather spend my time on the concrete case. This 
would hopefully result in some mime types actually being registered.

> In an earlier note I cited a recollection of resistance to new TLCTs.
> But I don't remember the reasons, though I seem to recall both substance
> and vigor to the reasons.

That has been reported elsewhere, see e.g.
http://lists.w3.org/Archives/Public/www-font/2009JulSep/1069.html

It would be good if we found an archive of this discussion, just so that 
we could understand what went on then. But if we can't find that 
anymore, then we shouldn't care too much. If we forgot why there was 
resistance, maybe it wasn't that important.

> We should try to resurrect or re-create a simple, pragmatic set of
> guidelines, and document them, so we don't have to go through the kind
> of vague exchanges happening now, with respect to arguing whether a new
> TLCT is appropriate.
>
> The purpose of this thread is to press for a brief document that
> resolves this kind of debate and that can be used going forward in
> designing the "structure" of new registration groups. This ought to be
> in terms of real pragmatics, rather than aesthetics or personal preference.

I'm going to discuss this with you here because I'm interested in the 
example at hand, not because I want to create a general theory.

> 1. Administrative convenience. A hierarchy permits administrative
> grouping and possibly the convenience of delegating subsets of
> registration. Obviously the DNS is the prime example. At very large
> scale, this is essential. At small scale, extra structure can be a hassle.
>
> others see the issue differently. That is, my sense is that the entire
> data base of content-types is small enough to a) remain centralized, and
> b) not otherwise needing to benefit from sub-setting.
>
> This calls to question why there are /any/ sub-types, of course, but I'd
> rather cast the issue in terms of going forward, rather than of history.

I agree that delegating subsets along major types doesn't make sense. 
But there's another administrative reason for having major/minor types: 
It's hopefully easier to find a type you're looking for, or check that a 
type hasn't been registered. As a clear example, for image formats, I 
know that http://www.iana.org/assignments/media-types/image/index.html 
is where I have to check.

> Any useful difference seems to be to fall under #1, above, rather than
> under a software behavior distinction. What am I missing?

I think that fonts are dispatched quite differently e.g. from 
application data. For application data, the general solution is to call 
up the respective application (or ask the user to select or confirm 
one). For font data, there are of course applications that work on the 
data directly (e.g. editors). But the main usage is that they are saved 
(or installed) separately and used as auxiliary documents.

> The 'model' document talks about a benefit of grouping sub-types, but I
> do not understand what the software benefit is, in terms of MIME
> Content-type. That is, the benefit seems to be a local benefit within
> the model world, rather than more generally within the MIME world.
> Again, what am I missing?

You may not be missing too much. It's easy to imagine a world with a 
flat type system, or with a type system that didn't have image/, audio/, 
and video/.

But the fact is that we have a major/minor type system. We can as well 
make reasonable use of it.

> To take the 'font' discussion as an immediate, real-world exemplar, I
> think the above says that a logical hierarchy should have the
> representation engine be superior to the typeface or font specification,
> since it affects software dispatch.

I'm not sure I understand this point. Can you elaborate. What do you 
mean by "representation engine"?

> Also, I don't see what the driving need for a TLCT is for font (or
> face...) It's aesthetically appealing, but I don't see the technical or
> administrative benefit. Hence, here's one last: what am I missing?

Glad you agree that it's aesthetically appealing. And please remember 
that people involved with fonts care quite a lot about aestetics.

As for the technical/administrative benefits, things such as easy 
searchability and easier negotiation (e.g. Accept: font/*) and despatch 
have already been mentioned.

These benefits are not necessarily tremendous, but the same applies to 
image/, video/, and audio/.

Regards,   Martin.

---

From: Barry Leiba <[email-masked]>
Date: Thu, 10 Nov 2011 01:02:05 -0500

>> under a software behavior distinction. What am I missing?
...
> You may not be missing too much. It's easy to imagine a world with a flat
> type system, or with a type system that didn't have image/, audio/, and
> video/.
>
> But the fact is that we have a major/minor type system. We can as well make
> reasonable use of it.

As I said in the other thread (on font/*), we already have the concept
of saying "This is text," "This is an image," and "This is audio," and
I see only negative points in insisting that everything other than the
few TLMTs that we're already defined have to be "application".  We
should do some filtering and apply some judgment, of course, but
making everything the equivalent to "application/image-jpeg" is silly
and non-useful.

The point of "image/xxx" was to allow a program to say "I do/don't
handle images," and have it send the media content over to its image
processing, or to shortcut things because it just doesn't do images at
all.  The same thinking applies to other things, and it's not just
application or administrative "convenience".  Of course, the problem
with "application/xxx-yyy" is that unless we define a hierarchy for
the subtype field, the processing I described above that could apply
to "image/jpeg" can't apply to "application/image-jpeg".

For what it's worth, I think it was a mistake to put office document
stuff into "application"; we should have made a TLMT for "document",
so we could have, say, "document/spreadsheet" or "document/MS-excel"
or "document/ODS", "document/PDF", "document/presentation", and so on.
 (For that matter, "text/html" seems a bit silly these days, and
"document/html" might be more appropriate, but I digress.)

I do understand that we don't want to have TLMTs proliferating out of
control.  On the other hand, I think we're being *way* too restrictive
about defining them when there's a reasonable argument why they make
sense.

And I'm happy to participate in a document that talks about this.

Barry

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 10 Nov 2011 14:59:44 +0800

Folks,

> making everything the equivalent to "application/image-jpeg" is silly

That states a basic direction/goal: To have a lower hurdle than perhaps was 
there before.

Does it have any basic downsides?

My sense from the latest round of postings, including Ned's, is that there is no 
current reason to impose a high hurdle on TLCTs.

      Consensus check
      ===============

      Is there anyone out there who believes that it is extremely expensive or 
dangerous to introduce new, top-level Content-Type and that, therefore, the 
barrier to new TLCTs should be (kept) high?

Assuming the answer and the consensus is no, that still leaves open what 
pragmatic guidance is to be used for accepting or rejecting a request for a 
TLCT.  (We could take the ICANN approach and just filter based on huge gobs of 
money, but I doubt that model will transfer to the IETF.)

So far, the most cogent guidance I've seen seems to be related to the 
justification in the model spec (RFC 2077):

   "The important fact is that these
    various subtypes can be converted between each other with less loss
    of information then to that of other primary types.  This fact groups
    these subtypes together into the model primary type.  All of the
    expected subtypes have several features in common and that are unique
    to this primary type."

I had suggested a guideline in terms of "dispatching" to different applications. 
  Ned's note offered a better framing, in terms of common code among a set of 
content sub-types.  A particular form of such code, implied by the model 
document, is for translating among sub-types.

Martin pointed out an "administrative" issue quite different from the basic 
registration scaling or decentralization I cited, namely the search for related 
types by an implementer, where separate sub-tables would be helpful.  He noted:

      http://www.iana.org/assignments/media-types/image/index.html

as a good example.

      Consensus Check
      ===============

      So, I think this leaves guidance in favor of a new top-level content-type 
if one or more of the following apply:

      1.  Strong semantic relationship among the sub-types

      2.  Likelihood of some common code for the set of sub-types

      3.  Expectation that implementors will benefit from easily discovering the 
current set of sub-types in the registry.

Does this summarize the guidance that should be offered for justifying a new TLCT?

d/

-- 

   Dave Crocker

---

From: Barry Leiba <[email-masked]>
Date: Thu, 10 Nov 2011 04:33:27 -0500

>     Consensus Check
>     ===============
>
>     So, I think this leaves guidance in favor of a new top-level
> content-type if one or more of the following apply:
>
>     3.  Expectation that implementors will benefit from easily discovering
> the current set of sub-types in the registry.
>
>

WFM.

[Quick and nitty terminology check: the term of art is "media type",
which is why I was calling them "TLMTs".  We all understand what
people mean when they say "MIME type" or "content type", but "media
type" is what's used in the registry.]

Barry

---

From: "t.petch" <[email-masked]>
Date: Thu, 10 Nov 2011 10:28:49 +0100

----- Original Message -----
From: "Dave CROCKER" <[email-masked]>
To: "Apps Discuss" <[email-masked]>
Sent: Thursday, November 10, 2011 7:59 AM
> current reason to impose a high hurdle on TLCTs.

Yes.  Based on FUD rather than concrete information, but what will all the
deployed software do when face with a new top-level Content-Type?  And how long
will it take for the makers of commercial software to incorporate this in their
products (1-2years) and how long will it take for that software to be widely
deployed(3-5 years)?

I just received an (antisocial) e-mail with .gif's attached, so I edited the
Content-Type to be font/various things, and my MUA took no notice, just
processed them as .gif.

I modified the Content-Description in the same way, same result.

Only when I modified the filename (eg to .ttf) did the MUA take any notice and
complain that the attachment was not a font, pdf etc.

I think we need to know this for commonly deployed platforms before we can say
it is not dangerous.

Tom Petch

>       http://www.iana.org/assignments/media-types/image/index.html
content-type
>       3.  Expectation that implementors will benefit from easily discovering
the

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 10 Nov 2011 19:53:32 +0900

On 2011/11/10 18:28, t.petch wrote:
> From: "Dave CROCKER"<[email-masked]>
> To: "Apps Discuss"<[email-masked]>
> Yes.  Based on FUD rather than concrete information,

It's nice for you to admit that your argument is based on FUD. But I 
think its a bad idea to use that for standards development.

Ned has given some very good arguments for why most if not all software 
uses the full (i.e. major/minor) type for dispatching,... All the 
experience I have (which I have to admit is way less than Ned) points in 
the same direction.

> but what will all the

If it will have taken the IETF more than 10 years to get around to 
finally approve the font/ top level type, we can't blame developers when 
it takes them 3-5 years to deploy that.

And the deployment time is the same if we register new types under 
application/.

Regards,   Martin.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Thu, 10 Nov 2011 20:03:50 +0900

On 2011/11/10 15:59, Dave CROCKER wrote:
> That states a basic direction/goal: To have a lower hurdle than perhaps
> was there before.
>

In particular for the example now at hand, not that I know.

> My sense from the latest round of postings, including Ned's, is that
> there is no current reason to impose a high hurdle on TLCTs.
>
>
> Consensus check
> ===============
>
> dangerous to introduce new, top-level Content-Type and that, therefore,
> the barrier to new TLCTs should be (kept) high?
>
>
> pragmatic guidance is to be used for accepting or rejecting a request
> for a TLCT. (We could take the ICANN approach and just filter based on
> huge gobs of money, but I doubt that model will transfer to the IETF.)

I hope it won't, too.

> I had suggested a guideline in terms of "dispatching" to different
> applications. Ned's note offered a better framing, in terms of common
> code among a set of content sub-types. A particular form of such code,
> implied by the model document, is for translating among sub-types.

That would definitely apply to the font/ proposal.

> Martin pointed out an "administrative" issue quite different from the
> basic registration scaling or decentralization I cited, namely the
> search for related types by an implementer, where separate sub-tables
> would be helpful. He noted:
>

I still think that past, present, and future examples of new top-level 
types were/are/will be so rare that having explicit guidelines doesn't 
make much sense.

But while we are at it, I'd suggest that we add some degree of 
parallelism with existing top level types (I think font/ fits in well 
with image/, audio/, and video/).

On the negative side, we could include:

- Expectation that there will only be a single subtype, or only a couple.

- Majority of existing types already registered under another type (e.g. 
application)

Regards,   Martin.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 10 Nov 2011 12:45:06 +0100

On 10 November 2011 10:28, t.petch wrote:

> I just received an (antisocial) e-mail with .gif's
> attached, so I edited the Content-Type to be
> font/various things, and my MUA took no notice,
> just processed them as .gif.

That's okay.  RFC 2049 maps "dunno" to "application
octet-stream", and your UA or OS can figure out that
it's actually GIF looking at the first bytes...

> Only when I modified the filename (eg to .ttf) did
> the MUA take any notice and complain that the
> attachment was not a font, pdf etc.

...or looking at the extension.

> I think we need to know this for commonly deployed > platforms before we can say it is not dangerous.

+1  I vaguely recall than some fonts on my platform
use the format of a DLL.

-Frank

---

From: "t.petch" <[email-masked]>
Date: Thu, 10 Nov 2011 11:48:50 +0100

Dave

In this thread, there have been a number of references to how easy it is, or
not, to implement which, for me, should always be secondary to how easy it is to
use.

There is a use which is perhaps being missed which, for me, is the most
important one, and that is in keeping spam out of the system.  Being able to say
.exe goes into the bin unread is very important.  I am out of touch with current
spam filtering but believe that Media Type is a significant part of it.
(Personally, I would like ietf lists to be restricted to text/plain and while no
list moderators go that far, I have seen posts, in the past, from some
explaining what they will not allow on WG lists).

In this context, as has been said already, perhaps for very different reasons,
html should have been top-level, but we are too late for that.

Tom Petch

----- Original Message -----
From: "Dave CROCKER" <[email-masked]>
To: "Apps Discuss" <[email-masked]>
Sent: Thursday, November 10, 2011 3:54 AM

> The font thread has re-raised the issue of registering a Top-Level
Content-Type
>     1.  Administrative convenience.  A hierarchy permits administrative
grouping

---

From: Nathaniel Borenstein <[email-masked]>
Date: Thu, 10 Nov 2011 09:02:27 -0500

One other pragmatic condition might indicate the desirability of a top-level type:  the possibility of defining a reasonable default behavior for unrecognized subtypes.

Thus, for text/*, it is sometimes reasonable to just show users the raw text.  For image/* with an unrecognized subtype, it is sometimes reasonable to invoke a "smarter" program like xv.  For multipart/* you can at least find the parts, even if you're not sure what to do with them.  For video/* you  might choose to reject or strip out the data in a resource-challenged environment.  And I suspect this argues in favor of "font" as a TLCT, as the mail code may often be able to pass the data on to a "smarter" font engine.

I believe this is related, but not identical, to #1 and #2 below.  -- Nathaniel

On Nov 10, 2011, at 1:59 AM, Dave CROCKER wrote:

>     So, I think this leaves guidance in favor of a new top-level content-type if one or more of the following apply:
> 
>     3.  Expectation that implementors will benefit from easily discovering the current set of sub-types in the registry.
> 
>

---

From: Ned Freed <[email-masked]>
Date: Thu, 10 Nov 2011 16:46:04 -0800 (PST)

> deployed software do when face with a new top-level Content-Type?

I'm pretty familiar with a lot of different software that handles media types,
and in every case I know of the answer is "it already supports it".

This is a consequence of the fact that it's easiest to treat leaf media type
names as simple case-insensitive strings. Anything else is more work for no
good purpose. And one thing you can usually count on implementors to do is not
go to a lot of extra effort only to make their software harder to use and more
fragile.

> And how long

I have yet to see any evidence that any software currently has problems of this
sort.

> Content-Type to be font/various things, and my MUA took no notice, just

A lot of agents, when presented with a type they don't know, will either
use the file extension, sniff the data, or both. 

Now, you may think this handling is a bad idea (or not), but it has almost
nothing to do with top-level type usage. In fact the same thing would almost
certainly have happened had you changed the media type to image/foo or
application/foo. 

The only thing you have demonstrated in regards to unknown top-level types is
that your application doesn't choke on them.

> I modified the Content-Description in the same way, same result.

I completely fail to see why you would expect any other result. It's a textual
description, not supposed to be machine actionable, you know.

> complain that the attachment was not a font, pdf etc.

In other words, you changed the file extension to one that your application
knows is associated with a font of some kind. So it tried to process it
as such and got an error.

And again, to the extent this has anything to do with top-level types, you have
just shown your application to be capable of handling previously unknown ones
without falling over. And that's all you have shown.

But if you want to actually perform a more complete test, you should have tried
to add a font/* entry to your application's media type tables and then tested
with content having that label. I've done that lots of times and previously
unknown top-level types have never been a problem.

> I think we need to know this for commonly deployed platforms before we can say

If you regard the behavior you describe as dangerous, well, we appear to be
working from very different definitions of the word "dangerous".

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 10 Nov 2011 18:29:36 -0800 (PST)

> barrier to new TLCTs should be (kept) high?

I want to be very very clear here: It depends critically on whether or not it
has additional special semantics that MIME processors have to understand. New
leaf types generally do not have such semantics even when they are associated
with a new top-level type (text is a possible exception here).

However, a new top-level type with multipart semantics would be a *huge* deal,
especially if it had a different syntax than the existing multipart type.

New types with message semantics are a little better, but only a little.
message/global is something of a pain for two reasons: (1) The differing
syntax (CTE allowed) has to be accomodated, and (2) You have to know when has
to be decoded and when it must not be.

The only way a new leaf top-level type can have similar impact is if it
needs to be handled specially in some way when doing basic display actions.
Changes to text/plain are potentially problematic for this reason - adding
format=flowed support was actually a fair bit of work.

But I don't see how any of this applies to font/*.

> money, but I doubt that model will transfer to the IETF.)

Oh, I don't know. We always seem to be strapped for cash...

> I had suggested a guideline in terms of "dispatching" to different applications.

As others have pointed out, it is also potentially helpful in certain sorts of
negotiation activities.

>       1.  Strong semantic relationship among the sub-types

Yes, this is the key.

>       2.  Likelihood of some common code for the set of sub-types

It's more like it's convenient to be able to treat them as a group. It doesn't
necessarily result in code sharing.

> current set of sub-types in the registry.

Certainly a nice to have. I would also add:

        4.  It makes sense politically, socially, or both.

The [un]happiana discussion has just as much to do with perceptions as with
reality. If putting some stuff under a new top-level helps get stuff
registered and has no real cost, then I'm all for it.
 
> Does this summarize the guidance that should be offered for justifying a new

Pretty close.

				Ned

---

From: "t.petch" <[email-masked]>
Date: Fri, 11 Nov 2011 19:51:25 +0100

----- Original Message -----
From: "Ned Freed" <[email-masked]>
To: "t.petch" <[email-masked]>
Cc: <[email-masked]>; "Apps Discuss" <[email-masked]>
Sent: Friday, November 11, 2011 1:46 AM
> > Sent: Thursday, November 10, 2011 7:59 AM
<snip>

> working from very different definitions of the word "dangerous".

On the contrary, it suggested to me that the top level type was ignored and so
it did not matter what it was; ie the behaviour I see is safe, not dangerous.
If there is a danger, it might lie in cleverer software that took more notice of
the top level typ.

Just to be clear, I tried a variety of combinations, including ones where the
type was one I would expect the MUA to understand, eg 'image', with a filename
of eg .ttf, and the MUA preferred the file name extension to the type.  Or
perhaps it treats everything not text the same.

Tom Petch

---

From: Nico Williams <[email-masked]>
Date: Fri, 11 Nov 2011 14:43:56 -0600

Maybe we should standardize file(1) magic, create a file magic
registry, and be done.

Nico
--

---

From: Larry Masinter <[email-masked]>
Date: Sat, 12 Nov 2011 02:21:58 -0800

Media type sniffing, http://tools.ietf.org/html/draft-ietf-websec-mime-sniff, standardizes the method for examining content and determining its type.

In issue http://trac.tools.ietf.org/wg/websec/trac/ticket/17  I proposed using the media type registry (after fixing it to be accurate) to be the source of the standard.

I don't see what "file magic numbers" are good for if they're not good for the standard for how to determine content-type when none is supplied.

Larry

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Nico Williams
Sent: Friday, November 11, 2011 12:44 PM
To: Apps Discuss
Subject: Re: [apps-discuss] seeking pragmatic guidelines for content-type'structure': when to go top-level?

Maybe we should standardize file(1) magic, create a file magic registry, and be done.

Nico
--
_______________________________________________
apps-discuss mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/apps-discuss