Subject: [apps-discuss] text/markdown draft-03 guide (and use-cases-00 guide)

From: Sean Leonard <[email-masked]>
Date: Fri, 17 Oct 2014 16:28:58 -0700

The following documents are about to be posted:

draft-ietf-appsawg-text-markdown-03
draft-seantek-text-markdown-use-cases-00

As discussed in the last couple of posts: to make the drafts shorter yet more informative, I split them in two. The secondary document has seven additional syntax registrations, background information, and preservation strategies. The main registration document registers the media type. They are meant to be read together; however, the secondary document makes no normative statements (no RFC 2119 key words).

draft-ietf-appsawg-text-markdown-03
   1. Introduction
   2. Markdown Media Type Registration Application
   3.  Optional Parameters
     3.1. syntax
     3.2. output-type
   4. Fragment Identifiers
   5.  Example

draft-seantek-text-markdown-use-cases-00
1. (Introduction)
     1.1. On Formats
     1.2. Markdown Design Philosophy
     1.3. Uses of Markdown
     1.4. Uses of Labeling Markdown Content as text/markdown
   2.  Strategies for Preserving Media Type and Parameters
   3.  Registration Templates for Common Markdown Syntaxes
     3.1. MultiMarkdown
     3.2. GitHub Flavored Markdown
     3.3. Pandoc
     3.4. Fountain (Fountain.io)
     3.5. CommonMark
     3.6. kramdown-rfc2629 (Markdown for RFCs)
     3.7. rfc7328 (Pandoc2rfc)
   4.  Examples for Common Markdown Syntaxes

Thank you,

Sean

---

From: Michel Fortin <[email-masked]>
Date: Fri, 17 Oct 2014 20:46:59 -0400

Last draft, I wrote those two comments on the list:

> Hence, I think the requirement that parser implementers register anything is deeply flawed. If there is a registry to be maintained, it should be maintained by the subset of users who care about having a things registered in a central place, which for the most part does not include implementers of Markdown parsers.

That comment still stands. Having a registry won't help you if no one bothers registering anything. And I don't see why I'd bother. I'm sure it'll be the same for other implementers (Fletcher seems to be of the same opinion already). Who do you plan is going to maintain that registry exactly?

> I'd question whether it is a good idea to say anything about the output. Has the desired output format anything to do with the type of the Markdown content?

I still think the "output type" shouldn't be part of the document type. It's weird.

 - - -

About the new section 4 (Fragment Identifiers), I don't understand why it's been added or how those predefined fragments identifiers are useful at all. It is potentially conflicting with the author's intent. Many Markdown processors accept this syntax:

	Orchids   {#o}
	=======

where {#o} in HTML becomes id="o", the intent being to make "o" a valid fragment identifier referring to the header.

 - - -

In your second document:

> [...] One CMS [RAILFROG] uses media types to select appropriate processing, so a media type is necessary for the safe and interoperable use of Markdown.

I think the core of the problem is this *requirement* you seem to be trying to satisfy.

Because some CMS (Railfrog, others?) uses media types to select the appropriate processing doesn't necessarily mean it is appropriate to put all the details about the parsing algorithm and the expected output type in the media type. Perhaps the CMS lacks a place to put this extra metadata, but that doesn't necessarily mean the standardized media type is the right place either.

You seem to be seeking interoperability by cataloging various Markdown implementations and then selecting the most appropriate one depending on what's available and what's known about the content and the expected output type. I believe this is a noble goal and that it'd make a good experiment or research project. It doesn't justify however dumping all this data in a media type.

For my part, I don't think it'll work so great as a way to achieve interoperability. I expect it'll work well in some cases and badly in others. Perhaps it'll fail a little less than using one or the other of the commonly used Markdown parsers for everything, but does this small improvement justify all this added complexity for everyone? And does your niche use case justify asking others to selflessly maintain a registry? I have my doubts.

Please, scale it down.

-- 
Michel Fortin
[email-masked]
https://michelf.ca

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Oct 2014 21:28:40 -0700

Speaking purely as a participant, except as noted:

On Fri, Oct 17, 2014 at 4:28 PM, Sean Leonard <[email-masked]> wrote:

> As discussed in the last couple of posts: to make the drafts shorter yet
> more informative, I split them in two. The secondary document has seven
> additional syntax registrations, background information, and preservation
> strategies. The main registration document registers the media type. They
> are meant to be read together; however, the secondary document makes no
> normative statements (no RFC 2119 key words).
>

This draft also contains all the IANA stuff except for what's been moved to
the other document.  My preference would be to keep them together (and
Section 4.2 of RFC5226/BCP26 appears to indicate the same preference); why
create the registry in one document, and populate it in another if you
already know what you're going to put there?  I guess there's nothing
"wrong" with doing it that way, it just seems unconventional.  Another
alternative would be to move the registry creation to the second document
so they're together, but then you have a document with a lot of
introduction about what Markdown is mixed with these registry actions, and
then I think we're heading in the direction of it being too big again.

<co-chair>
Along the same lines, the working group will need to decide on adopting
this second document.  So far I've seen a couple of people in favor only,
which is good, but I'd like to see more discussion before doing a Call For
Adoption.  The question I really want to know is about all the stuff you
have in Section 1 now: Does the WG feel that it wants to publish all of
this, perhaps as a basis choices made in the rest of these two documents?
My own feeling is that doing so isn't necessary, and that it really belongs
elsewhere, somewhere outside of the RFC series.
</co-chair>

Having given it only a cursory glance so far, I'd also say Section 1.2
could move to the new document, and Section 1.3 should be in its own
"Definitions" section.

I remain skeptical about the "output-type" parameter.  It seems to me the
consumer of the media type knows what output form it will need to produce;
this isn't context the generator can really impose.  I guess that means I
either didn't understand or don't agree with your previous response on this
point.  I understand that, for example, you don't envision email readers or
browsers as consumers of this media type, but what happens if such a thing
does land in one's inbox or on a web page?  Should browsers and MUAs ignore
them, return errors, what?

I'm also skeptical about what's at the end of Section 6.4, in terms of
insisting on prior authorization to make changes or updates to existing
names.  BCP26 doesn't seem to preclude it, but is there precedent for
operating a registry this way?

Finally, the syntax registry definition in the first document looks sort of
complex to me.  I browsed a random selection of existing IANA tables at
http://www.iana.org/protocols just now and all of them are pretty much in a
simple row-column format.  In contrast, the registry being created for
syntaxes here seems to be a lot more complex than what one could create
with a purely linear schema.  I can't speak for IANA but I have to wonder
if they can or would handle a registry that has a bunch of values, some of
which themselves contain a bunch of values.  You might want to ping someone
from IANA to ask about that.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 17 Oct 2014 21:30:28 -0700

On Fri, Oct 17, 2014 at 5:46 PM, Michel Fortin <[email-masked]>
wrote:

> > Hence, I think the requirement that parser implementers register
> anything is deeply flawed. If there is a registry to be maintained, it
> should be maintained by the subset of users who care about having a things
> registered in a central place, which for the most part does not include
> implementers of Markdown parsers.
>
> That comment still stands. Having a registry won't help you if no one
> bothers registering anything. And I don't see why I'd bother. I'm sure
> it'll be the same for other implementers (Fletcher seems to be of the same
> opinion already). Who do you plan is going to maintain that registry
> exactly?
>

Assuming you're referring to the registry of Markdown processors, I think
this was removed in -03.

> > I'd question whether it is a good idea to say anything about the output.
> Has the desired output format anything to do with the type of the Markdown
> content?
>

+1'd this in another post.

-MSK

---

From: "Fletcher T. Penney" <[email-masked]>
Date: Sat, 18 Oct 2014 08:37:19 -0400

> That comment still stands. Having a registry won't help you if no one bothers registering anything. And I don't see why I'd bother. I'm sure it'll be the same for other implementers (Fletcher seems…

Agreed.

> I still think the "output type" shouldn't be part of the document type. It's weird.

Agreed.

> For my part, I don't think it'll work so great as a way to achieve interoperability. I expect it'll work well in some cases and badly in others. Perhaps it'll fail a little less than using one or t…

Agreed.

I firmly believe this is headed down the path of being an overly 
complicated solution to what is actually a simple problem.

Additionally, to be pedantic about the whole thing....  If the media 
type is to be "text/markdown" then there really are no flavors. 
Everything that is not Markdown is, well, *not* Markdown.  This includes 
MultiMarkdown, PHP Markdown Extra, Github Flavored Markdown, CommonMark, 
etc.

If your goal is to allow a media type to accurately distinguish between 
all of these different, but related, syntaxes, then in my opinion there 
are two options:

1) "text/markdown", "text/multimarkdown", etc.  Just like "image/jpeg" 
is not "image/bmp", the current proposal is conflating multiple distinct 
things into a single media type.

2) rename this proposal to "text/markdown-like" (or something less 
ridiculous) and then include a "flavor" or "parser" option.  While just 
as cumbersome as the current proposal, it would be a more accurate 
representation of reality.

I think Michel's comments about Railfrog are on point -- I believe that 
part of the difference of opinion over this matter seems to be that the 
*stated purpose* of this effort (at least as I understood it) is to 
officially sanction a "text/markdown" media type to be a more universal 
identifier than "text/x-markdown".  I think this is totally fine.  I 
personally don't really care (I have no problem typing the two extra 
characters should I ever need to use this), but I also have no reason to 
object to it.

The *actual purpose* seems to be to create a complex method of 
identifying which of a long list of programs was used by one person to 
convert something that is sort of Markdown, but sort of not Markdown, 
into HTML, or something that is not HTML, so that another person can 
then wonder in frustration why a different piece of software can't 
replicate what the first person was doing. (Because you *know* this 
isn't going to work as well in practice as you hope it will in theory.)

If the goal is to get Railfrog to accurately choose a Markdown parser, 
then simply add support to Railfrog for "text/x-markdown", 
"text/x-php-markdown-extra", etc. and be done with it.  Problem solved. 
  The difficult part is not *specifying* which Markdown variant to use. 
  The difficult part is the code to match the specified variant with 
which programs are installed, and passing the appropriate variables to 
that program to get the output one desires.  This process is no more or 
less difficult regardless of which approach to media type one is using 
(though, if anything, I bet using distinct media types would actually be 
easier to code and to understand as a user.)  Therefore, I truly don't 
understand what the current proposal, in it's complex form, accomplishes.

I understand that Gruber has, in his way, consented to the concept of 
"text/markdown."  Has he consented to the idea of using "text/markdown" 
to refer to a bunch of things that are not Markdown?  Based on past 
history, I would bet not -- but you never know and clearly I cannot 
speak for him.  I would recommend asking him.  I would not support the 
idea of using "text/multimarkdown" to refer to things that are not 
MultiMarkdown.

Please.  Simplify this.  Create "text/markdown".  No options.  No 
parameters.  Everyone can get behind it.  If Railfrog, or someone else 
really needs more detail, they can use "text/x-multimarkdown", etc. 
until there is enough use to justify a separate IETF proposal.

Fletcher

-- 
Fletcher T. Penney
[email-masked]

---

From: Sean Leonard <[email-masked]>
Date: Sat, 18 Oct 2014 06:30:28 -0700

On 10/17/2014 9:28 PM, Murray S. Kucherawy wrote:
> On Fri, Oct 17, 2014 at 4:28 PM, Sean Leonard <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     As discussed in the last couple of posts: to make the drafts
>     shorter yet more informative, I split them in two. The secondary
>     document has seven additional syntax registrations, background
>     information, and preservation strategies. The main registration
>     document registers the media type. They are meant to be read
>     together; however, the secondary document makes no normative
>     statements (no RFC 2119 key words).
>
> This draft also contains all the IANA stuff except for what's been 
> moved to the other document.  My preference would be to keep them 
> together (and Section 4.2 of RFC5226/BCP26 appears to indicate the 
> same preference); why create the registry in one document, and 
> populate it in another if you already know what you're going to put 
> there?  I guess there's nothing "wrong" with doing it that way, it 
> just seems unconventional.  Another alternative would be to move the 
> registry creation to the second document so they're together, but then 
> you have a document with a lot of introduction about what Markdown is 
> mixed with these registry actions, and then I think we're heading in 
> the direction of it being too big again.

Splitting the document had two goals in mind:
1) The text in the secondary document is not "as normative" as the first 
document. In the first document, it says that the original [MDSYNTAX], 
"Original", MUST be supported. The others do not have to be supported.

2) If we include seven additional registrations in the first document, 
it becomes "too long". People thought the first document was getting too 
long.

> I remain skeptical about the "output-type" parameter.  It seems to me 
> the consumer of the media type knows what output form it will need to 
> produce; this isn't context the generator can really impose.

As the document revision went on, it became clear to me that a primary 
consumer of this media type is **Markdown editors**, not Markdown 
processors. When I picture a Markdown editor, I envision something like 
the attached puc.png. Just [Google it][] and you will see many prominent 
examples.

[Google it]: https://www.google.com/search?q=markdown+editor

Markup formats like HTML are publishing formats. Markdown is a writing 
format. People exchange Markdown content because they are 
collaboratively writing something. When they are done, it gets converted 
to text/html, application/pdf, text/plain (RFCs), e-Book, etc.

Markdown is a format of record only in the same way that shorthand is to 
a news article, Adobe InDesign (or Quark) files are to magazines, or 
nroff (or xml2rfc) files are to RFCs. It's the historical building 
block, and therefore is interesting to historians, as well as to people 
during the creative process.

We could rename the "output-type" parameter to "output", or simply, 
"preview". I could even rewrite Section 3.2 so that it more explicitly 
calls this "live preview" aspect out. That would capture all of the same 
semantics without confusing people about "forcing" the Markdown into one 
particular thing. Of course once a receiver gets the Markdown content, 
the receiver can choose how to turn it into whatever output format the 
receiver wants.

So maybe I should address the WG on this point: what is the feeling 
about renaming "output-type" to "preview", with refers to the live 
preview, or "right-hand side", of the Markdown editor?

>   I guess that means I either didn't understand or don't agree with 
> your previous response on this point.  I understand that, for example, 
> you don't envision email readers or browsers as consumers of this 
> media type, but what happens if such a thing does land in one's inbox 
> or on a web page?

With the prior explanation (and Section 1.2) in mind, easy: pass it to a 
Markdown editor. The Markdown editor could be desktop-based 
(MarkdownPad, emacs, vi), or could be some web app (Dillinger, 
StackEdit, Google Docs+Markdown).

Markdown is not a publishing format. Having a web browser or e-mail 
client merely render it to HTML and displaying the HTML would do no 
good. (If that's what the sender wanted, the sender ought to send it as 
HTML in the first place.) You want to open Markdown in such a way that 
you can edit it, or at least, see the source text with some syntax 
highlighting.

It's just like a forum post, where you click "Edit". You expect the same 
kinds of widgets to pop-up.

Sean

---

From: Sean Leonard <[email-masked]>
Date: Sat, 18 Oct 2014 06:35:02 -0700

On 10/17/2014 9:28 PM, Murray S. Kucherawy wrote:
> insisting on prior authorization to make changes or updates to 
> existing names. BCP26 doesn't seem to preclude it, but is there 
> precedent for operating a registry this way?

Honestly, not sure. I don't have a strong feeling on this point, so 
whichever way the WG wants to go on that is fine. Overall though we 
should anticipate that the syntax registrations will change over time if 
they are to have continuing value. The purpose was to avoid malicious 
bots spamming the registration webform.

>
> Finally, the syntax registry definition in the first document looks 
> sort of complex to me.  I browsed a random selection of existing IANA 
> tables at http://www.iana.org/protocols just now and all of them are 
> pretty much in a simple row-column format.  In contrast, the registry 
> being created for syntaxes here seems to be a lot more complex than 
> what one could create with a purely linear schema.  I can't speak for 
> IANA but I have to wonder if they can or would handle a registry that 
> has a bunch of values, some of which themselves contain a bunch of 
> values.  You might want to ping someone from IANA to ask about that.

On this point, we should all step back and note that the purpose of a 
syntax registration is to be easier to register than an Internet media 
type. There is a registry of Internet media types--and each media type 
has a very detailed registration template that must be followed. Note 
that for Internet media types there are also a lot of optional fields, 
but RFC 6838 does not explicitly mark them as {optional}.

Ditto for URNs and URIs. There are simple tables of identifiers, but the 
semantic meaning of the identifier is potentially very intricate. There 
is no difference here. Actually, a significant quantity of the IANA 
tables have three columns:
  identifier | prose-description-of-identified-thing | [references]

The [references] are necessarily intricate--like RFCs.

In draft-03, the Markdown Syntaxes Registry is supposed to be really 
easy to get things registered (First-Come, First-Served -- compare with 
the complexities of media type registration!), but the information 
requested is supposed to be in a sufficiently structured format to 
ensure GIGO--good in, good out (and garbage in, garbage out). If someone 
registers a poor or useless syntax identifier, it's their problem.

We can make the registration template simpler, but then to ensure 
quality we probably need Expert Review, which encumbers the process. 
What ends up happening when the template is simpler is that the 
commentary/prose fields get stuffed with more semantic discussion, 
because the semantics weren't expressible in the formal template fields 
in the first place.

Sean

---

From: Michel Fortin <[email-masked]>
Date: Sat, 18 Oct 2014 11:02:03 -0400

Le 18-oct.-2014 à 9:30, Sean Leonard <[email-masked]> a écrit :

> So maybe I should address the WG on this point: what is the feeling about renaming "output-type" to "preview", with refers to the live preview, or "right-hand side", of the Markdown editor?

But even for live previews, the output type is a mere technical detail of the processing required to produce the preview in the editor. It doesn't really matter if one editor converts to HTML and then to a screen representation and another editor converts to Latex and then to a screen representation, as the same content is shown in the preview.

-- 
Michel Fortin
[email-masked]
https://michelf.ca

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 18 Oct 2014 17:10:00 +0000

This and an earlier message today set me thinking. I am not sure the result is significant to this document, but allow me to present it.

Markdown is originally derived from conventional email formatting. And it really does look like email - I have seen countless messages that may be syntactically valid markdown source, and I expect so have everyone else on this list.

But those email messages don't have content-type text/markdown. They have type text/plain, perhaps with format=flowed. Markdown, as used in email, is a flavour of text/plain, and is properly described as plain so that renderers work in the  traditional manner while a markdown-capable receiver does the right thing.

I don't know what happens of one sends a message marked text/markdown. Perhaps most readers handle it as plain? I think this is a worthwhile experiment. I will send something to this list when I am back at my desk.

Arnt

---

From: Graham Klyne <[email-masked]>
Date: Sat, 18 Oct 2014 21:03:52 +0100

On 18/10/2014 14:30, Sean Leonard wrote:
> 2) If we include seven additional registrations in the first document, it 
> becomes "too long". People thought the first document was getting too long.
Speaking for myself, I think you misunderstand the concern.  "Too long" is often 
a proxy for "too complicated" or "too detailed" (I'm not sure I used that term, 
but if I did, that's what I would have meant.)  The critical thing is how much 
text a developer must read through to understand the basic concept.  Too much 
background detail/justification and too much complexity in the specified 
mechanism (both things I've previously argued against in earlier drafts) fall 
into these categories.

Having a list of registrations may technically increase the length of the 
document, but not in the important sense noted above, as they should be clearly 
distinct from the core mechanism description.  So I think it's fine for the 
registrations to be in the primary document.

(More generally, I think the the appswg should consider just the core document 
describing the content type, parameter registry and initial registrations.  I 
think the other document could be submitted as an independent stream 
informational document (or even as an IESG non-WG submission?).  I, for one, 
have little interest in reviewing this kind of additional content - but if you 
think it's useful to publish then I'm OK with that if it doesn't consume WG 
bandwidth.)

#g
--

---

From: Michel Fortin <[email-masked]>
Date: Sat, 18 Oct 2014 17:39:24 -0400

Le 18-oct.-2014 à 8:37, Fletcher T. Penney <[email-masked]> a écrit :

> I understand that Gruber has, in his way, consented to the concept of "text/markdown."  Has he consented to the idea of using "text/markdown" to refer to a bunch of things that are not Markdown?  Based on past history, I would bet not -- but you never know and clearly I cannot speak for him. I would recommend asking him. I would not support the idea of using "text/multimarkdown" to refer to things that are not MultiMarkdown.

I'd tend to define "text/markdown" as identifying documents using the Markdown syntax and its compatible derivatives. "Compatible" in the sense that they mostly follow the rules of the official syntax document on daringfireball.net.

Pragmatically, you can't define it that much differently from how people have been using the .md and .markdown file extensions, because everyone knows that those extensions are going to be mapped to "text/markdown" eventually (it's just obvious) regardless of what the RFC says. I think we should simply formalize the obvious.

-- 
Michel Fortin
[email-masked]
https://michelf.ca

---

From: Ned Freed <[email-masked]>
Date: Sat, 18 Oct 2014 17:10:35 -0700 (PDT)

> > Hence, I think the requirement that parser implementers register anything
> > is deeply flawed.

I see no requirement that anything be registered. The syntax parameter where
registered identifiers appear is optional. Additionally, I don't see a
requirement that the identifier you put in there MUST be registered.

> > If there is a registry to be maintained, it should be
> > maintained by the subset of users who care about having a things registered in
> > a central place, which for the most part does not include implementers of
> > Markdown parsers.

As opposed to using a neutral service that has been maintaining many comparable
registries for decades? This makes no sense to me at all.

> That comment still stands. Having a registry won't help you if no one bothers
> registering anything. And I don't see why I'd bother. I'm sure it'll be the
> same for other implementers (Fletcher seems to be of the same opinion already).
> Who do you plan is going to maintain that registry exactly?

It's an IANA registry so IANA maintains it. Along with hundreds of others,
including the media types registry.

You're worrying about issues that simply don't exist. Which doesn't mean the
registry necessarily makes sense, but operational issues of this sort are not a
factor.

> I stHill think the "output type" shouldn't be part of the document type. It's
> weird.

I think there's a fairly close parallel to be found in application/octet-stream
name parameter that was originally part of MIME.

The name parameter was used to specify a filename to associate with the object
in the event one was needed. But such a name isn't an attribute of the object
itself, rather, it's part of the context surrounding a particular use of that
object.

As such, it was replaced by the filename parameter that's part of the
content-disposition field. (Note that this made filename labels available
to any type.)

This seems to me to be very similar situation to output-type. The output-type
isn't a property of the markdown object, it's a property of the message or
whatever context that contains the object.

And similarly, if there's a case for providing this information, it seems
to me that it makes sense for it to be a content-disposition parameter.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sat, 18 Oct 2014 17:47:35 -0700 (PDT)

> them, return errors, what?

As I indicated in my previous message, I agree that output-type doesn't make
sense as a content-type parameter.

But that doesn't mean providing the information makes no sense. While it is
true that the consumer has to have the final say, I can easily imagine
scenarios where conversion to a particular type provides a better result,
and it would be helpful to be able to provide that information.

But the place for that is content-disposition, not content-type.

				Ned

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 19 Oct 2014 23:41:27 -0700

> But the place for that is content-disposition, not content-type.

I think I'd be fine with meta-data like this being presented in a way
that's clearly no more than a recommendation.  As written, "output-type" in
this draft sounds like something the consumer is basically required to do;
it reads normatively in -03, though it doesn't use RFC2119 language.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 19 Oct 2014 23:48:09 -0700

On Sat, Oct 18, 2014 at 5:10 PM, Ned Freed <[email-masked]> wrote:

> anything

I think this might have been a reference to the "processor" argument, which
was present in -02 but was removed in -03.  Moot now.

> bothers

I'm not so sure.  The work we're doing on draft-ietf-appsawg-uri-scheme-reg
was undertaken specifically because lots of people were inventing, and
putting into use, but not registering with IANA, URI schemes.  I think the
concern is the same here: If it's a pain to register, or the benefit is
virtually non-existent, people won't bother.  To me, that either means the
definition or purpose of the registry is broken, or the bar for
registrations is too high.  It's not a matter of the load on or
capabilities of IANA; these are problems on our side of that fence.

> [...]

I commented on this in reply to Ned's other post.  And I agree that
re-using or extending Content-Disposition is a good thing for this work to
consider.

-MSK

---

From: Graham Klyne <[email-masked]>
Date: Mon, 20 Oct 2014 08:17:32 +0100

> whatever context that contains the object.

+1

#g

---

From: Carsten Brandt <[email-masked]>
Date: Sat, 25 Oct 2014 23:08:21 +0200

> I'd tend to define "text/markdown" as identifying documents using the Markdown syntax and its compatible derivatives. "Compatible" in the sense that they mostly follow the rules of the official syn…

I think an important part of this document has to be a definition of
what it understands as Markdown.

Imo Markdown is a set of languages that share the Gruber Markdown syntax
as a common subset. There is no such thing as a standard or
specification because markdown is just plaintext with some additional
rules. Parsing Markdown is, when it comes to edge cases more like
natural language processing than following a spec (I have written a
markdown parser myself [1]). A good parsing result is in these cases the
one that most people would interpret from the written plain text.
A result from this is that every markdown parser will interpret a given
document in a slightly different way when the document is complex
enough. See babelmark example [2].

So the important part of the media type is to describe the content of a
file as "text/markdown" letting the consumer application know that it
can be presented by parsing it at least for the syntax described by
Gruber. As markdown is designed to look in text already similar to the
rendering result it should be generally fine to interpret markdown as
plain text and present it that way. If there is a known flavor attached
as an optional attribute the presenting application may choose to render
additional syntax elements according to the flavor. A library may
therefor support mulitple flavors to render markdown files as for
example my parser does [1].

Different flavors may be registered if they are common but it should be
allowed to use application or library specific flavors to indicate usage
 of a customized flavor which is common to the nature of markdown.

Output format is nothing that describes the content of the markdown file
or significatnly influences the way the document is presented so it
should not be part of the media type.

best regards,
Carsten

[1] https://github.com/cebe/markdown#readme
[2]
http://johnmacfarlane.net/babelmark2/?normalize=1&text=1.+a%0A-+x%0A-+y%0A%0A2.+b%0A-+x%0A-+y%0A

-- 
mail:  [email-masked]
phone: +49 176 / 96 52 999 7
www:   http://cebe.cc/
pgp:   http://cebe.cc/cebe_pub.asc
skype: skype://cebe08
irc:   cebe @ irc.freenode.net

---

From: Sean Leonard <[email-masked]>
Date: Sat, 25 Oct 2014 22:08:02 -0700

> But the place for that is content-disposition, not content-type.

In order to keep this draft moving, I'd like to focus on specific 
technical issues.

Based on feedback thus far it seems that there is some support for 
conveying an output media type, but it should not be part of the 
text/markdown media type's parameters.

Proposal:

    Content-Disposition: ... preview-type parameter. The preview-type
    indicates the media type (and optional parameters) that a receiving
    application may present to a user as a "preview" of the content. The
    preview-type parameter is advisory and does not dictate or constrain
    the semantics of the content in any way, but rather provides a
    useful way to duplicate the sender's intent with regard to how the
    content should look when processed. The syntax of the preview-type
    parameter is the same as the field value of the RFC 2045
    Content-Type header, i.e., a media type and optional parameters,
    delimited by semicolons.

The purpose of this parameter is for Markdown content. Obviously, it 
could be used for other formats, but I am not concerned with those at 
this time.

(The choice of "preview-type" for a parameter name is meant to 
distinguish this from other kinds of previews, such as "preview-image", 
"preview-thumbnail", "preview-soundbite", etc.)

1) Can we get agreement on this preview-type parameter?

2a) Shall this preview-type parameter be registered in the text/markdown 
RFC? Or:

2b) Shall this preview-type parameter be registered in a new, SHORT, 
informational RFC? (RFC 2183 does not require this; the standard appears 
to be First-Come, First-Served with a simple e-mail to IANA.)

3) How shall the text/markdown registration discuss the applicability of 
the preview-type parameter for Markdown editors (or other Markdown 
applications)?

Regards,

Sean

---

From: Michel Fortin <[email-masked]>
Date: Sun, 26 Oct 2014 07:54:27 -0400

Le 26-oct.-2014 à 1:08, Sean Leonard <[email-masked]> a écrit :

> "The preview-type parameter is advisory and does not dictate or constrain the semantics of the content in any way, but rather provides a useful way to duplicate the sender's intent with regard to how the content should look when processed."

An output type is a very poor way to define "how the content should look". What does it mean that the content should look like HTML? or look like Latex? or look like PDF?

I see no justification for any presentational attribute. The media type is there to tell you what that content is. How it is presented or processed is up to the user of that document.

-- 
Michel Fortin
[email-masked]
https://michelf.ca

---

From: "Fletcher T. Penney" <[email-masked]>
Date: Sun, 26 Oct 2014 14:42:19 -0400

> text/markdown media type's parameters.

It may be true that there is support for it, but there is also 
opposition to an output media type, regardless of what you call it.

Perhaps I am just dumb.  It seems to me that the purpose of a media type 
is to say, "What I am about to send/attach/etc. is digital content that 
represents a picture/sound/movie/file/etc."  I did not understand that a 
media type should then say, "Now that you have the file, here's what you 
have to do with it."

In this case, the content is plain text. The plain text happens to 
follow a set of rules/guidelines/whatever that allow some programs to do 
something with that text (e.g. convert to HTML).  *Whether* that happens 
is up to the user receiving the file, not the person sending it.

> 1) Can we get agreement on this preview-type parameter?

Unanimous agreement?  NO.  As I (and others) have said, it just doesn't 
make sense and isn't necessary.  That doesn't seem to be persuading you 
from moving forward with the mission of creating a needlessly 
complicated media type however.

It feels as though my difficulty understanding this proposal is that 
there is a "hidden" agenda that I am unaware of.  It's like being asked 
to pack a suitcase without knowing the destination.

Please explain to those of us who clearly don't understand the need for 
all of this what we are missing.  In aggregate there have been 100's of 
person-hours spent on discussing this.  To my knowledge, there is not a 
single instance of something like this "in the wild" that needs 
standardization.  We're creating a solution where no problem exists.

If "text/x-markdown" were being used all over the place, in countless 
incompatible and creative ways, then it would make sense to agonize over 
the best way to standardize.  But we're trying to standardize something 
where there's not even a working prototype, much less competing versions 
that are creating confusion and incompatibility.

> 2a) Shall this preview-type parameter be registered in the text/markdown

My vote?  NO.

> to be First-Come, First-Served with a simple e-mail to IANA.)

My vote?  NO.

> the preview-type parameter for Markdown editors (or other Markdown

My vote?  It doesn't.  The programmers of Markdown editors can do 
whatever they like with it, including the most likely thing should this 
come to exist (IMO)...  they ignore it.

I've been working with Markdown for a long time. I released one of the 
early Markdown derivatives (second after PHP Markdown Extra??).  I 
released one of the first dedicated Markdown/MultiMarkdown text editors. 
  And not once have I ever needed something like this, nor do I foresee 
any need to support it in the future.

Identifying content as "text/markdown" is a nice convenience, but hardly 
necessary ("text/x-markdown" works just fine).  Anything more 
complicated than that is really unnecessary in my view.  If I'm editing 
a text document, and want to preview it as HTML, I'm going to be pretty 
upset if the app says, "Sorry -- the author wants this previewed as a 
PDF, so I can't make this HTML as you requested."  It just doesn't make 
sense.

I understand that one can envision all sorts of hypothetical cases where 
a complex media type would be useful.  But those things don't exist, and 
IMO the logic for that behavior belongs in the application, not in a 
complicated media type definition.

Fletcher

---

From: "Fletcher T. Penney" <[email-masked]>
Date: Sun, 26 Oct 2014 14:42:52 -0400

Agreed.

On 10/26/14, 7:54 AM, Michel Fortin wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Ned Freed <[email-masked]>
Date: Sun, 26 Oct 2014 13:34:56 -0700 (PDT)

> media type should then say, "Now that you have the file, here's what you

Indeed. The name parameter was removed from application/octet-stream and moved
to content-disposition because it didn't fit this model.

But this is argument against preview-type as a media type parameter. It 
says nothing about it being defined as a content-disposition parameter, which
is what's being proposed here.

> is up to the user receiving the file, not the person sending it.

Here we disagree. The content is *not* plain text. It's text containing a type
of markup intended to be interpreted in a particular way. The fact that the
markup is defined in a fairly loose way doesn't make it unformatted a la
text/plain.

There's also lots of running code here. For example, comma separated values
have their own media type, but are also a subset of plain text.

> to pack a suitcase without knowing the destination.

The only agenda I see here is a desire to provide a label for something
that is different from any existing media type along with a parameter
completely separate from the media type that can be used by content
creator to suggest handling to someone processing the content.

> that are creating confusion and incompatibility.

You're conflating the issues with the parameter with issues with the type
itself, and also seem to be missing the fact that this isn't a proposal
to have preview-type as a content-disposition parameter.

> My vote?  NO.

(We don't vote.)

> My vote?  NO.

Have you read RFC 2183 to see whether or not this fits as a content-disposition
parameter? And if you have, why doesn't it fit?

Personally, I have no problem with the parameter although the word "preview"
doesn't seem to fit very well, and my preference is to keep the parameter in
this RFC.

				Ned

---

From: Sean Leonard <[email-masked]>
Date: Sun, 26 Oct 2014 19:26:08 -0700

On Oct 26, 2014, at 1:34 PM, Ned Freed <[email-masked]> wrote:

> But this is argument against preview-type as a media type parameter. It says nothing about it being defined as a content-disposition parameter, which

That is correct, and that is what I am proposing. This is not a media type parameter; it’s a Content-Disposition parameter.

RFC 2183 says:
   MIME specifies a standard format for encapsulating multiple pieces of
   data into a single Internet message. That document does not address
   the issue of presentation styles; it provides a framework for the
   interchange of message content, but leaves presentation issues solely
   in the hands of mail user agent (MUA) implementors.

   Two common ways of presenting multipart electronic messages are as a
   main document with a list of separate attachments, and as a single
   document with the various parts expanded (displayed) inline. The
   display of an attachment is generally construed to require positive
   action on the part of the recipient, while inline message components
   are displayed automatically when the message is viewed. A mechanism
   is needed to allow the sender to transmit this sort of presentational
   information to the recipient; the Content-Disposition header provides
   this mechanism, allowing each component of a message to be tagged
   with an indication of its desired presentation semantics.

> markup is defined in a fairly loose way doesn't make it unformatted a la

Right.

Note that any textual data can be relabeled as text/plain; this changes the textual data’s semantics. If you serve text/html as text/plain, it will show the source as plain text. There is differential handling involved.

> creator to suggest handling to someone processing the content.

That is correct.

Let me give two concrete examples:
Example #1: The Markdown Editor. That’s the example given in draft-03 (Section 1.3), and one that we are probably all familiar with. On the left-hand side, you see syntax highlighting on plain text. On the right-hand side, you see a preview of the processed content.

It is useful to use Markdown-like syntaxes for things other than text/html. For example, if you write a scholarly article with it, you might want application/pdf output (which looks sort of TeX-like). Or if you are writing an RFC, you might want text/plain output (in RFC style).

Assuming that your Markdown editor is modular, it’s quite easy to treat the right-hand side as a generic Web Browser object thing, and feed the web browser object some kind of labeled content (data:application/pdf,……). Then the component will magically present the right stuff.

This leads me directly to Example #2:
https://github.com/cabo/kramdown-rfc2629/blob/master/examples/stupid-s.mkd

In this case, GitHub recognizes that the thing is some kind of Markdown, so it shows a preview of it…a really poor one at that. Firstly, it’s not identified as kramdown, so GitHub runs its GitHub Flavored Markdown processor on it. Sure, you kind of get the gist, but the YAML is all screwed up and the {: … } attributes are showing.

Ideally, if this were tagged as “preview this as an RFC”, we could get something slightly more RFC-like. Note that the option to get the raw view is always there, and guess what, it’s the same content but it’s relabeled as text/plain (but obviously this is source code view, not text/plain-as-RFC-view).

Anyway one upside of putting this in the Content-Disposition header is that the media type registration template itself is one parameter shorter, and I take it that is a goal.

> doesn't seem to fit very well, and my preference is to keep the parameter in

Definitely open to suggestions on parameter names (to keep score: currently the ones on the table are “preview-type” and “output-type”).

-Sean

---

From: Michel Fortin <[email-masked]>
Date: Sun, 26 Oct 2014 23:22:44 -0400

> Let me give two concrete examples:

:-) I think examples are a good way to validate if this is useful. Let's see...

> It is useful to use Markdown-like syntaxes for things other than text/html. For example, if you write a scholarly article with it, you might want application/pdf output (which looks sort of TeX-lik…

Note that someone might also want more than one output to publish the thing in various formats. Also, why a "text/plain" output would be in RFC style? Personally I'd expect to see the raw markdown content in text/plain.

> Assuming that your Markdown editor is modular, it’s quite easy to treat the right-hand side as a generic Web Browser object thing, and feed the web browser object some kind of labeled content (data…

But how does the editor app get the preview-type in the first place? The only way that could potentially work is if the editor edits the file directly via HTTP (with a special WebDAV server maybe?), otherwise there is no Content-Disposition for local files.

It is also unlikely that an editor app will support more than one preview type (is there an app that does this currently?). Generally in Markdown editor apps the side view is there to validate that what you write is properly interpreted by the parser (it also makes the document easier to read should you want to review your prose). That said, if someone wants to preview the result within a particular presentation, he'll probably need much more than an "output type" to achieve that presentation. He'll want style sheets, fonts, etc., much more than a Markdown document in short. I think storing the output type is either too much information (not needed to validate the syntax and reread what you wrote) or not not even close to be enough (not enough to do a preview of the final result). So I don't see it as very useful in this scenario.

> In this case, GitHub recognizes that the thing is some kind of Markdown, so it shows a preview of it…a really poor one at that. Firstly, it’s not identified as kramdown, so GitHub runs its GitHub F…

Again, same fundamental problem as above with the file system: Git doesn't convey media types attributes, nor Content-Disposition. So there's no way Github could know how to parse this file better (other than having a kramdown-specific file extensions). Beside that, would Github be willing to run kramdown and potentially many other third-party Markdown parsers on their server and keep them up to date? This might require a security- and performance-focused review I suppose. I suspect they'd rather integrate the missing features in Github Flavored Markdown instead, as there is much less to worry about this way.

> Ideally, if this were tagged as “preview this as an RFC”, we could get something slightly more RFC-like. Note that the option to get the raw view is always there, and guess what, it’s the same cont…

If Github wanted to support various presentational themes for viewing Markdown files (such as RFC-like), they could already do it in a proprietary way (go to a file, choose a theme, done). So far they haven't, and I don't expect that's something they want to do.

> Anyway one upside of putting this in the Content-Disposition header is that the media type registration template itself is one parameter shorter, and I take it that is a goal.

Getting it out of the media type is a good thing. But I'm not convinced it should be part of Content-Disposition either. I don't see anything in those examples convincing me that preview-type would actually be useful to add this attribute to Content-Disposition.

-- 
Michel Fortin
[email-masked]
https://michelf.ca

---

From: Graham Klyne <[email-masked]>
Date: Mon, 27 Oct 2014 12:04:58 +0000

Sean,

I'm agnostic about whether or not it's useful to define a content-disposition 
parameter.  I don't see a particular use for it, but I don't see a problem or 
have any deep objection.

But it seems to me that the examples you give are not really about content 
disposition or output format, but refer to how the author intends the Markdown 
text to be interpreted (i.e. what variation(s) of Markdown is being assumed by 
the author).  As such, these examples are of things which I would expect to be 
communicated by a (simple!) "flavors" (or whatever it's called) parameter on the 
content type.

#g
--

---

From: "Fletcher T. Penney" <[email-masked]>
Date: Wed, 29 Oct 2014 21:32:27 -0400

FYI,

This project has grown to consume more of  my time and energy than I have to spare.  I am increasingly convinced that the end result is unlikely to be substantively different than the initial proposal.  I've explained my concerns about how I fear that certain aspects of this proposal could potentially harm the Markdown community by increasing confusion amongst users about what is or is not Markdown.  Hopefully that will not come to pass.

I will unsubscribe from the email lists.

I have emailed Sean Leonard to explain this as well, and to ask him to remove me as a reviewer of this project.

Thank you,

Fletcher Penney

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 30 Oct 2014 07:36:41 +0000

I reiterate my earlier proposal for an RFC consisting of four sentences plus IANA and IETF boilerplate.

Arnt

---

From: Michel Fortin <[email-masked]>
Date: Thu, 30 Oct 2014 16:32:59 -0400

> I reiterate my earlier proposal for an RFC consisting of four sentences plus IANA and IETF boilerplate.

I think I'd like that too.

-- 
Michel Fortin
[email-masked]
https://michelf.ca