Subject: [xml2rfc] Tables in XML V3 - A downgrade

From: Anders Rundgren <[email-masked]>
Date: Fri, 13 Sep 2019 16:18:03 +0200

Hi,

When switching to XML V3 the tables looked as bad as they always have so I reverted to my V2 <artwork> based design using the since long perfectly working table characters:
https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-08#appendix-B

I would like RrfcMarkup <table> to use these characters instead of 7-bit ASCII.
The conversion to XML V3 also caused lines to fold although they fitted (with auto undenting) using XML V2.

thanx
Anders

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 13 Sep 2019 17:00:42 +0200

Hi Anders,

On 2019-09-13 16:18, Anders Rundgren wrote:
> Hi,
> 
> When switching to XML V3 the tables looked as bad as they always have
> so I reverted to my V2 <artwork> based design using the since long
> perfectly working table characters: 

Which won't result in HTML tables in the HTML output.

Yes, the v3 text output for tables need more work.

That requires an update to RFC 7997, which can be done, of course, but
isn't part of the current v3 work.

> The conversion to XML V3 also caused lines to fold although they
> fitted (with auto undenting) using XML V2.

I'll need an XML source file to debug that.

Regards,

	Henrik

---

From: Anders Rundgren <[email-masked]>
Date: Fri, 13 Sep 2019 17:25:50 +0200

On 2019-09-13 17:00, Henrik Levkowetz wrote:
Hi Henrik,
> isn't part of the current v3 work.

Well, RFC 7997 only talks about Unicode in author-provided data which is not the case here.
The same goes for bullets lists.

It is not a big deal though, <artwork> saved my day anyway :-)

Cheers,
Anders

---

From: Julian Reschke <[email-masked]>
Date: Fri, 13 Sep 2019 18:45:48 +0200

On 13.09.2019 17:25, Anders Rundgren wrote:
> ...
> ...

I think we need to look at the role of the plain text output in the
future. The original plan was to make it as bare as possible, not even
paginated. Not sure why we deviated from that.

The format(s) to look for are IMHO HTML for on-screen, and HTML or PDF
for print. AFAIC, plain text is here because textual diffs work well,
and because it might have advantages when copy/pasted into plain text
email (or source code, or....). Making tables prettier doesn't seem to
be relevant for that (or might actually hurt).

> ...

IMHO the wrong decision. If it's a table, use a table. Otherwise you'll
end up with ugly output in the other output formats.

Best regards, Julian

---

From: Charlie Perkins <[email-masked]>
Date: Fri, 13 Sep 2019 09:52:22 -0700

Hello folks,

> paginated. Not sure why we deviated from that.
That looks a lot like penalizing people who like plain text. Like me, 
for instance.

On the other hand it does seem to explain a lot of problems I've been 
having.

Regards,
Charlie P.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Sep 2019 06:59:03 +0200

> That looks a lot like penalizing people who like plain text. Like me,

See <https://www.rfc-editor.org/rfc/rfc7990.html#section-7.3> and
earlier <https://www.rfc-editor.org/rfc/rfc6949.html#section-3.3>.

Best regards, Julian

---

From: Anders Rundgren <[email-masked]>
Date: Sat, 14 Sep 2019 07:08:13 +0200

> paginated. Not sure why we deviated from that.

Paginated RfcMarkup is rather reducing readability and browsers can print fairly well these days.

RfcMarkup seems to be the current standard for communicating RFC.

Making HTML the standard probably requires more work and PIs.  XHTML is deprecated.

New problems:
------------
BTW, my recent XML V3 submission did not generate an HTML version at all!
My previous XML V2 submissions did this automatically.

The HTML output for Web tool doesn't produce any output. It says it doesn't find the file :-(
The URL input began to fail:
https://raw.githubusercontent.com/cyberphone/ietf-json-canon/gh-pages/xmlv3/draft-rundgren-json-canonicalization-scheme.xml

Cheers,
Anders

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Sep 2019 07:25:55 +0200

> print fairly well these days.

Print plain text?

> Making HTML the standard probably requires more work and PIs.  XHTML is

The "standard" is XML, in that the XML document is normative. HTML will
be generated from it.

I don't get your point about PIs, nor the one about XHTML.

> New problems:

My local install of xml2rfc dies with a stack trace, one issue is the
use of
<https://greenbytes.de/tech/webdav/rfc7991.html#element.dl.attribute.spacing>
which has been removed/changed but not yet documented in rfc7991bis.

Even after removing those, xml2rfc dies with:

> Traceback (most recent call last):
>   File "/bin/xml2rfc", line 10, in <module>
>     sys.exit(main())
>   File "/usr/lib/python2.7/site-packages/xml2rfc/run.py", line 595, in main
>     writer.write(filename)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 253, in write
>     html_tree = self.html_tree()
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 231, in html_tree
>     html_tree = self.render(None, self.root)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 309, in render
>     res = func(h, x)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 625, in render_rfc
>     self.render(body, c)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 309, in render
>     res = func(h, x)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 1546, in render_front
>     entry(dl, 'Workgroup', wg.text)
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 1511, in entry
>     dl.append( build.dd(*values, classes=cls))
>   File "/usr/lib/python2.7/site-packages/xml2rfc/writers/html.py", line 93, in __call__
>     elem = super(ClassElementMaker, self).__call__(tag, *children, **attrib)
>   File "src/lxml/builder.py", line 226, in lxml.builder.ElementMaker.__call__
> TypeError: bad argument type: NoneType(None)

You should submit a bug report.

Best regards, Julian

---

From: Anders Rundgren <[email-masked]>
Date: Sat, 14 Sep 2019 07:37:39 +0200

> See <https://www.rfc-editor.org/rfc/rfc7990.html#section-7.3> and

"A Byte Order Mark (BOM) will be added at the start of each file"

I accidentally got a BOM in a file which cause GitHub/Jenkins to fail.
Not even Windows' ultra-primitive Notepad needs BOM these days.

Thanx,
Anders

---

From: Anders Rundgren <[email-masked]>
Date: Sat, 14 Sep 2019 08:03:29 +0200

Hi Julian,

>> print fairly well these days.

I just thought that pagination was intended for that purpose.
If not pagination has no purpose.

> The "standard" is XML, in that the XML document is normative. HTML will

Sure, XML is the standard for creating RFC, I was rather referring
to links to published RFCs used everywhere.

HTML is currently NOT generated for V3 submissions.

> I don't get your point about PIs, nor the one about XHTML.

Since you as an author would like your precious text to be rendered
as good as possible, I guess that SVG would not be a part of RfcMarkup
making "line-art" variants a necessity.

PIs could be used to keep the XML intact.

XHTML is a deprecated W3C standard currently used by RFCs.  It is not
a problem but switching to HTML5 is preferable.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Sep 2019 10:17:55 +0200

On 14.09.2019 08:03, Anders Rundgren wrote:
>>>>> Well, RFC 7997 only talks about Unicode in author-provided data
>>>>> which is
>>>>> not the case here.

With all due respect, it's pretty easy to make absolute statements like
these.

The current plan was agreed upon after long mailing list discussions,
many years ago. It's a compromise, yes.

People in fact do like pagination, even when not printing (I don't
belong to that group). It was very hard to get to where we are right
now, where pagination is just a service for certain output formats
(originally planned for PDF only), not something inherent in the
canonical RFC format.

That said, I still don't understand what you're referring to when you
say "Paginated RfcMarkup is rather reducing readability and browsers can
print fairly well these days."

> HTML is currently NOT generated for V3 submissions.

I'm sure it's planned. And yes, I agree that it should happen ASAP.

> making "line-art" variants a necessity.

I do not necessarily agree that I would always supply line art as well.

> PIs could be used to keep the XML intact.

I still don't get that.

> a problem but switching to HTML5 is preferable.

"currently used by RFCs" where?

Best regards, Julian

---

From: Anders Rundgren <[email-masked]>
Date: Sat, 14 Sep 2019 11:19:35 +0200

> With all due respect, it's pretty easy to make absolute statements like

Pardon me.

> many years ago. It's a compromise, yes.

I'm sure about it.

> belong to that group).

OK.

> It was very hard to get to where we are right

Well, having code examples and tables interspersed with pagination
information is not everybody's cup of tea :-)

Unpaginated Web pages print fairly well these days.  I believe there
even is a way to paginate specifically for printing.

>> PIs could be used to keep the XML intact.

Maybe there is something I have misunderstood.

The idea is simply that there would be a single master document (in XML) that would conditionally produce different output depending on directives given by the author.

Since the transition to new formats always take long time, PIs could ease that.
> "currently used by RFCs" where?

In RfcMarkup and HTML RFCs.

Cheers,
Anders

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Sep 2019 11:41:42 +0200

> even is a way to paginate specifically for printing.

Yes, but that requires the document to be HTML, not plain text. That's
what confused me.

rtfc2629.xslt generates the required CSS print information
(headers/footers etc). Unfortunately, browsers do not really care yet.
(PrinceXML does when generating PDF from HTML).

xml2rfc doesn't seem to generate it yet, but I'm sure it will at some point.

> ... >>> PIs could be used to keep the XML intact.
>>
>> I still don't get that.
>
> The idea is simply that there would be a single master document (in XML)
> that would conditionally produce different output depending on
> directives given by the author.
>

The vocabulary has supported plain text fallbacks for ages, see
<https://greenbytes.de/tech/webdav/rfc7749.html#rfc.section.2.5.p.5>.
xml2rfc-as-implemented introduces something new for that, see
<https://tools.ietf.org/html/draft-levkowetz-xml2rfc-v3-implementation-notes-09#section-3.1.1>.

> In RfcMarkup and HTML RFCs.

RfcMarkup has no official standing (and yes, I like it as well, and it
has served us well for a very long time). It apparently generates XHTML,
but the content is served as text/html on tools.ietf.org. It might be
good to change it to produce valid HTML5.

And when you say: "HTML RFCs", what are you referring to?

Best regards, Julian

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 16 Sep 2019 07:22:33 +0200

On 2019-09-14 11:41, Julian Reschke wrote:
<snip>
> good to change it to produce valid HTML5.

This in interesting and but also rather confusing since this is the by far most used method for communicating RFCs by developers.
> And when you say: "HTML RFCs", what are you referring to?

The format that is called "html" which for unknown reasons was removed from XML V3 submissions:
V2: https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-06
V3: https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-08

Regards,
Anders

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Sep 2019 10:09:36 +0200

On 16.09.2019 07:22, Anders Rundgren wrote:
> This in interesting and but also rather confusing since this is the by
> far most used method for communicating RFCs by developers.

Right now, but the whole point is that in the future we'll be able to
generate the HTML directly from XML.

> V2:
> https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-06
> V3:
> https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-08

OK. You'll need to ask the tools team about that.

Best regards, Julian

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 16 Sep 2019 10:27:31 -0400

> far most used method for communicating RFCs by developers.

Do you have any data to backup this claim?

	Thanks,
	Paul

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 16 Sep 2019 16:46:00 +0200

> Do you have any data to backup this claim?

No, OTOH, since I mostly work with JSON-based stuff, the documents are fairly recent.

What I'm sure that I have newer seen in wild are references to PDF RFCs (which apparently are in plain text).

Thanx,
Anders

> 
> 	Thanks,
> 	Paul
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Sep 2019 16:54:01 +0200

> (which apparently are in plain text).

I agree with that, but I don't see how that supports the statement about
RfcMarkup... (it may well be true, but RfcMarkup is something that was
created because we did not have "official" HTML renderings of the
documents, and this is going to change soon anyway).

Best regards, Julian

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 16 Sep 2019 11:32:47 -0400

> No, OTOH, since I mostly work with JSON-based stuff, the documents are 

I don't recall every having seen a reference to an RfcMarkup.

> What I'm sure that I have newer seen in wild are references to PDF RFCs 

Nor have I.

The most common references I see are to either the "HTMLized" form, the 
plain text, or the datatracker page.

Personally I much prefer to work with the HTMLized form. It gives easy 
ability to jump around in the document, the extra info in the header 
including links to related tools, etc. and yet the text is almost 
identical to the plain text form which makes it easy to jump back and 
forth between a diff and the full document. It is also convenient to cut 
and paste bits of the document into email. I much prefer this format to 
the HTML format generated directly from xml.

When a new version of a document is published, I first pull up the 
HTMLized format. Then from there I follow the link to open a diff in 
another tab. That way I can skim the changes and easily refer back to 
the full document when I need more context.

I hope that the HTMLized format will continue to be available. It could 
however be directly generated from the XML rather than from the 
plaintext. Generating it from the plaintext sometimes results in defects 
regarding references.

	Thanks,
	Paul

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 16 Sep 2019 17:33:55 +0200

On 2019-09-16 16:54, Julian Reschke wrote:
>> (which apparently are in plain text).
>> ...
> 

I don't think the motives for using HTML has been very strong.
Is there actually a single RFC out there using SVG and Unicode?

Anyway, I surely look forward to HTML!

Lists currently look a bit "crammed" in my opinion:
https://tools.ietf.org/id/draft-rundgren-signed-http-requests-00.html

Anders

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Sep 2019 17:43:27 +0200

> I don't think the motives for using HTML has been very strong.

And you did re-read the epic discussion that lead to this decision?

> Is there actually a single RFC out there using SVG and Unicode?

Yes, there is one for non-ASCII, if that's what you're asking. That
said, how is this relevant when in fact it wasn't possible to do so up
until now?

Looking at the source:

> <meta name="generator" content="xml2rfc version 2.21.0 - https://tools.ietf.org/tools/xml2rfc" />

The lists indeed look crammed, but what you're looking at is the HTML
output in the "v2" mode, which we'll hopefully won't be using anymore
soonish.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Sep 2019 17:53:36 +0200

> I don't recall every having seen a reference to an RfcMarkup.

Funny enough, that is the tool that is creating the HTNLized versions.

>> What I'm sure that I have newer seen in wild are references to PDF RFCs

PDF RFCs have been possible in the past, but certainly not encouraged.

> the HTML format generated directly from xml.

There are multiple tools that do this, so it would be good to be more
specific here (xml2rfc in v2 mode? in v3 mode? rfc2629.xslt?)

Best regards, Julian

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 16 Sep 2019 12:36:43 -0400

> PDF RFCs have been possible in the past, but certainly not encouraged.

In the PALS WG, we did an RFC where the PDF is the "canonical" output (
https://tools.ietf.org/pdf/rfc7893.pdf ). Check out the figures and you'll
see why. In the upcoming V3 world, we would be able to do this with the
HTML version as well. except that SVG figures will be black and white only
(a definite drawback in this case, esp. for figures 5-8).

Cheers,
Andy

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 16 Sep 2019 20:18:10 +0200

Hi Paul,

> I don't recall every having seen a reference to an RfcMarkup.

rfcmarkup is the script that does the htmlization of text drafts and
rfcs.  I've pulled the interesting bits out as a python lib,
https://pypi.org/project/rfc2html/ .

> I hope that the HTMLized format will continue to be available.

That is my plan, yes.  I see no reason to stop the scripts that does that
job.

> It could 

Yes.  When I can find the time, I thought I'd at least try to do a CSS
stylesheet that applies a style to the v3 html that makes it come out
very close to the htmlized version.

Generating html which has *exactly* the same line breaks as the text output
will require writing a almost completely new formatter that does the text
format line breaking, but inserts the proper links everywhere the regular
HTML v3 renderer does so.  I don't know when, if ever, I'd have that much
time to spare.

Best regards,

	Henrik

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 16 Sep 2019 14:35:22 -0400

> Funny enough, that is the tool that is creating the HTNLized versions.

Interesting. But what ends up being referenced is the resulting HTML.

> specific here (xml2rfc in v2 mode? in v3 mode? rfc2629.xslt?)

I'm speaking here mostly as a consumer of rfcs and drafts, though I also 
author them. As a consumer I just see the results, not how they were 
produced. But I presume most have been produced from v2 xml using 
xml2rfc in v2 mode.

I look forward to seeing what might be available as v3 comes into use. 
If there is a good side by side diff tool for the html format or 
directly for the xml, that will be a start. Even then it will be 
interesting to see how easy it is to move back and forth between the 
diff and the full text.

(Sometimes I would prefer the side by side diff to include the full text 
so that there is complete context. But other times that would make it 
very convenient to see what has changed.)

Note that I'm only relating this to the way *I* have learned to work 
using the available tools. Perhaps new and better ways to work will be 
possible with the new format and related tools.

	Thanks,
	Paul

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 16 Sep 2019 20:24:42 +0100

On 16/09/2019 17:36, Andrew G. Malis wrote:
> https://tools.ietf.org/pdf/rfc7893.pdf ). Check out the figures and 
> you'll see why. In the upcoming V3 world, we would be able to do this 
> with the HTML version as well. except that SVG figures will be black and 
> white only (a definite drawback in this case, esp. for figures 5-8).
> 
> Cheers,
> Andy
> 

.. and RFC5317 the text version being a pointer to the pdf which carried 
a set of slides jointly agreed with the ITU.

However the most famous was RFC1305 which defines NTPv3 (NTPv2 was 
published in PS format)

My experience has always been that if you had a good reason for 
publishing in pdf was always possible and perhaps authors should have 
pushed a bit harder.

- Stewart