Subject: [xml2rfc] Unicode box-drawing for a new --table-borders value?

From: Daniel Kahn Gillmor <[email-masked]>
Date: Fri, 29 Jan 2021 17:55:14 -0500

Hi xml2rfc folks--

Now that we can include unicode characters in our text documents, has
anyone considered adding a new type of --table-borders flag that uses
the unicode box-drawing characters?

In particular, i'm thinking of the U+2500..U+256C range:

          .0 .1 .2 .3 .4 .5 .6 .7 .8 .9 .A .B .C .D .E .F 
     250.  ─  ━  │  ┃  ┄  ┅  ┆  ┇  ┈  ┉  ┊  ┋  ┌  ┍  ┎  ┏ 
     251.  ┐  ┑  ┒  ┓  └  ┕  ┖  ┗  ┘  ┙  ┚  ┛  ├  ┝  ┞  ┟ 
     252.  ┠  ┡  ┢  ┣  ┤  ┥  ┦  ┧  ┨  ┩  ┪  ┫  ┬  ┭  ┮  ┯ 
     253.  ┰  ┱  ┲  ┳  ┴  ┵  ┶  ┷  ┸  ┹  ┺  ┻  ┼  ┽  ┾  ┿ 
     254.  ╀  ╁  ╂  ╃  ╄  ╅  ╆  ╇  ╈  ╉  ╊  ╋  ╌  ╍  ╎  ╏ 
     255.  ═  ║  ╒  ╓  ╔  ╕  ╖  ╗  ╘  ╙  ╚  ╛  ╜  ╝  ╞  ╟ 
     256.  ╠  ╡  ╢  ╣  ╤  ╥  ╦  ╧  ╨  ╩  ╪  ╫  ╬

Looking at the use of bchar in xml2rfc/writers/text.py, it looks to me
like it should be easy enough to map three of the four border
characters:

      '=': '═',
      '-': '─',
      '|': '│',

But the corner border character ('+') has many different
options/permutations:

     '┌', '┐', '├', '┤', '┬', '┴', '┼',
     '╒', '╕', '╘', '╛', '╞', '╡', '╤', '╧', '╪'

so it's this last bit that seems challenging to implement, since it
looks like the current code just says "corners are corners" and doesn't
distinguish between all these cases.

Should i report this as an RFE on
https://trac.tools.ietf.org/tools/xml2rfc/trac/newticket or is there
some other preferred way to track such a suggestion?
https://tools.ietf.org/tools/xml2rfc/trac/report is timing out for me so
i can't tell whether it's already been reported there.

            --dkg

PS i'm thinking this because we're using some box drawing characters
   already in some of the verbatim sections (e.g. § 4.2 "Packet Headers"
   and § 6.3.  "Encoding Binary in Radix-64") and it seems a bit clunky
   to have these clean boxes adjacent to the rather clumsier US-ASCII
   table borders.

---

From: Lars Eggert <[email-masked]>
Date: Sat, 30 Jan 2021 07:22:43 +0200

Hi,

> the unicode box-drawing characters?

I think that would be excellent, but AFAIK Unicode can (still) only be used in contact names (see the recent "Unicode in xml2rfc v3" thread over on rfc-interest@). I really hope we can just get to an agreement to allow Unicode anywhere in the document.

Lars

---

From: "John Levine" <[email-masked]>
Date: 30 Jan 2021 14:08:20 -0500

In article <[email-masked]> you write:
>-=-=-=-=-=-

The text version is basically to provide some degree of backward
compatibility. If you want a version that is easy to read, I encourage
you to open your web browser and look at the HTML version.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 30 Jan 2021 14:14:54 -0500

In article <[email-masked]> you write:
>I think that would be excellent, but AFAIK Unicode can (still) only be used in contact names (see the recent "Unicode in
>xml2rfc v3" thread over on rfc-interest@). I really hope we can just get to an agreement to allow Unicode anywhere in the
>document.

The problem is that laying out arbitrary Unicode is a nightmare.

It would be helpful if people could list some scenarios where it would
be useful. Carsten has (thanks!) but more voices would be nice.

The ones that occur to me beyond people's names are math symbols like a≥b and
words that are sometimes written with accents like coördinate.  For the math
symbols, I get the impression that people really want TeX style equation typesetting
which while not impossible, would be a major extension.

There's also typesetting stuff like “quotes” which we decided not to
do when this came up before but we might revisit.

What have I missed?

R's,
John

---

From: Tom Pusateri <[email-masked]>
Date: Sat, 30 Jan 2021 14:35:07 -0500

> On Jan 30, 2021, at 2:08 PM, John Levine <[email-masked]> wrote:
> 

In the past, we’ve been told that the text version was the official version and the HTML was not. It sounds like this is now being reversed. Backward compatible doesn’t sound like official. But “easy to read” doesn’t sound like it’s official either. Could you clarify?

Thanks,
Tom

---

From: "John R Levine" <[email-masked]>
Date: 30 Jan 2021 16:05:32 -0500

> In the past, we’ve been told that the text version was the official 
> version and the HTML was not.

That has not been true for quite a while.

Since RFC 8651 which was published in October 2019, the official version 
is the v3 XML.  The text, HTML, and PDF are mechanically derived from it 
by running it through xml2rfc.  As our tools improve, we might recreate 
the derived versions so they look better or the HTML works better.

This was widely publicised on the IETF list and other places so I'm 
suprised people still aren't aware of it.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Sun, 31 Jan 2021 02:26:23 -0500

> you to open your web browser and look at the HTML version.

I do ☺ But i hear that some people still end up reading the txt
version, and i'd like them to have cleanly-drawn tables when they do.

         --dkg

---

From: Julian Reschke <[email-masked]>
Date: Sun, 31 Jan 2021 15:43:50 +0100

> version, and i'd like them to have cleanly-drawn tables when they do.

Understood, but then there are *also* people who want the plain text
output change as little as possible compared to what we had before.

Best regards, Julian

---

From: Anders Rundgren <[email-masked]>
Date: Sun, 31 Jan 2021 16:22:41 +0100

> version, and i'd like them to have cleanly-drawn tables when they do.

I actually did this by creating a table as ascii-art but unfortunately
the box-characters didn't always follow the monospace scheme so I had to
revert to the standard table.

Here it is BTW: https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-09#appendix-B

It doesn't look OK on Android.

Anders

> 
>           --dkg
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: "John R Levine" <[email-masked]>
Date: 31 Jan 2021 11:41:27 -0500

> https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-09#appendix-B
>

Thanks for the pointer.  It doesn't look OK in Chrome on MacOS either.

Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 31 Jan 2021 18:21:18 +0100

> Thanks for the pointer.  It doesn't look OK in Chrome on MacOS either.

That seems to be a problem specific to the legacy Courier (as opposed to Courier New and a few other monospaced fonts I tried).
The current stylesheet seems to rely on the browsers' preset for “<pre>”.
Most other fonts that are tagged monospace in macOS Font Book work OK, including the font Chrome’s HTML source inspector uses...

I think what we can learn from this example is that, beyond ASCII, strict grid spacing assumptions about monospace fonts will earn some surprises.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 31 Jan 2021 18:32:58 +0100

On 31. Jan 2021, at 18:21, Carsten Bormann <[email-masked]> wrote:
> 
>  a problem specific to the legacy Courier 

(Which is here that the limited set of box drawing characters in that font are not (all?) mapped to Unicode code points, so there is some font substitution going on.)

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 31 Jan 2021 13:48:06 -0500

> (Which is here that the limited set of box drawing characters in that font are not (all?) mapped to Unicode code points, so there is some font substitution going on.)

Right.  Once again, if you want nice looking boxes, HTML draws them much 
better.

Regards,

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 31 Jan 2021 17:36:01 -0500

> you to open your web browser and look at the HTML version.

Of more concern, AFAIK the only reasonable diff tools work on the txt 
version. IIRC there was once talk of creating a diff tool that worked on 
the xml, but I haven't heard of any such thing materialized. I think 
that would present great challenges, especially if the xml is 
aesthetically revised while preserving the look. And I would still want 
to see the diff results in side-by-side form. Also that won't work when 
one of the versions being diffed predates v3 xml.

And when using the txt version for diffs it then the complete txt 
version you want to look at for broader context than the diff provides.

The only way I have seen to reasonably manage differences in documents 
with fancier formatting is with change history, such as provided by 
MSWord. We clearly don't have that with v3 xml.

	Thanks,
	Paul

---

From: "John Levine" <[email-masked]>
Date: 1 Feb 2021 02:35:08 -0000

In article <[email-masked]>,
Paul Kyzivat  <[email-masked]> wrote:
>the xml, but I haven't heard of any such thing materialized. 

Jim Schaad was working on it, but the stuff he wrote wasn't close to done.

>I think 

I agree there's no plausible way to do XML diffs on documents using
different XML vocabularies, but eventually v2 will die out. Our
tooling is a problem and we underestimated what we'd need beyond
xml2rfc to create a productive v3 xml environment.

>MSWord. We clearly don't have that with v3 xml.

Well, we do if we store them in github but that's its own can of worms.

R's,
John
-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly