Subject: Re: [xml2rfc] Insufficiency of txt format

From: Paul Kyzivat <[email-masked]>
Date: Mon, 1 Feb 2021 11:34:32 -0500

On 1/31/21 9:35 PM, John Levine wrote:
> In article <[email-masked]>,
> Paul Kyzivat  <[email-masked]> wrote:
>> Of more concern, AFAIK the only reasonable diff tools work on the txt
>> version. IIRC there was once talk of creating a diff tool that worked on
>> the xml, but I haven't heard of any such thing materialized.
> 
> Jim Schaad was working on it, but the stuff he wrote wasn't close to done.
> 
>> I think
>> that would present great challenges, especially if the xml is
>> aesthetically revised while preserving the look. And I would still want
>> to see the diff results in side-by-side form. Also that won't work when
>> one of the versions being diffed predates v3 xml.
> 
> I agree there's no plausible way to do XML diffs on documents using
> different XML vocabularies, but eventually v2 will die out. Our
> tooling is a problem and we underestimated what we'd need beyond
> xml2rfc to create a productive v3 xml environment.

I'll be dead before v2 xml is.

I worked on rfc4566bis for a long time, and it was just published as 
rfc8866. Is is very informative to look at the diff between the two.

And the txt diff will remain an important tool as long as there are 
pre-v3 documents that get bis versions.

>> The only way I have seen to reasonably manage differences in documents
>> with fancier formatting is with change history, such as provided by
>> MSWord. We clearly don't have that with v3 xml.
> 
> Well, we do if we store them in github but that's its own can of worms.

That will only show the changes in the xml. That isn't generally what I 
want to see when reviewing a new revision of a document. The 
side-by-side txt diff usually gives me exactly what I want to see when 
commenting on a new version. It would be nice if I could get comparable 
functionality between html or pdf renderings of the document, but I 
can't imagine how that would work. (Hopefully this is just a statement 
about my lack of imagination.)

	Thanks,
	Paul

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 1 Feb 2021 18:21:21 +0100

> I'll be dead before v2 xml is.

I hope not, but this is an indication of the timelines we have to think in here.

Also, our documents have drawn-out timelines, during which the authoring-format-du-jour will change (as will even the “canonical” format).

So txt-level comparison needs to be part of any sustainable long-term strategy.

The interesting part here is normalization.  Rfcdiff has been normalizing the txt all along by removing footers and headers.  It could add normalization steps such as turning box-drawing characters into legacy ASCII.  If thinking about the comparison story is part of the strategy, such a presentation change could become invisible to long-timespan diffs.

Doing XML-level comparison could be aided by converting to a format such as Pyx (line-based ESIS representation) first.  Again, further normalization is crucial (as we have seen in the example of random attribute order already). The most ugly part here is likely to be white-space handling, which is a very sorry story for XML in general, but could be handled in a schema-driven way for RFCXML.  Also, preptool-generated attributes could be removed in the normalization (but they also can carry useful information).

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 1 Feb 2021 18:41:02 +0100

On 2021-02-01, at 18:21, Carsten Bormann <[email-masked]> wrote:
> white-space handling

And while we are talking about this:

What the [insert Unicode characters here] is the story behind the two occurrences of xml:space in v3.rnc.

   spanx =
     element spanx { […]
       [ a:defaultValue = "preserve" ]
       attribute xml:space { "default" | "preserve" }?,

So, for a (legacy) <spanx> specifically, xml:space defaults to “preserve”.

For artwork:

   artwork =
     element artwork { […]
       attribute xml:space { text }?,

… it is allowed to set xml:space to any value (invalid in XML), but there is no default.  Also, xml:space is not allowed for <sourcecode>.

Contrast this to v2.rnc:

   spanx =
     element spanx {
       [ a:defaultValue = "preserve" ]
       attribute xml:space { "default" | "preserve" }?,

So this is unchanged in v3 from v2.

   artwork =
     element artwork {
       [ a:defaultValue = "preserve" ]
       attribute xml:space { "default" | "preserve" }?,

… makes sense if you think that xml:space is going to be useful at all.

This was discussed in <https://mailarchive.ietf.org/arch/msg/xml2rfc-dev/U7ounlQdYz95U3I-PofVgheYmPI>, but apart from agreeing that xml:space is useless, not much light has been generated.

There are a few more mentions further in history that I can find, but none of them really discussed how the situation of just <spanx> and <artwork> using xml:space and the v3 weirdness around this occurred.

RFC 7991 says:

   o  Deprecate the "xml:space" attribute in <artwork> because there was
      only one useful value.  […]

… and dutifully notes the above change in the RNC diff.  
Er, since only one value is useful, we allow any value and then let the default lapse to the non-useful one?

Most likely, there is no reason (because nothing actually is using this attribute), so this can be cleaned up, but I sure would like to know if there is.  Should an authoring tool generate any of these?

(There is also the SVG issue, which I’ll pass now.)

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Mon, 1 Feb 2021 19:25:01 +0100

Am 01.02.2021 um 18:41 schrieb Carsten Bormann:
> ...
> ...

Looks like a bug to me.

Best regards, Julian

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Mon, 01 Feb 2021 14:01:35 -0500

On Sun 2021-01-31 16:22:41 +0100, Anders Rundgren wrote:
> I actually did this by creating a table as ascii-art but unfortunately
> the box-characters didn't always follow the monospace scheme so I had to
> revert to the standard table.
>
> Here it is BTW: https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-09#appendix-B
>
> It doesn't look OK on Android.

I can confirm that both Chromium and Tor Browser appear to misrender the
box drawing characters on an Android-derived (https://calyxos.org/)
device I got a chance to look at.

This is presumably a font rendering issue, probably something comparable
to https://bugs.debian.org/981577, which i just reported on Debian.

I've reported the issue to CalyxOS as well:
https://gitlab.com/CalyxOS/calyxos/-/issues/334

If this is a problem for you on a platform that you care about, please
report it to them.  Feel free to use
https://dkg.fifthhorseman.net/table.txt as a reference for any bug
report -- i'll leave it up.

In the meantime, though, we shouldn't avoid using standard, well-defined
characters just because some font rendering paths screw them up.  those
are bugs that should be fixed in the appropriate places.

       --dkg