Subject: [xml2rfc] idnits 2.16.05 finds non-ascii character &mdash;

From: Roger Price <[email-masked]>
Date: Thu, 18 Feb 2021 21:57:27 +0100 (CET)

My XML markup includes an &mdash; in the <title>.  This looks fine in HTML, but 
idnits 2.16.05 finds non-ascii utf-8 e28084 in the txt file.

I fixed the problem by adding the line

   text = text.replace(u'\u2014', u'-')  # Replace &mdash; EM DASH with HYPHEN-MINUS

to file $PYTHON/site-packages/xml2rfc/writers/text.py immediately before line

   text = text.replace(u'\u00A0', u' ')

Could this be made a permanent change?

Perhaps there are other character entities which could profit from such 
replacement.

Roger

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 18 Feb 2021 22:12:07 +0100

> On 2021-02-18, at 21:57, Roger Price <[email-masked]> wrote:
> 
> My XML markup includes an &mdash; in the <title>.  

Which is rather surprising to me, as it should not be possible according to the currently prevailing restrictive interpretation of RFC 7997.

> This looks fine in HTML, but idnits 2.16.05 finds non-ascii utf-8 e28084 in the txt file.

This is intentional as long as we stick to the above restrictive interpretation.

> Could this be made a permanent change?

I hope not.  Em-dashes look OK in plaintext.   They might be harder to distinguish from en-dashes or neutral hyphen minus in monospaced fonts. 
If we do decide we need a surrogate for em-dashes in monospaced plaintext, that should be ---.
(V2 used -- for its ASCIIification, IIRC.)

> Perhaps there are other character entities which could profit from such replacement.

There are not too many characters that suffer much from a monospaced font.

Grüße, Carsten