Subject: [xml2rfc] BibTeX Citation in xml

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 09:00:45 -0400

I want to include the following as a reference:

https://eprint.iacr.org/eprint-bin/cite.pl?entry=2017/498

The actual paper is:

https://eprint.iacr.org/2017/498.pdf

How can this be done?  Is there a DOI method for it?

thanks

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 10:41:33 -0400

oops.  Hit the wrong key back there...

> How can this be done?  Is there a DOI method for it?

Joan just sent me the following DOI information on his paper:

Yes, I found it at:

https://link.springer.com/chapter/10.1007%2F978-3-319-70697-9_21

this is what it says:

https://doi.org/10.1007/978-3-319-70697-9_21

Now to get that working as I have the NIST stuff working...

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 12:56:33 -0400

> Now to get that working as I have the NIST stuff working...

When I try:

https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.10.1007/978-3-319-70697-9_21

I get

invalid DOI or type

What is the proper invocation?

Thanks

---

From: Alice Russo <[email-masked]>
Date: Wed, 25 Sep 2019 10:11:32 -0700

Hi,
https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.1007/978-3-319-70697-9_21.xml
seems to work.

template:
https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.<number>.xml

Alice

> On Sep 25, 2019, at 9:56 AM, Robert Moskowitz <[email-masked]> wrote:
> 
> 
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 13:31:00 -0400

> https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.1007/978-3-319-70697-9_21.xml

when I use

     <?rfc 
include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.1007/978-3-319-70697-9_21.xml?anchor=ASIACRYPT-2017"?>

xml2rfc gives me the warnings:

Warning: Illegal character replaced in string: &#128;
Warning: Illegal character replaced in string: &#147;

And the reference text becomes:

    [ASIACRYPT-2017]
               Daemen, J., Mennink, B., and G. Van Assche, "Full-State
               Keyed Duplex with Built-In Multi-user Support", Advances
               in Cryptology a&#128;&#147; ASIACRYPT 2017 pp. 606-637,
               DOI 10.1007/978-3-319-70697-9_21, 2017.

I am running the most recent version of xml2rfc.  When I add --v3, I get 
the substitutions without any warnings.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 Sep 2019 20:10:19 +0200

> 
>     <?rfc include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml7/reference.DOI.10.1007/978-3-319-70697-9_21.xml?anchor=ASIACRYPT-2017"?>
> 

Something is very broken with the XML2RFC you are using (or with a firewall you are going through).

The XML at the URL is:

<reference anchor='ASIACRYPT-2017' >
  <front>
    <title>Full-State Keyed Duplex with Built-In Multi-user Support</title>
    <author initials="J." surname="Daemen" fullname="Joan Daemen">
      <organization></organization>
    </author>
    <author initials="B." surname="Mennink" fullname="Bart Mennink">
      <organization></organization>
    </author>
    <author initials="G." surname="Van Assche" fullname="Gilles Van Assche">
      <organization></organization>
    </author>
    <date year="2017"/>
  </front>
  <seriesInfo name="Advances in Cryptology – ASIACRYPT 2017" value="pp. 606-637"/>
  <seriesInfo name="DOI" value="10.1007/978-3-319-70697-9_21"/>
</reference>

Note the en-dash (U+2013) before “ASIACRYPT”, which turns into mojibake on your system (looks like UTF-8 misinterpreted as Latin-1).

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 Sep 2019 20:18:48 +0200

I get

   [DUPLEX]   Daemen, J., Mennink, B., and G. Van Assche, "Full-State
              Keyed Duplex with Built-In Multi-user Support", Advances
              in Cryptology - ASIACRYPT 2017 pp. 606-637,
              DOI 10.1007/978-3-319-70697-9_21, 2017.

(Oops, should have used your label instead of DUPLEX.)

Oh, xml2rfc 2.26.0

Retrying with xml2rfc 2.30.0:

   [DUPLEX]   Daemen, J., Mennink, B., and G. Van Assche, "Full-State
              Keyed Duplex with Built-In Multi-user Support", Advances
              in Cryptology - ASIACRYPT 2017 pp. 606-637,
              DOI 10.1007/978-3-319-70697-9_21, 2017.

Hmm, works for me.  (This is with Python V3 on a High Sierra Mac.)

Grüße, Carsten

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 14:33:05 -0400

> Hmm, works for me.  (This is with Python V3 on a High Sierra Mac.)

I am using xml2rfc v2.30.0, Python V2 on my Fedora 30 system.

And when I try:  https://xml2rfc.tools.ietf.org/

I get the same problem:

    [ASIACRYPT-2017]

xml attached

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 Sep 2019 20:42:39 +0200

> And when I try:  https://xml2rfc.tools.ietf.org/

Yep, verified with kdrfc -r (which is a front end to calling kramdown-rfc and using the remote xml2rfc service).

   [ASIACRYPT-2017]

I can’t reproduce this with a local xml2rfc even when switching to Python v2 (I think).
The next thing I’d try would be different locale settings, but I’m too exhausted to do this now.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 Sep 2019 20:49:38 +0200

Hmm, it does not happen if I expand the reference to the XML myself.
A problem with <?rfc include?> ?

Grüße, Carsten

> On Sep 25, 2019, at 20:42, Carsten Bormann <[email-masked]> wrote:
>

---

From: Jim Schaad <[email-masked]>
Date: Wed, 25 Sep 2019 12:17:26 -0700

Run away from Python v2 as soon as you want something other than ASCII - it does not like Unicode by default and it is easy to get the programming wrong.

-----Original Message-----
From: xml2rfc <[email-masked]> On Behalf Of Robert Moskowitz
Sent: Wednesday, September 25, 2019 11:33 AM
To: Carsten Bormann <[email-masked]>
Cc: [email-masked]
Subject: Re: [xml2rfc] BibTeX Citation in xml

I get the same problem:

    [ASIACRYPT-2017]

xml attached

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 15:38:34 -0400

I lost my notes on how to run xml2rfc under python3.  Plus I probably 
have to run pip under P3 first?

---

From: Robert Moskowitz <[email-masked]>
Date: Wed, 25 Sep 2019 16:40:30 -0400

On 9/25/19 4:13 PM, Jim Schaad wrote:
> Should just need to
>
> 1) uninstall your current xml2rfc so it is not in P2
> 2) install xml2rfc using "pip3"
> 3) if the right directories are in you path then xml2rfc will run the 
> same way. If you prefix it with python then you need to prefix it with 
> python3

I did all that and still get the character error:

$ pip uninstall xml2rfc
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 
2020. Please upgrade your Python as Python 2.7 won't be maintained after 
that date. A future version of pip will drop support for Python 2.7.
Uninstalling xml2rfc-2.30.0:
   Would remove:
     /home/rgm/.local/bin/xml2rfc
/home/rgm/.local/lib/python2.7/site-packages/xml2rfc-2.30.0-py2.7.egg-info
     /home/rgm/.local/lib/python2.7/site-packages/xml2rfc/*
Proceed (y/n)? y
   Successfully uninstalled xml2rfc-2.30.0

[rgm@lx140e drafts]$ pip3 install --user xml2rfc
Collecting xml2rfc
   Using cached 
https://files.pythonhosted.org/packages/5f/e9/af485545982f617820814ba60d7908747233f18fc2502cdf2204234e6831/xml2rfc-2.30.0.tar.gz
Collecting google-i18n-address>=2.3.2 (from xml2rfc)
   Downloading 
https://files.pythonhosted.org/packages/b4/e6/581d5a206a3cb04cdb75d51a979487a5718aa0fd3b30caf762719e21b12a/google_i18n_address-2.3.5-py2.py3-none-any.whl 
(773kB)
     100% |████████████████████████████████| 778kB 3.3MB/s
Requirement already satisfied: html5lib>=1.0.1 in 
/usr/lib/python3.7/site-packages (from xml2rfc) (1.0.1)
Collecting intervaltree!=3.0.0,>=2.1.0 (from xml2rfc)
   Downloading 
https://files.pythonhosted.org/packages/e8/f9/76237755b2020cd74549e98667210b2dd54d3fb17c6f4a62631e61d31225/intervaltree-3.0.2.tar.gz
Collecting kitchen>=1.2.6 (from xml2rfc)
   Using cached 
https://files.pythonhosted.org/packages/d9/ca/3365cb1160533be8c8b57dbfd6502f367d35e30935ee89a003c664740714/kitchen-1.2.6.tar.gz
Requirement already satisfied: lxml!=4.3.1,>=2.2.8 in 
/usr/lib64/python3.7/site-packages (from xml2rfc) (4.2.5)
Collecting pycountry!=19.7.15,>=1.8 (from xml2rfc)
   Downloading 
https://files.pythonhosted.org/packages/16/b6/154fe93072051d8ce7bf197690957b6d0ac9a21d51c9a1d05bd7c6fdb16f/pycountry-19.8.18.tar.gz 
(10.0MB)
     100% |████████████████████████████████| 10.0MB 1.1MB/s
Collecting pyflakes>=0.8.1 (from xml2rfc)
   Downloading 
https://files.pythonhosted.org/packages/84/f2/ed0ffb887f8138a8fe5a621b8c0bb9598bfb3989e029f6c6a85ee66628ee/pyflakes-2.1.1-py2.py3-none-any.whl 
(59kB)
     100% |████████████████████████████████| 61kB 6.3MB/s
Requirement already satisfied: requests>=2.5.0 in 
/usr/lib/python3.7/site-packages (from xml2rfc) (2.21.0)
Requirement already satisfied: setuptools>=18.0 in 
/usr/lib/python3.7/site-packages (from xml2rfc) (40.8.0)
Requirement already satisfied: six>=1.4.1 in 
/usr/lib/python3.7/site-packages (from xml2rfc) (1.12.0)
Requirement already satisfied: webencodings in 
/usr/lib/python3.7/site-packages (from html5lib>=1.0.1->xml2rfc) (0.5.1)
Collecting sortedcontainers<3.0,>=2.0 (from 
intervaltree!=3.0.0,>=2.1.0->xml2rfc)
   Downloading 
https://files.pythonhosted.org/packages/13/f3/cf85f7c3a2dbd1a515d51e1f1676d971abe41bba6f4ab5443240d9a78e5b/sortedcontainers-2.1.0-py2.py3-none-any.whl
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in 
/usr/lib/python3.7/site-packages (from requests>=2.5.0->xml2rfc) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in 
/usr/lib/python3.7/site-packages (from requests>=2.5.0->xml2rfc) (2.7)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in 
/usr/lib/python3.7/site-packages (from requests>=2.5.0->xml2rfc) (1.24.3)
Installing collected packages: google-i18n-address, sortedcontainers, 
intervaltree, kitchen, pycountry, pyflakes, xml2rfc
   Running setup.py install for intervaltree ... done
   Running setup.py install for kitchen ... done
   Running setup.py install for pycountry ... done
   Running setup.py install for xml2rfc ... done
Successfully installed google-i18n-address-2.3.5 intervaltree-3.0.2 
kitchen-1.2.6 pycountry-19.8.18 pyflakes-2.1.1 sortedcontainers-2.1.0 
xml2rfc-2.30.0

[rgm@lx140e drafts]$ xml2rfc draft-moskowitz-hip-new-crypto-01.xml
Warning: Illegal character replaced in string: &#128;
Warning: Illegal character replaced in string: &#147;
  Created file draft-moskowitz-hip-new-crypto-01.txt

Note that my python2.7 dir is still there dispite it saying it would 
delete it..

ls -ls /home/rgm/.local/lib
total 8
4 drwx------. 3 rgm rgm 4096 Sep 12 09:06 python2.7
4 drwxrwxr-x. 3 rgm rgm 4096 Sep 25 16:16 python3.7

ls -ls /home/rgm/.local/lib/python2.7/site-packages/
total 4
4 drwxrwxr-x. 2 rgm rgm 4096 Sep 12 09:06 xml2rfc-2.27.1-py2.7.egg-info

> From: Robert Moskowitz <[email-masked]>
> Sent: Wednesday, September 25, 2019 12:39 PM
> To: Jim Schaad <[email-masked]>; 'Carsten Bormann' <[email-masked]>
> Cc: [email-masked]
>> Run away from Python v2 as soon as you want something other than 
>> ASCII - it does not like Unicode by default and it is easy to get the 
>> programming wrong.
>>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 Sep 2019 09:44:52 +0200

> Run away from Python v2 as soon as you want something other than ASCII - it does not like Unicode by default and it is easy to get the programming wrong.

Well, it is easy to get the programming wrong in any language :-)

The empirical data we have so far doesn’t point to Python v2 being the problem, this time.

There appears to be a real mojibake issue around <?rfc include=…?> in xml2rfc, and that needs to be fixed.

(There also needs to be a decision to make the authoring side of RFCXML beyond-ASCII-resilient in text-carrying attributes(*) and not just in text content, but that is a separate issue.)

Grüße, Carsten

(*) Which are a mistake that you are taught to avoid in XML 101, but it’s what we have in RFCXML today.