Subject: [Tools-discuss] Python code for extracting title from an RFC or Draft?

From: Paul Hoffman <[email-masked]>
Date: Mon, 4 Aug 2008 08:43:24 -0700

Greetings again. I looked on the tools site and don't see any source 
code. I would love a tool that, given a draft or RFC, extracts the 
title from the top of the first page. (Extra points for pulling out 
the abstract). If I have to do it as "given the filename, look in 
1id-abstracts.txt or rfc-index.xml", that's fine, but I would like 
the code for that as well. No wheel-recreating, if not needed.

As a side note, it seems odd that the code y'all have created isn't 
on tools.ietf.org...

--Paul Hoffman, Director
--VPN Consortium
_______________________________________________
Tools-discuss mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tools-discuss

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 04 Aug 2008 18:40:08 +0200

> As a side note, it seems odd that the code y'all have created isn't 

Maybe it's mostly a matter of a missing index page and documentation,
or something, I guess ...

The datatracker source is available through
  http://tools.ietf.org/tools/ietfdb

and the following covers a number of other known and unknown tools.
Not all will be up-to-date and used in generating the tools pages,
but you're welcome to browse and comment.

  http://tools.ietf.org/tools/doublespace
  http://tools.ietf.org/tools/getdrafts
  http://tools.ietf.org/tools/html-404
  http://tools.ietf.org/tools/htmlize
  http://tools.ietf.org/tools/idcomments
  http://tools.ietf.org/tools/idnits
  http://tools.ietf.org/tools/idreplaced
  http://tools.ietf.org/tools/id_rss
  http://tools.ietf.org/tools/idspell
  http://tools.ietf.org/tools/ietfcal
  http://tools.ietf.org/tools/libsmi
  http://tools.ietf.org/tools/loginmgr
  http://tools.ietf.org/tools/mini-index
  http://tools.ietf.org/tools/mq
  http://tools.ietf.org/tools/msglint
  http://tools.ietf.org/tools/postconfirmd
  http://tools.ietf.org/tools/pyht
  http://tools.ietf.org/tools/rfcdiff
  http://tools.ietf.org/tools/rfcindex
  http://tools.ietf.org/tools/rfcmarkup
  http://tools.ietf.org/tools/rfc_rss
  http://tools.ietf.org/tools/rfcstrip
  http://tools.ietf.org/tools/searchplugins
  http://tools.ietf.org/tools/sv
  http://tools.ietf.org/tools/wg-pages

Regards,

	Henrik

---

From: Joe Touch <[email-masked]>
Date: Mon, 04 Aug 2008 09:42:36 -0700

Henrik Levkowetz wrote:
|
|
| On 2008-08-04 17:43 Paul Hoffman said the following:
|> Greetings again. I looked on the tools site and don't see any source
|> code. I would love a tool that, given a draft or RFC, extracts the
|> title from the top of the first page. (Extra points for pulling out
|> the abstract). If I have to do it as "given the filename, look in
|> 1id-abstracts.txt or rfc-index.xml", that's fine, but I would like the
|> code for that as well. No wheel-recreating, if not needed.

Why not pull the info from the XML rfc-index?

Joe

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 4 Aug 2008 10:21:32 -0700

>Why not pull the info from the XML rfc-index?

As I said, I'm happy to. If someone has already written the code to 
do so, that would help. I haven't done any XML coding in Python 
before. It is actually more important for me to be able to pull it 
for Internet Drafts.

I don't see anything on Henrik's list that does either of these...

--Paul Hoffman, Director
--VPN Consortium

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 04 Aug 2008 20:53:20 +0200

> Why not pull the info from the XML rfc-index?

For RFCs specifically you could do that, and I also have the titles available,
in a variety of other formats, but that doesn't match Paul's specification of
being able to extract title from both Drafts and RFCs...

	Henrik

---

From: Joe Touch <[email-masked]>
Date: Mon, 04 Aug 2008 19:15:21 -0700

Henrik Levkowetz wrote:
|
|
| On 2008-08-04 18:42 Joe Touch said the following:
|>
|>
|> Henrik Levkowetz wrote:
|> |
|> |
|> | On 2008-08-04 17:43 Paul Hoffman said the following:
|> |> Greetings again. I looked on the tools site and don't see any source
|> |> code. I would love a tool that, given a draft or RFC, extracts the
|> |> title from the top of the first page. (Extra points for pulling out
|> |> the abstract). If I have to do it as "given the filename, look in
|> |> 1id-abstracts.txt or rfc-index.xml", that's fine, but I would like the
|> |> code for that as well. No wheel-recreating, if not needed.
|>
|> Why not pull the info from the XML rfc-index?
|
| For RFCs specifically you could do that, and I also have the titles
| available,
| in a variety of other formats, but that doesn't match Paul's
| specification of
| being able to extract title from both Drafts and RFCs...

Hmmm. Given the automation of the ID submission process, it might be
useful to ask the ID-administrator to support this rather than screen
scraping (or page scraping, as the case may be).

Joe

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 6 Aug 2008 08:42:02 -0700

(resending; mailman didn't like my .zip attachment.)

I have been using abstract.sed for awhile, and just whipped up title.sed
tonight for you.  Hope it helps, both are in http://www2.fuggles.com/paul.zip
(along with tighten.sed).

To use it:

  sed -f abstract.sed FILENAME | sed -f tighten.sed
  sed -f title.sed FILENAME

Examples with some RFCs and an I-D:

> sed -f abstract.sed rfc1122.txt
   This is one RFC of a pair that defines and discusses the requirements
   for Internet host software.  This RFC covers the communications
   protocol layers: link layer, IP layer, and transport layer; its
   companion RFC-1123 covers the application and support protocols.

> sed -f abstract.sed rfc5037.txt | sed -f tighten.sed
   The purpose of this memo is to document how some of the requirements
   specified in RFC 1264 for advancing protocols developed by working
   groups within the IETF Routing Area to Draft Standard have been
   satisfied by LDP (Label Distribution Protocol).  Specifically, this
   report documents operational experience with LDP, requirement 5 of
   section 5.0 in RFC 1264.

> sed -f title.sed rfc5037.txt
         Experience with the Label Distribution Protocol (LDP)

> sed -f title.sed draft-wing-avt-dtls-srtp-key-transport-02.txt
                        DTLS-SRTP Key Transport

-d

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Paul Hoffman
> Sent: Monday, August 04, 2008 8:43 AM
> To: [email-masked]
> Subject: [Tools-discuss] Python code for extracting title 
> from an RFC orDraft?
>