Subject: [Tools-discuss] Mining WG notes

From: Carsten Bormann <[email-masked]>
Date: Sat, 4 Sep 2021 18:23:30 +0200

In an AD review, a simple question came up:
“Has the WG discussed X?” 

Unfortunately, none of us remembers, because X seems to come up so often for different documents that it is hard to keep these discussions apart.

Theoretically, I should be able to find the answer from the notes of the various WG meetings (at an IETF meeting or as an interim).  How do I do this?
Of course, I could manually fetch the minutes of approximately 36 meetings (plus some safety) that have happened since we started the discussion of this document, and then start to apply some laptop-local searching on these.  Is there an rsync target I could use to get those (all?) minutes?

https://datatracker.ietf.org/meeting/past is almost useful as a starting point, except that it points to meeting materials and not proceedings (and then there is the notorious difficulty of extracting the current version of the notes), except for IETF meetings, where one has to click through the meeting agenda to find the CoRE meetings.  The mailing list is sometimes useful, but does not contain (the final version of) the meeting minutes.

There must be a better way to do this.

Grüße, Carsten

---

From: Robert Sparks <[email-masked]>
Date: Sat, 4 Sep 2021 12:29:46 -0500

It's a bit of soup.

You can use the api to strategically identify the minutes you want to 
find and pull the files.

The rsync endpoint you are looking for is at rsync.ietf.org::proceedings

You might find things like rsync.ietf.org::proceedings/111/minutes/ useful.

As you see, the files are in many, many formats, including pdf. The hunt 
will not necessarily be as easy as grep.

I wonder if we're close to the point where we can say "Keep your agendas 
and minutes going forward in markdown"?

RjS

On 9/4/21 11:23 AM, Carsten Bormann wrote:
> ___________________________________________________________
> Tools-discuss mailing list - [email-masked]
> This list is for discussion, not for action requests or bug reports.
> * Report datatracker and mailarchive bugs to: [email-masked]
> * Report tools.ietf.org bugs to: [email-masked]
> * Report all other bugs or issues to: [email-masked]
> List info (including how to Unsubscribe): https://www.ietf.org/mailman/listinfo/tools-discuss

---

From: Tim Wicinski <[email-masked]>
Date: Sat, 4 Sep 2021 13:35:47 -0400

> and minutes going forward in markdown"?
+1 on our Markdown overlords.

and offer different output formats via the API?

as much as I have long loved my rsync, I feel we are reaching
that tipping point of delivering everything via API.

tim

---

From: "Prof. Diego Dujovne" <[email-masked]>
Date: Sat, 4 Sep 2021 13:40:05 -0400

Hi.
Wouldn't it be nice to have the discussion in a kind of forum or a
discussion thread associated directly to the draft, instead of going to the
E-mail?
Thanks.
Regards,

        Diego Dujovne

El sáb., 4 de septiembre de 2021 13:36, Tim Wicinski <[email-masked]>
escribió:

> List info (including how to Unsubscribe):
> https://www.ietf.org/mailman/listinfo/tools-discuss
>

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 5 Sep 2021 18:05:30 +0200

> The rsync endpoint you are looking for is at rsync.ietf.org::proceedings

Thank you!  That was the missing link.

I set up a little tool for my personal use, “minutes-wurst.rb” (named such because it compiles all recent minutes for a WG into a big “Wurst”, which is a mostly unstructured text file (“Wurst”) intended to be in chronological order).

Example output at

https://tzi.de/~cabo/minutes-core.txt
https://tzi.de/~cabo/minutes-cbor.txt

(My broken web server serves these as text/plain as opposed to text/plain;charset=utf-8 — sorry about that.  Wget/curl-O and open locally.)

Tool source for your amusement at:
https://gist.github.com/cabo/8e99a77502e4920ee2c3d52227f97a1e
(Some assembly required.)
The tool requires rsyncing the proceedings first, which weigh at 50 GB — but selective rsyncing might speed this up (didn’t try to figure this out).

If people find this little tool useful, tell me; I might make this available in a more immediately useful way.
(I might also take a couple more minutes to implement PDF to text conversion, which I simply didn’t need for these two WGs.)

Grüße, Carsten