Subject: [Tools-discuss] Clickable URLs in slides displayed in Meetecho

From: Alexandre Petrescu <[email-masked]>
Date: Tue, 9 Nov 2021 13:52:57 +0100

Hi, Tools Discuss,

It would be great to have clickable URLs in the slides displayed in the 
Meetecho presentations.

Whenever a displayed slide shows an URL then a person assisting to the 
presentation should be able to click on it.

This is a very necessary feature, but nobody has implemented it in 
videoconf tools.  I suggested it to Webex and to others but each time I 
receive replies saying why it does not work.

On another hand, some pdf readers to recognize the text on images they 
display (use OCR - optical char recogn'n) and and make the URL links 
clickable in images.  Some times they clickwrong (the recognition was 
fooled) but in simpler cases it works fine.

So, if one uses free software OCR in Meetecho then it might work, more 
or less.  It would be a helpful feature.  It will help in very many 
contexts where people type the URLs in the chatbox, or where other 
people try to memorize quickly the URL links they see while the 
presenter shows the slide.

Alex

---

From: Tero Kivinen <[email-masked]>
Date: Tue, 9 Nov 2021 15:44:21 +0200

Alexandre Petrescu writes:
> This is a very necessary feature, but nobody has implemented it in videoconf
> tools.  I suggested it to Webex and to others but each time I receive replies
> saying why it does not work.
> 
> On another hand, some pdf readers to recognize the text on images they display
> (use OCR - optical char recogn'n) and and make the URL links clickable in
> images.  Some times they clickwrong (the recognition was fooled) but in
> simpler cases it works fine.
> 
> So, if one uses free software OCR in Meetecho then it might work, more or
> less.  It would be a helpful feature.  It will help in very many contexts
> where people type the URLs in the chatbox, or where other people try to
> memorize quickly the URL links they see while the presenter shows the slide.

Actually it would be much easier when using the slides uploaded to the
meetecho, i.e., the ones where chair upload pdf files and meetecho
converts them to pictures which it shares to users. From the pdf
finding links would be much easier, and generating image that has hot
spots that you can click should be possible.
-- 
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 9 Nov 2021 16:27:28 -0500

It appears that Alexandre Petrescu  <[email-masked]> said:
>-=-=-=-=-=-

In all of the sessions I have attended, the folder button at the top of the window links to copies
of the presentations so you can open them on your own computer and click the links to your heart's content.

I would think that recovering the links from a shared screen window
would be quite difficult, basically trying to do OCR in realtime and
figure out what looks like a link. Since the original links are
usually two clicks away, I can't get too upset about it.

R's,
John

---

From: Tero Kivinen <[email-masked]>
Date: Tue, 9 Nov 2021 23:34:02 +0200

> spots that you can click should be possible.

Playing around with python and poppler I made proof of concept
implementation of it...

Example output of that can be found from
https://kivinen.iki.fi/ietf-112-combined/slide-000.html 

Slide-001.html has links in the note well, and the next page as xmpp
meetecho and notes links. Unfortunately my ietf-112 slideset for
ipsecme had splitted the link for meetecho to two lines, and the
python code is smart enough to recognize that situation so the link
breaks in wrong location.

The code can recognize links written in the text (like on
slide-002.html), or as real links in pdf (see draft links on
slide-004.html).

By default it makes the box and very light blue background around
links, and adds the navigation buttons on the top, but both can be
disabled. This is proof of concept, so there is some error handling
missing. There are most likely some pdfs which are generated in such
way that it can't find the links (i.e., it now assumes the link text
inside the slide is as one text object, it does not try to merge
different text objects together to for a link).

It does calculate the resolution used for each page separately so that
the resulting image is around 3000 pixes wide, and then scales it down
to 1920x1080 resolution keeping the aspect ratio intact, so it should
work with different sized pdfs. As my combined pdf file
https://datatracker.ietf.org/doc/slides-112-ipsecme-chair-slides/ is
combination of several presentations each having different aspect
rations, and one of them having two pages very small page sizes
compared to other this was automatic calculation was actually
mandatory. 

The actual links are generated as an inline SVG overlay inside the
html file that is then placed over the png image of the slide...

So replacing the pdf -> imported slides program in meetecho with
something like this script would allow the links to be clickable...

----------------------------------------------------------------------
# Convert PDF to png images and html with inline svg that
# allows links on the image to be clicked.

import argparse
import re
import popplerqt5

def parseargs():
  parser = argparse.ArgumentParser(description='Extract pdf annotiations.')
  parser.add_argument('-d', '--debug', action='store_true')
  parser.add_argument('-n', '--navigation', action='store_true')
  parser.add_argument('-i', '--invisible', action='store_true')
  parser.add_argument('file')
  return parser.parse_args()

def main():
  args = parseargs()
  input_filename = args.file
  document = popplerqt5.Poppler.Document.load(input_filename)
  if document is None:
      exit(1)
  n_pages = document.numPages()

  css = """
<style>
.img-overlay-wrap { position: relative; display: inline-block; }
.img-overlay-wrap img { display: block; max-width: 100%; height: auto; z-index: 10; }
.img-overlay-wrap svg { position: absolute; top: 0; left: 0; z-index: 20; }
.links rect:hover { fill: yellow; fill-opacity: 0.8; }
  """

  if args.invisible:
      css += '.links rect { fill: white; fill-opacity: 0.0001; stroke:black;stroke-width:0; stroke-opacity:0.001; }\n'
  else:
      css += '.links rect { fill: blue; fill-opacity: 0.05; stroke:black;stroke-width:1;stroke-opacity:0.5; }\n'

  if args.navigation:
      css += """
.btns { position: absolute; z-index: 30; }
.btns a { font-size: 32px; font-weight: bold; color: #888;
          text-decoration: none; padding: 0px 10px;
          border: 2px solid #bbb; border-radius: 5px; }
.btns a:hover { background-color: #aaa; color: black; }
      """

  css += '</style>\n'
  
  rect = '''
  <a href="{4}" target="_blank">
  <rect x="{0}" y="{1}" width="{2}" height="{3}"/>
  </a>
  '''

  # Loop through all pages
  for i in range(n_pages):
    page = document.page(i)
    w = page.pageSize().width()
    h = page.pageSize().height()
    label = page.label()

    # Use width of 3000 and calculate resultion accordingly
    # and take suitable smaller resolution
    res = int(3000 / w * 72 / 25) * 25
    image = page.renderToImage(res, res)
    if image is None:
        print("Image rendering failed")
        exit(1)
        
    # last two arguments are KeepAspectRatio, SmoothTransformation
    image2 = image.scaled(1920, 1080, 1, 1)
    w2 = image2.width()
    h2 = image2.height()
    fn = 'image-{0:03d}.png'.format(i)
    fnh = 'slide-{0:03d}.html'.format(i)
    if not image2.save(fn):
        print('Saving image {0} failed'.format(i, fn))
        exit(1)
    if args.debug:
        print('Image {0} saved as {1} using resolution {2}'.
              format(i, fn, res)) 

    html = ''
    if args.navigation:
        html += '<div class="btns">\n'
        if i != 0:
            html += '<a href="slide-{0:03d}.html">&#8592</a>\n'.format(i - 1)
        if i != n_pages - 1:
            html += '<a href="slide-{0:03d}.html">&#8594</a>\n'.format(i + 1)
            html += '</div>\n'

    html += '<div class="img-overlay-wrap">\n'
    html += '<img src="' + fn + '">\n'
    svg = ''

    # Go through links in pdf
    links = page.links()
    for l in links:
        la = l.linkArea()
        lt = l.linkType()
        # Check that it is Browse
        if lt == 3:
            lu = l.url()
            lx = la.x() * w
            ly = la.y() * h
            lw = la.width() * w
            lh = la.height() * h
            if lh < 0:
                ly += lh
                lh = -lh
            if args.debug:
                print('Link on page {5} ({0:.2f},{1:.2f}) ({2:.2f},{3:.2f}) {4}'.
                      format(lx, ly, lw, lh, lu, label))
            svg += rect.format(int(lx / w * w2), int(ly / h * h2),
                               int(lw / w * w2), int(lh / h * h2), lu)

    # Go through texts on the page
    txts = page.textList()
    for t in txts:
        # Search for xmpp, ftp, http or https links, end it
        # ignore >, ., or ) at the end.
        m = re.search(r'((http|https|xmpp|ftp):[^\"\']*[^\>\)\.\"\'])', t.text())
        if m is not None:
            bb = t.boundingBox()
            lx = bb.x()
            ly = bb.y()
            lw = bb.width()
            lh = bb.height()
            u = m.group(1)

            if args.debug:
                print('Text link on page {5} ({0:.2f},{1:.2f}) ({2:.2f},{3:.2f}) {4}'.format(lx, ly, lw, lh, u, label))
            svg += rect.format(int(lx / w * w2), int(ly / h * h2), int(lw / w * w2), int(lh / h * h2), u)

    # Check if we had any links
    if svg != '':
        html += '<svg class="links" viewBox="0 0 {0} {1}">\n'.format(w2, h2)
        html += svg
        html += '</svg>\n'
    html += '</div>\n'
    f = open(fnh, "w")
    f.write('<!DOCTYPE html>\n<html>\n' + css + '<body>\n' +
            html + '</body>\n</html>')
    f.close()

if __name__ == "__main__":
  main()

-- 
[email-masked]

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 9 Nov 2021 22:36:39 +0100

> of the presentations so you can open them on your own computer and click the links to your heart's content.

This demonstrates that the feature is not “necessary”.

It sure would still be rather useful.

> figure out what looks like a link.

That sounds like a nice student assignment, but was not the suggestion.

The suggestion was to extract PDF links before rasterizing them and putting them back as links at the right place on the raster image (incredibly JPG right now…).
That is a tractable problem, but probably more than five minutes of work, even if most of it already has been done (e.g., [4]).

Grüße, Carsten

[1]: https://stackoverflow.com/questions/31436357/how-to-extract-all-links-from-pdf-file
The pdfx tool described there is GPLv3, but it should not be hard to extract the idea.
[2]: https://stackoverflow.com/questions/5973463/extracting-links-to-pages-in-another-pdf-from-pdf-using-python-or-other-method
[3]: https://www.metachris.com/pdfx/
[4]: https://github.com/metachris/pdfx

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 9 Nov 2021 22:49:17 +0100

On 2021-11-09, at 22:34, Tero Kivinen <[email-masked]> wrote:
> implementation of it…

Amazing.
Hats off to you…
(Please don’t forget to license the code.)

Being able to put user-contributed stuff like this into our day-to-day tools really makes a difference.
(Even if in this case it needs to go through meetecho.)

Grüße, Carsten

---

From: Tero Kivinen <[email-masked]>
Date: Wed, 10 Nov 2021 00:13:49 +0200

Carsten Bormann writes:
> That is a tractable problem, but probably more than five minutes of
> work, even if most of it already has been done (e.g., [4]).

Took 185 minutes from me, actual editing time... Not including the
time reading documentation and trying to find suitable pdf libraries
(I first tried another python poppler library, but that was using too
old poppler, and didn't have supprot for extracting text).
-- 
[email-masked]

---

From: Alexandre Petrescu <[email-masked]>
Date: Thu, 11 Nov 2021 10:57:09 +0100

Le 09/11/2021 à 22:27, John Levine a écrit :
>> It would be great to have clickable URLs in the slides displayed in
>> the Meetecho presentations.
> 
> In all of the sessions I have attended, the folder button at the top
>  of the window links to copies of the presentations so you can open 
> them on your own computer and click the links to your heart's 
> content.

YEs fair enough.  The links are reachable to be clicked, albeit through
a longer path.

> usually two clicks away, I can't get too upset about it.

OCR in realtime when the presenter advances the slide at the speed of 1
per second could be too difficult to keep up with, even though it
depends on the computing power.

But there seems to be now a phase of pre-upload of slides into meetecho
by the Chairs, so the OCR could be done in a pre-action manner
(preflight, etc.) at which time there would would be more time for OCR.

Alex

> R's, John

---

From: Alexandre Petrescu <[email-masked]>
Date: Thu, 11 Nov 2021 11:06:17 +0100

it works!

https://kivinen.iki.fi/ietf-112-combined/slide-001.html
and then below there is that ombudsteam link that seems to be output by
a process of recognition: it can be clicked.

(also the link works ok on IPv6)

Le 09/11/2021 à 22:34, Tero Kivinen a écrit :
> Tero Kivinen writes:
>> Actually it would be much easier when using the slides uploaded to
>> the meetecho, i.e., the ones where chair upload pdf files and
>> meetecho converts them to pictures which it shares to users. From
>> the pdf finding links would be much easier, and generating image
>> that has hot spots that you can click should be possible.
> 
> It does calculate the resolution used for each page separately so
> that the resulting image is around 3000 pixes wide, and then scales
> it down to 1920x1080 resolution keeping the aspect ratio intact, so
> it should work with different sized pdfs. As my combined pdf file 
> import argparse import re import popplerqt5
> 
> def parseargs(): parser =
> argparse.ArgumentParser(description='Extract pdf annotiations.') 
> parser.add_argument('file') return parser.parse_args()
> 
> def main(): args = parseargs() input_filename = args.file document =
> popplerqt5.Poppler.Document.load(input_filename) if document is
> None: exit(1) n_pages = document.numPages()
> 
> css = """ <style> .img-overlay-wrap { position: relative; display:
> inline-block; } .img-overlay-wrap img { display: block; max-width:
> 100%; height: auto; z-index: 10; } .img-overlay-wrap svg { position:
> absolute; top: 0; left: 0; z-index: 20; } .links rect:hover { fill:
> yellow; fill-opacity: 0.8; } """
> 
> if args.invisible: css += '.links rect { fill: white; fill-opacity:
> 0.0001; stroke:black;stroke-width:0; stroke-opacity:0.001; }\n' 
> else: css += '.links rect { fill: blue; fill-opacity: 0.05;
> stroke:black;stroke-width:1;stroke-opacity:0.5; }\n'
> 
> if args.navigation: css += """ .btns { position: absolute; z-index:
> 30; } .btns a { font-size: 32px; font-weight: bold; color: #888; 
> text-decoration: none; padding: 0px 10px; border: 2px solid #bbb;
> border-radius: 5px; } .btns a:hover { background-color: #aaa; color:
> black; } """
> 
> css += '</style>\n'
> 
> rect = ''' <a href="{4}" target="_blank"> <rect x="{0}" y="{1}"
> width="{2}" height="{3}"/> </a> '''
> 
> # Loop through all pages for i in range(n_pages): page =
> document.page(i) w = page.pageSize().width() h =
> page.pageSize().height() label = page.label()
> 
> # Use width of 3000 and calculate resultion accordingly # and take
> suitable smaller resolution res = int(3000 / w * 72 / 25) * 25 image
> = page.renderToImage(res, res) if image is None: print("Image
> rendering failed") exit(1)
> 
> # last two arguments are KeepAspectRatio, SmoothTransformation image2
> = image.scaled(1920, 1080, 1, 1) w2 = image2.width() h2 =
> image2.height() fn = 'image-{0:03d}.png'.format(i) fnh =
> 'slide-{0:03d}.html'.format(i) if not image2.save(fn): print('Saving
> image {0} failed'.format(i, fn)) exit(1) if args.debug: print('Image
> {0} saved as {1} using resolution {2}'. format(i, fn, res))
> 
> html = '' if args.navigation: html += '<div class="btns">\n' if i !=
> 0: html += '<a href="slide-{0:03d}.html">&#8592</a>\n'.format(i - 1) 
> if i != n_pages - 1: html += '<a
> href="slide-{0:03d}.html">&#8594</a>\n'.format(i + 1) html +=
> '</div>\n'
> 
> html += '<div class="img-overlay-wrap">\n' html += '<img src="' + fn
> + '">\n' svg = ''
> 
> # Go through links in pdf links = page.links() for l in links: la =
> l.linkArea() lt = l.linkType() # Check that it is Browse if lt == 3: 
> lu = l.url() lx = la.x() * w ly = la.y() * h lw = la.width() * w lh =
> la.height() * h if lh < 0: ly += lh lh = -lh if args.debug: 
> format(lx, ly, lw, lh, lu, label)) svg += rect.format(int(lx / w *
> w2), int(ly / h * h2), int(lw / w * w2), int(lh / h * h2), lu)
> 
> # Go through texts on the page txts = page.textList() for t in txts: 
> # Search for xmpp, ftp, http or https links, end it # ignore >, ., or
> ) at the end. m =
> re.search(r'((http|https|xmpp|ftp):[^\"\']*[^\>\)\.\"\'])',
> t.text()) if m is not None: bb = t.boundingBox() lx = bb.x() ly =
> bb.y() lw = bb.width() lh = bb.height() u = m.group(1)
> 
> if args.debug: print('Text link on page {5} ({0:.2f},{1:.2f})
> ({2:.2f},{3:.2f}) {4}'.format(lx, ly, lw, lh, u, label)) svg +=
> rect.format(int(lx / w * w2), int(ly / h * h2), int(lw / w * w2),
> int(lh / h * h2), u)
> 
> # Check if we had any links if svg != '': html += '<svg class="links"
> viewBox="0 0 {0} {1}">\n'.format(w2, h2) html += svg html +=
> '</svg>\n' html += '</div>\n' f = open(fnh, "w") f.write('<!DOCTYPE
> html>\n<html>\n' + css + '<body>\n' + html + '</body>\n</html>') 
> f.close()
> 
> if __name__ == "__main__": main()
> 
>

---

From: Alexandre Petrescu <[email-masked]>
Date: Thu, 11 Nov 2021 11:08:02 +0100

>> Playing around with python and poppler I made proof of concept

YEs me too hats off.  Happiness, thrillnes and a sentiment of 
recognition go all to you.

> ___________________________________________________________
> Tools-discuss mailing list - [email-masked]
> This list is for discussion, not for action requests or bug reports.
> * Report datatracker and mailarchive bugs to: [email-masked]
> * Report tools.ietf.org bugs to: [email-masked]
> * Report all other bugs or issues to: [email-masked]
> List info (including how to Unsubscribe): https://www.ietf.org/mailman/listinfo/tools-discuss
>

---

From: Meetecho IETF support <[email-masked]>
Date: Thu, 11 Nov 2021 18:16:25 +0100

Hi Tero,

thank you for this. The same kind of feature is already in the works - 
we are already grabbing all links from the pdf metadata when we convert 
them. We plan to ship this in the next major release of the platform.

Thanks!
Alessandro

---

From: Tero Kivinen <[email-masked]>
Date: Thu, 11 Nov 2021 21:40:57 +0200

> them. We plan to ship this in the next major release of the platform.

Good.

When I was testing this I noticed that pdf metadata links are not that
common in slides, quite often there is links just as a text on the
page, so thats why I added that step that finds the links from the
text too.

I was also really suprised how well the SVG overlay on top of the
image worked, so you should try that method when you implement this
feature.
-- 
[email-masked]

---

From: Alexandre Petrescu <[email-masked]>
Date: Thu, 11 Nov 2021 21:10:45 +0100

Clickable URLs is one thing, but 'hoverable' URL would be an advancement 
as well, probably.

Hovering over a URL which tells an RFC would conveniently display its 
title without actually having to click on it.  An example of such 
hoverable behaviour is the experience of browsing through wikipedia.

Le 11/11/2021 à 18:16, Meetecho IETF support a écrit :
>>        css += '.links rect { fill: white; fill-opacity: 0.0001; 
>> stroke:black;stroke-width:0; stroke-opacity:0.001; }\n'
>>    else:
>>        css += '.links rect { fill: blue; fill-opacity: 0.05; 
>>              html += '<a 
>> href="slide-{0:03d}.html">&#8592</a>\n'.format(i - 1)
>>          if i != n_pages - 1:
>>              html += '<a 
>> href="slide-{0:03d}.html">&#8594</a>\n'.format(i + 1)
>>              html += '</div>\n'
>>
>>          m = 
>> re.search(r'((http|https|xmpp|ftp):[^\"\']*[^\>\)\.\"\'])', t.text())
>>          if m is not None:
>>              bb = t.boundingBox()
>>              lx = bb.x()
>>              ly = bb.y()
>>              lw = bb.width()
>>              lh = bb.height()
>>              u = m.group(1)
>>
>>              if args.debug:
>>                  print('Text link on page {5} ({0:.2f},{1:.2f}) 
>> ({2:.2f},{3:.2f}) {4}'.format(lx, ly, lw, lh, u, label))
>> int(lw / w * w2), int(lh / h * h2), u)
>>

---

From: Tero Kivinen <[email-masked]>
Date: Fri, 12 Nov 2021 16:07:11 +0200

> Clickable URLs is one thing, but 'hoverable' URL would be an advancement 

Something like what is now in the
https://kivinen.iki.fi/ietf-112-combined/slide-001.html pages?

It does load the titles from the datatracker using datatracker api
when you first hover over the RFC or BCP name, and that will take some
time, so you might need to move mouse out from the link and back in to
get the tooltip visible (browsers wait for unspecified time before
they show the title when you stop, and depending on the network speed
and datatracker load the title might or might not be loaded at that
time).

The script to do conversion is in the same directory:
https://kivinen.iki.fi/ietf-112-combined/find-pdf-http-links3.py and
the javascript loading the titles is in the slides.js file in same
directory.

-- 
[email-masked]

---

From: Alexandre Petrescu <[email-masked]>
Date: Fri, 12 Nov 2021 18:36:43 +0100

Le 12/11/2021 à 15:07, Tero Kivinen a écrit :
>> Clickable URLs is one thing, but 'hoverable' URL would be an
>> advancement as well, probably.
> 

YEs, hovering a second time over the distinguished "BCP 79" does display
the tooltip "IPR in IETF Technos".

This is yes what I meant, and thank you very much.

This does help a lot during the meeting in order to understand what the
presenter means without having to download the pdf, click the URL, load
other pages.  It is a real shortcut.

Alex

> when you first hover over the RFC or BCP name, and that will take
> some time, so you might need to move mouse out from the link and back
> in to get the tooltip visible (browsers wait for unspecified time
> before they show the title when you stop, and depending on the
> network speed and datatracker load the title might or might not be
> loaded at that time).
> 
>> Hovering over a URL which tells an RFC would conveniently display
>> its title without actually having to click on it.  An example of
>> such hoverable behaviour is the experience of browsing through
>> wikipedia.