Subject: [Tools-discuss] polls "show of hands": can we include a "i don't understand" or "not enough information" option

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 18 Nov 2020 13:15:16 -0500

Currently, when there's a show of hands in meetecho, there are two
actions any participant can take:

 - raise my hand
 - do not raise my hand

in my experience, if a meeting has 100 participants, you're lucky to get
50 that select either of the above options.  At least half tend to give
no response at all.

There are two reasons to give neither response though:

 - the question is confusing, invalid, wrong, i didn't understand it, or
   i'm not sure of the answer

 - i am asleep/bored/not paying attention

It'd be nice to be able to to distinguish between these two large
classes of non-responding participants.  This is akin to including an
"N/A" or "Don't know" answer in a multiple choice survey.

Could such a option be added?  I don't want clutter with a dozen extra
options, but it would be useful to know how many people are paying
attention but object to the binary framing for whatever reason.

       --dkg

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 18 Nov 2020 13:26:32 -0600

FWIW, "binary questions hide information" matches my experience so far this
week.

I'm especially thinking of Gorry asking "who has read the draft?" in TSVWG
more than once, and realizing that he couldn't tell the difference between
"I haven't read the draft, but I care and I'm going to", and "I'm not going
to read the draft, because I don't care".

I am sympathetic to not dropping a dozen options for the group to read
through before clicking, but we can't even tell the difference between
packet congestion and packet corruption in TCP, if the only choices are
"ACKed" and "not ACKed".

I should also ask if this is a tool thng, or a social problem thing?

Best,

Spencer

On Wed, Nov 18, 2020 at 1:05 PM Daniel Kahn Gillmor <[email-masked]>
wrote:

> ___________________________________________________________
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
>
> Please report datatracker.ietf.org and mailarchive.ietf.org
> bugs at http://tools.ietf.org/tools/ietfdb
> or send email to [email-masked]
>
> Please report tools.ietf.org bugs at
> http://tools.ietf.org/tools/issues
> or send email to [email-masked]
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 18 Nov 2020 20:50:57 +0100

> I should also ask if this is a tool thng, or a social problem thing?

Yes.

Of course, a bespoke tool should have prepared questions for “did you read the draft” etc., with a useful choice of answers prepared.

Fire away with the requirements, I think your list is already 90 % there.

>  "I haven't read the draft, but I care and I'm going to", and "I'm not going to read the draft, because I don't care". 

Grüße, Carsten

---

From: Bob Hinden <[email-masked]>
Date: Wed, 18 Nov 2020 12:11:21 -0800

Spencer,

In my experience doing polls, asking three (or more) questions many times does not provide clear answers.

The current tool (and I note it is much improved from the last IETF hum tool), has three choices:

   Raise hand (aka Yes)
   Don’t raise hand (aka No)
   Nothing (aka Don’t care)

In my Monday session, we did a lot of “Have your read the draft” polls.    That was helpful to see as it told the chairs if enough people had read the draft (or not) to know that we were having a meaningful discussion.   I think it would be useful to get a sense of the room for things like adopting or advancing a draft.

I doubt we can build a tool that does much more than this, that is why we allow people to come to the mic and speak.

Bob

> On Nov 18, 2020, at 11:26 AM, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> FWIW, "binary questions hide information" matches my experience so far this week.
> 
> I'm especially thinking of Gorry asking "who has read the draft?" in TSVWG more than once, and realizing that he couldn't tell the difference between "I haven't read the draft, but I care and I'm going to", and "I'm not going to read the draft, because I don't care".
> 
> I am sympathetic to not dropping a dozen options for the group to read through before clicking, but we can't even tell the difference between packet congestion and packet corruption in TCP, if the only choices are "ACKed" and "not ACKed".
> 
> On Wed, Nov 18, 2020 at 1:05 PM Daniel Kahn Gillmor <[email-masked]> wrote:

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 18 Nov 2020 14:58:28 -0800

> attention but object to the binary framing for whatever reason.

Is there some reason why the responses have to be canned and can't be
provided by the chairs? Then we could just have as many as the chairs want
and with whatever content.

-Ekr

       --dkg

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 18 Nov 2020 18:23:08 -0800

On Wed, Nov 18, 2020 at 6:18 PM Daniel Kahn Gillmor <[email-masked]>
wrote:

> On Wed 2020-11-18 14:58:28 -0800, Eric Rescorla wrote:
> If we do this (i'd be happy with it, and we could provide some
> pre-canned choices for a WG chair that just wants something simple),
> it'd be nice to stage the polling options visibly so the group can
> preview them before the "hand raising" starts.
>
> My experience with in-person WG activity is that the chairs typically
> propose a hum with two options, look the (awake) people in the room in
> the eyes, and say "does this seem about right?" and about a quarter of
> the time someone says "hey, i think we need a third option, how about
> X".  Then the chairs agree or disagree, then the hum proceeds.
>
> That initial pause between the framing of the options and the conducting
> of the hum/poll is really useful to make sure that at least the awake
> participants are all trying to tackle roughly the same problem.  it'd be
> good to have something analogous in our online tooling.
>

This all sounds good to me.

-Ekr

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 18 Nov 2020 20:33:10 -0500

--dkg

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 19 Nov 2020 14:09:22 +1100

> preview them before the "hand raising" starts.

A related suggestion: have the poll take the place of the slides.  Or show it in an overlay.  The tabbed interface is quite poor for this. As a chair, I found that trying to construct a poll while also managing a queue and trying to monitor chat is impossible alone.  As a participant, with slightly less responsibility, but more engagement with discussion, it's nearly as bad.

Having chat visible while formulating the poll would be ideal as well, as suggestions for text could be more readily copied.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 18 Nov 2020 22:57:31 -0500

Maybe we need a "standard" set of emoji responses that WG chairs can put up for
each internet draft during the entire period of the discussion?

> "I haven't read the draft, but I care and I'm going to"
> "I'm not going to read the draft, because I don't care".
> "I read the draft, and as a result, I have no opinion"
...

It would ideally be attached to an agenda item.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Michael Richardson <[email-masked]>
Date: Thu, 19 Nov 2020 02:18:23 -0500

> A related suggestion: have the poll take the place of the slides.  Or
    > show it in an overlay.  The tabbed interface is quite poor for this. As

THIS.

---

From: Bob Hinden <[email-masked]>
Date: Wed, 18 Nov 2020 23:25:02 -0800

> On Nov 18, 2020, at 11:18 PM, Michael Richardson <[email-masked]> wrote:
> 
> Signed PGP part
> 

I agree.

Bob

---

From: Meetecho IETF support <[email-masked]>
Date: Thu, 19 Nov 2020 11:56:06 +0100

> Having chat visible while formulating the poll would be ideal as well, as suggestions for text could be more readily copied.

While this list is the right place to discuss change requests like the 
one above, OTOH I think the specs of the "show of hands tool" are 
captured in the draft below and comments on that should probably be 
addressed to the SHMOO list:

	https://tools.ietf.org/html/draft-duke-shmoo-show-of-hands-00

-Alessandro

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 20 Nov 2020 09:08:10 +1300

> A related suggestion: have the poll take the place of the slides.  Or show it in an overlay. 

Yes. That's essential. On at least two occasions I heard a chair reporting the result of a show of hands when I had no idea that it had taken place: nothing told me that I should have selected that tab. At least the tab should be forcibly popped on top, or it should start flashing, or something.

Regards
   Brian