Subject: [openpgp] Mailing lists

From: "Neal H. Walfield" <[email-masked]>
Date: Wed, 15 Jul 2015 16:05:00 +0200

Hi,

Encrypted mailing lists are currently difficult to do securely and
easily.  Either they trade security for usability by reencrypting mail
or they trade usability for security by requiring each poster to keep
a local list of subscribers up to date.

I think many cases can be easily accommodated at the OpenPGP level.
My proposal is relatively informal.  This is because: I haven't
actually implemented it yet; I would like some feedback on the idea
before pursuing it further; and, I don't have any experience writing
RFCs.  If the community agrees that the approach is reasonable, I'll
be happy to draft a more formal specification / patch for 4880bis.

The basic idea is to publish the list of subscribes as an OpenPGP
message.  Concretely:

  To create a mailing list, the mailing list administrator generates a
  new key pair in the usual way.

  Associated with the key pair is a list of encryption keys
  (Public-Key Packet, tag 6, or Public-Subkey Packet, tag 14).  Each
  key may optionally be preceded by a user id (User ID packet, tag
  13).  This list is stored in a signature subpacket with a new
  subpacket type.

    The list may be encrypted to hide the subscriber list.  In this
    case, the list will be encrypted to the set of authorized posters
    (which may or may not be the same as the set of subscribers).  The
    session key is encrypted for each poster and is saved in a
    public-key encrypted session key packet as usual.

    Since it is useful to hide the list of posters, as an extension to
    the public-key encrypted session key packet specification, if the
    first 6 octets of the key id are 0 and the last two are not, then
    the last two octets specify the last two octets of the key's id.
    I call these are "practically hidden key ids".  They are
    practically hidden, because 16-bits does not provide enough
    information to identify a key (it's trivial to create a collision
    unlike with a 64-bit id) and thus plausible deniability is
    ensured.  The 16-bits are useful, because there are potentially
    thousands of posters to a mailing list and if the public-key
    encrypted session key packets all have hidden key ids, it could
    take a long time to find the appropriate packet.  The 16-bits are
    a simple filter that reduces decryption time by a factor of 2^16.

    Keyservers / export commands should only include the most recent
    valid version of this packet.

  The key may contain two optional uids.  The first uid contains an
  email address that forwards the email to all of the mailing list's
  subscribers.  This user id has the following comment: "mailing
  list".  If there is no valid user id with the comment "mailing list"
  then the MUA should send the email to all of the UIDs listed in the
  encryption key list.  The second uid contains the email address of
  the list's owner.  It should have the following comment: "mailing
  list: owner".
  
  When encrypting, the implementation should make sure that the key is
  fresh.  It could do this by automatically downloading the key from a
  key server or, if the key hasn't been updated in the last 24 hours,
  by issuing a warning.

Thanks!

:) Neal

---

From: Bill Frantz <[email-masked]>
Date: Sat, 18 Jul 2015 15:09:03 -0700

>a local list of subscribers up to date.

A long time ago I was a member of an encrypted mailing list that 
used PGP. Every member of the list had a copy of the private PGP 
key (and its password). Messages to the list were encrypted 
using the public key and all the legitimate list members could 
decrypt using their copy of the private key.

It worked quite well. When someone was dropped from the list, 
new keys were needed and had to be distributed, which was a 
disadvantage, but practically resulted in somewhat regular key changes.

There may be better solutions, but this one worked with 
unmodified PGP.

Cheers - Bill

---------------------------------------------------------------------------
Bill Frantz        |"After all, if the conventional wisdom was 
working, the
408-356-8506       | rate of systems being compromised would be 
going down,
www.pwpconsult.com | wouldn't it?" -- Marcus Ranum

---

From: "Neal H. Walfield" <[email-masked]>
Date: Sun, 19 Jul 2015 00:54:48 +0200

Hi,

Thanks for your thoughts, I appreciate them.

At Sat, 18 Jul 2015 15:09:03 -0700,
Bill Frantz wrote:
> A long time ago I was a member of an encrypted mailing list that used
> PGP. Every member of the list had a copy of the private PGP key (and
> its password). Messages to the list were encrypted using the public
> key and all the legitimate list members could decrypt using their copy
> of the private key.
> 
> It worked quite well. When someone was dropped from the list, new keys
> were needed and had to be distributed, which was a disadvantage, but
> practically resulted in somewhat regular key changes.
> 
> There may be better solutions, but this one worked with unmodified
> PGP.

Yes, I think this is a reasonable solution, but it's not easy to use.

The solution I propose is easy to use and it largely backwards
compatible.

  - Only posters need to understand the new format; subscribers just
    decrypt as usual.

  - If a poster encrypts using the mailing list's key, the admin can
    always decrypt and reencrypt.  Depending on the security
    requirements, this can be completely or partially automated.

Thanks!

Neal

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 18 Jul 2015 22:28:58 -0400

I don't think this is a good time to do mailing lists in OpenPGP. A mailing
list is essentially a form of multi-access drop box. There is a list of
people who can write to the drop box and a list of people who can read.

The only thing that distinguishes a mailing list from a drop box is the way
the data is organized. But I think it much easier to organize the contents
of a shared drop box to be a mailing list than to try to do anything on top
of SMTP, let alone add security. The OpenPGP bit is fine, its the SMTP bit
that is the problem.

There are two basic ways a dropbox type scheme can be made to work with
standard public key

* There is a shared public key and everyone knows the private key. This is
changed each time a person drops off the list.

* Each person has an individual public key pair and the mailing list is
encrypted and sent out to each of them.

---

From: Bill Frantz <[email-masked]>
Date: Sun, 19 Jul 2015 06:39:09 -0700

On 7/18/15 at 7:28 PM, [email-masked] (Phillip 

>changed each time a person drops off the list.

This approach will probably scale to reasonable levels. When it 
gets to when you are sending out new keys several times a day, 
then batching the drops may be a viable solution.

>encrypted and sent out to each of them.

This approach has real scaling problems. Assume the mailing list 
software does the encryption. When you get a large list, then 
the CPU load of encrypting the symmetric key to each member will 
be quite high. The alternative seems to be to have the sender do 
the encryption, but then every list member needs to have every 
other's public key and a smart phone may be completely overwhelmed.

So the question is, how large a list do we need to support? The 
practical high water mark may come with a large organization 
that needs a mailing list for all its members. The internal 
mailing list of a corporation with 100,000 employees may be a 
good example. Of course, a secret which that many people know 
isn't very secret.

Cheers - Bill

-----------------------------------------------------------------------
Bill Frantz        | "The only thing we have to   | Periwinkle
(408)356-8506      | fear is fear itself." - FDR  | 16345 
Englewood Ave
www.pwpconsult.com | Inaugural address, 3/4/1933  | Los Gatos, 
CA 95032

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 19 Jul 2015 09:54:09 -0400

On Sun, Jul 19, 2015 at 9:39 AM, Bill Frantz <[email-masked]> wrote:

> On 7/18/15 at 7:28 PM, [email-masked] (Phillip Hallam-Baker) wrote:
>
> This approach will probably scale to reasonable levels. When it gets to
> when you are sending out new keys several times a day, then batching the
> drops may be a viable solution.

At some point though, the process of dropping the new keys becomes
equivalent to a cloud service with knowledge of the content and we lose end
to end.

> This approach has real scaling problems. Assume the mailing list software
> does the encryption. When you get a large list, then the CPU load of
> encrypting the symmetric key to each member will be quite high. The
> alternative seems to be to have the sender do the encryption, but then
> every list member needs to have every other's public key and a smart phone
> may be completely overwhelmed.
>
> So the question is, how large a list do we need to support? The practical
> high water mark may come with a large organization that needs a mailing
> list for all its members. The internal mailing list of a corporation with
> 100,000 employees may be a good example. Of course, a secret which that
> many people know isn't very secret.
>

100,000 subscribers does not need to be end to end. There is no security
advantage, there is a huge cost.

---

From: "Neal H. Walfield" <[email-masked]>
Date: Mon, 20 Jul 2015 12:02:42 +0200

Hi Phillip,

Thanks for the feedback.

At Sat, 18 Jul 2015 22:28:58 -0400,
Phillip Hallam-Baker wrote:
> I don't think this is a good time to do mailing lists in OpenPGP.

If now is not a good time to add mailing list support to OpenPGP, then
I'll retract my proposal.  What is the reason that now is not a good
time to discuss this issue?  Do you think this should wait until after
4880bis is finished and we should then recharter?

> A mailing list is essentially a form of multi-access drop box. There
> is a list of people who can write to the drop box and a list of
> people who can read.

> The only thing that distinguishes a mailing list from a drop box is
> the way the data is organized. But I think it much easier to organize
> the contents of a shared drop box to be a mailing list than to try to
> do anything on top of SMTP, let alone add security. The OpenPGP bit is
> fine, its the SMTP bit that is the problem.

I don't understand this.  Isn't a mailing list normally on top of
SMTP?  When you say that the OpenPGP bit is fine, but that the SMTP
bit is the problem, what do you mean?  Are you arguing that we should
abandon SMTP altogether and use some other transport?  I think this
will be a hard fight and not one that the OpenPGP WG should take on.
The reality is that people are trying to do secure mailing lists over
SMTP and it is currently hard.

> There are two basic ways a dropbox type scheme can be made to work
> with standard public key
> 
> * There is a shared public key and everyone knows the private key.
> This is changed each time a person drops off the list.

I think this is fine for small, technically apt groups, but I don't
this is scales or is suitable for more casual use.  Alternatively, I
may just not be aware of all of the tools that make this easy.  If so,
I'd appreciate some pointers.

> * Each person has an individual public key pair and the mailing list
> is encrypted and sent out to each of them.

There are two possibilities here.

First, the mailing list software reencrypts the message.  This means
that the plaintext is handled on the server and that either the
private key material is directly on the server or it is on a smartcard
permanently attached to the server.  In the former case, the key
material can be stolen and in the later case, an attacker can send out
fake messages until the break-in is discovered.  This implies absolute
trust in the provider.  These issues limit the applicability of this
solution.

The other possibility is that each poster encrypts to all recipients.
This works, but has horrible management issues.  In particular, the
posters need to synchronize the list of subscribers and keys.  This is
not easy.  There is another management problem as well, which is less
obvious.  The mailing list participants might want the list of
recipients to be hidden.  In GnuPG, this is done using the
--hidden-recipient option.  In this case, the public-key encrypted
session key packets don't contain the recipients' keyids.  Making sure
all posters remember to use this option is hard.  This is also not
scalable: the recipients will need to decrypt (on average) half of the
public-key encrypted session key packets before they find the packet
for them.

My proposal solves these problems.  There is no reencryption.
Distribution takes advantage of existing infrastructure, namely, key
servers.  The packet format includes mailing list options and provides
an optimization so that normally only a few public-key encrypted
session key packets need to be decrypted when hiding the recipients.
My proposal is also largely backwards compatible with existing OpenPGP
implementations.  In particular, it is completely backwards compatible
for recipients and requires the mailing list administrator to
reencrypt posts from posters if they are not using a compliant OpenPGP
implementation.

Now, my proposal is not perfect.  In particular, posters still need to
encrypt the session key for all of the subscribers, which can take a
while if the list is large.  But, this represents a security trade-off
and users can decide if this cost justifies the added security
relative to the convenience of a reencryption gateway.

Thanks,

Neal

---

From: "Neal H. Walfield" <[email-masked]>
Date: Mon, 20 Jul 2015 12:12:57 +0200

At Sun, 19 Jul 2015 06:39:09 -0700,
Bill Frantz wrote:
> practical high water mark may come with a large organization that
> needs a mailing list for all its members. The internal mailing list of
> a corporation with 100,000 employees may be a good example. Of course,
> a secret which that many people know isn't very secret.

I think this is a very good point.  But, once you get to mailing lists
with that many subscribers, I think the mode of operation changes.  In
particular, the number of posts per unit time is probably very small
and there are probably only a small number of authorized posters.
These posters are probably not posting from their phones, but sending
messages that were partially composed by the PR department.  As such,
computational cost is probably not a concern.

I think a solution that scales to thousands of subscribers is probably
more than adequate.

Neal

---

From: "Neal H. Walfield" <[email-masked]>
Date: Mon, 20 Jul 2015 12:14:18 +0200

At Sun, 19 Jul 2015 09:54:09 -0400,
Phillip Hallam-Baker wrote:
> equivalent to a cloud service with knowledge of the content and we
> lose end to end.

Can you elaborate on this?  If the drops are encrypted to all of the
posters why does the server know the content?

Thanks,

Neal

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Wed, 22 Jul 2015 01:30:36 +0200

> trust in the provider.  These issues limit the applicability of this

I think the goal of an attacker in most scenarios isn't key recovery,
but cleartext recovery.  an attack on the remailer service doesn't need
access to the key even if it's on a smartcard, they just want to
compromise the server enough to read the messages after the server
decrypts and before it encrypts.

> public-key encrypted session key packets before they find the packet

This approach is clunky, and i agree that your approach is superior to
it for key management.  however, both of these approaches also leak the
size of the subscriber list (as well as historical join and leave dates)
in the public keyring.  if they're re-using their normal
encryption-capable subkeys (or if they have User IDs attached) then
anyone who has access to the list's OpenPGP certificate is going to know
who all of the subscribers are.

There is a third approach, though, which is that used by PSELS:

       http://sels.ncsa.illinois.edu/about.html
       http://www.ncsa.illinois.edu/People/hkhurana/ICICS.pdf

I haven't seen anyone re-tread this idea to use more modern crypto, but
i think it might work.

The idea is (roughly, from memory) that you're using some sort of
discrete-log-based encryption key (e.g. el gamal, encryption-capable
ECC), and the list has a public key.  All correspondence to the list is
encrypted to the list's public key.

The (human, offline) mailing list manager knows a secret key for the
list, and when a new subscriber joins, the manager issues the new
subscriber a personalized secret key that has a specific randomized
offset from the list's secret key, and tells the list server the
subscriber's address, public key, and their offset from the list's key.

The (automated) mailing list server doesn't know the list's secret
material, but it knows the offset between all subscribers' public keys
and the encryption key, and modifies the message so that the subscriber
can decrypt it.

unsubscribe happens by removing the address and the offset from the
mailing list server.  End users send encrypted mail exactly as they
currently do, and the automated server never sees the cleartext.

one chokepoint in this system is certainly the mailing list manager, who
has access to the secret key material; but this is no different from
your proposal, where whoever has access to the list's primary key has
the same powers.

In the PSELS approach, if any subscriber can collude with the list
server, i think they can jointly reassemble the list's secret key, which
is an additional risk, i think.  though any subscriber on their own
already has the content of all the messages, so i'm not sure how
important the risk is.

Of course, none of this avoids the problem of a large "private" list
actually having the security of the weakest link in the group :/

         --dkg

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 22 Jul 2015 09:32:37 +0200

> already has the content of all the messages, so i'm not sure how

I was playing with a similar approach, it translates fine into any DH
system in any group (and you want to do DH in a group...).

This is another consequence of the DH key multiplication capability I will
be presenting an application of in CFRG later today. DH is actually a much
more interesting cipher system than RSA as far as 'exotic' approaches go.

If we have a Right private key R and a Left private key L, we can derive a
Joint key J as follows:

J = R+L
e^J = e^R . e^L

Further, if we perform the computations in a finite group it can be shown
that knowledge of e^R and e^L does not provide any information about J
unless knowledge of e^R discloses R (in which case DH is broken in that
group anyway).

The thing is that this is only a way to avoid the need to store the private
key for the mailing list on the system. Everything else is completely
unchanged. The end users can use their regular DH key.

This is a very very useful property and one that the folk doing PERC should
probably be looking at.

It isn't one that has any impact on OpenPGP (or S/MIME) because it can be
implemented regardless of the base spec. The only constraint being that the
mailing list members use a DH based key.

---

From: "Neal H. Walfield" <[email-masked]>
Date: Sat, 25 Jul 2015 13:10:58 +0200

Hi Daniel,

Thanks for your comments!

At Wed, 22 Jul 2015 01:30:36 +0200,
> decrypts and before it encrypts.

I wonder if key recovery is as unimportant as you suggest.  If I were
an attacker, I wanted to spy on a mailing list, and I got local access
to the box doing the reencryption, I'd rather get the private key and
get out than maintain a presence on the box.  This would seem to
reduce the chance that I was discovered.  But, perhaps I'm
misunderstanding something.

> in the public keyring.

Good point.  This can be mitigated by adding cover traffic.
Concretely: we update the subscriber list with a fixed frequency
(e.g., everyday at midnight) and we add / remove a number of
subscribers drawn from an exponential distribution (or vice versa).
If there aren't enough real new subscribers, then we add some dummy
subscribers.

Of course, an adversary can more or less figure out the subscriber
list by watching the SMTP traffic.  My understanding is that although
it is possible to encrypt SMTP traffic, no one actually verifies the
keys, which makes the whole thing susceptible to MITM attacks.  Of
course, a MITM attack requires significantly more resources than
simply analyzing data that is explicitly stored in a public database.

> if they're re-using their normal

I think what you are saying is that all posters know who all of the
subscribers are (rather than just the list admin).  I'm not sure this
is really a problem in practice.  It's normal that subscribers know
more or less who the posters are.

> encrypted to the list's public key.

Thanks for pointing this out.  I wasn't aware of this work.

> actually having the security of the weakest link in the group :/

True.  But that is a people problem and there is nothing we can do
about that at the OpenPGP level.

I spent some time reviewing the PLES paper that you linked to and I
see major security and usability problems with it.

The first problem is a security problem.

  - Since the user has a new private key for each mailing list, it is
    not practical to store all the keys on a smart card.  Thus, the
    keys will probably be stored on the user's hard drive.  This
    significantly lowers the practical upper bound of the security of
    the system.

The rest of the problems are usability problems.

  - When the user subscribes to a mailing list, the list manager sends
    the subscriber the private key.  The subscriber needs to import
    this.  I don't think current MUAs make this easy.

  - If the user wants to use the key from multiple machines, she needs
    to import it on all of them.  It is currently not easy to securly
    move one private key from one machine to another.  Moving many
    keys would be a serious pain.

  - It encourages a bad mind set: we tell users that private keys
    should never be exported and yet here is a counter example.

You identified two weaknesses with my proposed approach:

  - It exposes the subscriber list to subscribers (including
    historical join / leave times).

  - It exposes the size of the subscriber list to the world.

I think these weaknesses are genuine, but not severe.  Regarding the
first issue, in practice, subscribers know who the other subscribers
are (or, at least the posters).  With respect to the second issue,
this can be mitigated using cover traffic (although this would place a
large burden on the keyservers).  Nevertheless, there is no way to
stop a well funded adversary who can monitor SMTP traffic, which is a
problem for both systems.

Do you still think we should drop my proposal?

Thanks,

Neal