Subject: [openpgp] Web Key Directory I-D -07

From: Werner Koch <[email-masked]>
Date: Tue, 13 Nov 2018 15:02:04 +0100

Hi!

A new revision of the Web Key Directory I-D has been published:

  https://www.ietf.org/id/draft-koch-openpgp-webkey-service-07.txt

Changes since -06 are:

- Specify the advanced method with the openpgpkey sub-domain.
- Specify the l=LOCAL-PART query parameter.
- Require the provider to filter the key for publication.
- Drop the use of DNS SRV records.

See below for the gist of the change.  GnuPG master implements the new
advanced method.  You may use my address for testing.  For now the SRV
method is still used as a fallback by GnuPG.

Note that the domain name is now also part of the file name if the
openpgpkey sub-domain is used.  This should make it easier to server the
directory for several domains from a single server.  This sub-domain
approach is similar to Mozilla's mail auto configuration [1].

Shalom-Salam,

   Werner

--8<---------------cut here---------------start------------->8---
   There are two variants on how to form the request URI: The advanced
   and the direct method.  Implementations MUST first try the advanced
   method.  Only if the required sub-domain does not exist, they SHOULD
   fall back to the direct method.

   The advanced method requires a sub-domain with the fixed name
   "openpgpkey" is created and queried.  It constructs the URI from the
   concatenation of these items:

   o  The scheme "https://",

   o  the domain-part,

   o  the string "/.well-known/openpgpkey/",

   o  the domain-part in lowercase,

   o  the string "/hu/",

   o  the above constructed 32 octet string,

   o  the unchanged local-part as a parameter with name "l" using proper
      percent escaping.

   An example for such an advanced method URI to lookup the key for
   [email-masked] is:

     https://openpgpkey.example.org/.well-known/openpgpkey/
     example.org/hu/iy9q119eutrkn8s1mk4r39qejnbu3n5q?l=Joe.Doe

   (line has been wrapped for rendering purposes)

   The direct method requires no additional DNS entries and constructs
   the URI from the concatenation of these items:

   o  The scheme "https://",

   o  the domain-part,

   o  the string "/.well-known/openpgpkey/hu/",

   o  the above constructed 32 octet string,

   o  the unchanged local-part as a parameter with name "l" using proper
      percent escaping.

   Example for a direct method URI:

     https://example.org/.well-known/openpgpkey/
     hu/iy9q119eutrkn8s1mk4r39qejnbu3n5q?l=Joe.Doe

   (line has been wrapped for rendering purposes)

[...]
   The benefit of the advanced method is its greater flexibility in
   setting up the Web Key Directory in environments where more than one
   mail domain is hosted.  DNS SRV resource records, as used in earlier
   specifications of this protocol, posed a problem for implementations
   which have only limited access to DNS resolvers.  The direct method
   is kept for backward compatibility and to allow providing a Web Key
   Directory even with without DNS change requirements.
--8<---------------cut here---------------end--------------->8---

[1] <https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration>

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

---

From: Bart Butler <[email-masked]>
Date: Tue, 13 Nov 2018 20:35:15 +0000

Hi Werner,

Thanks for making these changes! On first read my only comment is about this line:

"The key needs to carry a
   User ID packet ([RFC4880]) with that mail address."

I think this statement is a little vague, as one of the reasons for providing the unchanged local part was to allow the server to do routing in the same way for WKD as it does for incoming mail. As such, things like case, subaddresses with +, catch-all, etc. will necessarily sometimes return a key with a UserID packet which does not exactly match the local part used to query.

I was wondering what you think about saying something like:

The key MUST carry a
   User ID packet ([RFC4880]) with what the server considers the canonical form of the requested mail address.

So if I request from ProtonMail [email-masked], I would get a key back with [email-masked], and the clients could then prompt on unrecognized types of mismatches if desired because they would know that the server is returning the canonical form of the address.

-Bart

Sent from ProtonMail, encrypted email based in Switzerland.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> [1]https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration
>

---

From: Werner Koch <[email-masked]>
Date: Tue, 13 Nov 2018 22:13:46 +0100

On Tue, 13 Nov 2018 21:35, bartbutler=[email-masked]
said:

> routing in the same way for WKD as it does for incoming mail. As such,
> things like case, subaddresses with +, catch-all, etc. will

We had some internal discussion and came to the conclusion that it is
best to not care about sub-addresses in the protocol.  It should be a
MUA only thing and nobody should create a key for a subaddress.

With the help of Kristian I took a look at the 5.3 million keys on the
SKS servers and we found only 3055 unique mailboxes with a '+' in it.
After removing leading and trailing '+' as well as multiple '+'
(e.g. "c++" or "foo+bar+baz") 2697 were left which seem to be valid
sub-addresses. 

Now this is definitely a minority and there oweners can be asked (or
gpg-wks-client does it on the fly) to create another user-id without the
subaddress.

To help MUAs, I started to change gpg to strip off sub-addresses; at
least for WKD queries.

> So if I request from ProtonMail [email-masked], I would
> get a key back with [email-masked], and the clients could

I doubt that we can do anything about this except for adding another
user id to the key.  There would be just too many cases and that simple
protocol would be much complex to implement and also fully lose the
property of a simple one to one match.

Shalom-Salam,

   Werner

---

From: Bart Butler <[email-masked]>
Date: Tue, 13 Nov 2018 21:37:33 +0000

Hi Werner,

I don't think I was very clear before so I'll try to clarify a bit. I think we are in agreement that the MUA should do all of this kind of processing exclusively and we should not do any funny business in the WKD spec about secondary UserIDs or the like. My concern was that this language:

could be read as requiring that the UserID packet in the key match the queried address exactly, and I'd like to relax that requirement to make it clear that servers can essentially serve up whichever key they choose based on how they want to route mail. Whatever checks the MUA does on this UserID is not the business of the WKD spec. Here's another suggestion:

"The key MUST carry a User ID packet ([RFC4880]) containing the email address to which mail sent to the queried email address will be routed."

which should take care of this case without implying that requested mail address must match the mail address in the UserID.

-Bart

> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>

---

From: Werner Koch <[email-masked]>
Date: Wed, 14 Nov 2018 11:09:30 +0100

> "The key MUST carry a User ID packet ([RFC4880]) containing the email address to which mail sent to the queried email address will be routed."

You are talking about how mail is routed, the spec is about discovering
the one and only key to be used for a given mail address.  And by key I
mean the OpenPGP keyblock, that is the public key plus one user ID (or
several if they have the same addrspec part).

A mail address is here considered as an identifier for an entity and not
as an addressing scheme for mails.  An entity may have several
identifiers like Werner.Koch@foo, Werner_Koch@foo, wernerkoch@foo,
wk@foo, koch@foo.  That is a pretty normal but there is no way a sender
can decide whether they are all the same; for example the last two of
the list could also identify my brother.

A sender gets hold of one mail address and that must have been relayed
(direct or indirectly) to them by the owner of that mail address.  The
recipient needs to take care that a key exists for that very mail
address.

Sub-addresses are different and we can find a way to handle this common
case. 

Salam-Shalom,

   Werner

---

From: Paul Wouters <[email-masked]>
Date: Wed, 14 Nov 2018 10:26:58 -0500 (EST)

> So if I request from ProtonMail [email-masked], I would get a key back with [email-masked], and the clients could then prompt on unrecognized types of mismatches if desired because they would know…

We went through this with OPENPGPKEY and SMIMEA. You will get the SMTP
people blocking your draft when you try to interpret (or like above,
even rewrite) addresses based on "common mail provider rewrites". Their
firm believe is only receiving SMTP servers can interpret email addresses.

Perhaps you want to take a look at the language used in those two
RFCs with respcet to mapping email addresses to DNS. Some of it
applies here too, and you will have some new ones with special
characters in URL's.

This took a year to resolve for RFC 7929 and RFC 8162. So I recommend
you look at previous discussion on this topic on the openpgp and dane
mailing lists.

Paul

---

From: Phil Pennock <[email-masked]>
Date: Wed, 14 Nov 2018 22:03:07 -0500

> firm believe is only receiving SMTP servers can interpret email addresses.

Bart isn't suggesting encoding common rewrites into OpenPGP, although I
believe that Werner might be.  As an MTA maintainer, I think Bart's
suggestion is entirely appropriate and inline with the ethos of
federated SMTP.  I'll proceed on the assumption that my analysis of
Bart's intent is correct.  Bart, please correct me if wrong.

In Bart's suggestion, the WKD server, run within the same autonomous
mail-system (or email administrative domain) as the SMTP servers, can
decide to map addresses and return a key suitable for use for that email
address.  This does not change where the MUA should send email, but does
adjust how a well-integrated MUA might choose an appropriate OpenPGP key
for use for a given email address.

This is entirely in line with the federated design and only the owner of
the systems at that location getting to make such decisions.  There are
implications around caching, because such a rewrite becomes a long-term
commitment if it's to be stored as ancillary data in the keyring for
choosing the correct key.

Thus if presented with a new address [email-masked] and needing
to get a key for it, with Bart's proposal, the MUA and the OpenPGP
client software can make no assumptions.  It must not normalize anything
to the left of the '@' sign.  But the MUA can use WKD and get back a key
for <[email-masked]>; the software can then record a mapping of
[email-masked] -> [email-masked] in OpenPGP recipient key
selection preferences.  When later sending email to
[email-masked], the SMTP transaction proceeds unmodified: the MUA
does not rewrite the recipient, you have to preserve the address
as-given.  The remapped OpenPGP key selection proceeds as suggested
though.  If sending email to [email-masked] then another WKD
lookup needs to be made.  (Future work might look at protocols for
indicating patterns to avoid repeated lookups).

Because server-side mapping decisions are being held elsewhere, and
because people do change PGP keys (in particular: security teams which
cycle their keys annually) this does suggest that the HTTP cache control
expiration headers might play into SHOULD controls on how the OpenPGP
client expires mappings and puts in place refresh checks.

The spirit of Bart's proposal, as I read it, is a great fit for the
autonomy of mail administrative domains.

-Phil

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 14 Nov 2018 22:57:43 -0600

> indicating patterns to avoid repeated lookups).

I'm probably confused, but is this implying that WKD would insert a new
"lookup" operation such that a compromised WKD could cause me to encrypt a
message to an attacker-controlled key (with different UID) when I am trying
to encrypt to a non-attacker peer?

Thanks,

Ben

---

From: azul <[email-masked]>
Date: Thu, 15 Nov 2018 10:13:21 +0100

Hello,

> recipient needs to take care that a key exists for that very mail
I think the last sentence exactly captures the difference between the
two approaches.
Who needs to take care that a key exists / pick an existing key for a
different email address routed to the same mailbox.

One of the use cases I have for the '+' syntax is registering to
services with email addresses that allow me to remember where i used
them and filter mails based on them.

One example would be using [email-masked]' to register
to a conference.
It's low overhead. I can just come up with a addition to my email
address when filling in a web form. If I later receive spam to that
address I know who leaked my email address.

Providing a key for the new address or even publish all those addresses
in my key do not seem like an option for me. Therefor the party I
submitted the form to will not be able to get back to me with an
encrypted email even if i have a key in the wkd.

I don't quite understand the downside of relaxing the spec in this
regard. As far as i understand it it would not require implementations
to lookup which mailbox an address would route to. It would just allow
it. For some providers / implementations this may be trivial and other
may choose to not do it.

Where does the need for the matching user_id come from?

One downside I see is that the mechanism could be used to detect where
an email address routes to. In the case of azul+conf this may be easy to
guess. But for aliases users do not expect them to be linkable without
any interaction on their side.

Cheers,
 Azul

---

From: azul <[email-masked]>
Date: Thu, 15 Nov 2018 10:16:55 +0100

Hi,

> to encrypt to a non-attacker peer?
As far as i understand the compromised WKD could easily send you
an attacker controlled key with a valid UID as well.

Why would the different UID make the attack any worse?

Cheers,
 Azul

---

From: Werner Koch <[email-masked]>
Date: Thu, 15 Nov 2018 12:22:42 +0100

> It's low overhead. I can just come up with a addition to my email

Right, and we more or less agreed that this is a good and useful
extension.  I merely don't think that it should go into the protocol,
because it is a MUA thing.  GnuPG will very likely help MUAs by taking
care to strip of such sub-addresses when looking for keys in a remote
directory [1].

Salam-Shalom,

   Werner

[1] See
<https://dev.gnupg.org/rG6b9f772914624cc673ba26d49b6e3adc32dd7e0a>
for a start

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 15 Nov 2018 13:42:36 -0600

> Why would the different UID make the attack any worse?

It seems like there's a transparency difference --if the WKD gives me a
totally spoofed key (valid/expected UID but attacker-controlled key
material) then that key could/should be on the keyservers, and the actual
holder of that UID can notice it and take action.  If I as the WKD consumer
get wholly redirected to the attacker's real key/UID, the actual intended
recipient doesn't have any way to discover that there was an attack.

-Ben

---

From: Bart Butler <[email-masked]>
Date: Thu, 15 Nov 2018 20:45:50 +0000

> any interaction on their side.

If they use a shared key for this purpose they are theoretically linkable via fingerprint anyway. In the case of things like catch-all addresses, I think this can be left up to the MTA to implement how they see fit. I think in most cases for catch-all the destination address is not sensitive.

---

From: Bart Butler <[email-masked]>
Date: Thu, 15 Nov 2018 20:48:27 +0000

Hi Phil,

You got it spot on, and were much more articulate than I was able to be despite multiple attempts. Thank you!

Yes, all I want to do is relax that one sentence in the spec so that no UserID match, however that is defined, is required in terms of the key returned by the WKD server. We can still require a valid UserID packet of course.

I also like leveraging HTTP cache control headers in the way you suggested, though that can be discussed separately at a later date.

-Bart

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, November 14, 2018 7:03 PM, Phil Pennock <[email-masked]> wrote:

> for [email-masked]; the software can then record a mapping of
> openpgp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/openpgp

---

From: Bart Butler <[email-masked]>
Date: Thu, 15 Nov 2018 20:49:09 +0000

> recipient doesn't have any way to discover that there was an attack.

The MUA could always have some kind of warning in this situation if the UserID match isn't recognized ("recognized" matches could include subaddresses, etc. but would be at the MUA's discretion). I'd leave this up to the MUA implementation.

-Bart

---

From: Paul Wouters <[email-masked]>
Date: Fri, 16 Nov 2018 02:03:14 -0500 (EST)

> The MUA could always have some kind of warning in this situation if the UserID match isn't recognized ("recognized" matches could include subaddresses, etc. but would be at the MUA's discretion). I…

Requiring the MUA to do this is wrong. It will break many potential use
cases. Take for example my phone mail client. It is hard to support PGP,
but it is easy to send it over TLS to my MTA. My MTA can then do all
the work to PGP encrypt it. But there are no humans in this process.

Please ensure this feature works without humans.

Paul

---

From: Bart Butler <[email-masked]>
Date: Fri, 16 Nov 2018 17:09:54 +0000

> Paul

I'm not proposing that we require the MUA to do anything. All I'm saying is the the MUA could implement such validation if they want to, otherwise the key returned by WKD could just be used, and either way, we don't make any sort of UserID email address matching part of the WKD spec that the server has to enforce.

-Bart