Subject: [openpgp] OpenPGP Web Key Directory I-D

From: Ian Jackson <[email-masked]>
Date: Wed, 7 Nov 2018 19:49:19 +0000

I was referred here
  https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/

I'm not sure exactly what the status of this I-D is or whether the
openpgp ietf list is the right place, but it seems to be the best
place to send comments.

I. URL final pathname component format

It specifies a URL format ending in a base-32-encocded SHA-1 of a
mangled version of the email address associated with the PGP key.

This seems rather odd.

1. SHA-1 is obsolete.

2. The use of a cryptographic hash here makes it harder for a server
   to conduct an appropriate lookup.  For example, if a server defines
   that all email addresses
       alice+<something>@example.com
   are owned by Alice, and Alice tells the server `please advertise
   my one OpenPGP key for all my email addresses', it is not clear how
   the server could implement that.

2a. The cryptographic hash does not, however, provide any significant
   degree of useful obfuscation since a search will usually be able to
   reverse it.

2b. The cryptographic hash is not needed for space reasons since URLs
   can easily be as long as email addresses.

3. Supposing the hash were to be retained, choice of base-32 rather
  than base-64 is unusual and needs to be justified.

4. The lowercasing of the email address is contrary to the Internet
   mail specifications, where case-sensitivity of the email address
   is up to the mail domain in question.  If the email address were
   not obfuscated by hashing it would be easy for the webserver to
   handle case-sensitivity by URL remapping.

Suggested modification: Replace this part of the URL with the
URL-encoded email address.

II. URL domain name part

The mail system for some domain, and its web server, are not
necessarily on the same host or under the same practical
administration.  Often webservers are outsourced.

Trying to provide this .well-known/openpgpkey subpath may therefore
involve complicated interactions between different teams or even
different organisations entirely.

Furthermore, the webserver may be less secure than the mail system;
whereas this protocol assumes that it is at least as secure.

Suggested modification: the domain name part should have a fixed
string prepended.

III. Normative status of this document

I was referred to this I-D from this trail of web pages:
  https://wiki.gnupg.org/EasyGpg2016/PubkeyDistributionConcept
  https://wiki.gnupg.org/WKDDetails
which I reached from someone who asked whether they should
deploy this system.

This seems a bit odd.

Ian.

-- 
Ian Jackson <[email-masked]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

---

From: NIIBE Yutaka <[email-masked]>
Date: Thu, 08 Nov 2018 10:17:08 +0900

Hello,

> II. URL domain name part
[...]
> Suggested modification: the domain name part should have a fixed

In page 4, the draft suggests possible use of "wkd" sub-domain by DNS
SRV resource record.  The example says: "wkd.example.org" for
"example.org".

Does it work for your intended use case?

Or... should we care about some broken edge router implementation which
behaves wrongly about DNR SRV RR query?
--

---

From: "brian m. carlson" <[email-masked]>
Date: Thu, 8 Nov 2018 01:25:59 +0000

>    handle case-sensitivity by URL remapping.

I definitely agree that lowercasing the address is wrong.  The RFCs say
that the local part is case sensitive, and there are many case-sensitive
systems on the Internet today.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

---

From: Werner Koch <[email-masked]>
Date: Thu, 08 Nov 2018 07:59:24 +0100

> systems on the Internet today.

Please tell me a single public accessible system which is
case-sensitive.  Ask any non-hacker about mail addresses; almost
everyone enters mail addresses in whatever case they like.  I have seen
many business cards which spell [email-masked] despite that the
canonical address is [email-masked].  See also the OpenPGP DANE
RFC for this.

Shalom-Salam,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

---

From: Werner Koch <[email-masked]>
Date: Thu, 08 Nov 2018 08:18:34 +0100

> 1. SHA-1 is obsolete.

|   The use of SHA-1 for the mapping of the local-part to a fixed string
|   is not a security feature but merely used to map the local-part to a
|   fixed-sized string made from a well defined set of characters.  It is
|   not intended to conceal information about a mail address.

>    the server could implement that.

The problem here is that the client needs to know how the server handles
this.  This is because the client should filter the returned key for a
matching mail-addresses.  Actually we have had a discussion about this
recently in Brussels with the outcome that for experiments we now
appened "?l=joe.hacker" to the request.  Currently implemented servers
will just ignore these parameters.

>    degree of useful obfuscation since a search will usually be able to

See above

>    can easily be as long as email addresses.

Right.  A constant length can make implementations easier. YMMV.
Further there is no limit on the length of the local part of the
addrspec.  It may well exceed the max. length of file names.

>   than base-64 is unusual and needs to be justified.

Easy: The standard Base-64 includes a slash and thus the has can't be
used as a file name.

>    handle case-sensitivity by URL remapping.

See my reply to Brian and above.

> URL-encoded email address.

Nope: That breaks existing implementations and would not allow to serve
the data from static files.

> administration.  Often webservers are outsourced.

That is rarely the case but given that I know a pretty large provider
which has this problem, SRV records are used to overcome this problem.

> Suggested modification: the domain name part should have a fixed

This is an open question but for a different reason: Current web
browsers have no way to lookup SRV records.

> III. Normative status of this document

|   Internet-Drafts are draft documents valid for a maximum of six months
|   and may be updated, replaced, or obsoleted by other documents at any
|   time.  It is inappropriate to use Internet-Drafts as reference
|   material or to cite them other than as "work in progress."

OTOH, it is a standard practise that RFCs are based on existing
implementations and we have that implemented in in GnUPG 2.1.12 (May
2016) and thus for example also in Debian Stretch

Salam-Shalom,

   Werner

---

From: Paul Fawkesley <[email-masked]>
Date: Thu, 8 Nov 2018 11:08:23 +0000

On 08/11/2018 07:18, Werner Koch wrote:
> matching mail-addresses.

The requirement for the client to filter the return key for matching
user IDs is a fairly big blocker for us.

We (and previous large organisations I've worked in) use + aliases
feature extensively to distinguish different services, for example:

* [email-masked]
* [email-masked]

>  Actually we have had a discussion about this

For experimenting, fine, but I'd prefer to get to a "production"
solution ASAP.

> the data from static files.

To stay compatible, keep supporting static files *and* resolve the case
and aliasing issues, I'd support:

1. keep supporting sha1-z-base-32 identifier

2. additionally start supporting z-base-32 identifier with no SHA1

   2.a maybe put them at different path
   2.b don't lowercase anything before searching for these
   2.c in the client, don't filter on the user id for these

Ian, apart from z-base-32 not being exactly mainstream, does 2. work for
you?

> which has this problem, SRV records are used to overcome this problem.

Given that SRV records can't be accessed by Javascript implementations,
I don't think we can point to that as a solution

I'm recalling a large organisation I worked in that were heavy users of
PGP. I can see two options for how I could have implemented WKD there:

1. request an HTTP redirect on the root domain (this one change would
take weeks of "change control" process, with lots of signing off by line
managers etc.) Then I can run a WKD service wherever I like.

  (the hosting for different subdomains and mail server were sprawling
across many different systems and providers)

2. use a subdomain e.g. wkd.example.com (also through a lengthy change
control process to get the subdomain)

> browsers have no way to lookup SRV records.

I'd support this.

I'd also support dropping the whole SRV records part to simplify the spec.

Paul

---

From: Werner Koch <[email-masked]>
Date: Thu, 08 Nov 2018 13:00:31 +0100

> user IDs is a fairly big blocker for us.

We can't request a certain address and then work with any address
returned from the server.  I am pretty sure that there will be server's
which store the entire key and not used the key stripped off all other
user-ids.  Thus the filtering on the client site is important.

> feature extensively to distinguish different services, for example:

Yes, I know.  We discussed this withe protonmail folks.  One idea was to
ask the server for the rules it uses to canonicalize the address.  That
will be quite complex and needs additional roundtrips.  So the other
idea is to simply go with the '+' separator and have a special case for
it.  Implementation wise it this requires changes at several palce to
have a consistent user experience.

> 1. keep supporting sha1-z-base-32 identifier

Already doing that.

>    2.b don't lowercase anything before searching for these

GnuPG 2.2.11  does this as descipned in my last mail.
For example the URI to lookup the key for [email-masked] is now

     https://example.org/.well-known/openpgpkey/
     hu/iy9q119eutrkn8s1mk4r39qejnbu3n5q?l=[email-masked]

No hashing is required, the case is not changed; the parameters undergo
the usual percent-quoting.

>    2.c in the client, don't filter on the user id for these

That is the harder part.  The matching in gpg is anyway case-insensitive
and I would like to implement '+' style sub-addresses; that is ignoring
everything from '+' to '@'.

> I don't think we can point to that as a solution

It is a pity that solid network techniques need to be dropped in favor
of large but uncooperative browser vendors.  Adding an interface for SRV
records would be really easy and helps to support flexible network
administration.  Note that I don't wan't them to use SRV records for
generic http requests but extensions should be able to use it instead of
resorting to external DNS lookup providers.  [Arggh, it is the whole
everything must be http thing - I bet we will sooner or later see a new
IP protocol number for HTTP-ng].

> I'd also support dropping the whole SRV records part to simplify the spec.

I recently looked into a protocol *forgot which one) which didn't used
SRV but a domain name prefix.  So, do you think that a strategy of:

First try

     https://openpgpkey.example.org/.well-known/openpgpkey/...

if that host can't be resolved (or accessed?), fallback to

     https://example.org/.well-known/openpgpkey/...

be a practical replacement for SRV records here?

Shalom-Salam,

   Werner

---

From: Ian Jackson <[email-masked]>
Date: Fri, 9 Nov 2018 10:50:29 +0000

> the data from static files.

It certainly would allow serving the data from static files.  If you
wanted case-insensitivity and can't configure your webserver to smash
the case, then then in practice you could make three files.

Overall, I described this protocol to Simon Tatham (author of PuTTY)
in the pub last night and he spluttered into his cider and said "is it
April the 1st in a different timezone?"

> 2016) and thus for example also in Debian Stretch

I don't think most of the Debian folks realise they are participating
in a protocol design experiment.

Since you are still in the protocol design phase, you would no doubt
welcome implementation and deployment of an alternative simpler
protocol ?

Ian.

---

From: Werner Koch <[email-masked]>
Date: Fri, 09 Nov 2018 12:17:50 +0100

> wanted case-insensitivity and can't configure your webserver to smash

It is not only about the case but about allowed characters in a file
name.  In particular '/' and depending on file system the length.  Noet
that '/' is a valid character in the local part of the addrspec.

> welcome implementation and deployment of an alternative simpler

Nope.  It is in use for more than 2 years.

The only simpler thing which could have been done would be to skip the
hashing and directly use the z-base-32 encoding.  The only drawback
would have been that very long addresses won't work on all file systems.

Changes to the SVR record thing should be possible because I doubt that
this is widely used (Caesonia specifies it use, though).

Salam-Shalom,

   Werner

---

From: Wiktor Kwapisiewicz <[email-masked]>
Date: Fri, 9 Nov 2018 12:37:07 +0100

> would have been that very long addresses won't work on all file systems.

There is also base64*url* variant that is "URL-safe":

https://en.wikipedia.org/wiki/Base64#URL_applications

It's widely used in several "Web" technologies (e.g. JSON Web Tokens RFC
7519).

> Nope.  It is in use for more than 2 years.

I agree. "?l=" solves most practical problems with WKD for
service-providers in a backwards-compatible way and there is a lot of
software using WKD (GnuPG, Enigmail, OpenKeychain, Openpgpjs, Mailpile...).

If I'd be designing such a protocol now I'd probably overlay it on top
of WebFinger (RFC 7033) but currently the benefits would not
counter-balance added work for everyone involved.

Kind regards,
Wiktor

-- 
https://metacode.biz/@wiktor

---

From: Werner Koch <[email-masked]>
Date: Fri, 09 Nov 2018 14:41:52 +0100

> counter-balance added work for everyone involved.

I actually looked into WebFinger but figured that there is no need to
employ yet another protocol format (JSON) and possible require special
handling on the server side.

Salam-Shalom,

   Werner

---

From: Wiktor Kwapisiewicz <[email-masked]>
Date: Fri, 9 Nov 2018 15:16:26 +0100

> handling on the server side.

Good points, now WKD is single-roundtrip and easy to deploy in static
setups although at the cost of making "special handling on the server
side" harder (due to SHA-1).

Looking forward to the I-D updated with "?l="!

Kind regards,
Wiktor

---

From: "brian m. carlson" <[email-masked]>
Date: Fri, 9 Nov 2018 21:27:05 +0000

> canonical address is [email-masked].  See also the OpenPGP DANE

My mail system is case sensitive.

Even ignoring me as an example, there's now SMTPUTF8, which means that
case folding is nontrivial.  Turkish has a dotted I and dotless I, and
case folding a Turkish email address in the traditional ASCII way could
produce invalid results even if the system is case-insensitive.  Greek
sigma case folds differently depending on position.  Moreover, I expect
some SMTPUTF8-capable systems don't case fold non-ASCII characters.

Even if you think this is not an issue, RFC 5321 requires that the
local-part "MUST be…assigned semantics only by the host specified", and
we should not knowingly violate other IETF RFCs in writing our own.
This is a MUST directive; it is not optional.

If you adopted Ian Jackson's suggestion to not hash the name, then case
sensitivity wouldn't be a concern; you could simply choose to let the
remote system accept whichever case you wanted.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

---

From: Bart Butler <[email-masked]>
Date: Fri, 09 Nov 2018 23:18:57 +0000

Hi all,

Based on discussions in Brussels, our current plan at ProtonMail is to implement, on the 'serving' side:

.well-known/openpgpkey/hu/<hash>?l=[email-masked]

accepting everything including and empty string for the hash and using the 'l' query string parameter exclusively for lookup for all the reasons previously mentioned in this thread and discussed in Brussels (case sensitivity, +aliases/subaddresses, Unicode, catch-all addresses). The hash would be ignored.

I think that long-term, two parameters that do the same thing and could conflict is bad and that while compatibility is a good short-term goal, we should try drop the hash and to migrate to this final form as soon as possible:

.well-known/openpgpkey/hu/?l=[email-masked]

While this doesn't allow a filesystem to serve WKD directly, I'm fairly certain that one could do it reasonably easily with, say mod_rewrite and some minimal transformations or use RewriteMap with a script which does whichever transformation you'd like (lowercase, remove subaddresses, etc).

As for the SRV record discussion, I agree that JS support is a problem and unfortunately that means supporting HTTPS. I'm wondering if we should simplify this and simply mandate the 'wkd' subdomain, full stop, rather than having a fallback mechanism to the main domain. The reason for this is that a client could determine whether the HTTP lookup is worth doing from whether the WKD subdomain resolves rather than making an experimental HTTP request for every domain periodically and contributing to the internet's 404 rate.

With the current scheme, relatively high-volume clients like us would probably need to do requests for the policy document periodically for each domain we send to and cache the result to avoid getting banned for generating too many 404s. The TTL for that caching would be decided by the us, the client. With the 'wkd' subdomain all clients would get that kind of caching for free via DNS, with a TTL controlled by the WKD server, which is both simpler and more logical I think.

-Bart Butler

Sent from ProtonMail, encrypted email based in Switzerland.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> > To stay compatible, keep supporting static files and resolve the case

> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>

---

From: Bart Butler <[email-masked]>
Date: Fri, 09 Nov 2018 23:50:00 +0000

I also wanted to clarify in the WKD I-D that requests should follow HTTP redirects (301, 302). As I provider we would like this so that a client using us to host their email could just set up a redirect to our WKD server. It needs to be at the HTTP level rather than using a CNAME record because we do not want to have to host our clients' TLS certificates.

-Bart Butler

> ..well-known/openpgpkey/hu/<hash>?l=[email-masked]
> 

> ..well-known/openpgpkey/hu/?l=[email-masked]
> 

> On Thursday, November 8, 2018 4:00 AM, Werner Koch [email-masked] wrote:
>

---

From: Benjamin Kaduk <[email-masked]>
Date: Fri, 9 Nov 2018 23:00:50 -0600

> Nope.  It is in use for more than 2 years.

I feel some obligation to push back on this -- if there is no willingness
to deviate from the deployed implementation, why not just document the
existing implementation behavior as part of the implementation's
documentation and move on?

-Ben

---

From: Werner Koch <[email-masked]>
Date: Sat, 10 Nov 2018 11:13:08 +0100

> existing implementation behavior as part of the implementation's

Simply because there are deployed services which required quite some
in-person discussion.  Telling them that they need to replace it by
something new is not a convincing argeument for running a web key
directory.

Sure over time it can eventually be removed.

Salam-Shalom,

   Werner

---

From: Werner Koch <[email-masked]>
Date: Sat, 10 Nov 2018 11:25:21 +0100

On Sat, 10 Nov 2018 00:18, bartbutler=[email-masked]
said:

> reasons previously mentioned in this thread and discussed in Brussels
> (case sensitivity, +aliases/subaddresses, Unicode, catch-all
> addresses). The hash would be ignored.

BTW, the sub-addressing does not seem to be a real problem.  A cursory
inspection of some large keyrings showed that user-ids with
sub-addresses are quite rare and there is always the opportunity for the
user (or a tool) to create another user-id w/o the sub-address.  Thus
the sub-addresses can be handled in the MUA and won't need protocol
support.

> ..well-known/openpgpkey/hu/?l=[email-masked]

I disagree but I don't think it is the time to discuss this now.  Let us
first deploy a useful key discovery and then see how it can be
improved/changed.

> should simplify this and simply mandate the 'wkd' subdomain, full
> stop, rather than having a fallback mechanism to the main domain. The

I concur.  Given that we need to drop the SRV records for silly reasons
anyway, we can also demand a fixed subdomain.  Given that I don't like
the "wkd" acronym, I would prefer to use a different name, like
"openpgpkey".

And regarding your other mail: Sure, a redirection can only be allowed
to use a http redirect and not with a CNAME.

Shalom-Salam,

   Werner

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 10 Nov 2018 04:30:30 -0600

> Sure over time it can eventually be removed.

Thanks for the clarification -- I probably misread the context in which the
original message was given.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 11 Nov 2018 23:58:15 -0600

> the "wkd" acronym, I would prefer to use a different name, like

I'll note that the IESG is generally not super-keen on reserved leaf names
in the DNS, though it is not something that is entirely disallowed when
there are not usable alternatives.
https://tools.ietf.org/html/draft-moonesamy-dnsop-special-use-label-registry-00
is (IIUC) supposed to be a way forward to at least provide a discoverable
registry of these "reserved" names.

-Ben

> _______________________________________________
> openpgp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/openpgp

---

From: Bjarni Runar Einarsson <[email-masked]>
Date: Mon, 12 Nov 2018 16:33:33 -0000

Hello all!

Werner Koch <[email-masked]> wrote:
> It is a pity that solid network techniques need to be dropped
> in favor of large but uncooperative browser vendors. Adding an
> interface for SRV records would be really easy and helps to
> support flexible network administration. Note that I don't
> wan't them to use SRV records for generic http requests but
> extensions should be able to use it instead of resorting to
> external DNS lookup providers.

On behalf of Mailpile, I'd just like to speak up and say I am
very glad to hear we are moving away from SRV records.

Mailpile tries very hard to protect the privacy of its users;
when fetching things from the public Internet (usually the web),
the easiest way to do that is to route the requests over Tor.

This is very easy for HTTPS - so looking up keys at predictable,
stable HTTPS URLs (exactly what the format of the URL is, doesn't
matter much) is something we are very comfortable and happy to
do. This means WKD (w/o SRV records) is currently our preferred
mechanism for discovering keys that we haven't already received
over AutoCrypt or imported by hand.

If I were to implement support for SRV records, that would mean I
can no longer rely on Tor to do that for us, but need to start
thinking about DNS-over-HTTPS or other emerging standards (or,
hacks ;-) in order to maintain the same security/privacy
guarantees. It's just more complicated to do in a
privacy-preserving fashion, which means in any resource-starved
project (aren't we all?) it might not happen at all.

I'm very happy not to have to deal with that.

> be a practical replacement for SRV records here?

This works well for Mailpile.

I might be tempted to suggest trying the bare domain first, and
openpgpkey.example.org as a fallback, simply because from a
privacy point of view that leaks less information about what the
client is doing.

But I understand this would probably waste more bytes/CPU cycles
and it requires more complicated logic to determine when to fall
back, and when to treat the server's 404 as final. So it may not
be worth it.

I am also quite happy with the ?l=... solution to expose the
original e-mail address to servers who want to be smarter than
can be accommodated with a fixed hash in the filesystem.

In general, I applaud the fact that this appears to be converging
on something that is dead-simple to implement both on the client
and the server, even if the "fixed subdomain" is a hack from a
protocol-purity point of view. It's pragmatic and it works, which
is (IMO) exactly what we need in this space.

All the best,
 - Bjarni

- -- 
PageKite.net lets your personal computer be part of the web

---

From: Werner Koch <[email-masked]>
Date: Tue, 13 Nov 2018 15:15:05 +0100

> thinking about DNS-over-HTTPS or other emerging standards (or,

Well, GnuPG implements a full DNS resolver over Tor (but w/o DNSSEC).
This was required to properly implement access to the keyserver pools.
If there is a need we coul turn this into a public API.

> I'm very happy not to have to deal with that.

Mailpile will also like it.

> This works well for Mailpile.

I changed this in the -07 I-D to 

  https://openpgpkey.example.org/.well-known/openpgpkey/example.org/...

to make it easier to host several domains and to convey the domain info
without resorting to HTTP header info.

> privacy point of view that leaks less information about what the

But in this regard it is not different from SRV RRs.  The requests
should anyway be easy to identify because the reply is pretty small or
by utilizing the fact that an encrypted mail is anyway soon send to the
same provider.

> protocol-purity point of view. It's pragmatic and it works, which

Right, but Mozilla and MS Exchange do something very similar to ease the
configuraion of a mail account.

Salam-Shalom,

   Werner