Subject: [openpgp] Can the OpenPGP vs. S/MIME situation be fixed?

From: Hanno Böck <[email-masked]>
Date: Fri, 1 Jul 2016 15:33:04 +0200

Hi,

Maybe this is a crazy idea, but I wanted to throw it into the
discussion.

IMHO a big problem with e-mail encryption is that there are two
competing "official" standards: OpenPGP and S/MIME. Both are RFCs, so
both have a kinda "official" IETF approval.
I think it was a big mistake to create two competing standards in the
first place, but that was back in the 90s. So we may ask if we want to
live forever with this situation or if it can be fixed.

One of the most common explanations for the two standards I hear
is that S/MIME is the solution for business communications while
OpenPGP is more for private users. This never made a lot of sense to
me, because there are plenty of situations where "business" people may
have to communicate with "private" people. And the requirements aren't
any different. E-Mail encryption is supposed to ensure that no
unauthorized people can read or manipulate your mail, that doesn't
change whether you're using E-Mail for private or business
communication. So essentially I think there is no rational case for
competing standards.

So the question is: Instead of making RFC4880bis a "new OpenPGP
standard", could it instead be a successor of both OpenPGP and S/MIME?
Maybe it needs a new name, maybe not. There seems to be an smime working
group and there is still some activity, although the last RFC was
published in 2009. Things would obivously have to be coordinated so
that there is wide acceptance of the new standard.

Technically it would probably mean to create a compatibility layer to
be able to use both X.509 certificates and PGP keys to encrypt. But
that shouldn't be too hard, as the keys itself are just numbers, the
major difference is just the storage format.

Maybe this is a crazy idea, but maybe this could also be a chance to
fix one of the biggest mistakes in email encryption.

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: [email-masked]
GPG: BBB51E42

---

From: Thijs van Dijk <[email-masked]>
Date: Fri, 1 Jul 2016 17:01:27 +0200

Hanno,

This is not a crazy idea at all.
I would welcome and applaud this effort.

It's an idea I've been mulling over as well; in fact I'd envisioned taking
it one step further and introducing a new (v5?) key format that can
transparently embed an X.509 certificate.
This way, key signatures as we in the PGP universe know them can work their
way into the X.509 world, as a way for a "corporate entity" (for lack of a
better word") to endorse an individual, or the other way around. This may
have profound implications for the way we anchor trust in, for instance,
TLS, as individuals can certify e.g. *"I've verified that this certificate
is valid for HTTPS connections to this site"* or even *"I've inspected the
operation of this CA and know them to have their act together"*.
The main things separating the above fantasy from reality are the fact that
   a) this would be hugely impractical *even if* the entire world would use
PGP, and
   b) the entire world does not use PGP.

Having said that, I would still consider unifying PGP and S/MIME a very
worthy direction for 4880bis to take, even if it isn't a prelude to the
above "web of trust ALL the things" daydream.

--
Thijs van Dijk

6A94 F9A2 DFE5 40E3 067E  C282 2AFE 9EFA 718B 6165

On 1 July 2016 at 15:33, Hanno Böck <[email-masked]> wrote:

> _______________________________________________
> openpgp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/openpgp
>
>

---

From: Derek Atkins <[email-masked]>
Date: Fri, 01 Jul 2016 11:10:29 -0400

Hi Hanno,

Hanno Böck <[email-masked]> writes:

[snip]
> that there is wide acceptance of the new standard.

Unfortunately from a process standpoint that is not an option.  That's
not to say that we cannot write such a draft/document, but it cannot be
"4880bis".

> fix one of the biggest mistakes in email encryption.

-derek
-- 
       Derek Atkins                 617-623-3745
       [email-masked]             www.ihtfp.com
       Computer and Internet Security Consultant

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 1 Jul 2016 12:48:58 -0400

I have wanted this for a long time. there are actually three separate
problems to be solved.

1) How to make S/MIME work with OpenPGP credentials

2) How to make OpenPGP work with S/MIME credentials

3) How to merge the two specifications into one.

The conditions for making the first two happen are easy. If there is a
will, IETF will find a way. We already have an OpenPGP group. The
chartering of a SPASM group is in the works. I do not think it is going to
be at all difficult to get ADs or IESG to approve work to make two existing
IETF standards interoperate. That is what IETF is for.

The third is still hard because it requires existing infrastructures to
merge and that is a long and difficult process unless you can deliver a
major improvement in functionality. I just can't see merger of two email
security standards offering encryption and signature into one offering that
incentive.

But that doesn't have to be what we do. I think we do have an option that
would be Blu-Ray to OpenPGP and S/MIME's Betamax and VHS. There are in fact
three technologies we can build on that offer dramatic improvements in
functionality.

1) Linked Timestamp (aka Blockchain)

Forget Bitcoin for a minute and proof of work. Linked Timestamps improve
the Work Factor of any PKI and you do not in fact need proof of work to
guarantee that. There are better, cheaper options to achieve the same
result.

Let us imagine for a moment that we upgraded the MIT Key Server
infrastructure that supports OpenPGP to a similar infrastructure that
included technology similar to Certificate Transparency. As soon as a key
signing or certificate or whatever is enrolled and the infrastructure
synchronizes, the Work Factor for backdating a forgery of that assertion to
before the enrollment date goes to 2^256. That is real cryptographic power.

2) Combining the Web of Trust and Brokered Trust (CA) models

People have fixed on the idea of one model or the other. What if we choose
both. The work factor of the resulting Webs of trust becomes very high very
quickly and more importantly the work factor values become objective.

3) Proxy Re-Encryption

[NB IPR encumbrance for the next 18 months]

Using Recryption, a user can encrypt a document to be read by a named group
of users (e.g. [email-masked]) using the public key for that group
and upload it to a server. The server can then create decryption keys for
each of the users that have been granted access by the administrator by
converting the decryption blob for the group into a decryption blob for
each authorized recipient. But the server can't decrypt the document itself.

Recryption is very very powerful and we should make it the heart and soul
of the next generation of message security infrastructure.

* Chat rooms which can only be accessed by people who are on the list
   *These can be text, voice, video, naturally
* Dropbox style document repositories
* Next generation email
* Internal document distribution.

Recryption offers real power and we have been ignoring it for too long.

Now a program of the type I am describing is obviously not something for
SPASM or OpenPGP to discuss. It is way beyond their charter. In fact some
folk will probably argue that this is IRTF work, not IETF.

But I do have the start of open source (MIT license) code for a system that
I believe could grow into this. And the code is almost on the verge of
working cross platform. It uses all the modern platforms you would expect,
JSON over HTTPS, consensus crypto algorithms, etc. etc.

I am trying to follow the path that Tim laid out for the deployment of the
Web - start off by concentrating on how to add value to existing code
bases. The early Web users weren't actually using HTTP very much. Most of
the information they were getting came from FTP, NNTP, WAIS and so on. The
main use of HTTP and HTML was to provide a common interchange format for
gateways to access legacy gateways.

So right now, all the Mathematical Mesh is focused on is making S/MIME and
OpenPGP and SSH and Web Usernames/Passwords easy to use. I am working to
make existing crypto applications as easy to use as legacy ones. This isn't
'OK usability' meaning follow a long list of instructions. As you all know,
I am an obsessive and a perfectionist when it comes to usability. This is
security that you won't know is there unless you are asking yourself if
something is safe and start looking into it.

But if the Mesh succeeds then we get to a point where a significant
userbase has private keys established on every single device they use. We
have a large client side PKI that can establish trust through Web of Trust,
PKI or hybrid methods. Once you have that in place, developing new
cryptographic applications to leverage that infrastructure is really
straightforward.

I could use some help.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 1 Jul 2016 14:27:59 -0400

Just a note to add that the way to get OpenPGP to work with S/MIME would be
for experts to get together and write drafts proposing a way to do it. If
we have drafts that have support from experts in both camps then getting
them through IETF process should not be a roadblock.

We don't need to charter or recharter before starting that type of work. We
only need to charter before we adopt it somewhere as a working group item.
The precondition for that being 'do you have a plan'.

---

From: Peter Gutmann <[email-masked]>
Date: Sat, 2 Jul 2016 15:08:55 +0000

>3) How to merge the two specifications into one.

The first two are pretty easy, I've been doing that for years.  For S/MIME,
use the subjectKeyIdentifier form of the key ID.  For PGP, use an
issuerAndSerialNumber in a type-and-value subpacket.

The third is impossible.  While at an abstract level PGP and S/MIME do the
same thing, the bit-bagging formats used to encode the abstraction are
completely incompatible.  You can't make them compatible without either moving
S/MIME to the PGP format or PGP to the S/MIME format.  I can't see either of
those happening...

Peter.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 2 Jul 2016 20:24:04 -0400

> S/MIME to the PGP format or PGP to the S/MIME format.  I can't see either

​
That would clearly be impossible if it was what was being proposed.

What I am suggesting is rather different, A new application for managing
encrypted content, Word, Powerpoint, PDF, etc. that has crypto designed
into the metal and also provides a messaging capability.

I am suggesting Blu Ray, not trying to develop adapters to play VHS on
Betamax.
​

---

From: Vincent Breitmoser <[email-masked]>
Date: Sun, 03 Jul 2016 09:43:05 +0200

*queue n+1 competing standards xkcd

 -  V

On 3 July 2016 02:24:04 CEST, Phillip Hallam-Baker <[email-masked]> wrote:
>> completely incompatible.  You can't make them compatible without
>either
>> moving
>> S/MIME to the PGP format or PGP to the S/MIME format.  I can't see
>either
>> of
>> those happening...
>
>
>​

---

From: Watson Ladd <[email-masked]>
Date: Sun, 3 Jul 2016 07:47:01 -0700

>> S/MIME to the PGP format or PGP to the S/MIME format.  I can't see either

The other issue, which is sadly overlooked, is implementation
simplicity and UX. Both OpenPGP and S/MIME fail hard on this front for
reasons that cannot be fixed easily. If Adam Langley and I cannot
easily send encrypted emails to each other (fun story about a summer
internship I had) without screwing up multiple times, what hope for
the rest of us? Has anyone fuzzed S/MIME clients to see if they parse
X509 correctly? My guess is some do, most don't, and you will find
exploitable bugs.

> encrypted content, Word, Powerpoint, PDF, etc. that has crypto designed into
> the metal and also provides a messaging capability.
>

-- 
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Sun, 03 Jul 2016 19:26:19 -0400

> live forever with this situation or if it can be fixed.

I agree with Hanno that this is a real concern, but we're currently
chartered with a simpler goal: revising the OpenPGP standard to use
sensible modern crypto going forward.  If we can do that well, then i'd
be all for thinking about a PGP/MIME update also, but i'd rather not
hold up 4880bis on this.

I think we should be clear about what it would take to do what you're
proposing; there are two main angles:

* certificate interoperability (OpenPGP certs vs. X.509 certs)

* message interoperability (PGP/MIME vs. S/MIME)

We should avoid foreclosing either form of interop with 4880bis, and if
simple modifications to 4880bis point the way toward easier future
interop without bogging down the process, including them would be fine.
But anything that obstructs or delays the goals of the charter should
probably be put off for future work.

(and remember: if we sort out 4880bis rapidly, "future work" doesn't
have to mean "the far future" -- let's show that we can get a
straightforward 4880bis done this year or early 2017 at the latest!)

      --dkg

---

From: Peter Gutmann <[email-masked]>
Date: Mon, 4 Jul 2016 03:41:18 +0000

>* certificate interoperability (OpenPGP certs vs. X.509 certs)

This is easily solved in a technical spec, just define (to use the approach
I've been using in my code, which as worked more or less seamlessy for some
years), the use of sKID for S/MIME and issuerAndSerialNumber for PGP.

>* message interoperability (PGP/MIME vs. S/MIME)

This can't be solved by a technical spec, it's an application issue which you
resolve by e.g. writing a PGP plugin for Outlook.

Peter.

---

From: Andrey Jivsov <[email-masked]>
Date: Sun, 03 Jul 2016 21:05:26 -0700

> years), the use of sKID for S/MIME and issuerAndSerialNumber for PGP.

Commercial PGP products used this type of "same key, two certificates" 
paradigm for over a decade. Some of this is documented in 
http://www.ietf.org/mail-archive/web/openpgp/current/msg01742.html 
(that's what PGP Corp. did; this write-up is incomplete).

One issue with storing OpenPGP KeyID in X.509 Subject Key Identifier 
(SKI) is that over the last decade and earlier popular S/MIME clients 
were not using SKI to identify a recipient. Instead, they were using the 
X.509 cert's Issuer and SN. Therefore, one will have to encode OpenPGP 
keyID into the SN of the X.509 cert to be able to locate the OpenPGP key 
later from the encrypted S/MIME message. This works if the ecosystem 
owns an issuing X.509 Sub-CA, so that it's possible to control the SNs.

---

From: Peter Gutmann <[email-masked]>
Date: Mon, 4 Jul 2016 04:14:39 +0000

Andrey Jivsov <[email-masked]> writes:

>One issue with storing OpenPGP KeyID in X.509 Subject Key Identifier (SKI) is
>that over the last decade and earlier popular S/MIME clients were not using
>SKI to identify a recipient. Instead, they were using the X.509 cert's Issuer
>and SN. Therefore, one will have to encode OpenPGP keyID into the SN of the
>X.509 cert to be able to locate the OpenPGP key later from the encrypted
>S/MIME message. This works if the ecosystem owns an issuing X.509 Sub-CA, so
>that it's possible to control the SNs.

We'd really need to get more data on what can handle sKID, since in my case
the use is all closed environments (banking, embedded, SCADA, etc) it's easy
enough to simply specify that the implementation needs to support sKID but
there's no current data (that I know of) on general support.  In any case I
think getting a small number of implementations to support sKID is going to be
vastly easier than asking CAs to put PGP IDs into certs.

In any case it doesn't cost anything to put the sKID/iAndS details into the
spec, and if you want it you've at least got an interoperable way of doing it.

Peter.

---

From: Jon Callas <[email-masked]>
Date: Mon, 04 Jul 2016 19:29:23 -0700

To chime in with Peter and Andrey, this is something that can done in software.

Not everything needs to be done in protocol. 

Whatever the details, one can (and perhaps should) use the same key material and dress it up in whatever uniform one wants, OpenPGP or S/MIME. 

While on the surface, it kinda seems like a good idea to unify the two in protocol, that's a different task than either group has. A new protocol would want to be a new protocol. Despite each protocol being used in much the same way (especially in email), there are a lot of things that would have to be re-hashed out.

There's how you issue certificates (the whole CA/introducer issue(s)), whether certs contain one key or key sets, how they are transported (S/MIME puts them in the message, OpenPGP in directories etc.), and even the role of the internal layering. Note that OpenPGP is a binary (and UTF-8 is still binary) object protocol with a drizzling of MIME-encoding frosting over the top. That frosting is subject to its own interpretations. S/MIME in contrast *starts* with the email and MIME object and underneath there's CMS, usually almost as an afterthought. (Did you have a momentary "huh?" in your head when you read CMS? Many people do, and that's the point.) S/MIME starts at the top, OpenPGP starts at the bottom.

And oh, there are also other things that have to be re-hashed like ASN.1 all over again and the things it drags along like encoding rules. This is a good deal why perhaps its better to just push the other things up into software. The reason that there are the two standards is that they address different views of the world, technical as well as political. 

At the end of the day, there are many things you *have* to push up into software. Consider the case where I am sending an email (which often happens, but may not even be the primary case in OpenPGP, merely the one that comes first to mind) to Alice, Bob, and Charlie. It's indeed irritating that Alice has an OpenPGP key and Bob an S/MIME certificate, and I am thus  going to have to code up two copies of the message. It is, however, straightforward. I know what to do. The subtlety comes from the fact that Charlie is being BCCed. It doesn't matter what happens with Charlie, whatever encoding we use (even plaintext) we have to send that message separately. You have to handle this at the software level no matter what. Even with a unified crypto standard, messaging isn't just crypto.

Unless the unifying protocol is so compelling that people of all stripes can agree that we should drop the old ones and go to this, we merely have a reification of an XKCD cartoon -- we'll have *three* protocols that have to exercised at the proper software level in exactly the same way you'd have to hand it with two. Trying to simplify will almost certainly just make things more complex.

	Jon

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 5 Jul 2016 19:05:20 -0400

On Mon, Jul 4, 2016 at 10:29 PM, Jon Callas <[email-masked]> wrote:

> Whatever the details, one can (and perhaps should) use the same key
> material and dress it up in whatever uniform one wants, OpenPGP or S/MIME.
>
> While on the surface, it kinda seems like a good idea to unify the two in
> protocol, that's a different task than either group has. A new protocol
> would want to be a new protocol. Despite each protocol being used in much
> the same way (especially in email), there are a lot of things that would
> have to be re-hashed out.
>

​
The only reason to introduce a new protocol would be to introduce features
that aren't currently supported and would require a substantial
re-engineering of the legacy protocols.

Now I think I have actually found such a feature and might even write a
client to demonstrate it. But that would be in addition to supporting
S/MIME and OpenPGP etc. Because the lesson we learned on the Web was that
the gateways to legacy systems were what allowed the Web to beat systems
like HyperG etc. that most independent observers would have said were
'better' at the time.

I think that the idea of OpenPGP Key Server + Linked Timestamp (aka
Blockchain) is very powerful. I also think that I don't like ASN.1 or PGP
data encoding and a modern protocol should have a really good reason for
not using JSON or JSON plus extensions to support binary blobs without
Base64 armoring.

[OK I lied about saying I don't like ASN.1, I utterly despise it, there are
few things I loathe more]

> There's how you issue certificates (the whole CA/introducer issue(s)),
> whether certs contain one key or key sets, how they are transported (S/MIME
> puts them in the message, OpenPGP in directories etc.), and even the role
> of the internal layering. Note that OpenPGP is a binary (and UTF-8 is still
> binary) object protocol with a drizzling of MIME-encoding frosting over the
> top. That frosting is subject to its own interpretations. S/MIME in
> contrast *starts* with the email and MIME object and underneath there's
> CMS, usually almost as an afterthought. (Did you have a momentary "huh?" in
> your head when you read CMS? Many people do, and that's the point.) S/MIME
> starts at the top, OpenPGP starts at the bottom.
>
> And oh, there are also other things that have to be re-hashed like ASN.1
> all over again and the things it drags along like encoding rules. This is a
> good deal why perhaps its better to just push the other things up into
> software. The reason that there are the two standards is that they address
> different views of the world, technical as well as political.
>

​Two views of the world that are rather absolutist and thus wrong. Some
parts of the world are hierarchical, others are not. A trust infrastructure
needs to support both. But it isn't clear such infrastructure is best
implemented inside a client.
​

> At the end of the day, there are many things you *have* to push up into
> software. Consider the case where I am sending an email (which often
> happens, but may not even be the primary case in OpenPGP, merely the one
> that comes first to mind) to Alice, Bob, and Charlie. It's indeed
> irritating that Alice has an OpenPGP key and Bob an S/MIME certificate, and
> I am thus  going to have to code up two copies of the message. It is,
> however, straightforward. I know what to do. The subtlety comes from the
> fact that Charlie is being BCCed. It doesn't matter what happens with
> Charlie, whatever encoding we use (even plaintext) we have to send that
> message separately. You have to handle this at the software level no matter
> what. Even with a unified crypto standard, messaging isn't just crypto.
>
> Unless the unifying protocol is so compelling that people of all stripes
> can agree that we should drop the old ones and go to this, we merely have a
> reification of an XKCD cartoon -- we'll have *three* protocols that have to
> exercised at the proper software level in exactly the same way you'd have
> to hand it with two. Trying to simplify will almost certainly just make
> things more complex.
>

​If it is only a question of merging S/MIME and OpenPGP functionality, I
don't see it.

But Proxy Re-Encryption is a lot more powerful. Essentially it is the next
logical step in crypto. The trick in crypto is that we introduce a new key
for each function. Public key crypto is more powerful because encryption is
separate from decryption. With Proxy Re-Encryption we have Lora who is
managing a mailing list and she has a key that can add new members to the
list, and we have the mailing list encryption key and we have the per-user
decryption keys. Going from two key crypto to three key crypto is powerful

---

From: Derek Atkins <[email-masked]>
Date: Wed, 06 Jul 2016 10:59:50 -0400

Phillip Hallam-Baker <[email-masked]> writes:

>     whether certs contain one key or key sets, how they are transported (S/
>     MIME puts them in the message, OpenPGP in directories etc.), and even the
>     role of the internal layering. Note that OpenPGP is a binary (and UTF-8 is
>     still binary) object protocol with a drizzling of MIME-encoding frosting
>     over the top. That frosting is subject to its own interpretations. S/MIME
>     in contrast *starts* with the email and MIME object and underneath there's
>     CMS, usually almost as an afterthought. (Did you have a momentary "huh?"
>     in your head when you read CMS? Many people do, and that's the point.) S/
>     MIME starts at the top, OpenPGP starts at the bottom.
>    
>     all over again and the things it drags along like encoding rules. This is
>     a good deal why perhaps its better to just push the other things up into
> ​Two views of the world that are rather absolutist and thus wrong. Some parts
> of the world are hierarchical, others are not. A trust infrastructure needs to
> support both. But it isn't clear such infrastructure is best implemented
> inside a client.

OpenPGP can support hierarchical certificate deployments just fine (my
company is building one) as well as the Web of Trust model.  X.509
cannot support a Web of Trust deployment, period.

So there is a clear winner here.

-derek

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 6 Jul 2016 18:12:25 -0400

On Wed, Jul 6, 2016 at 10:59 AM, Derek Atkins <[email-masked]> wrote:

> parts

​
You can in fact make X.509 do Web of trust. You simply give each user their
own CA root and cross certify.

I was doing that for quite a while till I realized that the legacy stuff
was hurting rather than helping. Yes you can get the protocols to do more
than the apps let them. But you don't have the advantage of legacy platform
support or legacy platform ignoring your stuff in a predictable way.

---

From: Derek Atkins <[email-masked]>
Date: Thu, 07 Jul 2016 10:45:17 -0400

> own CA root and cross certify.

I guess X.509v3 does, theoretically, allow multiple signatures on a
certificate, but I was under the impression that zero implementations
actually supported that?

> I was doing that for quite a while till I realized that the legacy stuff was
> hurting rather than helping. Yes you can get the protocols to do more than the
> apps let them. But you don't have the advantage of legacy platform support or
> legacy platform ignoring your stuff in a predictable way.

The nice thing here is that legacy OpenPGP apps DO support hierarchical
deployments without any changes.  The only thing you need to do for
OpenPGP is that you need to tell the program to trust the CA.  This
does have the benefit (or I suppose if you come from an X.509 world it's
a drawback) that each user needs to declare which CAs are trusted.

I am curious in what way you found the legacy OpenPGP deployments didn't
support hierarchical trust?   Or are you saying that legacy X.509 didn't
support a Web of Trust model (which, honestly, doesn't surprise me).

-derek

---

From: Werner Koch <[email-masked]>
Date: Thu, 07 Jul 2016 17:28:22 +0200

> certificate, but I was under the impression that zero implementations

I can't see how this is possible within an X.509 certificate:

   Certificate  ::=  SEQUENCE  {
        tbsCertificate       TBSCertificate,
        signatureAlgorithm   AlgorithmIdentifier,
        signatureValue       BIT STRING  }

There is only one signature for the to-be-signed-Certificate.

Salam-Shalom,

   Werner

-- 
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.
 /* Join us at OpenPGP.conf  <https://openpgp-conf.org> */