Subject: Re: [openpgp] "OpenPGP Simple"

From: Peter Gutmann <[email-masked]>
Date: Tue, 17 Mar 2015 02:10:16 +0000

David Shaw <[email-masked]> writes:
>On Mar 16, 2015, at 5:15 PM, David Leon Gil <[email-masked]> wrote:
>> Partial lengths are really a nuisance to parse.
>
>No argument there...

The whole bizarro sort-of-fixed-point encoding of lengths is a pain (this is a
cue for Jon to do his "every bit is sacred" dance).  If the format is revised,
there should be only two lengths, a 16-bit one for almost everything (keyring
data, signatures, etc), and a 32-bit one for payloads and partial lengths that
are going to exceed 16-bit lengths.  Length-decoding shouldn't be any more
complicated than:

read tag;
if( tag & length_32_flag )
  length = read32();
else
  length = read16();

While I'm venting, shall I get started on the MDC kludge?

Peter.

---

From: David Shaw <[email-masked]>
Date: Mon, 16 Mar 2015 23:05:40 -0400

> if( tag & length_32_flag )

I'm fine with that, but I do want to keep the concept of partial lengths, as you did above.  People do encrypt things without knowing how large they are ahead of time.  I'm fine with a restriction (which already exists today) that only payloads can use partial lengths.

David

---

From: Jon Callas <[email-masked]>
Date: Mon, 16 Mar 2015 23:59:59 -0700

> On Mar 16, 2015, at 7:10 PM, Peter Gutmann <[email-masked]> wrote:
> 
> are going to exceed 16-bit lengths.  

Okay... NOOOOOOOOO!!!!! For the love of God, Montressor, only *one* type of length. You’re spending more space in the parsing code and sooner or later, someone’s going to screw it up and there will be a stupid ass security problem that could have been solved by just spending the two damned extra bytes.

> While I'm venting, shall I get started on the MDC kludge?

Sure. Go right ahead.

But when you do, take into account that MDC pre-dates HMAC and at the time, one of the major objections was a "why would you want to have symmetric crypto protection when you could just sign it" whine, and the other was excessive worry about single-pass processing that got so irrational we couldn’t work through it.

Standards are compromises, and a good compromise leaves everyone a bit grumpy. Since those days, I’ve developed an affection for MDC because it sits in a nether world where related concepts like deniable encryption that also sound good until you think about them for long enough. And it doesn’t hurt anything, because if you really want it protected, just sign the darned thing.

But please, please, go right ahead.

	Jon

---

From: Phill <[email-masked]>
Date: Tue, 17 Mar 2015 10:30:00 -0400

> On Mar 16, 2015, at 11:05 PM, David Shaw <[email-masked]> wrote:
> 

+1 

But that 32 bit length really worries me. Just because people can’t send 4GB messages today does not mean that they can’t or won’t in the future. Do not build OpenPGP around assumptions based on SMTP continuing forever. Use today is not limited to mail in any case.

If I have a 1TB archive file I am not going to want to break it into chunks just to encrypt it.

I am not convinced a complete overhaul of the messaging format is desirable. But if people were going to completely revise the message format, the field is moving towards JSON for almost everything.

The only thing JSON does not do that is essential for crypto is length prefixed binary encoding of binary blobs. Base64 encoding is not so bad if you do it only once. But a 33% penalty for each time a file is wrapped starts to add up. When folk were suggesting yet another data encoding, several of us who only want one data model suggested just adding code points for binary blobs to JSON. It does get the job done.

Again, I am not sure that a complete overhaul is desirable. Just pruning back the unnecessary features is probably sufficient. But if a change is made, best to pick something that looks as much like the standard the rest of the protocol stack above layer 5 is converging on as possible.

---

From: Stephen Paul Weber <[email-masked]>
Date: Tue, 17 Mar 2015 10:21:13 -0500

>I am not convinced a complete overhaul of the messaging format is 
>desirable.

I agree.

>the field is moving towards JSON for almost everything.

*shudders*

>Again, I am not sure that a complete overhaul is desirable. Just pruning 
>back the unnecessary features is probably sufficient.

Yes.

-- 
Stephen Paul Weber, @singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph

---

From: Derek Atkins <[email-masked]>
Date: Tue, 17 Mar 2015 11:33:02 -0400

Phill <[email-masked]> writes:

> But that 32 bit length really worries me. Just because people can’t
> send 4GB messages today does not mean that they can’t or won’t in the
> future. Do not build OpenPGP around assumptions based on SMTP
> continuing forever. Use today is not limited to mail in any case.
>

That's what partial (indefinite) lengths are for.  But the point is that
each "size" parameter is 32 bits, always, instead of having a 1, 2, or
5-byte length parameter.

[snip]
> Again, I am not sure that a complete overhaul is desirable. Just
> pruning back the unnecessary features is probably sufficient. 

+1

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [email-masked]                        PGP key available

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 17 Mar 2015 12:44:52 -0400

On Tue, Mar 17, 2015 at 11:33 AM, Derek Atkins <[email-masked]> wrote:

> That's what partial (indefinite) lengths are for.

Having to cut the dinner up into bite sized portions is a bit of a pain.

It also means that special code is required to do pass along data that has
already been encrypted in a different format.

Partial lengths are necessary to support streaming where the length is
indefinite. They can be used for large chunks but that is a hack.

> But the point is that

This is 2015, either make the size 64 bits or if you really think that the
space is a critical issue then use a self describing length format. The
body of any modern email is going to be UTF8 after all.

Even 64 bits starts to become an issue in exabyte stores. But only the NSA
and possibly Google have got one of those to date.

-- 
Website: http://hallambaker.com/

---

From: Christoph Anton Mitterer <[email-masked]>
Date: Wed, 18 Mar 2015 23:51:49 +0100

On Tue, 2015-03-17 at 10:30 -0400, Phill wrote:
> Do not build OpenPGP around assumptions based on SMTP continuing
> forever.
+1, especially since I there are still many people here mentioning
mail/SMTP and their restrictions/issues/problems quite often.

A future OpenPGP (core) standard should try to avoid solving the
issues/limitations of any other standard/protocols.

That being said, I also don't like the idea of mail header
signing/encryption.
It's simply not OpenPGP's task and one should rather create an amendment
for the mail standards or MIME which describes how an OpenPGP
signed/encrypted headers would be embedded in an mail.

Cheers,
Chris.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 18 Mar 2015 19:32:58 -0400

> signed/encrypted headers would be embedded in an mail.

I think that it is now widely agreed that if SMTP was to be revised, it
would be separated into two layers with the message routing headers in one
section and the content meta-data in another.

Subject, To, From, Content-Type, etc. are all content headers. They have
nothing to do with SMTP. They are only used by the MUA.

As a general rule, if a fix works for S/MIME and OpenPGP, then we can
probably be fairly confident it will work for SMTPvNext as well.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 18 Mar 2015 19:33:31 -0400

On Mar 16, 2015, at 10:10 PM, Peter Gutmann <[email-masked]>
wrote:

No argument there...

The whole bizarro sort-of-fixed-point encoding of lengths is a pain (this
is a
cue for Jon to do his "every bit is sacred" dance).  If the format is
revised,
there should be only two lengths, a 16-bit one for almost everything
(keyring
data, signatures, etc), and a 32-bit one for payloads and partial lengths
that
are going to exceed 16-bit lengths.  Length-decoding shouldn't be any more
complicated than:

read tag;

I'm fine with that, but I do want to keep the concept of partial lengths,
as you did above.  People do encrypt things without knowing how large they
are ahead of time.  I'm fine with a restriction (which already exists
today) that only payloads can use partial lengths.

+1

But that 32 bit length really worries me. Just because people can’t send
4GB messages today does not mean that they can’t or won’t in the future. Do
not build OpenPGP around assumptions based on SMTP continuing forever. Use
today is not limited to mail in any case.

If I have a 1TB archive file I am not going to want to break it into chunks
just to encrypt it.

I am not convinced a complete overhaul of the messaging format is
desirable. But if people were going to completely revise the message
format, the field is moving towards JSON for almost everything.

The only thing JSON does not do that is essential for crypto is length
prefixed binary encoding of binary blobs. Base64 encoding is not so bad if
you do it only once. But a 33% penalty for each time a file is wrapped
starts to add up. When folk were suggesting yet another data encoding,
several of us who only want one data model suggested just adding code
points for binary blobs to JSON. It does get the job done.

Again, I am not sure that a complete overhaul is desirable. Just pruning
back the unnecessary features is probably sufficient. But if a change is
made, best to pick something that looks as much like the standard the rest
of the protocol stack above layer 5 is converging on as possible.

---

From: Stephan Somogyi <[email-masked]>
Date: Sat, 21 Mar 2015 18:34:29 -0700

I’m enjoying this exchange tremendously, but Phill just said something that overcame my reticence to pipe up.

On Mar 17, 2015, at 09:44, Phillip Hallam-Baker <[email-masked]> wrote:

> The body of any modern email is going to be UTF8 after all.

There’s a fair bit of email still being exchanged that isn't UTF8, and I expect this’ll continue for as long as we’re still using SMTP. Everyone really should read <http://en.wikipedia.org/wiki/Mojibake> in the context of making these types of assumptions. I still wish the charset header had been added to OpenPGP sooner, and made to have teeth.

When we started work on End-To-End, it’d’ve been easier in many ways to start from scratch, invent our own spiffy new protocol, and evade the tyranny of the installed base entirely. But interop matters. As do maturity and understood-ness of protocols. We made some implementation-specific choices (eg only ever generating ECC keys), but we very concertedly set out not to break anything. David’s declaration of intent to deprecate is worded a bit more strongly than I would’ve done, but I certainly don't disagree with the idea of considered algorithmic tidying in order to encourage modern OpenPGP implementations to generate contemporarily appropriate keys and messages.

When it comes to decryption, I’m (unsurprisingly) in vigorous agreement with Derek that this is a software problem, not a human problem. The question is whether we need a more usable universally backward compatible OpenPGP decryptor for our older data.

s.

---

From: Christoph Anton Mitterer <[email-masked]>
Date: Mon, 23 Mar 2015 19:29:40 +0100

On Wed, 2015-03-18 at 19:32 -0400, Phillip Hallam-Baker wrote: 
> I think that it is now widely agreed that if SMTP was to be revised,
> it would be separated into two layers with the message routing headers
> in one section and the content meta-data in another.
And you really think this is every going to happen in our life time? ;-)

(And standardising something is one thing, but having it effectively
used on millions of mail systems/software/etc. - a completely different
topic).

Cheers,
Chris.

---

From: Christoph Anton Mitterer <[email-masked]>
Date: Mon, 23 Mar 2015 19:33:41 +0100

+1

+1

> the field is moving towards JSON for almost everything.
I'm not so convinced about that... there are still many areas where
completely other things (e.g. XML) are used.

> Again, I am not sure that a complete overhaul is desirable.
When I originally wrote about "complete overhaul" in my wishlist, I
didn't meant to say whether it's necessary or not.
But it would at least be a nice way to get rid of all legacy cruft that
may have came together over the years. A fresh start often allows one to
employ much cleaner designs :)

Cheers

---

From: ianG <[email-masked]>
Date: Mon, 23 Mar 2015 19:52:01 +0000

> are going to exceed 16-bit lengths.

Crazy * 2 :)

One int is good enough for anyone.  Just use an expanding 7 bit form, 
where the hi bit says, grab the following byte.

> Length-decoding shouldn't be any more
> complicated than:
>
> read tag;

Simplify!  Simplify!

iang

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 23 Mar 2015 14:57:12 -0500

One of the thing I really hate in asinine one is the way they managed to take a bunch of good ideas and trash them.... 

Yes one integer format that expands as needed would be the way to go IFF this is reopened.

Sent from my difference engine

> On Mar 23, 2015, at 14:52, ianG <[email-masked]> wrote:
> 
> One int is good enough for anyone.  Just use an expanding 7 bit form, where the hi bit says, grab the following byte.
> 
> 
> _______________________________________________
> openpgp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/openpgp