Subject: Re: Clarification needed on compressed messages

From: "Michael Young" <[email-masked]>
Date: Fri, 1 Aug 2003 11:24:30 -0400

Derek Atkins wrote:
> I believe it is the intent, and in the SIG+(COMPRESSED(LITERAL) the
> SIG should be issued over the COMPRESSED(LITERAL).  The only special
> case that I know of is SIG+LITERAL, where the SIG is over the data
> inside the literal and doesn't include the literal packet itself.

to which "David Shaw" <[email-masked]> responds:
> This sounds very reasonable to me.  I think a word or two to make that
> clear in the draft would be helpful: something that indicates that

I have mixed feelings about Derek's interpretation, but if that's
the intent, then I agree with David that this must be made clear
in the draft.  There is definitely a special case here.

Why mixed feelings?  On the one hand, I don't like special cases.  I
also find it surprising that one would want to sign the COMPRESSED
packet.  (It's less to hash, but that hardly seems meaningful.)
On the other hand, it is a little disturbing that the LITERAL
packet headers are ignored, and including them in the signature (by
way of hashing the entire COMPRESSED packet) would overcome that
deficiency.

Note that both of my concerns could be addressed by a different rule
that has no special case: the signature hash is computed over the
CONTENTS of the FOLLOWING packet (*not* recursively).  In the original
PGP case, this would be the contents of the literal packet.  In the
COMPRESSED(LITERAL(x)) case, it would be the LITERAL(x).  [One could
use an "uncompressed" COMPRESSED packet to intentionally capture the
LITERAL header information.]

Whatever we do, I expect that
    ONEPASS COMPRESSED(LITERAL(x)) SIGNATURE
would be treated the same as
    OLD-SIG COMPRESSED(LITERAL(x))
Reasonable?

Of course, adopting the "be liberal in what you accept" principle, an
implementation *could* do parallel hashes against all of these
possibilities, and report what got signed :-).

Before passing final judgement, I'd be curious to know what the
known implementation that uses SIG+COMPRESSED(LITERAL(x)) did
with the construct?  What did it sign?

---

From: David Shaw <[email-masked]>
Date: Fri, 1 Aug 2003 12:51:59 -0400

> packet.  (It's less to hash, but that hardly seems meaningful.)

It's not just signing the COMPRESSED packet.  Using Derek's
interpretation, you can also do possibly useful things as
sign-encrypt-sign or encrypt-sign-encrypt a message and have the
parser handle it automatically.

The special case (and I agree it is a special case) doesn't bother me
too much.  I agree it would have been nice if the signature had always
been over the complete literal packet, headers and all, but history
ruled otherwise.

It is not terribly complicated to say "always hash the complete
OpenPGP object unless it is a literal packet, in which case hash the
contents".  As you point out, if an implementation wanted to force
signing the complete literal packet, it could just encapsulate the
literal packet into a compressed data packet.

> LITERAL header information.]

This seems a bit like simplifying the hashing rule by adding
complexity somewhere else.  For starters, we would lose the current
ability to do encrypt-sign-encrypt (the signature would become
effectively a detached signature over the encrypted data) and
sign-encrypt-sign (the outer signature would become a notary signature
in effect).

I suppose the user could create
LITERAL(SIGN(ENCRYPT(SIGN(LITERAL(x))))) or
LITERAL(ENCRYPT(SIGN(ENCRYPT(LITERAL(x))))) and sign that, but we're
getting complex again since the parser shouldn't be looking inside a
*literal* packet for more OpenPGP data to parse.

>     OLD-SIG COMPRESSED(LITERAL(x))

I agree, but on the subject of the SIG+LITERAL (or SIG+OPENPGPOBJECT)
format, I'd actually like to see it deprecated (in the "understand
this, but please don't generate it" sense).  The ONEPASS signature
method is far superior, and we can at least start the slow process of
getting all implementations to use it.

The ONEPASS method can also easily handle such constructions as
ONEPASS+OBJECT+OBJECT+OBJECT+SIG, which SIG+OBJECT cannot.

> with the construct?  What did it sign?

It signed COMPRESSED(LITERAL(x)) - the whole packet.  Note also that
PGP can correctly verify such a message, so Derek's interpretation is
supported by working code... though possibly because Derek worked on
the PGP parser at one point. ;)

David

---

From: "Michael Young" <[email-masked]>
Date: Fri, 1 Aug 2003 13:42:47 -0400

> parser handle it automatically.
...
> *literal* packet for more OpenPGP data to parse.

I'd use COMPRESSED (with no compression) here instead, which
generally are parsed for other packets.  But, I agree that this
looks messy for SES or ESE applications.

I can live with the special case.  (The fact that the new
implementation conformed to Derek's rule provides further persuasion.)

> getting all implementations to use it.

I concur.

> ONEPASS+OBJECT+OBJECT+OBJECT+SIG, which SIG+OBJECT cannot.

Only for non-signature OBJECTs.

I'd rather not complicate the rules any further to allow this
construction.  (It is not permitted now.)

---

From: David Shaw <[email-masked]>
Date: Fri, 1 Aug 2003 15:27:29 -0400

> Only for non-signature OBJECTs.

True.  There would have to be some encapsulation in that case, perhaps
using your compressed packet suggestion:
ONEPASS+COMPRESSED(SIG)+SIG.

> construction.  (It is not permitted now.)

It's not clear whether it is permitted now or not.

The general question of multiple packets (whether in a one-pass
signature, compressed data packet, or encrypted data packet) is
somewhat hazy in the draft.  Speaking only about literal packets for
now, sections 5.6, 5.7, and 5.13 all say yes (using the plural
"literal data packets").  Section 10.2 says no.

I actually requested a clarification whether LITERAL+LITERAL was valid
a few weeks ago: http://www.imc.org/ietf-openpgp/mail-archive/msg05537.html

The reason why I was thinking about LITERAL+LITERAL in the first place
was Jon Callas' comments about OpenPGP as an archival primitive.  It
would be Very Useful to be able to store more than one file into a
single OpenPGP message.  I don't think OpenPGP should be setting out
to replace tar or zip, but it's handy nonetheless.  An archive program
with strong encryption whose results can be de-archived with any
OpenPGP program is compelling.

Note that both PGP and GnuPG already do the right thing with
ENCRYPTED(LITERAL+LITERAL) messages.

Google says Hal Finney argued for this interpretation in 2000:
 http://cert.uni-stuttgart.de/archive/ietf-openpgp/2000/05/msg00032.html

All that said, I think that LITERAL+LITERAL should probably be legal,
but either way, the draft shouldn't say both yes and no.

Come to think, your suggestion of using a compressed data packet to
encapsulate could be useful here as well:
ENCRYPTED(COMPRESSED(LITERAL+LITERAL)).

David

---

From: "Michael Young" <[email-masked]>
Date: Sat, 2 Aug 2003 16:39:26 -0400

> > > ONEPASS+OBJECT+OBJECT+OBJECT+SIG, which SIG+OBJECT cannot.
... [to which I replied, and then he replied again:]
...
> "literal data packets").  Section 10.2 says no.

Those sections refer to sequences inside compressed and the two
flavors of encrypted data packets (respectively), not signatures.
My claim that this is not allowed for signatures was based on the
grammar in 10.2.

...
> ENCRYPTED(COMPRESSED(LITERAL+LITERAL)).

Yes, it's pretty clear that this is legal.  The grammar doesn't
cover the contents of compressed packets, so the language in
section 5.6 would seem to govern.  (Curiously, that section
suggests that COMPRESSED packets might live directly in
signatures, which is what kicked off this whole discussion.)

I don't really think that sequences of LITERAL packets are a good
thing.  I really don't like them for general archiving, as they don't
carry nearly enough information.  I also see no reason to avoid using
an external archiver (tar, zip, or any of many others).  I was going
to argue against allowing sequences of LITERAL because they complicate
decryption: implementations will likely want to provide a user
interface to control which LITERALs should be processed.  But then,
David pointed out that we already have this problem inside COMPRESSED
packets.  I'd be happy to withdraw that ability, but I'm sure others
will object, so I'll admit defeat in advance.

[Note that one can always encrypt each file separately, and if you're
worried about the public-key encryption cost, your implementation
could reuse the session key (but not IV) during multiple-file
encryption, and recognize that case on decryption.  I'm not advocating
this -- I prefer using a real (external) archiver instead -- but
simply pointing out an option that would be "conservative in what you
generate".]

---

From: David Shaw <[email-masked]>
Date: Sat, 2 Aug 2003 17:37:53 -0400

> signatures, which is what kicked off this whole discussion.)

Well, I agree with your end result, but I'm not quite sure I agree
with the path you took to get there.  In fact, the grammar *does*
cover the contents of compressed and encrypted packets:

   In addition, decrypting a Symmetrically Encrypted Data Packet or a
   Symmetrically Encrypted Integrity Protected Data Packet as well as
   decompressing a Compressed Data packet must yield a valid OpenPGP
   Message.

Thus, COMPRESSED(LITERAL+LITERAL) is valid only if LITERAL+LITERAL is
valid.

So what this all comes down to is that 5.6 says that
COMPRESSED(LITERAL+LITERAL) is a valid construction.  10.2 says that
LITERAL+LITERAL isn't a valid construction.  Conflict: they can't both
be right.  Repeat as needed for section 5.7 and 5.13 - the problem is
identical.

This is why I have been suggesting a minor change to 10.2 to make it
match 5.6, 5.7 and 5.13:

The current draft says:

 Literal Message :- Literal Data Packet.

I'd like to change that to:

  Literal Message :- Literal Data Packet |
                     Literal Message, Literal Data Packet.

The draft, as it stands now, is internally inconsistent.  I'd like to
fix that.

David

---

From: Jon Callas <[email-masked]>
Date: Mon, 04 Aug 2003 17:45:19 -0700

> The draft, as it stands now, is internally inconsistent.  I'd like to

Changed.

    Jon