Subject: [openpgp] Chunked OpenPGP streams

From: Guilhem Moulin <[email-masked]>
Date: Wed, 30 Dec 2015 12:51:30 +0100

Hi there,

It's convenient to use OpenPGP streams to avoid creation of temporary
files, as illustrated in the following pipeline:

    tar -c / | gpg -r $KEYID --sign --encrypt | ssh remote.example.org 'cat >backup.tar.gpg'

(using GnuPG [0] as OpenPGP implementation).  Unfortunately, since
integrity mechanisms are appended at the very end of the data stream,
the same technique cannot be used when decrypting and/or verifying a
data stream.  For instance

    gpg --decrypt backup.tar.gpg | tar -C /restore -x

could crash the system if an attacker has replaced ‘backup.tar.gpg’ with
a tar bomb.  The problem arises because unverified data is written to
the standard output, which is hard to avoid since the hash is computed
on the full data stream and is not available until the very end of the
stream.

I wonder if chunked streams could make their way to RFC4880bis instead.
The verification mechanism (MDC or data signature) would be added to
each chunk using the intermediate hash value, and a compatible
implementation would cache each chunk, update the hash context, and
*then* write down the chunk to the output File Descriptor *only* after
verifying its integrity.  (If there is not enough RAM to cache a chunk
one could always fallback to the creation of a temporary file.)
AFAICT using intermediate hash values is enough to cryptographically
glue the chunks together in an order preserving way.

What do you folks think?
-- 
Guilhem.

[0] https://gnupg.org/

---

From: Nils Durner <[email-masked]>
Date: Wed, 30 Dec 2015 14:01:46 +0100

Hi,

> each chunk using the intermediate hash value,

I think this goes in the same direction that OAED or online
authenticating cipher modes are being considered for - see the recording
of the last IETF meeting at
http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF94_OPENPGP&chapter=chapter_1

Regarding the potential use of online authenticating cipher modes, it
was discussed during that meeting that there is *some* research on modes
that *might* be usable with PGP. If anyone can share papers (or
references), I would appreciate it.

Regards,

Nils

---

From: Guilhem Moulin <[email-masked]>
Date: Wed, 30 Dec 2015 15:28:12 +0100

Hi,

> http://recs.conf.meetecho.com/Playout/watch.jsp?recording=IETF94_OPENPGP&chapter=chapter_1

Thanks for the pointer.  I think however that this could be useful for
detached sigs, too.  For instance assuming a remote tarball, a local
detached signature, and an OpenPGP implementation that would copy
*verified* data to the output File Descriptor, one could write

    ssh remote.example.org cat /path/to/backup |
    gpg --verify /path/to/backup.sig - |
    tar -x

without fear of race condition and without the inconvenience of creating
a temporary local file.

Cheers,
-- 
Guilhem.