Subject: [openpgp] review of the SOP draft

From: Antoine Beaupré <[email-masked]>
Date: Mon, 11 Nov 2019 12:57:51 -0500

Hi,

I've taken some time to do a a short review of dkg's "SOP" interface
specification, version 01, as provided here:

https://tools.ietf.org/html/draft-dkg-openpgp-stateless-cli-01

First, I want to say this is excellent and much-needed work. OpenPGP
interfaces have traditionnally been hard to use and a major obstacle to
adoption of strong cryptography in our communities. Having a standard
and *simple* way of interoperating with the underlying OpenPGP
primitives would go a *long* way in significantly improving OpenPGP
adoption and online privacy in general. So I salute this (first) effort
in improving this situation.

Second, I have proposed a series of changes to the document here:

https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9

Patch set:

https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9.patch

Those are mostly small fixes that might not require a full discussion
here, although I of course welcome feedback on those here as well. It
might be preferable to comment on the issue directly if you have a
GitLab account, obviously.

Finally, there are some things about the document I wanted to comment on
and that I don't have an easy patch for, so I will comment on the
document, inline, here instead. I hope that is proper form, let me know
if there is a better way to do this.

> Introduction
> ============

[...]

> This separation should make it easier to provide interoperability testing for the object security work, and to allow implementations to consume and produce new cryptographic primitives as needed.
 
I don't understand the part after ", and allow..." what does that
actually mean? What do we mean by "new cryptographic primitives" here
exactly?

[...]

> Obviously, the user will need to manage their secret keys (and their
> peers' certificates) somehow, but the goal of this interface is to
> separate out that task from the task of interacting with OpenPGP
> messages.
 
It's unclear to me how or if the SOP specification takes into account
the current design of GnuPG, specifically the part where secrets are
handled by a separate process, gpg-agent, which is designed to be
separate from the other parts of gnupg. From what I understand, the
"agents" keep the secrets and do the operations on behalf of other parts
of gnupg. But here sop would do so. Are we designing an agent here?

What about OpenPGP cards like the Yubikey? How does sop interoperate
with those?

[...]

> Examples
> ========
> 
> These examples show no error checking, but give a flavor of how `sop` might be used in practice from a shell.
> 
> The key and certificate files described in them (e.g. `alice.sec`) could be for example those found in {{I-D.draft-bre-openpgp-samples-00}}.
> 
> ~~~
> sop generate-key "Alice Lovelace <[email-masked]>" > alice.sec
> sop extract-cert < alice.sec > alice.pgp
> 
> sop sign --as=text alice.sec < announcement.txt > announcement.txt.asc
> sop verify announcement.txt.asc alice.pgp < announcement.txt
> 
> sop encrypt --sign-with=alice.sec --as=mime bob.pgp < msg.eml > encrypted.asc
> sop decrypt alice.sec < ciphertext.asc > cleartext.out
> ~~~

I find those examples confusing. Multiple arguments, in particular,
seems ambiguous. Is it "CERT DATA"? or "CERT DATA"?

There should be *mandatory* commandline *options* instead, that clearly
state the purpose of (say) the "CERT" argument. It's a common in APIs,
to rely on the order of arguments for meaning, and I think it's often a
mistake. We should explicitely use *options* instead of *arguments* (as
in `--foo=bar` instead of just `bar`) for critical parameters like
secret or verification keys.

Then "arguments" are left for more regular file parameters, the primary
purpose of the command (e.g. "sign, encrypt, decrypt this file", with
"verify" being of course the tricky bit).

So instead of:

> sop sign --as=text alice.sec < announcement.txt > announcement.txt.asc

I would suggest:

> sop sign --as=text --sign-with=alice.sec < announcement.txt >announcement.txt.asc

For example. The idea would be to use `--PURPOSE-with` pattern (where
PURPOSE is `sign`, `verify`, etc) that is already used in `encrypt
--sign-with`. Maybe it could be shortened to just --with in some cases
(like decrypt, sign and verify). The above example would become:

sop generate-key "Alice Lovelace <[email-masked]>" > alice.sec
sop extract-cert < alice.sec > alice.pgp

sop sign --as=text --sign-with=alice.sec < announcement.txt > announcement.txt.asc
sop verify announcement.txt.asc --verify-with=alice.pgp < announcement.txt

sop encrypt --sign-with=alice.sec --as=mime bob.pgp < msg.eml > encrypted.asc
sop decrypt --decrypt-with=alice.sec < ciphertext.asc > cleartext.out

Or, as a diff:

@@ -99,11 +103,11 @@ The key and certificate files described in them (e.g. `alice.sec`) could be for
 sop generate-key "Alice Lovelace <[email-masked]>" > alice.sec
 sop extract-cert < alice.sec > alice.pgp
 
-sop sign --as=text alice.sec < announcement.txt > announcement.txt.asc
-sop verify announcement.txt.asc alice.pgp < announcement.txt
+sop sign --as=text --sign-with=alice.sec < announcement.txt > announcement.txt.asc
+sop verify announcement.txt.asc --verify-with=alice.pgp < announcement.txt
 
 sop encrypt --sign-with=alice.sec --as=mime bob.pgp < msg.eml > encrypted.asc
-sop decrypt alice.sec < ciphertext.asc > cleartext.out
+sop decrypt --decrypt-with=alice.sec < ciphertext.asc > cleartext.out
 ~~~
 
 Subcommands

> Subcommands
> ===========
> 
> `sop` uses a subcommand interface, similar to those popularized by systems like `git` and `svn`.
>
> If the user supplies a subcommand that `sop` does not implement, it
> fails with a return code of 69.  If a `sop` implementation does not
> handle a supplied option for a given subcommand, it fails with a
> return code of 37.
 
In general, I feel using the numeric error codes in the document make it
(needlessly?) harder to read. When i got to this section, my first
reaction was: "69?? why 69? and why 37? where the heck do those come
from and why do they matter?" We should at least include a reference to
the "Failure modes section" in the Introduction section. In Terminology
maybe? And maybe refer to it here.

In general, I'm worried there might be inconsistencies between the table
in the "Failure modes" section and the various hardcoded integers
peppered through the document. This practice also makes the document
more difficult to review and maintain in the future. We might instead
use constant names like `SUCCESS`, `NO_GOOD_SIG` that *then* have
integer values in the later section. This could also provide for good
constants to use in a library implementation.

> For all commands that have an `--armor|--no-armor` option, it defaults
> to `--armor`, meaning that any output OpenPGP material should be
> ASCII-armored (section 6 of {{I-D.ietf-openpgp-rfc4880bis}})
> by default.
 
Is this on input or output? or both? It's clarified later, I
think, but it should be made explicit here as well.

[...]

> generate-key: Generate a Secret Key {#generate-key}
> -----------------------------------
> 
>     sop generate-key [--armor|--no-armor] [--] [USERID…]
> 
>  - Standard Input: ignored
>  - Standard Output: `KEY` ({{key}})
> 
> Generate a single default OpenPGP certificate with zero or more User
> IDs.

[...]

How do we generate purpose-specific subkeys?

[...]

> sign: Create a Detached Signature {#sign}
> ---------------------------------
> 
>     sop sign [--armor|--no-armor]
>          [--as={binary|text}] [--] KEY [KEY...]
> 
>  - Standard Input: `DATA` ({{data}})
>  - Standard Output: `SIGNATURE` ({{signature}})
> 
> `--as` defaults to `binary`.  If `--as=text` and the input `DATA` is
> not valid `UTF-8`, `sop sign` fails with a return code of 53.
 
Why do we mandate UTF-8 here? Explain.

In general, I find the `--as` arguments to be a little confusing and I
don't undrestand what they bring to the table.
 
> Example:
> 
>     $ sop sign --as=text alice.sec < message.txt >message.txt.asc
>     $ head -n1 < message.txt.asc
>     -----BEGIN PGP SIGNATURE-----
>     $

Another good example of the "argument vs option" problem. If I would see
a `sop sign` command, the first thing I would try would be:

    sop sign document

and expect it to find the right private key to sign the document
with. Of course, we don't do this in sop, which is fine, but I'll note
that we allow implementations to do so. By forcing the arguments here to
be the signing key, we make it difficult to let the implementation pick
the right key.

We should follow POLA (Principle Of Least Astonishment) here and allow
users to provide the document as an argument, and use something like
`--sign-with=KEY` (possibly multiple times) to provide the private key
material.

[...]

> encrypt: Encrypt a Message {#encrypt}
> --------------------------
> 
>     sop encrypt [--as={binary|text|mime}]
>         [--armor|--no-armor]
>         [--with-password=PASSWORD...]
>         [--sign-with=KEY...]
>         [--] [CERTS...]

[...]

> `--as` defaults to `binary`.

[...]

> If `--as` is set to either `text` or `mime`, then `--sign-with`
> will sign as a canonical text document.  In this case, if the input
> `DATA` is not valid `UTF-8`, `sop encrypt` fails with a return code of
> 53.
 
What is `mime` here? Why is it necessary? Expand.

[...]

> Example:
> 
> (In this example, `bob.bin` is a file containing Bob's binary-formatted OpenPGP certificate.
> Alice is encrypting a message to both herself and Bob.)
> 
>     $ sop encrypt --as=mime --sign-with=alice.key alice.asc bob.bin < message.eml >encrypted.asc
>     $ head -n1 encrypted.asc
>     -----BEGIN PGP MESSAGE-----
>     $

This is another good example of how the arguments and options can become
confusing. Looking at the above commandline, I can't tell what alice.asc
or bob.bin are. Is bob.bin a private key? Maybe not, because it's a
`.bin`. But what if it was also named `.asc`? What if I reverse the two
arguments by mistake? I could encrypt to bob instead of alice! Or wait,
those are *both* keys I encrypt to! Phew, I'm safe... But wait, what if
I forgot the `<`!!

See where I'm going? :)

Having an explicit --encrypt-with=alice.asc (or --encrypt-to?) would be
much better here. It would also make much more sense in an API.
 
> decrypt: Decrypt a Message {#decrypt}
> --------------------------
> 
>     sop decrypt [--session-key-out=SESSIONKEY]
>         [--with-session-key=SESSIONKEY...]
>         [--with-password=PASSWORD...]
>         [--verify-out=VERIFICATIONS
>          [--verify-with=CERTS...]
>          [--verify-not-before=DATE]
>          [--verify-not-after=DATE] ]
>         [--] [KEY...]
> 
>  - Standard Input: `CIPHERTEXT` ({{ciphertext}})
>  - Standard Output: `DATA` ({{data}})
> 
> `--session-key-out` can be used to learn the session key on
> successful decryption.

"learn"? What does that mean? It seems it means "write to a file". 
If so that should be said explicitely here.
 
> If `sop decrypt` fails for any reason and the identified `--session-key-out`
> file already exists in the filesystem, the file will be unlinked.
 
This seems dangerous! Why do we delete a file we haven't created?
Explain.

> `--session-key-in` enables decryption of the `CIPHERTEXT` using the session key directly against the `SEIPD` packet.
> This option can be used multiple times if several possible session keys should be tried.

What happens if both "in" and "out" are provided? I can venture a guess,
but it would be important to make that explicit as there can be horrible
bugs there.
 
> `--with-password` enables decryption based on any `SKESK` packets in the `CIPHERTEXT`.
> This option can be used multiple times if the user wants to try more than one password.
 
We should include SKESK in terminology, because it's the first time we
encounter it here and I have close to no idea what it means.
 
> If `sop decrypt` tries and fails to use a supplied `PASSWORD`, and it
> observes that there is trailing `UTF-8` whitespace at the end of the
> `PASSWORD`, it will retry with the trailing whitespace stripped.
 
Explain why we do magic things with whitespace. Consider not doing magic
at all as magic can be evil.
 
> `--verify-out` produces signature verification status to the
> designated file.
> 
> `sop decrypt` does not fail (that is, the return code is not modified)
> based on the results of signature verification.  The caller MUST check
> the returned `VERIFICATIONS` to confirm signature status.  An empty
> `VERIFICATIONS` output indicates that no valid signatures were found.
> If `sop decrypt` itself fails for any reason, and the identified
> `VERIFICATIONS` file already exists in the filesystem, the file will
> be unlinked.
> 
> `--verify-with` identifies a set of certificates whose signatures would be
> acceptable for signatures over this message.

Not failing explicitely on verification seems very dangerous. It relies
on callers properly reading the spec and realizing this is the only
exception where exit codes don't suffice in providing a general state of
the program. I would strongly recommend failing here, just like regular
verify.

As an aside, why can't we compose verify and decrypt here and just keep
"verify" out of "decrypt" altogether? I would guess that's (a
limitation?) part of the OpenPGP standard, but maybe it would be nice to
explicitely expand on this here as well.

> If the caller is interested in signature verification, both
> `--verify-out` and at least one `--verify-with` must be supplied.  If
> only one of these arguments is supplied, `sop decrypt` fails with a
> return code of 23.
 
Another argument for failing on bad signatures: if we fail on bad
arguments of --verify, why don't we fail on bad signatures?

> armor: armor: Add ASCII Armor
> -----------------------------

[...]

> If the incoming data is already armored, and the `--allow-nested` flag
> is not specified, the data MUST be output with no modifications.
> Data is considered ASCII armored iff the first 14 bytes are exactly
> `-----BEGIN PGP`. This operation is thus idempotent by default.
 
Explain why we want idempotent and why we want to do this guessing game.

[...]

> Input/Output Indirect Types
> ===========================
> 
> Some material is passed to `sop` indirectly, typically by referring to
> a filename containing the data in question.  This type of data may
> also be passed to `sop` on Standard Input, or delivered by `sop` to
> Standard Output.
> 
> If the filename for any indirect material used as input has the
> special form `@ENV:xxx`, then contents of environment variable `$xxx`
> is used instead of looking in the filesystem.
> 
> If the filename for any indirect material used as either input or
> output has the special form `@FD:nnn` where `nnn` is a decimal
> integer, then the associated data is read from file descriptor `nnn`.

This @ENV: and @FD: stuff really makes me uncomfortable. It's a neat
hack for commandline applications, but it would break down when
designing a library API, as the "type" of data passed around the API
would be ambiguous, or at least with possible side effects. That feels
like "design smell" here and I would like this to be changed.

I would recommend using equivalent environment variables to the
parameters instead, for example SIGN_WITH for --sign-with and so
on. This would, of course, require switching positional arguments to
options but I already explain why that would be a good idea anyways
earlier.

File descriptors could be passable as distinct options, like
--sign-with-fd for --sign-with.

I've dealt with commandline applications that have special meanings with
@, and in retrospect, it was a bad idea. In particular, Python's
argparse module supports using a prefix argument to mean "read options
from this file" and I've used it to implement crude configuration file
support for monkeysign and other programs. It's confusing for users and
does not work very well.

Specifically in this case, I would also worry about security
vulnerabilities with untrusted filenames being passed to the program.

[...]

> CERTS {#certs}
> -----
> 
> One or more OpenPGP certificates (section 11.1 of {{I-D.ietf-openpgp-rfc4880bis}}), aka "Transferable Public Key".
> May be armored.
> 
> Although some existing workflows may prefer to use one `CERTS` object with multiple certificates in it (a "keyring"), supplying exactly one certificate per `CERTS` input will make error reporting clearer and easier.

This last bit is in contradiction with `extract-cert` command
documentation which says it will "only contain one cert". Maybe we
should just pick one and stick with it here?

[...]

> VERIFICATIONS {#verifications}
> -------------
> 
> One line per successful signature verification.  Each line has three
> structured fields delimited by a single space, followed by arbitrary
> text to the end of the line.
> 
>  - ISO-8601 UTC datestamp
>  - Fingerprint of the signing key (may be a subkey)
>  - Fingerprint of primary key of signing certificate (if signed by primary key, same as the previous field)
>  - arbitrary text
 
That last part doesn't *look* like "arbitrary text" to me. It looks like
some explanatory message of the operation. If that's the case, we should
make that explicit and say why the text is present at all. Calling it a
"note" or "message" would already be an improvement.

> Example:
> 
>     2019-10-24T23:48:29Z C90E6D36200A1B922A1509E77618196529AE5FF8 C4BC2DDB38CCE96485EBE9C2F20691179038E5C6 certificate from dkg.asc

For the record, arbitrary text looks like:

> This is some garbage I just jklfa bldasjkl ajklblablabla.

I can provide more samples or arbitrary text as required. I'm an
experience freelance writer and large samples can be found on
https://anarc.at/ ;)

[...]

> Failure Modes
> =============
> 
> When `sop` succeeds, it will return 0 and emit nothing to Standard
> Error.  When `sop` fails, it fails with a non-zero return code, and
> emits one or more warning messages on Standard Error.  Known return
> codes include:
> 
> Return | Meaning
> ---:|--------------------------------------------------
>  0 | Success
>  3 | No acceptable signatures found (`sop verify`)
> 13 | Asymmetric algorithm unsupported (`sop encrypt`)
> 17 | Certificate not encryption-capable (e.g., expired, revoked, unacceptable usage flags) (`sop encrypt`)
> 19 | Missing required argument
> 23 | Incomplete verification instructions (`sop decrypt`)
> 29 | Unable to decrypt (`sop decrypt`)
> 31 | Non-`UTF-8` password (`sop encrypt`)
> 37 | Unsupported option
> 41 | Invalid data type (no secret key where `KEY` expected, etc)
> 53 | Non-text input where text expected
> 69 | Unsupported subcommand

I already mentioned some problems with those failure codes, but I will
repeat here the suggestion that symbolic names instead of integers
should be used for primary referencing in the document here again.

> A `sop` implementation MAY return other error codes than those listed
> above.
 
This sounds like a bad idea. I interpret that as meaning that I can
return an error code 2 instead of error code 3 if i fancy. If we're
going to pick numbers, we should either enforce them or not, but don't
dance around the issue and encourage people to diverge from the spec.

Or at least, if you allow divergence, explain why it can be allowed.
 
It would also be great if we could explain where those magic numbers
come from in the first place. I suspect they were chosen to not overlap
with existing error codes, but that's just a guess.

[...]

> Detached Signatures {#detached-signatures}
> -------------------
> 
> `sop` deals with detached signatures as the baseline form of OpenPGP signatures.
> 
> The main problem this avoids is the trickiness of handling a signature that is mixed inline into the data that it is signing.

Should we expand on "trickiness" here?
 
Also: how *do* we deal with inline signatures? Are those deprecated now?

[...]

> Guidance for Consumers
> ======================
> 
> While `sop` is originally conceived of as an interface for interoperability testing, it's conceivable that an application that uses OpenPGP for object security would want to use it.
> 
> FIXME: more guidance for how to use such a tool safely and efficiently goes here.
> 
> FIXME: if an encrypted OpenPGP message arrives without metadata, it is difficult to know which signers to consider when decrypting.
> How do we do this efficiently without invoking `sop decrypt` twice, once without `--verify-*` and again with the expected identity material?
 
Maybe we could use a "sop probe" command for this and other things?

> Security Considerations
> =======================
> 
> The OpenPGP object security model is typically used for confidentiality and authenticity purposes.
> 
> Signature Verification {#signature-verification}
> ----------------------
> 
> In many contexts, an OpenPGP signature is verified to prove the origin and integrity of an underlying object.
> 
> When `sop` checks a signature (e.g. via `sop verify` or `sop decrypt --verify-with`), it MUST NOT consider it to be verified unless all of these conditions are met:
> 
>  * The signature must be made by a signing-capable public key that is present in one of the supplied certificates
>  * The certificate and signing subkey must have been created before or at the signature time
>  * The certificate and signing subkey must not have been expired at the signature time
>  * The certificate and signing subkey must not be revoked with a "hard" revocation
>  * If the certificate or signing subkey is revoked with a "soft" revocation, then the signature time must predate the revocation
>  * The signing subkey must be properly bound to the primary key, and cross-signed
>  * The signature (and any dependent signature, such as the cross-sig or subkey binding signatures) must be made with strong cryptographic algorithms (e.g., not `MD5` or a 1024-bit `RSA` key)
 
Latter seems to contradict section 7.5, which says:

>    For performance reasons, an implementation may choose to ignore
>    validation on certificate and key material supplied to it.  The
>    security implications are of doing so depend on how the certs and
>    keys are managed outside of "sop".

So should we check supplied keys or not? but I guess that's covered by...

[...]

> Signature validity is a complex topic, and this documentation cannot list all possible details.

Is there a reference we could add here to cover that topic?

> Compression {#compression}
> -----------
> 
> The interface as currently specified does not allow for control of compression.
> Compressing and encrypting data that may contain both attacker-supplied material and sensitive material could leak information about the sensitive material (see the CRIME attack).
> 
> Unless an application knows for sure that no attacker-supplied material is present in the input, it should not compress during encryption.

How about decryption? Do we attempt decompression during decrypt?

That's about it! My comments might be a little dry and are the results
of notes I jotted down on "paper" (a PDF really), a copy of which is
available here:

https://paste.anarc.at/publish/2019-11-10-sxrDKhJL9R0/sop-Exported.pdf

I hope the comments are still useful and please don't interpret them as
me being upset or too critical. I love this work and it's only in a
spirit of collaboration that I bring up those concerns. :)

Thank you for your work!

A.

PS: I mistakenly took the modified version of the document, as available
on the HEAD of the merge request, to comment on the document. I have
tried, as much as possible, to revert changes to the original in the
examples, but some other changes I suggested might have crept up in the
quoted text. That shouldn't affect the comments I have brought up in any
other way.

-- 
Antoine Beaupré
torproject.org system administration

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Mon, 11 Nov 2019 18:00:17 -0500

Hi Antoine--

Thanks for the thoughtful review.  It was super long though!  I've
opened a bunch of issues from the stuff you raised here, but more
dicsussion follows inline below.

> https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9

I've reviewed and merged the changes requested there, definitely useful
to have this clarifying editorial work done as patches, thanks!

> actually mean? What do we mean by "new cryptographic primitives" here

The intent is that if every OpenPGP implementation provides a `sop`
interface (or potentially a superset of it), then we can write
interoperabiity tests and the like that drive all of them in a reliable
way.

We can, for example, generate new OpenPGP objects that incorporate new
primitives, and feed them to a stable of `sop` implementations, to
determine whether those implementations can consume them.

Or, we can drive them with simple inputs, and see which cryptographic
primitives they choose to use as they produce output.

If you think this text is clearer, i can incorporate it directly in the
draft.

> What about OpenPGP cards like the Yubikey? How does sop interoperate

sop is not GnuPG, and it doesn't claim or intend to be.

An implementation of sop *may* choose to support other forms of secret
key access (the "@FOO:" namespace is carved out to allow for that), but
the goal of sop is that it is simple enough that every implementation
that can handle OpenPGP secret keys and certificates should be capable
of implementing the interface.

> seems ambiguous. Is it "CERT DATA"? or "CERT DATA"?

???  i think those are the same thing, but i'll just assume you meant
"DATA CERTS" at the end.  The answer is that there must be exactly one
SIGNATURE object to verify and there may be multiple certs, so the only
possible way to do it is SIGNATURE first, then CERTS.

> secret or verification keys.

I would welcome a MR that makes this change across the entire spec, to
see whether it looks reasonable.  If people on this list prefer that
structure, i would be fine with adopting it, though i think it makes
the CLI more verbose than i would like.

I've trimmed out a lot of your comments below that were to do with
whether positional arguments are sensible or not.  that's not because i
don't care, but i don't think arguing about it in this one message is
helpful.

i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/7 to
record the overall concern.  if you decide to make an MR for it, please
reference that issue in the MR!

> constants to use in a library implementation.

I really like this idea, and i encourage someone™ who also likes it to
make a MR that implements it.

i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/1 to
keep track of it.

> think, but it should be made explicit here as well.

the text you've quoted says "any output OpenPGP material".  I'm not sure
how to make that more visible, but i welcome proposed edits.

> How do we generate purpose-specific subkeys?

With `sop`, you do not ;)

If you want to do fancy OpenPGP certificate generation, you do that with
your toolkit's own fancy features.

I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/2 to
track that maybe we do want some rough guidance about what kinds of
secret key capabilities we want any `sop` to be able to generate here
though.

> […]
>> If `--as=text` and the input `DATA` is

We don't mandate UTF-8 unless the signer claims that the thing being
signed is text.  If so, it really does need to be UTF-8.  I have no
patience for non-UTF-8-encoded text in 2019.

OpenPGP embeds UTF-8 explicitly in its User ID formatting.  Any OpenPGP
implementation must already handle UTF-8.

if anyone thinks that dealing with different character encodings is a
good idea, please consider that the character encoding is not recorded
in the signature itself, leading charset-switching attacks like those in
https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/

Do you think this information belongs in this document?

> don't undrestand what they bring to the table.

OpenPGP has two different forms of cryptographic signature, which are
dealt with by recipients differently.  The textual form canonicalizes
line-endings and the binary does not.  When signing a document, you need
to indicate to the thing doing the signing which form you are expecting
to create.

If you have a different suggestion, i'm happy to hear it.

> that we allow implementations to do so.

We deliberately *do not* allow implementations to do so.  sop requires
that you indicate which secret keys you want to sign with.  it is one or
more KEY objects, not zero or more.

If you think that `sop` is supposed to sign with other objects, then
i've done a bad job at drafting this proposal.  I've opened
https://gitlab.com/dkg/openpgp-stateless-cli/issues/5 to try to clarify this.

> By forcing the arguments here to be the signing key, we make it
> difficult to let the implementation pick the right key.

The entire point of `sop` is that `sop` *does not know* anything about
the keys.  it is stateless, and you have to tell it explicitly which key
to use.

> What is `mime` here? Why is it necessary? Expand.

i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/6 to
track this.

> If so that should be said explicitely here.

*What* the user of sop does is that they learn of the session key that
was used for the encrypted message.  *How* they learn it is by receiving
it from the program, either via the filesystem, or via @FD:NNN.

If you think that's unclear, i'd welcome a clarifying patch.

> This seems dangerous! Why do we delete a file we haven't created?

We don't want the user to run `sop`, and then inspect a file that was
already in the filesystem thinking that it is `sop`s output.  If you
think that's a bad decision, please suggest what we should do
differently.

>> [`--with-session-key`] enables decryption of the `CIPHERTEXT` using the session key directly against the `SEIPD` packet.

Please do venture a guess, in the form of proposed text! I'd also love
to hear what the horrible bugs are.  I don't see them.

> encounter it here and I have close to no idea what it means.

https://gitlab.com/dkg/openpgp-stateless-cli/issues/12

I'm not sure we should put it in "terminology", but we surely should
document it better the first time it appears (it is documented the
second time it appears in the text.

> at all as magic can be evil.

I expect the following use case to be common:

    echo correct horse battery staple > password.txt
    sop decrypt --with-password=password.txt < ciphertext > cleartext

If we don't strip whitespace, it will fail on one side or the other.

sop tries to define what sensible magic should be here, and i think i've
gotten it right.

as for magic being evil: if we don't try to do magic, then we will be
failing in ways that users don't understand, and implementers will get
bug reports that they respond to with "have you tried trimming trailing
whitespace and then trying again?"  that kind of round trip is pointless
when the tool could just avoid the problem in the first place by being
reasonable about what kinds of things people tend to do.

    https://gitlab.com/dkg/openpgp-stateless-cli/issues/13

> the program. I would strongly recommend failing here, just like regular

if the person invokes `sop decrypt` with no arguments, should it fail?

if the person invokes `sop decrypt` with `--verify-out` and
`--verify-with`, should `sop decrypt` produce any output?

I'm trying to imagine using this in a MUA.  In my MUA i have what i
think are the keys for my peer, and i see a message from them.

I don't yet know whether it's signed.

I want to decrypt the message to read it, and in the process, i want to
find out whether it has been signed.  I'd prefer to avoid two passes in
this common use case.

If i supply the --verify-* arguments, and sop fails, then i don't get
the cleartext of the message (not in any reliable way, anyhow).  If i
don't supply the --verify-* arguments, and sop succeeds, i've lost any
signature data.

The subcommand is "decrypt", so `sop` treats successful decryption as a
success.  If you want to understand some additional state, you have to
inspect that state somewhere else, and that's in the contents of
`--verify-out`.

does that make sense?  do you think any of that explanation belongs in
the document itself?

> explicitely expand on this here as well.

Do you want to propose a way to compose them?  I suppose we could have
`sop decrypt` offer a `--signatures-to` argument, which the sender could
then use for `sop verify`, if anything comes out.  That's kind of an
interesting suggestion, and it might reduce the overall complexity of
`sop`.

However:

 * It means that a caller would need to handle the data twice (a minor
   inefficiency)

 * the signatures might not be verifiable if the thing that is signed is
   not a simple literal data packet, but is, say, a compressed data
   packet wrapped around a literal data packet.

So i don't see how to do this safely (or efficiently, but the
verifiability of the signature is more important than the efficiency
argument)

> arguments of --verify, why don't we fail on bad signatures?

failing on bad arguments is "you've asked an ill-formed question".
That's legitimate and useful to let the operator know that things are
not what they seem.

Failing on a bad signature would be "the answer is no".

If the primary operation you're asking for is verification, then failing
on bad signatures is a reasonable outcome -- only succeed if you succeed
in verifying.

But if the primary operation is decryption, i don't think we should fail
on signature validity for reasons outlined above.

> Explain why we want idempotent and why we want to do this guessing game.

I'm at a bit of a loss here, because these seem obvious to me.

We want a guessing game because users who don't know what they want are
going to guess anyway, and they're likely to guess wrong.  We might as
well guess on their behalf if they don't know.

We want idempotent because "ensure this thing is armored" is a
reasonable semantic request -- indeed, it's probably the *only*
reasonable semantic request.  Perhaps we could drop --allow-nested?

> like "design smell" here and I would like this to be changed.

FWIW, the @ENV: and @FD: conventions are specifically CLI conventions
and i don't expect them to be translated to any programmatic API.

for example, in the pythonic variant of this framework that i'm working
on, i've treated these objects as labeled bytestreams. ("labeled"
meaning -- if you get a set of these objects, each one has a bytes-like
thing, and a textual name that you can use in error reporting)

I admit that i'm struggling a bit with whether i want to pass around
bytes objects directly (simple, what i've got now) or some sort of
asyncio handle that the bytes are readable from (fancier, probably nicer
to use in the kinds of programs that i will eventually want to write
with this). https://gitlab.com/dkg/python-sop/issues/1 But that has
nothing to do with @ENV and @FD, as those aren't exposed to the python
functions at all.

> options but I already explain why that would be a good idea anyways

environment variables won't work for arguments that can be supplied
multiple times, because then we have to invent a new delimiting scheme.
I definitely don't want to do that.

> --sign-with-fd for --sign-with.

This is an interesting proposal, though i don't see how --sign-with=@FD:3
is much different from --sign-with-fd=3  -- i guess it lets you use
files that are literally named @FD:3 ?  Is that important?

If you could open a merge request that proposes this change i'd be happy
to consider it.

I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/14 to
keep track of it.

> support for monkeysign and other programs. It's confusing for users and

afaict, this is not mandatory in argparse, and i wouldn't recommend it
for any sop implementation:
https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars

I think the way it's specified in `sop` right now is pretty pincipled
and hard to screw up, but i could be wrong.

> vulnerabilities with untrusted filenames being passed to the program.

can you explain this more?

> should just pick one and stick with it here?

I have carefully considered this, and i do not think these are in
contradiction.

extract-cert explicitly says it will contain only one cert *because* the
other places where `CERTS` might be supplied could contain more than
one.

The fact is that people use "keyrings" today, including some that
contain hundreds or thousands of keys.  If a distro, for example, wants
to use `sop` to verify that a package is signed by one of their
developers, i don't want the distro to need to put each developer's key
in a separate file.

> "note" or "message" would already be an improvement.

patches welcome, particularly for this kind of editorial cleanup :)

> Or at least, if you allow divergence, explain why it can be allowed.

I've documented this concern as
https://gitlab.com/dkg/openpgp-stateless-cli/issues/10

> with existing error codes, but that's just a guess.

Justus picked 69 in his OpenPGP Interoperability Test Suite.  I chose
the others as "reasonable-sized primes" just for fun.  I don't think
this information belongs in this document, as it doesn't matter.

> Also: how *do* we deal with inline signatures? Are those deprecated now?

From my POV, yes, inline signatures are recipes for disaster, because
the difficulty of figuring out what specifically was signed when you
have an inline signature is not at all obvious.  I've noted that this
needs to be reflected in the spec at:

  https://gitlab.com/dkg/openpgp-stateless-cli/issues/9

That said, i've heard from some of the folks handling package managers
who have specific reasons for wanting inline signatures (also, Neal
Walfield appears to think they're useful in his
https://gitlab.com/sequoia-pgp/pgpcat).  so we probably need to tackle
them somehow here.  (it's even in "future work"!)  I've recorded that
concern here:

   https://gitlab.com/dkg/openpgp-stateless-cli/issues/11

> Maybe we could use a "sop probe" command for this and other things?

I don't understand what you think a "sop probe" command would do.  if
you'd like to propose it as an MR, i'd consider it, though.

>> Compression {#compression}
[…]
> How about decryption? Do we attempt decompression during decrypt?

It will be interesting to see what implementers do!  I've left `sop`
deliberately agnostic there, and i would like to learn from test suites
what the answer is.

     --dkg

---

From: Antoine Beaupré <[email-masked]>
Date: Tue, 12 Nov 2019 12:26:00 -0500

> dicsussion follows inline below.

No problem! It's a long draft in the first place (24 pages PDF), and
talks about something I actually have (had) lots of ideas about in the
past, so I guess that's normal...

> to have this clarifying editorial work done as patches, thanks!

Great!

> If you think this text is clearer, i can incorporate it directly in the

That would be great, thanks! Providing examples like this would go a
long way...

> sop is not GnuPG, and it doesn't claim or intend to be.

thank you for that too. ;)

> of implementing the interface.

I guess what I'm wondering is how I would make this work with my yubikey
at all. Or maybe I got this backwards and the yubikey interface is what
should implement sop directly?

> possible way to do it is SIGNATURE first, then CERTS.

Ah, yes, sorry about this. I was specifically refering to:

And `"CERT DATA" or "DATA CERT"`?

And I guess where we differ is I am not sure it's that clear that the
first argument of a series can be different from the rest...

> reference that issue in the MR!

I have a patch for this and will submit it. It was more invasive than
the others so I figured I would bring it up as a discussion first.

> i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/1 to

I can try to tackle this as well.

> how to make that more visible, but i welcome proposed edits.

I made this MR for this:

https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/10

> With `sop`, you do not ;)

Sad.

> secret key capabilities we want any `sop` to be able to generate here

Commented on that. Would still love to see a more decent way to handle
subkeys because that's a really hard thing to do in existing
implementations.

At least creating split subkeys by default would be a great start, IMHO.

> Do you think this information belongs in this document?

Absolutely, otherwise it looks like an arbitrary decision.

I am not, for the record, arguing for other encodings. UTF-8 is fine and
diverging is a recipe for disaster, indeed.

> to indicate to the thing doing the signing which form you are expecting

The intricacies of OpenPGP never cease to amaze me.

> If you have a different suggestion, i'm happy to hear it.

I wish we didn't have to deal with this distinction, but if so, maybe we
should clarify the source of it here. Otherwise it comes as a surprise
to me, an experience OpenPGP user.

> https://gitlab.com/dkg/openpgp-stateless-cli/issues/5 to try to clarify this.

I understand where you're coming from here, and it's good to make sure a
stateless implementation.

What I'm saying is the `sop sign` example is error prone. Forget the `<`
and the mandated order and you might reverse the signing key and the
message.

> the keys.  it is stateless, and you have to tell it explicitly which key

I guess I'm wondering if implementations would be allow to diverge on
that matter at all.

[...]

> If you think that's unclear, i'd welcome a clarifying patch.

https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/11

> think that's a bad decision, please suggest what we should do

Maybe we should not overwrite existing files at all and fail earlier?

> to hear what the horrible bugs are.  I don't see them.

I would argue that both options should not be provided at once. One
implementation that could come up would be that the program attempts to
read the file as it's writing it, truncating the precious key before it
has time to read it.

[...]

> sop tries to define what sensible magic should be here, and i think i've

We can continue the discussion in issue #13, but the TL;DR: is that I
agree that stripping trailing control characters is a good idea, but
disagree about whitespace in general.

> if the person invokes `sop decrypt` with no arguments, should it fail?

no.

> `--verify-with`, should `sop decrypt` produce any output?

yes, except if verification fails.

> don't supply the --verify-* arguments, and sop succeeds, i've lost any

I don't know how OpenPGP packets are built. Can't we show the signature
on the output of decrypt?

> does that make sense?  do you think any of that explanation belongs in

Maybe there are some obvious assumptions about the OpenPGP message
structure that make it clear this option is necessary. It's not to me.

> verifiability of the signature is more important than the efficiency

You very likely know better. ;) It was just an idea, and probably
impractical in reality.

> on signature validity for reasons outlined above.

But that assumes decryption is the primary operation. In the context
where all my email traffic is encrypted with OpenPGP, for example,
decryption is not the primary operation anymore. I *do* want to fail
properly on signature validity, it becomes a primary operation when
encryption is "default"...

> reasonable semantic request.  Perhaps we could drop --allow-nested?

I am now also confused here and ready to drop this argument. :)

> and i don't expect them to be translated to any programmatic API.

Of course.

> nothing to do with @ENV and @FD, as those aren't exposed to the python

I'm not sure asyncio is useful here. Callers can wrap around the byte
stream as they see fit, no?

(I really dislike this "parallel universe" thing they built in Python 3
for asyncio, it makes every API more complicated than it needs to
be. But that's another story...)

> I definitely don't want to do that.

True. That is a significant limitation. I am not sure how to work around
that problem.

> files that are literally named @FD:3 ?  Is that important?

It's less magic, more explicit, and correlates better with other
commandline APIs I have encountered.

> I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/14 to

I'll see what I can do.

> can you explain this more?

Say you think you are in a trusted directory with "CERTS" that you want
to encrypt to. You call:

  sop encrypt * < /tmp/file > /tmp/file.pgp

Except you made a mistake and the attacker has control of the current
directory, and injects a file named (say) @ENV:SOMETHING. Assuming they
have control over the SOMETHING environment, they can now add an
encryption key to the message.

Control of the environment is kind of a stretch, I must admit, but in
certain environments (most notably web servers), a *lot* of stuff can
end up there and it shouldn't be completely trusted this way.

> developers, i don't want the distro to need to put each developer's key

Understood.

> patches welcome, particularly for this kind of editorial cleanup :)

https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/12

[...]

> this information belongs in this document, as it doesn't matter.

I love this kind of information in text, it makes it less dull. :p

[...]

> you'd like to propose it as an MR, i'd consider it, though.

`sop probe` would do the minimal amount of work required to determine
which keys ("signers") to consider  when decrypting, then call `decrypt`
properly.

`sop probe` could also do the general task of parsing OpenPGP messages
into packets and stuff like that.

> deliberately agnostic there, and i would like to learn from test suites

Should we make that decision clearer in the document?

Thanks again!

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Thu, 14 Nov 2019 03:18:07 +0300

I've incorporated some of your suggestions from this e-mail directly in
the draft now.  Thanks a lot for them!  more discussion below…

> should implement sop directly?

I have no idea about how to make it work with hardware tokens.  I remain
frankly unconvinced about the seurity tradeoffs for most uses of
hardware tokens (something i guess i need to actually write down more
formally), and i'm unlikely to spend a lot of time developing how to
integrate them with `sop`.

I would only assume that some `sop` implementation would choose to carve
out `@MYHSM:xxx` as an input space for `KEY`-style inputs, where `xxx`
is some form of addressing scheme that indicates which secret key on
which device should be used to interact with the device.

But sorting out the addressing scheme alone is problematic enough don't
plan to incorporate any of that detail in this draft.  A well-formed,
thorough yet compact merge request might convince me otherwise, but
color me skeptical at the moment.

> first argument of a series can be different from the rest...

well, the middle arguments of a series are *definitely* hard to
distinguish, so the only plausible distinctions when you've got
one-vs-many positional arguments is whether the "one" goes first or
last.  But let's follow up on that over at

    https://gitlab.com/dkg/openpgp-stateless-cli/issues/7

and in particular, on your ongoing merge request at:

    https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/13

If anyone else has strong feelings about this choice, please take a look
over there and follow up, either here on list, or on those tickets.

>> With `sop`, you do not ;)

I be

> At least creating split subkeys by default would be a great start, IMHO.

This is exactly the sort of decision that i want to see implementers
make, so we can document their choices.  `sop` is not about fancy key
management, nor should it be.  As i wrote in
https://gitlab.com/dkg/openpgp-stateless-cli/issues/2, i do not want
`sop` to place any detailed constraints here, i just want the generated
key to be functional for use with `sop`.

> Absolutely, otherwise it looks like an arbitrary decision.

I've just added the following subsection with "Guidance for
Implementers":

    Text is always UTF-8 {#utf8}
    --------------------

    Various places in this specification require UTF-8 {{RFC3629}} when encoding text. `sop` implementations SHOULD NOT consider textual data in any other character encoding.

    OpenPGP Implementations MUST already handle UTF-8, because various parts of {{RFC4880}} require it, including:

     - User ID
     - Notation name
     - Reason for revocation
     - ASCII-armor Comment: header

    Dealing with messages in other charsets leads to weird security failures like {{Charset-Switching}}, especially when the charset indication is not covered by any sort of cryptographic integrity check.
    Restricting textual data to `UTF-8` universally across the OpenPGP ecosystem eliminates any such risk without losing functionality, since `UTF-8` can encode all known characters.

If any thinks that's either wrong or insufficient, please send
corrections/improvements!

> to me, an experience OpenPGP user.

As a user, you shouldn't ever need to see it.  As an implementer, you
do need to think about it.

I've added the following text to the discussion of `sop sign`:

    `--as=binary` SHOULD result in an OpenPGP signature of type 0x00 ("Signature of a binary document").
    `--as=text` SHOULD result in an OpenPGP signature of type 0x01 ("Signature of a canonical text document").
    See section 5.2.1 of {{RFC4880}} for more details.

And i've added a new subsection in "Guidance for Conumers":

    Choosing between `--as=text`  and `--as=binary`
    ------------------------------------------------------

    A program that invokes `sop` to generate an OpenPGP signature typically needs to decide whether it is making a text or binary signature.

    By default, `sop` will make a binary signature.
    The caller of `sop sign` should choose `--as=text` only when it knows that:
     - the data being signed is in fact textual, and encoded in `UTF-8`, and
     - the signed data might be transmitted to the recipient (the verifier of the signature) over a channel that has the propensity to transform line-endings.

    Examples of such channels include FTP ({{RFC959}}) and SMTP ({{RFC5321}}).

> and the mandated order and you might reverse the signing key and the

sure. if you screw up any API, you can screw up any API :)

> Maybe we should not overwrite existing files at all and fail earlier?

I think you're proposing that if the `--sessionkey-out` file already
exists in the filesystem, that should be an error in the first place.
I'd be happy to entertain that idea, if anyone wants to provide text for
it.

If you decide to try to write it up, please think about how it works for
the other scenarios where `sop` can produce output on more than stdout.
it would be nice if these mechanisms all had the same behavior.

> read the file as it's writing it, truncating the precious key before it

Ah, you're not talking about providing both options -- you're talking
about providing both options pointing *at the same file*.  i agree, that
sounds like a bad idea, but it's a bad idea for *any* pair of input and
output fields.

> disagree about whitespace in general.

I hope other folks will weigh in on #13.  There's interesting discussion
going on there about what properties it's reasonable to expect from a
"well-formed" password.

> on the output of decrypt?

Absolutely not.  Mixing the cleartext output with the signature
verification stream is a classic cause of failures.  What if the
cleartext data happens to "look like" a signature verification?  how is
the consumer supposed to distinguish between them?

It is critical to keep them separate.

> But that assumes decryption is the primary operation.

The subcommand is "sop decrypt".  By definition, "decrypt" is the
primary operation.

> In the context where all my email traffic is encrypted with OpenPGP,
> for example, decryption is not the primary operation anymore. I *do*
> want to fail properly on signature validity, it becomes a primary
> operation when encryption is "default"...

You want a *failure* in the sense that you think that an MUA shouldn't
show the user the cleartext of the message if no valid signature can be
found?

This is suprising to me, and i know of no MUA that does this.

> commandline APIs I have encountered.

it looks to me like it would make the description of the command line
significantly more verbose, but i'm willing to consider it if someone
wants to propose a specific textual change.

> encryption key to the message.

if the attacker has control of the directory, they can inject an
encryption key in the first place, right?  i don't think @ENV makes this
any worse...

> end up there and it shouldn't be completely trusted this way.

perhaps when an `@`-prefixed argument is supplied, if a file with a
matching literal name exists, `sop` should fail with an error because of
the ambiguity?  This seems like an unlikely and unusual situation, but i
can see how it might be worth thinking about it.

Perhaps it would be interesting to contrast a MR that contains this
guidance with a MR that switches over to the --with-$foo-fd= approach
you've suggested.

> https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/12

thanks, merged ;)

> I love this kind of information in text, it makes it less dull. :p

i think we have a difference of opinion here.  maybe this is ok in an
acknowlegements section, but i definitely don't want to read discursive
stories when i'm trying to extract technical information.

If you want to supply a patch for the acknowledgements section, i'd be
willing to consider merging it.

> which keys ("signers") to consider  when decrypting, then call `decrypt`

I don't think this is "stateless", and i don't think it's
well-specified.

I also don't think it's particularly useful to know *that* a thing was
signed (by some arbitrary certificates) if you haven't already made some
determination that the certificate is meaningful in the current context.

> into packets and stuff like that.

this doesn't sound like a subcommand at the same level of abstraction
that `sop` is aiming for.

So i'm still not convinced.  But if you (or anyone) wants to make a
merge request that proposes new subcommands, i'm definitely up for
reviewing them.

> Should we make that decision clearer in the document?

it's not really a decision :) Hopefully my description of how it would
work as part of an interop suite will give a hint of this kind of
approach.

Thanks for the discussion!

        --dkg

---

From: Antoine Beaupré <[email-masked]>
Date: Thu, 14 Nov 2019 14:52:20 -0500

> integrate them with `sop`.

I'm not sure it would be productive for me to engage at that level here
right now, given how long this thread is already. :) Suffice to say I
believe there's enough of a use case of those devices that we might want
to think about how this standards interoperates with them.

I guess we don't have good answers for this in any case. At least I
don't know how to materialize those things in standards right now
anyways, but I would love for this to change in the future.

[...]

[all good stuff elided]

[...]

> sure. if you screw up any API, you can screw up any API :)

Some APIs are harder to screw up than others. Naming things,
specifically, forces people to think about meaning. Ordering them
doesn't.

> I'd be happy to entertain that idea, if anyone wants to provide text for

Yes, i think that might be preferable.

> it would be nice if these mechanisms all had the same behavior.

Okay, I'll keep that in mind.

> sounds like a bad idea, but it's a bad idea for *any* pair of input and

This is something I came up with recently while writing another
program. I started by writing something that would read parameters from
a config file, then realized I could also *save* parameters to that same
file (or another). So I have two parameters, usually pointing at the
same file (but not necessarily).

The way "conflicts" are handled is simple: the file is first read, and
closed. Then it is written to. As long as that's done in order and
there's no parallelism, it works correctly.

I was wondering if we might want to do such a suggestion specifically
for the session key because it's this one case where you have a state
that can be read and written. Sure, everything in sop read and writes to
stuff, but in general they are different objects (cleartext vs
encrypted) that won't be written to the same file unless the user does a
mistake.

[...]

Understood.

[...]

> This is suprising to me, and i know of no MUA that does this.

Didn't think of that pattern. The system you're proposing makes more
sense in that context.

[... possibility of making yet another patch to support -fd ...]

> encryption key in the first place, right?  i don't think @ENV makes this

I guess so. It still makes me iffy even if I can't put my finger on it.

[...]

> I don't think this is "stateless", and i don't think it's

Sure, it's stateless. The input is external and the output is purely
informative. But of coruse, it's not well-specified. It was just an idea
to work around the double-pass problem you identified.

[...]

> Thanks for the discussion!

You're welcome! I'm not sure I will have much more time to provide
patches and more work, but at least that was something. :)

a.

---

From: Clint Adams <[email-masked]>
Date: Sat, 16 Nov 2019 17:08:53 +0000

> reasonable semantic request.  Perhaps we could drop --allow-nested?

As an implementer, I don't want this because this kind of run-time
detection is annoying, though it would be more annoying if the check
weren't explicit.

As a user, I don't want this because it violates the POLA.  Why would
I be directing something to "ensure this thing is armored"?  If I piped
something to base64 and it came out unchanged I would be shocked and
horrified.

Along those lines, having some subcommands default to binary output
and others default to ASCII Armor also seems confusing except in cases
like `armor` and `dearmor` where the intent would seem obvious
(modulo this whole idempotency thing).

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Sun, 17 Nov 2019 01:27:14 +0300

On Thu 2019-11-14 14:52:20 -0500, Antoine Beaupré wrote:
> [ dkg wrote: ]

i think you're right.  thanks for that.  I've committed that change in
d027f6e3b29417387daf541d251a541e179898b1.

> Okay, I'll keep that in mind.

On inspection, only `sop decrypt` has multiple outputs, so it wasn't as
wide a change as i'd feared.

> encrypted) that won't be written to the same file unless the user does a

This is bizarre stuff, and i'm struggling to imagine any use case for it
with `sop` as it is currently defined, just for supplying both
--with-session-key and --session-key-out at the same time (let alone
pointing them both at the same fle).

If a user invokes `sop decrypt` with a known session key, presumably it
is because they already know the session key.  Why, in that
circumstance, would they ask `sop decrypt` for the session key?

Are you imagining a situation where the user of `sop decrypt` is
*guessing* at the session-key, but is unsure?  For example, supplying
multiple --with-session-key arguments, or --with-session-key alongside a
KEY or a --with-password argument?

I guess i can see it for some situation where "i've got a pile of
session keys and a pile of messages, and i want to figure out which
session key belongs to which message".

Anyway, I've just added an explicit failure mode when an indirect input
is pointed to a file that does not exist (in
c7c05598a5532c203de28b0dc6e1bfb69eecf549).  And given that it's now an
error to point --with-session-key at an existing file, it should be
impossible to point both options to the same file.  so hopefully this
concerns is now moot.

         --dkg

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Sun, 17 Nov 2019 22:19:05 +0800

Hi Clint--

Thanks for your feedback!

> detection is annoying, though it would be more annoying if the check

I understand that you'd like to implement something more minimal, but
sop is trying to supply plausible interfaces for use cases for a
developer who doesn't know all the grubby details of OpenPGP but still
wants to work with confidential data as framed by OpenPGP.

> something to base64 and it came out unchanged I would be shocked and

The only use case i'm currently imagining for using `sop armor` is that
the user has come across some OpenPGP content in some environment, and
they want to transmit it over a channel that is not 8-bit clean (or
perhaps they want to put it into a text-based revision control system
that would rather not host binary blobs).

They have two choices:

 - detect whether it is already armored, and if not, invoke `sop armor`
 - unconditionally invoke `sop armor`, to ensure that it is armored.

The latter sounds simpler and easier, and it really isn't surprising or
astonishing, so i'm not sure it really violates POLA.

Vincent Breitmoser proposed the idempotency requirement over in
https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/3 and i
merged it because it seemed plausible to me.

I'm currenly thinking of dropping --allow-nested entirely, but i see
you've given that a thumbs-down on
https://gitlab.com/dkg/openpgp-stateless-cli/issues/15

if you want me to consider that more, a clearer use for nested armoring
would be welcome.

Do you have another use case that we should consider?

> (modulo this whole idempotency thing).

I believe all the subcommands default to armored output (with the
exception of "sop dearmor").  Are you seeing somewhere where that's not
the case?

    --dkg