From nobody Tue Nov 5 14:35:18 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B73A1120025 for ; Tue, 5 Nov 2019 14:35:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N2K-N_H66vg2 for ; Tue, 5 Nov 2019 14:35:14 -0800 (PST) Received: from mail.dasr.de (mail.dasr.de [217.69.77.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 21C8A120018 for ; Tue, 5 Nov 2019 14:35:13 -0800 (PST) Received: from p54abdd01.dip0.t-ipconnect.de ([84.171.221.1] helo=forster.huenfield.org) by mail.dasr.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1iS7Pr-0005aM-RG; Tue, 05 Nov 2019 22:35:11 +0000 Received: from grit.huenfield.org ([192.168.20.9] helo=grit.walfield.org) by forster.huenfield.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1iS7Pr-0005Kl-78; Tue, 05 Nov 2019 23:35:11 +0100 Date: Tue, 05 Nov 2019 23:35:11 +0100 Message-ID: <87v9rydk9s.wl-neal@walfield.org> From: "Neal H. Walfield" To: Daniel Kahn Gillmor Cc: openpgp@ietf.org In-Reply-To: <87woe7zx7o.fsf@fifthhorseman.net> References: <87woe7zx7o.fsf@fifthhorseman.net> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 192.168.20.9 X-SA-Exim-Mail-From: neal@walfield.org X-SA-Exim-Scanned: No (on forster.huenfield.org); SAEximRunCond expanded to false Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2019 22:35:17 -0000 I'm considering using the following "grammar". (I've put grammar in scare quotes, because it is not a valid grammar according to RFC 5322 due to several ambiguities. In particular, the production "*WS [name] *WS" is ambiguous when applied to a string containing a single whitespace character: the whitespace character could match the first WS or the second one. In practice, this ambiguity doesn't matter, because we only care about what the "name", "comment-content" and "addr-spec" productions match.) WS =3D 0x20 (space character) comment-specials =3D "<" / ">" / ; RFC 2822 specials - "(" and ")" "[" / "]" / ":" / ";" / "@" / "\" / "," / "." / DQUOTE atext-specials =3D "(" / ")" / ; RFC 2822 specials - "<" and ">". "[" / "]" / ":" / ";" / "@" / "\" / "," / "." / DQUOTE atext =3D ALPHA / DIGIT / ; Any character except contro= ls, "!" / "#" / ; SP, and specials. "$" / "%" / ; Used for atoms "&" / "'" / "*" / "+" / "-" / "/" / "=3D" / "?" / "^" / "_" / "`" / "{" / "|" / "}" / "~" / \u{80}-\u{10ffff} ; Non-ascii, non-control UTF-8 name-char-start =3D atext / atext-specials name-char-rest =3D atext / atext-specials / WS name =3D name-char-start *name-char-rest comment-char =3D atext / comment-specials / WS comment-content =3D *comment-char comment =3D "(" *WS comment-content *WS ")" addr-spec =3D dot-atom-text "@" dot-atom-text pgp-uid-convention =3D addr-spec / *WS [name] *WS [comment] *WS "<" addr-spec ">" / *WS name *WS [comment] *WS Beyond being more fleshed out, this grammar is different from the grammar in dkg's second proposal in a few ways. First, it matches comments. dkg made this a non-goal. Given that people who add comments intend them as comments and not as part of their name, it seems reasonable to me to not display comments in places where only the user's name is desired. And, since it turns out that matching non-nested comments is relatively straightforward, why not? Note: doing this might actually help deprecate comments, because they won't be shown as often. The grammar more carefully handles whitespace. It ignores whitespace at the beginning of the User ID (this is what motivates the name-char-start production) and between the individual components in the pgp-uid-convention production. As is, the grammar only ignores the 0x20 space character. We may also want to include the tab character, unicode's NO-BREAK SPACE (U+00A0) character and its IDEOGRAPHIC SPACE (U+3000) character for thoroughness. But, since software will normally concatenate the individual components, just recognizing the ASCII space character here is probably fine. Whatever the case, I think we can safely ignore the rest of unicode's whitespace characters: https://en.wikipedia.org/wiki/Whitespace_character My pgp-uid-convention production also matches user ids without email addresses, e.g., "Daniel Kahn Gillmor". This is convenient. Instead of having to figure out why parsing failed (is it not valid UTF-8? is it just missing an addr-spec?), we explicitly cover this common pattern in the grammar. I think this will significantly simplify code that uses this interface: if there is an error, then the code can just assume the User ID is trash and can be ignored. In RFC 2822, "specials" are only allowed in a display name if they are quoted. dkg removes this requirements. I think this is mostly sensible, but it means that we can have User IDs like: " " where the first is the display name and the second is the addr-spec. I think we should exclude angle brackets from the display name. In my grammar, I have an "atext-specials" which is just RFC 2822 specials without the angle brackets. I'm a bit concerned about allowing the backslash character: with this grammar, it is just a normal character, but for an RFC 2822 parser, it's an escape character. Since User IDs may be used in contexts where RFC 2822 things are expected, we should be careful. But, I fear that if we reject it, we'll end up gratuitiously rejecting some emojis. =C2=AF\_(=E3=83=84)_/=C2=AF. :) Neal From nobody Tue Nov 5 16:06:25 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA14E120121 for ; Tue, 5 Nov 2019 16:06:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (3072-bit key) header.d=crustytoothpaste.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jVTu8nx3m_PH for ; Tue, 5 Nov 2019 16:06:22 -0800 (PST) Received: from injection.crustytoothpaste.net (injection.crustytoothpaste.net [192.241.140.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 534AE1200C3 for ; Tue, 5 Nov 2019 16:06:22 -0800 (PST) Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id D08A06044D; Wed, 6 Nov 2019 00:05:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1572998751; bh=STFiEiGxNWcMeMEbxvXASzGJ/IDBpg6WEd7SjiNAweg=; h=Date:From:To:Cc:Subject:References:Content-Type: Content-Disposition:In-Reply-To:From:Reply-To:Subject:Date:To:CC: Resent-Date:Resent-From:Resent-To:Resent-Cc:In-Reply-To:References: Content-Type:Content-Disposition; b=CygPpAtI7UP/y9j2iMgT9QfLJIhs5xaZo8dvcxfo3Z5q6F9E3TGYOK/oPv4svwFuX X8K4Hz0R3vmMX/sS27naRyzZeh79k1YJ0xA7SpXuryBkSelyfIsUy3hcGiQTNs3b8D 7DOctoHbPwStCMtfB8u9nL7KmF2I7TKPIQjk4sYo7J51opsIJCDBqBO9/1SsGyLprD +VH0dd8cawf2ZvSTF3o8fcXFXRBliAmVWhNFXdnsKOS7wU7nR0yeTsP0UxamC8580u pX4luhNIt23MC24RUQd2PKyd7dfM/BZOK+EfixJM7YYBoAcYEIhriVFOgYvMDVAid7 EQ3CuP9dei7MCVRh8E48y+gXdKZKOTuMUKV9eN2Yk96e+OmdqdcGJG81H8kFblsGGV whnPjEK2aRstxwEimGvAlQMdm6myAhjBacxyoFbHK2oipIurXBbjbGkPytNDt9YvTt Ic5NHe7LcEUCMq2HFEYeiH10gyxzg14wfWdFu3Gfzg+57WPnIx8 Date: Wed, 6 Nov 2019 00:05:46 +0000 From: "brian m. carlson" To: "Neal H. Walfield" Cc: Daniel Kahn Gillmor , openpgp@ietf.org Message-ID: <20191106000546.GE32531@camp.crustytoothpaste.net> References: <87woe7zx7o.fsf@fifthhorseman.net> <87v9rydk9s.wl-neal@walfield.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="X3gaHHMYHkYqP6yf" Content-Disposition: inline In-Reply-To: <87v9rydk9s.wl-neal@walfield.org> X-Machine: Running on camp using GNU/Linux on x86_64 (Linux kernel 5.3.0-1-amd64) User-Agent: Mutt/1.12.2 (2019-09-21) Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 00:06:24 -0000 --X3gaHHMYHkYqP6yf Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2019-11-05 at 22:35:11, Neal H. Walfield wrote: > I'm considering using the following "grammar". (I've put grammar in > scare quotes, because it is not a valid grammar according to RFC 5322 > due to several ambiguities. In particular, the production "*WS > [name] *WS" is ambiguous when applied to a string containing a single > whitespace character: the whitespace character could match the first > WS or the second one. In practice, this ambiguity doesn't matter, > because we only care about what the "name", "comment-content" and > "addr-spec" productions match.) >=20 > WS =3D 0x20 (space character) >=20 > comment-specials =3D "<" / ">" / ; RFC 2822 specials - "(" and "= )" > "[" / "]" / > ":" / ";" / > "@" / "\" / > "," / "." / > DQUOTE >=20 > atext-specials =3D "(" / ")" / ; RFC 2822 specials - "<" and "= >". > "[" / "]" / > ":" / ";" / > "@" / "\" / > "," / "." / > DQUOTE >=20 > atext =3D ALPHA / DIGIT / ; Any character except cont= rols, > "!" / "#" / ; SP, and specials. > "$" / "%" / ; Used for atoms > "&" / "'" / > "*" / "+" / > "-" / "/" / > "=3D" / "?" / > "^" / "_" / > "`" / "{" / > "|" / "}" / > "~" / > \u{80}-\u{10ffff} ; Non-ascii, non-control UTF-8 >=20 > name-char-start =3D atext / atext-specials >=20 > name-char-rest =3D atext / atext-specials / WS >=20 > name =3D name-char-start *name-char-rest >=20 > comment-char =3D atext / comment-specials / WS >=20 > comment-content =3D *comment-char >=20 > comment =3D "(" *WS comment-content *WS ")" >=20 > addr-spec =3D dot-atom-text "@" dot-atom-text dot-atom-text isn't defined here, so it isn't clear to me what it includes. Does it permit UTF-8 in addresses according to the SMTPUTF8 RFCs? --=20 brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204 --X3gaHHMYHkYqP6yf Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.2.17 (GNU/Linux) iQIzBAABCgAdFiEEX8OngXdrJt+H9ww3v1NdgR9S9osFAl3CDloACgkQv1NdgR9S 9ovEsQ/+PRgMTRTyOXuTpibUItIIPTq8XpfUTu1FMQ2q1aMe0QIvmewJUJvdWTGg NWcmFwfZzM8NOmsHFrXWM2c2pkR5r9WBu/t8WWYX6FxYZ2rd5vgqcfbxIE0Mfr+L a+GsOmmkUmPFtdGrpau4j+mtJCxuF7Khe3ArYfsLASWPnbV5lBFxv1ZriJlpoOj0 oKw8ZJ2g/RsLQapqINuceVXVkrewlv32y6QIf1/1BQiJVL3mb/ncu5GShiT8Aod3 mZIk1HoO8Vv9J6f6c9e4d2HH3oC/XtSqFiJDA7tzmSbsm3blf1hi4R1UdjvY3Uza NGdzpN6V32pQN/Ib8czD6g+DUrFSzjzGuwpyFJ/0Meti1xGu1ESoIj467raoOJ6R 6bXpzJq/sjSSCGmRt89tKEcFJTGpCEgWWWUhiEDBRQuTRuV0IqhqHdtkHHKzaMlg Uyi6q214h9gzKuU8p77MGJ5X5S0DLnkAWVEO6BTK8SuMQVfnj8qg4H4+NPV97gB0 1LJs+wyzRCLDYOFCY8QOIOBSbzSK45ddMXjbwT3V2dquqRZTm4UMQzdDMon+9vBN f2ABcClbDvd1JSEJHhAdFL5PAt8TZ14zg9ENCv2bh/Abs0oWQI7TQirZLSBKwrd4 vyM4fUO9L96AvN+cBqej9A5smQ/SUcBac8TrdWFJ4/czY8K+tj8= =bGlY -----END PGP SIGNATURE----- --X3gaHHMYHkYqP6yf-- From nobody Tue Nov 5 23:37:35 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 936D7120122 for ; Tue, 5 Nov 2019 23:37:33 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4MpHvndXhudb for ; Tue, 5 Nov 2019 23:37:31 -0800 (PST) Received: from mail.dasr.de (mail.dasr.de [217.69.77.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4524712004F for ; Tue, 5 Nov 2019 23:37:31 -0800 (PST) Received: from p54abdd01.dip0.t-ipconnect.de ([84.171.221.1] helo=forster.huenfield.org) by mail.dasr.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1iSFsZ-0002Pg-KL; Wed, 06 Nov 2019 07:37:23 +0000 Received: from grit.huenfield.org ([192.168.20.9] helo=grit.walfield.org) by forster.huenfield.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1iSFsY-0005oF-Va; Wed, 06 Nov 2019 08:37:23 +0100 Date: Wed, 06 Nov 2019 08:37:22 +0100 Message-ID: <87tv7he9ql.wl-neal@walfield.org> From: "Neal H. Walfield" To: "brian m. carlson" Cc: Daniel Kahn Gillmor , openpgp@ietf.org In-Reply-To: <20191106000546.GE32531@camp.crustytoothpaste.net> References: <87woe7zx7o.fsf@fifthhorseman.net> <87v9rydk9s.wl-neal@walfield.org> <20191106000546.GE32531@camp.crustytoothpaste.net> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 192.168.20.9 X-SA-Exim-Mail-From: neal@walfield.org X-SA-Exim-Scanned: No (on forster.huenfield.org); SAEximRunCond expanded to false Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 07:37:33 -0000 Hi Brian, On Wed, 06 Nov 2019 01:05:46 +0100, brian m. carlson wrote: > On 2019-11-05 at 22:35:11, Neal H. Walfield wrote: > > I'm considering using the following "grammar". (I've put grammar in > > scare quotes, because it is not a valid grammar according to RFC 5322 > > due to several ambiguities. In particular, the production "*WS > > [name] *WS" is ambiguous when applied to a string containing a single > > whitespace character: the whitespace character could match the first > > WS or the second one. In practice, this ambiguity doesn't matter, > > because we only care about what the "name", "comment-content" and > > "addr-spec" productions match.) > > > > WS = 0x20 (space character) > > > > comment-specials = "<" / ">" / ; RFC 2822 specials - "(" and ")" > > "[" / "]" / > > ":" / ";" / > > "@" / "\" / > > "," / "." / > > DQUOTE > > > > atext-specials = "(" / ")" / ; RFC 2822 specials - "<" and ">". > > "[" / "]" / > > ":" / ";" / > > "@" / "\" / > > "," / "." / > > DQUOTE > > > > atext = ALPHA / DIGIT / ; Any character except controls, > > "!" / "#" / ; SP, and specials. > > "$" / "%" / ; Used for atoms > > "&" / "'" / > > "*" / "+" / > > "-" / "/" / > > "=" / "?" / > > "^" / "_" / > > "`" / "{" / > > "|" / "}" / > > "~" / > > \u{80}-\u{10ffff} ; Non-ascii, non-control UTF-8 > > > > name-char-start = atext / atext-specials > > > > name-char-rest = atext / atext-specials / WS > > > > name = name-char-start *name-char-rest > > > > comment-char = atext / comment-specials / WS > > > > comment-content = *comment-char > > > > comment = "(" *WS comment-content *WS ")" > > > > addr-spec = dot-atom-text "@" dot-atom-text > > dot-atom-text isn't defined here, so it isn't clear to me what it > includes. Does it permit UTF-8 in addresses according to the SMTPUTF8 > RFCs? Thanks for catching that. When turning my code into a grammar, I somehow forgot that production. The dot_atom_text is unchanged from e.g. RFC 2822: dot_atom_text = 1*atext *("." *atext) But since we've extended atext to include non-control UTF-8 characters, this should allow international email addresses. RFC 6531 (the SMTPUTF8 RFC) extends atext as follows: atext =/ UTF8-non-ascii ; extend the implicit definition of atext in ; RFC 5321, Section 4.1.2, which ultimately points to ; the actual definition in RFC 5322, Section 3.2.3 https://tools.ietf.org/html/rfc6531#section-3.3 which, I think, is what I did above. But, I've only skimmed RFC 6531 so I might have missed something else. Thanks! :) Neal From nobody Wed Nov 6 03:49:14 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B033120869 for ; Wed, 6 Nov 2019 03:49:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KF__9-MlLTPE for ; Wed, 6 Nov 2019 03:49:10 -0800 (PST) Received: from mail.dasr.de (mail.dasr.de [217.69.77.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BF49A120861 for ; Wed, 6 Nov 2019 03:49:10 -0800 (PST) Received: from p54abdd01.dip0.t-ipconnect.de ([84.171.221.1] helo=forster.huenfield.org) by mail.dasr.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1iSJoC-0006Cp-F2; Wed, 06 Nov 2019 11:49:08 +0000 Received: from grit.huenfield.org ([192.168.20.9] helo=grit.walfield.org) by forster.huenfield.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1iSJoC-00026i-2J; Wed, 06 Nov 2019 12:49:08 +0100 Date: Wed, 06 Nov 2019 12:49:08 +0100 Message-ID: <87r22ldy2z.wl-neal@walfield.org> From: "Neal H. Walfield" To: Daniel Kahn Gillmor Cc: openpgp@ietf.org In-Reply-To: <87v9rydk9s.wl-neal@walfield.org> References: <87woe7zx7o.fsf@fifthhorseman.net> <87v9rydk9s.wl-neal@walfield.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 192.168.20.9 X-SA-Exim-Mail-From: neal@walfield.org X-SA-Exim-Scanned: No (on forster.huenfield.org); SAEximRunCond expanded to false Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 11:49:13 -0000 On Tue, 05 Nov 2019 23:35:11 +0100, Neal H. Walfield wrote: > I'm a bit concerned about allowing the backslash character: with this > grammar, it is just a normal character, but for an RFC 2822 parser, > it's an escape character. Since User IDs may be used in contexts > where RFC 2822 things are expected, we should be careful. But, I fear > that if we reject it, we'll end up gratuitiously rejecting some > emojis. =C2=AF\_(=E3=83=84)_/=C2=AF. The double quote character has a similar problem. From nobody Wed Nov 6 09:52:44 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1977E12081B for ; Wed, 6 Nov 2019 09:52:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.457 X-Spam-Level: X-Spam-Status: No, score=-0.457 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DATE_IN_PAST_06_12=1.543, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=4o02UaLm; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=U612i2uT Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jPR6ytPPesGs for ; Wed, 6 Nov 2019 09:52:38 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [IPv6:2001:470:1:116::7]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC70712013F for ; Wed, 6 Nov 2019 09:52:38 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1573062756; h=from : to : cc : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=osfgQeX0XKQDpsMEs64BEkezZWIFyAjs/kNmfLM6v/0=; b=4o02UaLmDbABHgqwHi5zjhhd9Kg0kW/vHiW35IMLc/Rpz/TRBSHY6ZTO 668LB1kRVrtyxVpFyGDePRM5l0zCAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1573062756; h=from : to : cc : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=osfgQeX0XKQDpsMEs64BEkezZWIFyAjs/kNmfLM6v/0=; b=U612i2uTj9IktB8SiTuEuLgam7OsddzEQ6ljTi6kEfF+M6PWJnHC0vFo STfITFdEZfpqNYe2A8BJikHd/v5P1OjQlwNTGCry9Xi8LA/g0RGQ0RFjQ7 kaFvtSbICtpX6Kq2xBijb6uC0kL2Vxnt4EO3CzBQoFyRkrpRaNfGQ/k7/O bnjcoV/tJcjo1q2mFBmZjfMqOTitnNOhe+1oaL0NBOYpZL8fxbmW/LG7fw N/ClUE9BTebqeGkLZze6ChR0WoS5JziNs0atfwwqJQe62qUCQmHeKN03vF qtd+tg14r7/YD7mUsOVCEd+MOUptcvFU+QSP4gxssoVgrUIFbFnN0A== Received: from fifthhorseman.net (unknown [38.88.5.182]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 8F107F9A7; Wed, 6 Nov 2019 12:52:36 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id C97582038D; Wed, 6 Nov 2019 01:37:14 -0500 (EST) From: Daniel Kahn Gillmor To: "Neal H. Walfield" Cc: openpgp@ietf.org In-Reply-To: <87v9rydk9s.wl-neal@walfield.org> References: <87woe7zx7o.fsf@fifthhorseman.net> <87v9rydk9s.wl-neal@walfield.org> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Wed, 06 Nov 2019 01:37:14 -0500 Message-ID: <878soto6hx.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 17:52:41 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Neal-- Thanks for this thoughtful writeup! On Tue 2019-11-05 23:35:11 +0100, Neal H. Walfield wrote: > Beyond being more fleshed out, this grammar is different from the > grammar in dkg's second proposal in a few ways. > > First, it matches comments. dkg made this a non-goal. Given that > people who add comments intend them as comments and not as part of > their name, it seems reasonable to me to not display comments in > places where only the user's name is desired. And, since it turns out > that matching non-nested comments is relatively straightforward, why > not? Note: doing this might actually help deprecate comments, because > they won't be shown as often. User IDs are full UTF-8 strings. The idea that any part of that string would be hidden from the user is pretty disturbing to me. Consider the situation where someone is certifying a user ID on an OpenPGP certificate. If the comment is hidden, do they know what identity assertion they're making? I'd much rather have comments be deprecated *because they are weird and show up in places where you'd think your name should go* rather than have them be some vestigial thing that people don't even notice any longer. I would recommend dropping the comment from your grammar and letting the "name" part subsume it, when you're splitting out e-mail address from the rest of the user ID. Furthermore, because you've allowed "(" and ")" in atext-specials, it looks to me like your proposed grammar is ambiguous: bob (joe) is either: name: "bob (joe)" comment: None addr-spec: "bob@example.net" or: name: "bob" comment: "joe" addr-spec: "bob@example.net" I don't think this is helpful to anyone. > The grammar more carefully handles whitespace. It ignores whitespace > at the beginning of the User ID (this is what motivates the > name-char-start production) and between the individual components in > the pgp-uid-convention production. As is, the grammar only ignores > the 0x20 space character. We may also want to include the tab > character, unicode's NO-BREAK SPACE (U+00A0) character and its > IDEOGRAPHIC SPACE (U+3000) character for thoroughness. But, since > software will normally concatenate the individual components, just > recognizing the ASCII space character here is probably fine. Whatever > the case, I think we can safely ignore the rest of unicode's > whitespace characters: > > https://en.wikipedia.org/wiki/Whitespace_character I'm fine with being judicious about selecting whitespace characters. In addition to tab (U+0009, ascii "HT"), i note that you've declined to include U+000A and U+000D (ascii "LF" and "CR") in the grammar at all. I like that kind of opinionated decision, as unprintable symbols like this are likely to be problematic in many ways (hard for users to distinguish at least!) I also think that whitespace at the beginning of a user ID is asking for trouble, and would be happy with a grammar that considers that user ID non-conventional. Is there a use case for leading whitespace in a user ID? > My pgp-uid-convention production also matches user ids without email > addresses, e.g., "Daniel Kahn Gillmor". This is convenient. Instead > of having to figure out why parsing failed (is it not valid UTF-8? is > it just missing an addr-spec?), we explicitly cover this common > pattern in the grammar. I think this will significantly simplify code > that uses this interface: if there is an error, then the code can just > assume the User ID is trash and can be ignored. I should be clear that i intended my earlier proposal specifically to match OpenPGP User ID conventions *that have an e-mail address in them*. There are indeed other User ID conventions (like "Daniel Kahn Gillmor", or "ssh://foo.example") that aren't covered by this, and i thought i would be doing folks a favor by focusing on the e-mail address side of things specifically. My thought was that common interfaces would allow for matching against a User ID that has an e-mail address, and then they would have other matchers for other common conventions that they could try applying if this convention didn't match. This is probably an implementation detail, though. > In RFC 2822, "specials" are only allowed in a display name if they are > quoted. dkg removes this requirements. I think this is mostly > sensible, but it means that we can have User IDs like: > " " where the first > is the display name and the second is the addr-spec. > I think we should exclude angle brackets from the display name. In my > grammar, I have an "atext-specials" which is just RFC 2822 specials > without the angle brackets. I totally agree with this constraint. If you're doing away with comments (as i recommend above) then you would have to prohibit angle brackets in commas too, which seems fine to me. Even if you decide to go ahead with splitting out comments, I would go so far as to ban them in comments too. is there any plausible reason for including angle brackets in a comment? Simplify simplify :) > I'm a bit concerned about allowing the backslash character: with this > grammar, it is just a normal character, but for an RFC 2822 parser, > it's an escape character. Since User IDs may be used in contexts > where RFC 2822 things are expected, we should be careful. But, I fear > that if we reject it, we'll end up gratuitiously rejecting some > emojis. =C2=AF\_(=E3=83=84)_/=C2=AF. There are all kinds of things that will break if implementations casually stick OpenPGP user IDs into an e-mail header, not just backslashes. for example, commas are likely to cause a problem. consider trying to mail two people whose OpenPGP certificates have these User IDs: Lucy Hernandez, MD Chuck Wilson, Jr. A simple concatenation with commas yields the disastrous: To: Lucy Hernandez, MD , Chuck Wilson, Jr. and DQUOTE is just as bad if not worse :) So i have no problem with including backslash in the display name area. --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXcJqGgAKCRB2GBllKa5f +L51AP91tPQWrZ3uD+ZW0qcJmg3PGDiXHNXnd/hp3vAVdeAi9QEA4RlnvILc36KE n22H5IUTyWFP46sEnqBtlIGi3x3a9gE= =02CL -----END PGP SIGNATURE----- --=-=-=-- From nobody Wed Nov 6 15:33:04 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E3281201B7 for ; Wed, 6 Nov 2019 15:33:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (3072-bit key) header.d=crustytoothpaste.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DOcgZlPHg2j8 for ; Wed, 6 Nov 2019 15:32:58 -0800 (PST) Received: from injection.crustytoothpaste.net (injection.crustytoothpaste.net [192.241.140.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3EBCE1200A4 for ; Wed, 6 Nov 2019 15:32:58 -0800 (PST) Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E870A60424 for ; Wed, 6 Nov 2019 23:32:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1573083174; bh=zjwQ0XBjio91+VH0s6tntgQjlwX0fqYDK+hghVfKhm0=; h=Date:From:To:Subject:References:Content-Type:Content-Disposition: In-Reply-To:From:Reply-To:Subject:Date:To:CC:Resent-Date: Resent-From:Resent-To:Resent-Cc:In-Reply-To:References: Content-Type:Content-Disposition; b=f33K3VdHBfN3fDYTNMsNsTvjujLDX1UCJMNS3lcq3ZCDclugT4OgXN0Sos8A1vn/F aKAH71hBwD4meI0nuLfU4mm3mu8N4H9WsTtEZ3Ypwez5kAsiPUV+qB4Occ+kLnNEMa mWPT3w8hQ2ugjYMPbw8m2RWSozWqedz0nNEJA2jkNzt06qmM9h/dUSLrSiFPFpPmT6 NEQtmpNxpldqpk3vtHTIEn0DbuEX+IpK48/aw/dz/Ibqw5q3Q71obRkQfh0BdLqivV bCcXkH6D7ggd6vPjV4EO2hxf16v1Ammk2Qht5Jv5//AVCfDjukvXYcLexAgSIJMmtA 1fopHe504Q1KMg5RTCIOSNsQ4/cXvSdTxZTUk5i6FtrSZvIaZZBB3jgfxKnBLgwBks eDMBUEtFtB9c4HWSeM/mniDz9ZPndAgZ3G139J9G7BxIX/BBfoOcTdNOWB4MxifErq JgobNRA5L6+c5XWOg1mHTOqllfjHjgIxDJOP+UYUR0YrfG3L/Q8 Date: Wed, 6 Nov 2019 23:32:49 +0000 From: "brian m. carlson" To: openpgp@ietf.org Message-ID: <20191106233249.GG32531@camp.crustytoothpaste.net> References: <87woe7zx7o.fsf@fifthhorseman.net> <87v9rydk9s.wl-neal@walfield.org> <20191106000546.GE32531@camp.crustytoothpaste.net> <87tv7he9ql.wl-neal@walfield.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="aYDVKSzuImP48n7V" Content-Disposition: inline In-Reply-To: <87tv7he9ql.wl-neal@walfield.org> X-Machine: Running on camp using GNU/Linux on x86_64 (Linux kernel 5.3.0-1-amd64) User-Agent: Mutt/1.12.2 (2019-09-21) Archived-At: Subject: Re: [openpgp] User ID conventions (it's not really a RFC2822 name-addr) X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 23:33:00 -0000 --aYDVKSzuImP48n7V Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2019-11-06 at 07:37:22, Neal H. Walfield wrote: > Thanks for catching that. When turning my code into a grammar, I > somehow forgot that production. >=20 >=20 > The dot_atom_text is unchanged from e.g. RFC 2822: >=20 > dot_atom_text =3D 1*atext *("." *atext) >=20 > But since we've extended atext to include non-control UTF-8 > characters, this should allow international email addresses. >=20 > RFC 6531 (the SMTPUTF8 RFC) extends atext as follows: >=20 > atext =3D/ UTF8-non-ascii > ; extend the implicit definition of atext in > ; RFC 5321, Section 4.1.2, which ultimately points to > ; the actual definition in RFC 5322, Section 3.2.3 >=20 > https://tools.ietf.org/html/rfc6531#section-3.3 >=20 > which, I think, is what I did above. >=20 > But, I've only skimmed RFC 6531 so I might have missed something else. Yup, in that case, I have no objections to your grammar. It seems fine to me. --=20 brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204 --aYDVKSzuImP48n7V Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.2.17 (GNU/Linux) iQIzBAABCgAdFiEEX8OngXdrJt+H9ww3v1NdgR9S9osFAl3DWCEACgkQv1NdgR9S 9otphw//dprC0ky686KnhqSCRilzQzLGel9aLWrOOHrmv5oFPFH9MIzrD5tIY3bV INQ/K2yT0toI5WcGXsPn/m0g+nXBxI2U+CtWXECpUKEKRCBTlrHDsQTsKq6tfRlw X9vLwOorhDdgQybdR0xUleK8pvC/I2x+XHHPxdf3qJXC73r2fujtb9KmmeAC04+3 GFQXG5be9xARQM4DpuKsyPBoxTBujUW0+V/1cJT73JIpt0mVwDwkmJ/Sm26cFSXN /WPtOV6B0eyDAGmrLkNcdkEiOSK3XBA/d8qBFOo1Kj7uE8FMO6ffUylLldPcbNzi q6MFkRACXcsHbTtT43r+S/U6g45Kw3mBqMcPyWqascyMyEblSM2Pp/jZh89LoWo4 Tgax8zyivKY98VswbA5VdAvcz+f5JqVJjldxTgQy82DcwV2ANT0K2M/CwVqxBfmE Cf4RqL2yosIsvi8hK5ahgPv1qdEkEbxQcBshDR9XyLA7XvFk5NoLvAsSS04zve5l cxg24p5gZ2YcVNGM+mhKaYhKMidZuaEh+w1zvS72QQqnc9ISww925pmFl9iROmKF R+wnK45dm4gcxIHU0AbUAe/gqKouRmdpNRYiXKsj/1wrHssA30VLf7YKnAACOj83 eol6tCdS7VaRUWjA9OyCqNvlw1HdkmNWGxIH3gvV02tK9/YB2rg= =eyJP -----END PGP SIGNATURE----- --aYDVKSzuImP48n7V-- From nobody Sat Nov 9 12:56:40 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1BA2912008D for ; Sat, 9 Nov 2019 12:56:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.3 X-Spam-Level: X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=fc5eSZRa; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=ZLYMerC8 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2c4V64CHV5Wv for ; Sat, 9 Nov 2019 12:56:36 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 57BA3120048 for ; Sat, 9 Nov 2019 12:56:36 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1573332995; h=from : to : cc : subject : date : message-id : mime-version : content-type : from; bh=pac3p+37dQakGmsPTVO1rET5wBUZ/luR0Hn4lpqlcNs=; b=fc5eSZRaqMZJMsEa4ZWSMVllHax/k8dzKlDxgu9jzCLjfx8ldgYKPZ6k ag2cGtxAj2ki3xf4cyxzMGL4/a02Bg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1573332995; h=from : to : cc : subject : date : message-id : mime-version : content-type : from; bh=pac3p+37dQakGmsPTVO1rET5wBUZ/luR0Hn4lpqlcNs=; b=ZLYMerC8ZVi7crDFV3a+P8a4U0KeXqUOCu6NI4GIoCw0y5G3/LeTPeT9 4nD9TStK1wcUScv0uK74oT35GUNjYqDy6hBkSmMeak0atgqoD1D+Vkk31E zhlOMM6i0dDFspvCZAvczsCfl2aqD/0olIJDVyfcvI7LG1wZ91evJ1/J+3 7sDI73yQSL/lKIM7H2MKqAYZ28f0+bDr0ri2J0x0atk3eEsfpSUcV5AfO2 m/vKSJbEyVBf6hbXNcidkh8apyplqPYYlU0jbAHMKjJRF6ZkcbTTpi+/1n 2pkXDiuGYToiI2C5uozGQ+yoKMPjgXciki09933N64z50YpMy3vcBg== Received: from fifthhorseman.net (unknown [IPv6:2001:470:1f07:60d:54c4:58ff:fe31:1452]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 86974F9A6; Sat, 9 Nov 2019 15:56:34 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id 6881C203ED; Sat, 9 Nov 2019 15:56:25 -0500 (EST) From: Daniel Kahn Gillmor To: openpgp@ietf.org Cc: Werner Koch Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Sat, 09 Nov 2019 15:56:24 -0500 Message-ID: <87o8xklqfb.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: [openpgp] Web Key Directory (WKD) draft home on gitlab X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Nov 2019 20:56:39 -0000 --=-=-= Content-Type: text/plain Hi OpenPGP folks-- One of the key takeaways for me from the OpenPGP E-mail Summit that happened in Berlin last month [0] was that Werner Koch's Web Key Directory (WKD) [1] is clearly a desirable and useful standard for the community of developers who work on OpenPGP-enabled e-mail clients. [0] https://wiki.gnupg.org/OpenPGPEmailSummit201910 [1] https://datatracker.ietf.org/doc/draft-koch-openpgp-webkey-service/ WKD was discussed in a dedicated session [1], but it was also threaded throughout the discussions over the days of the summit. [2] https://wiki.gnupg.org/OpenPGPEmailSummit201910Notes#Workshop:_WKD It was apparent from the lively conversation that WKD is considered part of the critical infrastructure for OpenPGP e-mail clients these days, and that there are subtle nuances to it that people actively wanted to discuss. After talking with Werner about it, i've set up a location where we can keep track of outstanding issues with WKD on gitlab: [3] https://gitlab.com/openpgp-wg/webkey-directory I hope that folks who have ideas, suggestions, or questions about WKD will use the issue tracker there to help make sure their concerns are addressed in future revisions of the draft. I've opened three issues as a start, covering themes I heard voiced at the summit: * Focus on WKD retrieval by splitting out the WKD Update Protocol to a separate draft * Size-based metadata leakage: padding concerns * Nuances about fallback from "advanced" to "direct" URLs Hopefully others who have been using WKD will record their own concerns on the issue tracker as well as bringing them up on the list here. One helpful job would be for anyone who was at the WKD session at the summit to go over the notes in [2] and transfer any relevant concerns to issues in the gitlab issue tracker. A note about the git repository hosted at gitlab: While the WKD draft has been developed by Werner in his gnupg-doc repo at https://dev.gnupg.org/source/gnupg-doc.git, that repository contains lots of things that are not WKD-specific. I used "git filter-branch" on the gnupg-doc repo to pull out just the commits relating to the wkd draft, and what's on gitlab is the result of that extraction. I did this because i thought it would be simpler for other people to engage with the draft if they don't have to worry about the rest of gnupg-doc. But if Werner doesn't want to switch to that repository, I am fine with taking it down and pointing people to the gnupg-doc repo instead. Werner, please let me know if you want me to do that. Regards, --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXccn+QAKCRB2GBllKa5f +EKgAQDEKbMBPuVR+kVJNwMrwiw5v7WYavVX0lvMKA4Wxy+8HAD9E1BH2OIupuf3 TGAQdMMK6eTjn/WAnSzMdWdnq4p64QY= =L1Iq -----END PGP SIGNATURE----- --=-=-=-- From anarcat@torproject.org Mon Nov 11 09:58:06 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7CF83120B15 for ; Mon, 11 Nov 2019 09:58:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BcqCeiAUiC1F for ; Mon, 11 Nov 2019 09:58:01 -0800 (PST) Received: from marcos.anarc.at (marcos.anarc.at [206.248.172.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 385F9120AFB for ; Mon, 11 Nov 2019 09:57:53 -0800 (PST) Received: by marcos.anarc.at (Postfix, from userid 1000) id 6110D10E087; Mon, 11 Nov 2019 12:57:52 -0500 (EST) Received: by curie.anarc.at (Postfix, from userid 1000) id 842711208E2; Mon, 11 Nov 2019 12:57:51 -0500 (EST) From: =?utf-8?Q?Antoine_Beaupr=C3=A9?= To: openpgp@ietf.org Organization: Tor Date: Mon, 11 Nov 2019 12:57:51 -0500 Message-ID: <87mud28fds.fsf@curie.anarc.at> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: X-Mailman-Approved-At: Mon, 11 Nov 2019 10:57:01 -0800 Subject: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Nov 2019 18:00:09 -0000 Hi, I've taken some time to do a a short review of dkg's "SOP" interface specification, version 01, as provided here: https://tools.ietf.org/html/draft-dkg-openpgp-stateless-cli-01 First, I want to say this is excellent and much-needed work. OpenPGP interfaces have traditionnally been hard to use and a major obstacle to adoption of strong cryptography in our communities. Having a standard and *simple* way of interoperating with the underlying OpenPGP primitives would go a *long* way in significantly improving OpenPGP adoption and online privacy in general. So I salute this (first) effort in improving this situation. Second, I have proposed a series of changes to the document here: https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9 Patch set: https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9.patch Those are mostly small fixes that might not require a full discussion here, although I of course welcome feedback on those here as well. It might be preferable to comment on the issue directly if you have a GitLab account, obviously. Finally, there are some things about the document I wanted to comment on and that I don't have an easy patch for, so I will comment on the document, inline, here instead. I hope that is proper form, let me know if there is a better way to do this. > Introduction > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 [...] > This separation should make it easier to provide interoperability testing= for the object security work, and to allow implementations to consume and = produce new cryptographic primitives as needed. =20 I don't understand the part after ", and allow..." what does that actually mean? What do we mean by "new cryptographic primitives" here exactly? [...] > Obviously, the user will need to manage their secret keys (and their > peers' certificates) somehow, but the goal of this interface is to > separate out that task from the task of interacting with OpenPGP > messages. =20 It's unclear to me how or if the SOP specification takes into account the current design of GnuPG, specifically the part where secrets are handled by a separate process, gpg-agent, which is designed to be separate from the other parts of gnupg. From what I understand, the "agents" keep the secrets and do the operations on behalf of other parts of gnupg. But here sop would do so. Are we designing an agent here? What about OpenPGP cards like the Yubikey? How does sop interoperate with those? [...] > Examples > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > These examples show no error checking, but give a flavor of how `sop` mig= ht be used in practice from a shell. >=20 > The key and certificate files described in them (e.g. `alice.sec`) could = be for example those found in {{I-D.draft-bre-openpgp-samples-00}}. >=20 > ~~~ > sop generate-key "Alice Lovelace " > alice.sec > sop extract-cert < alice.sec > alice.pgp >=20 > sop sign --as=3Dtext alice.sec < announcement.txt > announcement.txt.asc > sop verify announcement.txt.asc alice.pgp < announcement.txt >=20 > sop encrypt --sign-with=3Dalice.sec --as=3Dmime bob.pgp < msg.eml > encry= pted.asc > sop decrypt alice.sec < ciphertext.asc > cleartext.out > ~~~ I find those examples confusing. Multiple arguments, in particular, seems ambiguous. Is it "CERT DATA"? or "CERT DATA"? There should be *mandatory* commandline *options* instead, that clearly state the purpose of (say) the "CERT" argument. It's a common in APIs, to rely on the order of arguments for meaning, and I think it's often a mistake. We should explicitely use *options* instead of *arguments* (as in `--foo=3Dbar` instead of just `bar`) for critical parameters like secret or verification keys. Then "arguments" are left for more regular file parameters, the primary purpose of the command (e.g. "sign, encrypt, decrypt this file", with "verify" being of course the tricky bit). So instead of: > sop sign --as=3Dtext alice.sec < announcement.txt > announcement.txt.asc I would suggest: > sop sign --as=3Dtext --sign-with=3Dalice.sec < announcement.txt >announce= ment.txt.asc For example. The idea would be to use `--PURPOSE-with` pattern (where PURPOSE is `sign`, `verify`, etc) that is already used in `encrypt --sign-with`. Maybe it could be shortened to just --with in some cases (like decrypt, sign and verify). The above example would become: sop generate-key "Alice Lovelace " > alice.sec sop extract-cert < alice.sec > alice.pgp sop sign --as=3Dtext --sign-with=3Dalice.sec < announcement.txt > announcem= ent.txt.asc sop verify announcement.txt.asc --verify-with=3Dalice.pgp < announcement.txt sop encrypt --sign-with=3Dalice.sec --as=3Dmime bob.pgp < msg.eml > encrypt= ed.asc sop decrypt --decrypt-with=3Dalice.sec < ciphertext.asc > cleartext.out Or, as a diff: @@ -99,11 +103,11 @@ The key and certificate files described in them (e.g. = `alice.sec`) could be for sop generate-key "Alice Lovelace " > alice.sec sop extract-cert < alice.sec > alice.pgp =20 -sop sign --as=3Dtext alice.sec < announcement.txt > announcement.txt.asc -sop verify announcement.txt.asc alice.pgp < announcement.txt +sop sign --as=3Dtext --sign-with=3Dalice.sec < announcement.txt > announce= ment.txt.asc +sop verify announcement.txt.asc --verify-with=3Dalice.pgp < announcement.t= xt =20 sop encrypt --sign-with=3Dalice.sec --as=3Dmime bob.pgp < msg.eml > encryp= ted.asc -sop decrypt alice.sec < ciphertext.asc > cleartext.out +sop decrypt --decrypt-with=3Dalice.sec < ciphertext.asc > cleartext.out ~~~ =20 Subcommands > Subcommands > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > `sop` uses a subcommand interface, similar to those popularized by system= s like `git` and `svn`. > > If the user supplies a subcommand that `sop` does not implement, it > fails with a return code of 69. If a `sop` implementation does not > handle a supplied option for a given subcommand, it fails with a > return code of 37. =20 In general, I feel using the numeric error codes in the document make it (needlessly?) harder to read. When i got to this section, my first reaction was: "69?? why 69? and why 37? where the heck do those come from and why do they matter?" We should at least include a reference to the "Failure modes section" in the Introduction section. In Terminology maybe? And maybe refer to it here. In general, I'm worried there might be inconsistencies between the table in the "Failure modes" section and the various hardcoded integers peppered through the document. This practice also makes the document more difficult to review and maintain in the future. We might instead use constant names like `SUCCESS`, `NO_GOOD_SIG` that *then* have integer values in the later section. This could also provide for good constants to use in a library implementation. > For all commands that have an `--armor|--no-armor` option, it defaults > to `--armor`, meaning that any output OpenPGP material should be > ASCII-armored (section 6 of {{I-D.ietf-openpgp-rfc4880bis}}) > by default. =20 Is this on input or output? or both? It's clarified later, I think, but it should be made explicit here as well. [...] > generate-key: Generate a Secret Key {#generate-key} > ----------------------------------- >=20 > sop generate-key [--armor|--no-armor] [--] [USERID=E2=80=A6] >=20 > - Standard Input: ignored > - Standard Output: `KEY` ({{key}}) >=20 > Generate a single default OpenPGP certificate with zero or more User > IDs. [...] How do we generate purpose-specific subkeys? [...] > sign: Create a Detached Signature {#sign} > --------------------------------- >=20 > sop sign [--armor|--no-armor] > [--as=3D{binary|text}] [--] KEY [KEY...] >=20 > - Standard Input: `DATA` ({{data}}) > - Standard Output: `SIGNATURE` ({{signature}}) >=20 > `--as` defaults to `binary`. If `--as=3Dtext` and the input `DATA` is > not valid `UTF-8`, `sop sign` fails with a return code of 53. =20 Why do we mandate UTF-8 here? Explain. In general, I find the `--as` arguments to be a little confusing and I don't undrestand what they bring to the table. =20 > Example: >=20 > $ sop sign --as=3Dtext alice.sec < message.txt >message.txt.asc > $ head -n1 < message.txt.asc > -----BEGIN PGP SIGNATURE----- > $ Another good example of the "argument vs option" problem. If I would see a `sop sign` command, the first thing I would try would be: sop sign document and expect it to find the right private key to sign the document with. Of course, we don't do this in sop, which is fine, but I'll note that we allow implementations to do so. By forcing the arguments here to be the signing key, we make it difficult to let the implementation pick the right key. We should follow POLA (Principle Of Least Astonishment) here and allow users to provide the document as an argument, and use something like `--sign-with=3DKEY` (possibly multiple times) to provide the private key material. [...] > encrypt: Encrypt a Message {#encrypt} > -------------------------- >=20 > sop encrypt [--as=3D{binary|text|mime}] > [--armor|--no-armor] > [--with-password=3DPASSWORD...] > [--sign-with=3DKEY...] > [--] [CERTS...] [...] > `--as` defaults to `binary`. [...] > If `--as` is set to either `text` or `mime`, then `--sign-with` > will sign as a canonical text document. In this case, if the input > `DATA` is not valid `UTF-8`, `sop encrypt` fails with a return code of > 53. =20 What is `mime` here? Why is it necessary? Expand. [...] > Example: >=20 > (In this example, `bob.bin` is a file containing Bob's binary-formatted O= penPGP certificate. > Alice is encrypting a message to both herself and Bob.) >=20 > $ sop encrypt --as=3Dmime --sign-with=3Dalice.key alice.asc bob.bin <= message.eml >encrypted.asc > $ head -n1 encrypted.asc > -----BEGIN PGP MESSAGE----- > $ This is another good example of how the arguments and options can become confusing. Looking at the above commandline, I can't tell what alice.asc or bob.bin are. Is bob.bin a private key? Maybe not, because it's a `.bin`. But what if it was also named `.asc`? What if I reverse the two arguments by mistake? I could encrypt to bob instead of alice! Or wait, those are *both* keys I encrypt to! Phew, I'm safe... But wait, what if I forgot the `<`!! See where I'm going? :) Having an explicit --encrypt-with=3Dalice.asc (or --encrypt-to?) would be much better here. It would also make much more sense in an API. =20 > decrypt: Decrypt a Message {#decrypt} > -------------------------- >=20 > sop decrypt [--session-key-out=3DSESSIONKEY] > [--with-session-key=3DSESSIONKEY...] > [--with-password=3DPASSWORD...] > [--verify-out=3DVERIFICATIONS > [--verify-with=3DCERTS...] > [--verify-not-before=3DDATE] > [--verify-not-after=3DDATE] ] > [--] [KEY...] >=20 > - Standard Input: `CIPHERTEXT` ({{ciphertext}}) > - Standard Output: `DATA` ({{data}}) >=20 > `--session-key-out` can be used to learn the session key on > successful decryption. "learn"? What does that mean? It seems it means "write to a file".=20 If so that should be said explicitely here. =20 > If `sop decrypt` fails for any reason and the identified `--session-key-o= ut` > file already exists in the filesystem, the file will be unlinked. =20 This seems dangerous! Why do we delete a file we haven't created? Explain. > `--session-key-in` enables decryption of the `CIPHERTEXT` using the sessi= on key directly against the `SEIPD` packet. > This option can be used multiple times if several possible session keys s= hould be tried. What happens if both "in" and "out" are provided? I can venture a guess, but it would be important to make that explicit as there can be horrible bugs there. =20 > `--with-password` enables decryption based on any `SKESK` packets in the = `CIPHERTEXT`. > This option can be used multiple times if the user wants to try more than= one password. =20 We should include SKESK in terminology, because it's the first time we encounter it here and I have close to no idea what it means. =20 > If `sop decrypt` tries and fails to use a supplied `PASSWORD`, and it > observes that there is trailing `UTF-8` whitespace at the end of the > `PASSWORD`, it will retry with the trailing whitespace stripped. =20 Explain why we do magic things with whitespace. Consider not doing magic at all as magic can be evil. =20 > `--verify-out` produces signature verification status to the > designated file. >=20 > `sop decrypt` does not fail (that is, the return code is not modified) > based on the results of signature verification. The caller MUST check > the returned `VERIFICATIONS` to confirm signature status. An empty > `VERIFICATIONS` output indicates that no valid signatures were found. > If `sop decrypt` itself fails for any reason, and the identified > `VERIFICATIONS` file already exists in the filesystem, the file will > be unlinked. >=20 > `--verify-with` identifies a set of certificates whose signatures would be > acceptable for signatures over this message. Not failing explicitely on verification seems very dangerous. It relies on callers properly reading the spec and realizing this is the only exception where exit codes don't suffice in providing a general state of the program. I would strongly recommend failing here, just like regular verify. As an aside, why can't we compose verify and decrypt here and just keep "verify" out of "decrypt" altogether? I would guess that's (a limitation?) part of the OpenPGP standard, but maybe it would be nice to explicitely expand on this here as well. > If the caller is interested in signature verification, both > `--verify-out` and at least one `--verify-with` must be supplied. If > only one of these arguments is supplied, `sop decrypt` fails with a > return code of 23. =20 Another argument for failing on bad signatures: if we fail on bad arguments of --verify, why don't we fail on bad signatures? > armor: armor: Add ASCII Armor > ----------------------------- [...] > If the incoming data is already armored, and the `--allow-nested` flag > is not specified, the data MUST be output with no modifications. > Data is considered ASCII armored iff the first 14 bytes are exactly > `-----BEGIN PGP`. This operation is thus idempotent by default. =20 Explain why we want idempotent and why we want to do this guessing game. [...] > Input/Output Indirect Types > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >=20 > Some material is passed to `sop` indirectly, typically by referring to > a filename containing the data in question. This type of data may > also be passed to `sop` on Standard Input, or delivered by `sop` to > Standard Output. >=20 > If the filename for any indirect material used as input has the > special form `@ENV:xxx`, then contents of environment variable `$xxx` > is used instead of looking in the filesystem. >=20 > If the filename for any indirect material used as either input or > output has the special form `@FD:nnn` where `nnn` is a decimal > integer, then the associated data is read from file descriptor `nnn`. This @ENV: and @FD: stuff really makes me uncomfortable. It's a neat hack for commandline applications, but it would break down when designing a library API, as the "type" of data passed around the API would be ambiguous, or at least with possible side effects. That feels like "design smell" here and I would like this to be changed. I would recommend using equivalent environment variables to the parameters instead, for example SIGN_WITH for --sign-with and so on. This would, of course, require switching positional arguments to options but I already explain why that would be a good idea anyways earlier. File descriptors could be passable as distinct options, like --sign-with-fd for --sign-with. I've dealt with commandline applications that have special meanings with @, and in retrospect, it was a bad idea. In particular, Python's argparse module supports using a prefix argument to mean "read options from this file" and I've used it to implement crude configuration file support for monkeysign and other programs. It's confusing for users and does not work very well. Specifically in this case, I would also worry about security vulnerabilities with untrusted filenames being passed to the program. [...] > CERTS {#certs} > ----- >=20 > One or more OpenPGP certificates (section 11.1 of {{I-D.ietf-openpgp-rfc4= 880bis}}), aka "Transferable Public Key". > May be armored. >=20 > Although some existing workflows may prefer to use one `CERTS` object wit= h multiple certificates in it (a "keyring"), supplying exactly one certific= ate per `CERTS` input will make error reporting clearer and easier. This last bit is in contradiction with `extract-cert` command documentation which says it will "only contain one cert". Maybe we should just pick one and stick with it here? =20 [...] > VERIFICATIONS {#verifications} > ------------- >=20 > One line per successful signature verification. Each line has three > structured fields delimited by a single space, followed by arbitrary > text to the end of the line. >=20 > - ISO-8601 UTC datestamp > - Fingerprint of the signing key (may be a subkey) > - Fingerprint of primary key of signing certificate (if signed by primar= y key, same as the previous field) > - arbitrary text =20 That last part doesn't *look* like "arbitrary text" to me. It looks like some explanatory message of the operation. If that's the case, we should make that explicit and say why the text is present at all. Calling it a "note" or "message" would already be an improvement. > Example: >=20 > 2019-10-24T23:48:29Z C90E6D36200A1B922A1509E77618196529AE5FF8 C4BC2DD= B38CCE96485EBE9C2F20691179038E5C6 certificate from dkg.asc For the record, arbitrary text looks like: > This is some garbage I just jklfa bldasjkl ajklblablabla. I can provide more samples or arbitrary text as required. I'm an experience freelance writer and large samples can be found on https://anarc.at/ ;) [...] > Failure Modes > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > When `sop` succeeds, it will return 0 and emit nothing to Standard > Error. When `sop` fails, it fails with a non-zero return code, and > emits one or more warning messages on Standard Error. Known return > codes include: >=20 > Return | Meaning > ---:|-------------------------------------------------- > 0 | Success > 3 | No acceptable signatures found (`sop verify`) > 13 | Asymmetric algorithm unsupported (`sop encrypt`) > 17 | Certificate not encryption-capable (e.g., expired, revoked, unaccept= able usage flags) (`sop encrypt`) > 19 | Missing required argument > 23 | Incomplete verification instructions (`sop decrypt`) > 29 | Unable to decrypt (`sop decrypt`) > 31 | Non-`UTF-8` password (`sop encrypt`) > 37 | Unsupported option > 41 | Invalid data type (no secret key where `KEY` expected, etc) > 53 | Non-text input where text expected > 69 | Unsupported subcommand I already mentioned some problems with those failure codes, but I will repeat here the suggestion that symbolic names instead of integers should be used for primary referencing in the document here again. > A `sop` implementation MAY return other error codes than those listed > above. =20 This sounds like a bad idea. I interpret that as meaning that I can return an error code 2 instead of error code 3 if i fancy. If we're going to pick numbers, we should either enforce them or not, but don't dance around the issue and encourage people to diverge from the spec. Or at least, if you allow divergence, explain why it can be allowed. =20 It would also be great if we could explain where those magic numbers come from in the first place. I suspect they were chosen to not overlap with existing error codes, but that's just a guess. [...] > Detached Signatures {#detached-signatures} > ------------------- >=20 > `sop` deals with detached signatures as the baseline form of OpenPGP sign= atures. >=20 > The main problem this avoids is the trickiness of handling a signature th= at is mixed inline into the data that it is signing. Should we expand on "trickiness" here? =20 Also: how *do* we deal with inline signatures? Are those deprecated now? [...] > Guidance for Consumers > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > While `sop` is originally conceived of as an interface for interoperabili= ty testing, it's conceivable that an application that uses OpenPGP for obje= ct security would want to use it. >=20 > FIXME: more guidance for how to use such a tool safely and efficiently go= es here. >=20 > FIXME: if an encrypted OpenPGP message arrives without metadata, it is di= fficult to know which signers to consider when decrypting. > How do we do this efficiently without invoking `sop decrypt` twice, once = without `--verify-*` and again with the expected identity material? =20 Maybe we could use a "sop probe" command for this and other things? > Security Considerations > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > The OpenPGP object security model is typically used for confidentiality a= nd authenticity purposes. >=20 > Signature Verification {#signature-verification} > ---------------------- >=20 > In many contexts, an OpenPGP signature is verified to prove the origin an= d integrity of an underlying object. >=20 > When `sop` checks a signature (e.g. via `sop verify` or `sop decrypt --ve= rify-with`), it MUST NOT consider it to be verified unless all of these con= ditions are met: >=20 > * The signature must be made by a signing-capable public key that is pre= sent in one of the supplied certificates > * The certificate and signing subkey must have been created before or at= the signature time > * The certificate and signing subkey must not have been expired at the s= ignature time > * The certificate and signing subkey must not be revoked with a "hard" r= evocation > * If the certificate or signing subkey is revoked with a "soft" revocati= on, then the signature time must predate the revocation > * The signing subkey must be properly bound to the primary key, and cros= s-signed > * The signature (and any dependent signature, such as the cross-sig or s= ubkey binding signatures) must be made with strong cryptographic algorithms= (e.g., not `MD5` or a 1024-bit `RSA` key) =20 Latter seems to contradict section 7.5, which says: > For performance reasons, an implementation may choose to ignore > validation on certificate and key material supplied to it. The > security implications are of doing so depend on how the certs and > keys are managed outside of "sop". So should we check supplied keys or not? but I guess that's covered by... [...] > Signature validity is a complex topic, and this documentation cannot list= all possible details. Is there a reference we could add here to cover that topic? > Compression {#compression} > ----------- >=20 > The interface as currently specified does not allow for control of compre= ssion. > Compressing and encrypting data that may contain both attacker-supplied m= aterial and sensitive material could leak information about the sensitive m= aterial (see the CRIME attack). >=20 > Unless an application knows for sure that no attacker-supplied material i= s present in the input, it should not compress during encryption. How about decryption? Do we attempt decompression during decrypt? That's about it! My comments might be a little dry and are the results of notes I jotted down on "paper" (a PDF really), a copy of which is available here: https://paste.anarc.at/publish/2019-11-10-sxrDKhJL9R0/sop-Exported.pdf I hope the comments are still useful and please don't interpret them as me being upset or too critical. I love this work and it's only in a spirit of collaboration that I bring up those concerns. :) Thank you for your work! A. PS: I mistakenly took the modified version of the document, as available on the HEAD of the merge request, to comment on the document. I have tried, as much as possible, to revert changes to the original in the examples, but some other changes I suggested might have crept up in the quoted text. That shouldn't affect the comments I have brought up in any other way. --=20 Antoine Beaupr=C3=A9 torproject.org system administration From nobody Mon Nov 11 15:00:30 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F16BE12082B for ; Mon, 11 Nov 2019 15:00:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.3 X-Spam-Level: X-Spam-Status: No, score=-4.3 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=dcqfYdTs; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=VK9uGmsz Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hmkk2PS8gpnu for ; Mon, 11 Nov 2019 15:00:23 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EABC1120806 for ; Mon, 11 Nov 2019 15:00:22 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1573513221; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=HgkeXvrx9RYgtfL/H4YcmtpZAnzM4CXf0hk9AKhTzLw=; b=dcqfYdTscRrH+Rh9eyQgdsV78YgOQNAGXpjk0AAco/2aQLd3PXXF4A2M 3z+8HwUWRPfuKIKJj6c6OaXRJlU8Ag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1573513221; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=HgkeXvrx9RYgtfL/H4YcmtpZAnzM4CXf0hk9AKhTzLw=; b=VK9uGmsz49QKM+fySjW97y5NTk7qsdhWkyEvhNwGGXKgNymzsVjB1xyL Ig57lxZJql/V6n800xZQQhyE4X4jstcYhyt/64iL1wLwCRm5sMSbR6YMLE 9eZYt4VbDdh0BSG0TzuvbEBGPzabsuazkwSZ2qrA8K8urKWo/qkTAgxskB w4ikpW4/NCi99R71aurTZwQoGd3yL1sCjESRZQf6AVWLdbhZtwNJOvLaaZ Bv6uqgqg+tbpPcLwM7F6rn9rakq6kJnRmknZeyfv2XUfepInw4w/JxoZK+ hjoUlly4PILhBfDxFpoYqrVBs+7C6a32sNVWJX4KEqY8Z9rYGsfOAA== Received: from fifthhorseman.net (unknown [IPv6:2001:470:1f07:60d:4494:50ff:fe83:821a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 3A335F9A6; Mon, 11 Nov 2019 18:00:20 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id C9DAF20482; Mon, 11 Nov 2019 18:00:17 -0500 (EST) From: Daniel Kahn Gillmor To: Antoine =?utf-8?Q?Beaupr=C3=A9?= , openpgp@ietf.org In-Reply-To: <87mud28fds.fsf@curie.anarc.at> References: <87mud28fds.fsf@curie.anarc.at> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Mon, 11 Nov 2019 18:00:17 -0500 Message-ID: <87h83arpby.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Nov 2019 23:00:28 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Antoine-- Thanks for the thoughtful review. It was super long though! I've opened a bunch of issues from the stuff you raised here, but more dicsussion follows inline below. On Mon 2019-11-11 12:57:51 -0500, Antoine Beaupr=C3=A9 wrote: > https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9 I've reviewed and merged the changes requested there, definitely useful to have this clarifying editorial work done as patches, thanks! >> This separation should make it easier to provide interoperability testin= g for the object security work, and to allow implementations to consume and= produce new cryptographic primitives as needed. >=20=20 > I don't understand the part after ", and allow..." what does that > actually mean? What do we mean by "new cryptographic primitives" here > exactly? The intent is that if every OpenPGP implementation provides a `sop` interface (or potentially a superset of it), then we can write interoperabiity tests and the like that drive all of them in a reliable way. We can, for example, generate new OpenPGP objects that incorporate new primitives, and feed them to a stable of `sop` implementations, to determine whether those implementations can consume them. Or, we can drive them with simple inputs, and see which cryptographic primitives they choose to use as they produce output. If you think this text is clearer, i can incorporate it directly in the draft. > [...] > >> Obviously, the user will need to manage their secret keys (and their >> peers' certificates) somehow, but the goal of this interface is to >> separate out that task from the task of interacting with OpenPGP >> messages. >=20=20 > It's unclear to me how or if the SOP specification takes into account > the current design of GnuPG, specifically the part where secrets are > handled by a separate process, gpg-agent, which is designed to be > separate from the other parts of gnupg. From what I understand, the > "agents" keep the secrets and do the operations on behalf of other parts > of gnupg. But here sop would do so. Are we designing an agent here? > > What about OpenPGP cards like the Yubikey? How does sop interoperate > with those? sop is not GnuPG, and it doesn't claim or intend to be. An implementation of sop *may* choose to support other forms of secret key access (the "@FOO:" namespace is carved out to allow for that), but the goal of sop is that it is simple enough that every implementation that can handle OpenPGP secret keys and certificates should be capable of implementing the interface. > I find those examples confusing. Multiple arguments, in particular, > seems ambiguous. Is it "CERT DATA"? or "CERT DATA"? ??? i think those are the same thing, but i'll just assume you meant "DATA CERTS" at the end. The answer is that there must be exactly one SIGNATURE object to verify and there may be multiple certs, so the only possible way to do it is SIGNATURE first, then CERTS. But that doesn't address your larger point: > There should be *mandatory* commandline *options* instead, that clearly > state the purpose of (say) the "CERT" argument. It's a common in APIs, > to rely on the order of arguments for meaning, and I think it's often a > mistake. We should explicitely use *options* instead of *arguments* (as > in `--foo=3Dbar` instead of just `bar`) for critical parameters like > secret or verification keys. I would welcome a MR that makes this change across the entire spec, to see whether it looks reasonable. If people on this list prefer that structure, i would be fine with adopting it, though i think it makes the CLI more verbose than i would like. I've trimmed out a lot of your comments below that were to do with whether positional arguments are sensible or not. that's not because i don't care, but i don't think arguing about it in this one message is helpful. i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/7 to record the overall concern. if you decide to make an MR for it, please reference that issue in the MR! > In general, I feel using the numeric error codes in the document make it > (needlessly?) harder to read. When i got to this section, my first > reaction was: "69?? why 69? and why 37? where the heck do those come > from and why do they matter?" We should at least include a reference to > the "Failure modes section" in the Introduction section. In Terminology > maybe? And maybe refer to it here. > > In general, I'm worried there might be inconsistencies between the table > in the "Failure modes" section and the various hardcoded integers > peppered through the document. This practice also makes the document > more difficult to review and maintain in the future. We might instead > use constant names like `SUCCESS`, `NO_GOOD_SIG` that *then* have > integer values in the later section. This could also provide for good > constants to use in a library implementation. I really like this idea, and i encourage someone=E2=84=A2 who also likes it= to make a MR that implements it. i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/1 to keep track of it. >> For all commands that have an `--armor|--no-armor` option, it defaults >> to `--armor`, meaning that any output OpenPGP material should be >> ASCII-armored (section 6 of {{I-D.ietf-openpgp-rfc4880bis}}) >> by default. >=20=20 > Is this on input or output? or both? It's clarified later, I > think, but it should be made explicit here as well. the text you've quoted says "any output OpenPGP material". I'm not sure how to make that more visible, but i welcome proposed edits. > How do we generate purpose-specific subkeys? With `sop`, you do not ;) If you want to do fancy OpenPGP certificate generation, you do that with your toolkit's own fancy features. I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/2 to track that maybe we do want some rough guidance about what kinds of secret key capabilities we want any `sop` to be able to generate here though. >> sign: Create a Detached Signature {#sign} > [=E2=80=A6] >> If `--as=3Dtext` and the input `DATA` is >> not valid `UTF-8`, `sop sign` fails with a return code of 53. > > Why do we mandate UTF-8 here? Explain. We don't mandate UTF-8 unless the signer claims that the thing being signed is text. If so, it really does need to be UTF-8. I have no patience for non-UTF-8-encoded text in 2019. OpenPGP embeds UTF-8 explicitly in its User ID formatting. Any OpenPGP implementation must already handle UTF-8. if anyone thinks that dealing with different character encodings is a good idea, please consider that the character encoding is not recorded in the signature itself, leading charset-switching attacks like those in https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/ Do you think this information belongs in this document? > In general, I find the `--as` arguments to be a little confusing and I > don't undrestand what they bring to the table. OpenPGP has two different forms of cryptographic signature, which are dealt with by recipients differently. The textual form canonicalizes line-endings and the binary does not. When signing a document, you need to indicate to the thing doing the signing which form you are expecting to create. If you have a different suggestion, i'm happy to hear it. >> Example: >>=20 >> $ sop sign --as=3Dtext alice.sec < message.txt >message.txt.asc >> $ head -n1 < message.txt.asc >> -----BEGIN PGP SIGNATURE----- >> $ > > Another good example of the "argument vs option" problem. If I would see > a `sop sign` command, the first thing I would try would be: > > sop sign document > > and expect it to find the right private key to sign the document > with. Of course, we don't do this in sop, which is fine, but I'll note > that we allow implementations to do so. We deliberately *do not* allow implementations to do so. sop requires that you indicate which secret keys you want to sign with. it is one or more KEY objects, not zero or more. If you think that `sop` is supposed to sign with other objects, then i've done a bad job at drafting this proposal. I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/5 to try to clarify thi= s. > By forcing the arguments here to be the signing key, we make it > difficult to let the implementation pick the right key. The entire point of `sop` is that `sop` *does not know* anything about the keys. it is stateless, and you have to tell it explicitly which key to use. >> If `--as` is set to either `text` or `mime`, then `--sign-with` >> will sign as a canonical text document. In this case, if the input >> `DATA` is not valid `UTF-8`, `sop encrypt` fails with a return code of >> 53. >=20=20 > What is `mime` here? Why is it necessary? Expand. i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/6 to track this. >> `--session-key-out` can be used to learn the session key on >> successful decryption. > > "learn"? What does that mean? It seems it means "write to a file".=20 > If so that should be said explicitely here. *What* the user of sop does is that they learn of the session key that was used for the encrypted message. *How* they learn it is by receiving it from the program, either via the filesystem, or via @FD:NNN. If you think that's unclear, i'd welcome a clarifying patch. >> If `sop decrypt` fails for any reason and the identified `--session-key-= out` >> file already exists in the filesystem, the file will be unlinked. >=20=20 > This seems dangerous! Why do we delete a file we haven't created? > Explain. We don't want the user to run `sop`, and then inspect a file that was already in the filesystem thinking that it is `sop`s output. If you think that's a bad decision, please suggest what we should do differently. >> [`--with-session-key`] enables decryption of the `CIPHERTEXT` using the = session key directly against the `SEIPD` packet. >> This option can be used multiple times if several possible session keys = should be tried. > > What happens if both "in" and "out" are provided? I can venture a guess, > but it would be important to make that explicit as there can be horrible > bugs there. Please do venture a guess, in the form of proposed text! I'd also love to hear what the horrible bugs are. I don't see them. >> `--with-password` enables decryption based on any `SKESK` packets in the= `CIPHERTEXT`. >> This option can be used multiple times if the user wants to try more tha= n one password. >=20=20 > We should include SKESK in terminology, because it's the first time we > encounter it here and I have close to no idea what it means. https://gitlab.com/dkg/openpgp-stateless-cli/issues/12 I'm not sure we should put it in "terminology", but we surely should document it better the first time it appears (it is documented the second time it appears in the text. >> If `sop decrypt` tries and fails to use a supplied `PASSWORD`, and it >> observes that there is trailing `UTF-8` whitespace at the end of the >> `PASSWORD`, it will retry with the trailing whitespace stripped. >=20=20 > Explain why we do magic things with whitespace. Consider not doing magic > at all as magic can be evil. I expect the following use case to be common: echo correct horse battery staple > password.txt sop decrypt --with-password=3Dpassword.txt < ciphertext > cleartext If we don't strip whitespace, it will fail on one side or the other. sop tries to define what sensible magic should be here, and i think i've gotten it right. as for magic being evil: if we don't try to do magic, then we will be failing in ways that users don't understand, and implementers will get bug reports that they respond to with "have you tried trimming trailing whitespace and then trying again?" that kind of round trip is pointless when the tool could just avoid the problem in the first place by being reasonable about what kinds of things people tend to do. https://gitlab.com/dkg/openpgp-stateless-cli/issues/13 >> `--verify-out` produces signature verification status to the >> designated file. >>=20 >> `sop decrypt` does not fail (that is, the return code is not modified) >> based on the results of signature verification. The caller MUST check >> the returned `VERIFICATIONS` to confirm signature status. An empty >> `VERIFICATIONS` output indicates that no valid signatures were found. >> If `sop decrypt` itself fails for any reason, and the identified >> `VERIFICATIONS` file already exists in the filesystem, the file will >> be unlinked. >>=20 >> `--verify-with` identifies a set of certificates whose signatures would = be >> acceptable for signatures over this message. > > Not failing explicitely on verification seems very dangerous. It relies > on callers properly reading the spec and realizing this is the only > exception where exit codes don't suffice in providing a general state of > the program. I would strongly recommend failing here, just like regular > verify. if the person invokes `sop decrypt` with no arguments, should it fail? if the person invokes `sop decrypt` with `--verify-out` and `--verify-with`, should `sop decrypt` produce any output? I'm trying to imagine using this in a MUA. In my MUA i have what i think are the keys for my peer, and i see a message from them. I don't yet know whether it's signed. I want to decrypt the message to read it, and in the process, i want to find out whether it has been signed. I'd prefer to avoid two passes in this common use case. If i supply the --verify-* arguments, and sop fails, then i don't get the cleartext of the message (not in any reliable way, anyhow). If i don't supply the --verify-* arguments, and sop succeeds, i've lost any signature data. The subcommand is "decrypt", so `sop` treats successful decryption as a success. If you want to understand some additional state, you have to inspect that state somewhere else, and that's in the contents of `--verify-out`. does that make sense? do you think any of that explanation belongs in the document itself? > As an aside, why can't we compose verify and decrypt here and just keep > "verify" out of "decrypt" altogether? I would guess that's (a > limitation?) part of the OpenPGP standard, but maybe it would be nice to > explicitely expand on this here as well. Do you want to propose a way to compose them? I suppose we could have `sop decrypt` offer a `--signatures-to` argument, which the sender could then use for `sop verify`, if anything comes out. That's kind of an interesting suggestion, and it might reduce the overall complexity of `sop`. However: * It means that a caller would need to handle the data twice (a minor inefficiency) * the signatures might not be verifiable if the thing that is signed is not a simple literal data packet, but is, say, a compressed data packet wrapped around a literal data packet. So i don't see how to do this safely (or efficiently, but the verifiability of the signature is more important than the efficiency argument) >> If the caller is interested in signature verification, both >> `--verify-out` and at least one `--verify-with` must be supplied. If >> only one of these arguments is supplied, `sop decrypt` fails with a >> return code of 23. >=20=20 > Another argument for failing on bad signatures: if we fail on bad > arguments of --verify, why don't we fail on bad signatures? failing on bad arguments is "you've asked an ill-formed question". That's legitimate and useful to let the operator know that things are not what they seem. Failing on a bad signature would be "the answer is no". If the primary operation you're asking for is verification, then failing on bad signatures is a reasonable outcome -- only succeed if you succeed in verifying. But if the primary operation is decryption, i don't think we should fail on signature validity for reasons outlined above. >> armor: armor: Add ASCII Armor >> ----------------------------- > > [...] > >> If the incoming data is already armored, and the `--allow-nested` flag >> is not specified, the data MUST be output with no modifications. >> Data is considered ASCII armored iff the first 14 bytes are exactly >> `-----BEGIN PGP`. This operation is thus idempotent by default. >=20=20 > Explain why we want idempotent and why we want to do this guessing game. I'm at a bit of a loss here, because these seem obvious to me. We want a guessing game because users who don't know what they want are going to guess anyway, and they're likely to guess wrong. We might as well guess on their behalf if they don't know. We want idempotent because "ensure this thing is armored" is a reasonable semantic request -- indeed, it's probably the *only* reasonable semantic request. Perhaps we could drop --allow-nested? > This @ENV: and @FD: stuff really makes me uncomfortable. It's a neat > hack for commandline applications, but it would break down when > designing a library API, as the "type" of data passed around the API > would be ambiguous, or at least with possible side effects. That feels > like "design smell" here and I would like this to be changed. FWIW, the @ENV: and @FD: conventions are specifically CLI conventions and i don't expect them to be translated to any programmatic API. for example, in the pythonic variant of this framework that i'm working on, i've treated these objects as labeled bytestreams. ("labeled" meaning -- if you get a set of these objects, each one has a bytes-like thing, and a textual name that you can use in error reporting) I admit that i'm struggling a bit with whether i want to pass around bytes objects directly (simple, what i've got now) or some sort of asyncio handle that the bytes are readable from (fancier, probably nicer to use in the kinds of programs that i will eventually want to write with this). https://gitlab.com/dkg/python-sop/issues/1 But that has nothing to do with @ENV and @FD, as those aren't exposed to the python functions at all. > I would recommend using equivalent environment variables to the > parameters instead, for example SIGN_WITH for --sign-with and so > on. This would, of course, require switching positional arguments to > options but I already explain why that would be a good idea anyways > earlier. environment variables won't work for arguments that can be supplied multiple times, because then we have to invent a new delimiting scheme. I definitely don't want to do that. > File descriptors could be passable as distinct options, like > --sign-with-fd for --sign-with. This is an interesting proposal, though i don't see how --sign-with=3D@FD:3 is much different from --sign-with-fd=3D3 -- i guess it lets you use files that are literally named @FD:3 ? Is that important? If you could open a merge request that proposes this change i'd be happy to consider it. I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/14 to keep track of it. > I've dealt with commandline applications that have special meanings with > @, and in retrospect, it was a bad idea. In particular, Python's > argparse module supports using a prefix argument to mean "read options > from this file" and I've used it to implement crude configuration file > support for monkeysign and other programs. It's confusing for users and > does not work very well. afaict, this is not mandatory in argparse, and i wouldn't recommend it for any sop implementation: https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars I think the way it's specified in `sop` right now is pretty pincipled and hard to screw up, but i could be wrong. > Specifically in this case, I would also worry about security > vulnerabilities with untrusted filenames being passed to the program. can you explain this more? >> CERTS {#certs} >> ----- >>=20 >> One or more OpenPGP certificates (section 11.1 of {{I-D.ietf-openpgp-rfc= 4880bis}}), aka "Transferable Public Key". >> May be armored. >>=20 >> Although some existing workflows may prefer to use one `CERTS` object wi= th multiple certificates in it (a "keyring"), supplying exactly one certifi= cate per `CERTS` input will make error reporting clearer and easier. > > This last bit is in contradiction with `extract-cert` command > documentation which says it will "only contain one cert". Maybe we > should just pick one and stick with it here? I have carefully considered this, and i do not think these are in contradiction. extract-cert explicitly says it will contain only one cert *because* the other places where `CERTS` might be supplied could contain more than one. The fact is that people use "keyrings" today, including some that contain hundreds or thousands of keys. If a distro, for example, wants to use `sop` to verify that a package is signed by one of their developers, i don't want the distro to need to put each developer's key in a separate file. > That last part doesn't *look* like "arbitrary text" to me. It looks like > some explanatory message of the operation. If that's the case, we should > make that explicit and say why the text is present at all. Calling it a > "note" or "message" would already be an improvement. patches welcome, particularly for this kind of editorial cleanup :) >> A `sop` implementation MAY return other error codes than those listed >> above. >=20=20 > This sounds like a bad idea. I interpret that as meaning that I can > return an error code 2 instead of error code 3 if i fancy. If we're > going to pick numbers, we should either enforce them or not, but don't > dance around the issue and encourage people to diverge from the spec. > > Or at least, if you allow divergence, explain why it can be allowed. I've documented this concern as https://gitlab.com/dkg/openpgp-stateless-cli/issues/10 > It would also be great if we could explain where those magic numbers > come from in the first place. I suspect they were chosen to not overlap > with existing error codes, but that's just a guess. Justus picked 69 in his OpenPGP Interoperability Test Suite. I chose the others as "reasonable-sized primes" just for fun. I don't think this information belongs in this document, as it doesn't matter. >> Detached Signatures {#detached-signatures} >> ------------------- >>=20 >> `sop` deals with detached signatures as the baseline form of OpenPGP sig= natures. >>=20 >> The main problem this avoids is the trickiness of handling a signature t= hat is mixed inline into the data that it is signing. > > Should we expand on "trickiness" here? >=20=20 > Also: how *do* we deal with inline signatures? Are those deprecated now? From=20my POV, yes, inline signatures are recipes for disaster, because the difficulty of figuring out what specifically was signed when you have an inline signature is not at all obvious. I've noted that this needs to be reflected in the spec at: https://gitlab.com/dkg/openpgp-stateless-cli/issues/9 That said, i've heard from some of the folks handling package managers who have specific reasons for wanting inline signatures (also, Neal Walfield appears to think they're useful in his https://gitlab.com/sequoia-pgp/pgpcat). so we probably need to tackle them somehow here. (it's even in "future work"!) I've recorded that concern here: https://gitlab.com/dkg/openpgp-stateless-cli/issues/11 >> FIXME: if an encrypted OpenPGP message arrives without metadata, it is d= ifficult to know which signers to consider when decrypting. >> How do we do this efficiently without invoking `sop decrypt` twice, once= without `--verify-*` and again with the expected identity material? >=20=20 > Maybe we could use a "sop probe" command for this and other things? I don't understand what you think a "sop probe" command would do. if you'd like to propose it as an MR, i'd consider it, though. >> Compression {#compression} [=E2=80=A6] > How about decryption? Do we attempt decompression during decrypt? It will be interesting to see what implementers do! I've left `sop` deliberately agnostic there, and i would like to learn from test suites what the answer is. --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXcnoAQAKCRB2GBllKa5f +A+tAQDmyrTrGMbona8GsjNATNBHmVA3gpKXqsdLOYZ1F8ix+QD9FAlMYggsob78 mnpnT1hUOC6myYXS3+SoSM6X2mSpRg4= =uNiB -----END PGP SIGNATURE----- --=-=-=-- From nobody Tue Nov 12 09:26:09 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40BDD12082B for ; Tue, 12 Nov 2019 09:26:07 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WzESDLkpXGlp for ; Tue, 12 Nov 2019 09:26:03 -0800 (PST) Received: from marcos.anarc.at (marcos.anarc.at [206.248.172.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2DEE012021C for ; Tue, 12 Nov 2019 09:26:02 -0800 (PST) Received: by marcos.anarc.at (Postfix, from userid 1000) id 201FB10E081; Tue, 12 Nov 2019 12:26:01 -0500 (EST) Received: by curie.anarc.at (Postfix, from userid 1000) id 3364D120E74; Tue, 12 Nov 2019 12:26:00 -0500 (EST) From: =?utf-8?Q?Antoine_Beaupr=C3=A9?= To: Daniel Kahn Gillmor , openpgp@ietf.org In-Reply-To: <87h83arpby.fsf@fifthhorseman.net> Organization: Tor References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> Date: Tue, 12 Nov 2019 12:26:00 -0500 Message-ID: <87r22dm2fr.fsf@curie.anarc.at> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Nov 2019 17:26:07 -0000 On 2019-11-11 18:00:17, Daniel Kahn Gillmor wrote: > Hi Antoine-- > > Thanks for the thoughtful review. It was super long though! I've > opened a bunch of issues from the stuff you raised here, but more > dicsussion follows inline below. No problem! It's a long draft in the first place (24 pages PDF), and talks about something I actually have (had) lots of ideas about in the past, so I guess that's normal... > On Mon 2019-11-11 12:57:51 -0500, Antoine Beaupr=C3=A9 wrote: >> https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/9 > > I've reviewed and merged the changes requested there, definitely useful > to have this clarifying editorial work done as patches, thanks! Great! >>> This separation should make it easier to provide interoperability testi= ng for the object security work, and to allow implementations to consume an= d produce new cryptographic primitives as needed. >>=20=20 >> I don't understand the part after ", and allow..." what does that >> actually mean? What do we mean by "new cryptographic primitives" here >> exactly? > > The intent is that if every OpenPGP implementation provides a `sop` > interface (or potentially a superset of it), then we can write > interoperabiity tests and the like that drive all of them in a reliable > way. > > We can, for example, generate new OpenPGP objects that incorporate new > primitives, and feed them to a stable of `sop` implementations, to > determine whether those implementations can consume them. > > Or, we can drive them with simple inputs, and see which cryptographic > primitives they choose to use as they produce output. > > If you think this text is clearer, i can incorporate it directly in the > draft. That would be great, thanks! Providing examples like this would go a long way... >> [...] >> >>> Obviously, the user will need to manage their secret keys (and their >>> peers' certificates) somehow, but the goal of this interface is to >>> separate out that task from the task of interacting with OpenPGP >>> messages. >>=20=20 >> It's unclear to me how or if the SOP specification takes into account >> the current design of GnuPG, specifically the part where secrets are >> handled by a separate process, gpg-agent, which is designed to be >> separate from the other parts of gnupg. From what I understand, the >> "agents" keep the secrets and do the operations on behalf of other parts >> of gnupg. But here sop would do so. Are we designing an agent here? >> >> What about OpenPGP cards like the Yubikey? How does sop interoperate >> with those? > > sop is not GnuPG, and it doesn't claim or intend to be. thank you for that too. ;) > An implementation of sop *may* choose to support other forms of secret > key access (the "@FOO:" namespace is carved out to allow for that), but > the goal of sop is that it is simple enough that every implementation > that can handle OpenPGP secret keys and certificates should be capable > of implementing the interface. I guess what I'm wondering is how I would make this work with my yubikey at all. Or maybe I got this backwards and the yubikey interface is what should implement sop directly? >> I find those examples confusing. Multiple arguments, in particular, >> seems ambiguous. Is it "CERT DATA"? or "CERT DATA"? > > ??? i think those are the same thing, but i'll just assume you meant > "DATA CERTS" at the end. The answer is that there must be exactly one > SIGNATURE object to verify and there may be multiple certs, so the only > possible way to do it is SIGNATURE first, then CERTS. Ah, yes, sorry about this. I was specifically refering to: sop verify announcement.txt.asc alice.pgp < announcement.txt And `"CERT DATA" or "DATA CERT"`? And I guess where we differ is I am not sure it's that clear that the first argument of a series can be different from the rest... > But that doesn't address your larger point: > >> There should be *mandatory* commandline *options* instead, that clearly >> state the purpose of (say) the "CERT" argument. It's a common in APIs, >> to rely on the order of arguments for meaning, and I think it's often a >> mistake. We should explicitely use *options* instead of *arguments* (as >> in `--foo=3Dbar` instead of just `bar`) for critical parameters like >> secret or verification keys. > > I would welcome a MR that makes this change across the entire spec, to > see whether it looks reasonable. If people on this list prefer that > structure, i would be fine with adopting it, though i think it makes > the CLI more verbose than i would like. > > I've trimmed out a lot of your comments below that were to do with > whether positional arguments are sensible or not. that's not because i > don't care, but i don't think arguing about it in this one message is > helpful. > > i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/7 to > record the overall concern. if you decide to make an MR for it, please > reference that issue in the MR! I have a patch for this and will submit it. It was more invasive than the others so I figured I would bring it up as a discussion first. >> In general, I feel using the numeric error codes in the document make it >> (needlessly?) harder to read. When i got to this section, my first >> reaction was: "69?? why 69? and why 37? where the heck do those come >> from and why do they matter?" We should at least include a reference to >> the "Failure modes section" in the Introduction section. In Terminology >> maybe? And maybe refer to it here. >> >> In general, I'm worried there might be inconsistencies between the table >> in the "Failure modes" section and the various hardcoded integers >> peppered through the document. This practice also makes the document >> more difficult to review and maintain in the future. We might instead >> use constant names like `SUCCESS`, `NO_GOOD_SIG` that *then* have >> integer values in the later section. This could also provide for good >> constants to use in a library implementation. > > I really like this idea, and i encourage someone=E2=84=A2 who also likes = it to > make a MR that implements it. > > i've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/1 to > keep track of it. I can try to tackle this as well. >>> For all commands that have an `--armor|--no-armor` option, it defaults >>> to `--armor`, meaning that any output OpenPGP material should be >>> ASCII-armored (section 6 of {{I-D.ietf-openpgp-rfc4880bis}}) >>> by default. >>=20=20 >> Is this on input or output? or both? It's clarified later, I >> think, but it should be made explicit here as well. > > the text you've quoted says "any output OpenPGP material". I'm not sure > how to make that more visible, but i welcome proposed edits. I made this MR for this: https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/10 >> How do we generate purpose-specific subkeys? > > With `sop`, you do not ;) Sad. > If you want to do fancy OpenPGP certificate generation, you do that with > your toolkit's own fancy features. > > I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/2 to > track that maybe we do want some rough guidance about what kinds of > secret key capabilities we want any `sop` to be able to generate here > though. Commented on that. Would still love to see a more decent way to handle subkeys because that's a really hard thing to do in existing implementations. At least creating split subkeys by default would be a great start, IMHO. >>> sign: Create a Detached Signature {#sign} >> [=E2=80=A6] >>> If `--as=3Dtext` and the input `DATA` is >>> not valid `UTF-8`, `sop sign` fails with a return code of 53. >> >> Why do we mandate UTF-8 here? Explain. > > We don't mandate UTF-8 unless the signer claims that the thing being > signed is text. If so, it really does need to be UTF-8. I have no > patience for non-UTF-8-encoded text in 2019. > > OpenPGP embeds UTF-8 explicitly in its User ID formatting. Any OpenPGP > implementation must already handle UTF-8. > > if anyone thinks that dealing with different character encodings is a > good idea, please consider that the character encoding is not recorded > in the signature itself, leading charset-switching attacks like those in > https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/ > > Do you think this information belongs in this document? Absolutely, otherwise it looks like an arbitrary decision. I am not, for the record, arguing for other encodings. UTF-8 is fine and diverging is a recipe for disaster, indeed. >> In general, I find the `--as` arguments to be a little confusing and I >> don't undrestand what they bring to the table. > > OpenPGP has two different forms of cryptographic signature, which are > dealt with by recipients differently. The textual form canonicalizes > line-endings and the binary does not. When signing a document, you need > to indicate to the thing doing the signing which form you are expecting > to create. The intricacies of OpenPGP never cease to amaze me. > If you have a different suggestion, i'm happy to hear it. I wish we didn't have to deal with this distinction, but if so, maybe we should clarify the source of it here. Otherwise it comes as a surprise to me, an experience OpenPGP user. >>> Example: >>>=20 >>> $ sop sign --as=3Dtext alice.sec < message.txt >message.txt.asc >>> $ head -n1 < message.txt.asc >>> -----BEGIN PGP SIGNATURE----- >>> $ >> >> Another good example of the "argument vs option" problem. If I would see >> a `sop sign` command, the first thing I would try would be: >> >> sop sign document >> >> and expect it to find the right private key to sign the document >> with. Of course, we don't do this in sop, which is fine, but I'll note >> that we allow implementations to do so. > > We deliberately *do not* allow implementations to do so. sop requires > that you indicate which secret keys you want to sign with. it is one or > more KEY objects, not zero or more. > > If you think that `sop` is supposed to sign with other objects, then > i've done a bad job at drafting this proposal. I've opened > https://gitlab.com/dkg/openpgp-stateless-cli/issues/5 to try to clarify t= his. I understand where you're coming from here, and it's good to make sure a stateless implementation. What I'm saying is the `sop sign` example is error prone. Forget the `<` and the mandated order and you might reverse the signing key and the message. >> By forcing the arguments here to be the signing key, we make it >> difficult to let the implementation pick the right key. > > The entire point of `sop` is that `sop` *does not know* anything about > the keys. it is stateless, and you have to tell it explicitly which key > to use. I guess I'm wondering if implementations would be allow to diverge on that matter at all. [...] >>> `--session-key-out` can be used to learn the session key on >>> successful decryption. >> >> "learn"? What does that mean? It seems it means "write to a file".=20 >> If so that should be said explicitely here. > > *What* the user of sop does is that they learn of the session key that > was used for the encrypted message. *How* they learn it is by receiving > it from the program, either via the filesystem, or via @FD:NNN. > > If you think that's unclear, i'd welcome a clarifying patch. https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/11 >>> If `sop decrypt` fails for any reason and the identified `--session-key= -out` >>> file already exists in the filesystem, the file will be unlinked. >>=20=20 >> This seems dangerous! Why do we delete a file we haven't created? >> Explain. > > We don't want the user to run `sop`, and then inspect a file that was > already in the filesystem thinking that it is `sop`s output. If you > think that's a bad decision, please suggest what we should do > differently. Maybe we should not overwrite existing files at all and fail earlier? >>> [`--with-session-key`] enables decryption of the `CIPHERTEXT` using the= session key directly against the `SEIPD` packet. >>> This option can be used multiple times if several possible session keys= should be tried. >> >> What happens if both "in" and "out" are provided? I can venture a guess, >> but it would be important to make that explicit as there can be horrible >> bugs there. > > Please do venture a guess, in the form of proposed text! I'd also love > to hear what the horrible bugs are. I don't see them. I would argue that both options should not be provided at once. One implementation that could come up would be that the program attempts to read the file as it's writing it, truncating the precious key before it has time to read it. [...] >>> If `sop decrypt` tries and fails to use a supplied `PASSWORD`, and it >>> observes that there is trailing `UTF-8` whitespace at the end of the >>> `PASSWORD`, it will retry with the trailing whitespace stripped. >>=20=20 >> Explain why we do magic things with whitespace. Consider not doing magic >> at all as magic can be evil. > > I expect the following use case to be common: > > echo correct horse battery staple > password.txt > sop decrypt --with-password=3Dpassword.txt < ciphertext > cleartext > > If we don't strip whitespace, it will fail on one side or the other. > > sop tries to define what sensible magic should be here, and i think i've > gotten it right. We can continue the discussion in issue #13, but the TL;DR: is that I agree that stripping trailing control characters is a good idea, but disagree about whitespace in general. >>> `--verify-out` produces signature verification status to the >>> designated file. >>>=20 >>> `sop decrypt` does not fail (that is, the return code is not modified) >>> based on the results of signature verification. The caller MUST check >>> the returned `VERIFICATIONS` to confirm signature status. An empty >>> `VERIFICATIONS` output indicates that no valid signatures were found. >>> If `sop decrypt` itself fails for any reason, and the identified >>> `VERIFICATIONS` file already exists in the filesystem, the file will >>> be unlinked. >>>=20 >>> `--verify-with` identifies a set of certificates whose signatures would= be >>> acceptable for signatures over this message. >> >> Not failing explicitely on verification seems very dangerous. It relies >> on callers properly reading the spec and realizing this is the only >> exception where exit codes don't suffice in providing a general state of >> the program. I would strongly recommend failing here, just like regular >> verify. > > if the person invokes `sop decrypt` with no arguments, should it fail? no. > if the person invokes `sop decrypt` with `--verify-out` and > `--verify-with`, should `sop decrypt` produce any output? yes, except if verification fails. > I'm trying to imagine using this in a MUA. In my MUA i have what i > think are the keys for my peer, and i see a message from them. > > I don't yet know whether it's signed. > > I want to decrypt the message to read it, and in the process, i want to > find out whether it has been signed. I'd prefer to avoid two passes in > this common use case. > > If i supply the --verify-* arguments, and sop fails, then i don't get > the cleartext of the message (not in any reliable way, anyhow). If i > don't supply the --verify-* arguments, and sop succeeds, i've lost any > signature data. I don't know how OpenPGP packets are built. Can't we show the signature on the output of decrypt? > The subcommand is "decrypt", so `sop` treats successful decryption as a > success. If you want to understand some additional state, you have to > inspect that state somewhere else, and that's in the contents of > `--verify-out`. > > does that make sense? do you think any of that explanation belongs in > the document itself? Maybe there are some obvious assumptions about the OpenPGP message structure that make it clear this option is necessary. It's not to me. >> As an aside, why can't we compose verify and decrypt here and just keep >> "verify" out of "decrypt" altogether? I would guess that's (a >> limitation?) part of the OpenPGP standard, but maybe it would be nice to >> explicitely expand on this here as well. > > Do you want to propose a way to compose them? I suppose we could have > `sop decrypt` offer a `--signatures-to` argument, which the sender could > then use for `sop verify`, if anything comes out. That's kind of an > interesting suggestion, and it might reduce the overall complexity of > `sop`. > > However: > > * It means that a caller would need to handle the data twice (a minor > inefficiency) > > * the signatures might not be verifiable if the thing that is signed is > not a simple literal data packet, but is, say, a compressed data > packet wrapped around a literal data packet. > > So i don't see how to do this safely (or efficiently, but the > verifiability of the signature is more important than the efficiency > argument) You very likely know better. ;) It was just an idea, and probably impractical in reality. >>> If the caller is interested in signature verification, both >>> `--verify-out` and at least one `--verify-with` must be supplied. If >>> only one of these arguments is supplied, `sop decrypt` fails with a >>> return code of 23. >>=20=20 >> Another argument for failing on bad signatures: if we fail on bad >> arguments of --verify, why don't we fail on bad signatures? > > failing on bad arguments is "you've asked an ill-formed question". > That's legitimate and useful to let the operator know that things are > not what they seem. > > Failing on a bad signature would be "the answer is no". > > If the primary operation you're asking for is verification, then failing > on bad signatures is a reasonable outcome -- only succeed if you succeed > in verifying. > > But if the primary operation is decryption, i don't think we should fail > on signature validity for reasons outlined above. But that assumes decryption is the primary operation. In the context where all my email traffic is encrypted with OpenPGP, for example, decryption is not the primary operation anymore. I *do* want to fail properly on signature validity, it becomes a primary operation when encryption is "default"... >>> armor: armor: Add ASCII Armor >>> ----------------------------- >> >> [...] >> >>> If the incoming data is already armored, and the `--allow-nested` flag >>> is not specified, the data MUST be output with no modifications. >>> Data is considered ASCII armored iff the first 14 bytes are exactly >>> `-----BEGIN PGP`. This operation is thus idempotent by default. >>=20=20 >> Explain why we want idempotent and why we want to do this guessing game. > > I'm at a bit of a loss here, because these seem obvious to me. > > We want a guessing game because users who don't know what they want are > going to guess anyway, and they're likely to guess wrong. We might as > well guess on their behalf if they don't know. > > We want idempotent because "ensure this thing is armored" is a > reasonable semantic request -- indeed, it's probably the *only* > reasonable semantic request. Perhaps we could drop --allow-nested? I am now also confused here and ready to drop this argument. :) >> This @ENV: and @FD: stuff really makes me uncomfortable. It's a neat >> hack for commandline applications, but it would break down when >> designing a library API, as the "type" of data passed around the API >> would be ambiguous, or at least with possible side effects. That feels >> like "design smell" here and I would like this to be changed. > > FWIW, the @ENV: and @FD: conventions are specifically CLI conventions > and i don't expect them to be translated to any programmatic API. Of course. > for example, in the pythonic variant of this framework that i'm working > on, i've treated these objects as labeled bytestreams. ("labeled" > meaning -- if you get a set of these objects, each one has a bytes-like > thing, and a textual name that you can use in error reporting) > > I admit that i'm struggling a bit with whether i want to pass around > bytes objects directly (simple, what i've got now) or some sort of > asyncio handle that the bytes are readable from (fancier, probably nicer > to use in the kinds of programs that i will eventually want to write > with this). https://gitlab.com/dkg/python-sop/issues/1 But that has > nothing to do with @ENV and @FD, as those aren't exposed to the python > functions at all. I'm not sure asyncio is useful here. Callers can wrap around the byte stream as they see fit, no? (I really dislike this "parallel universe" thing they built in Python 3 for asyncio, it makes every API more complicated than it needs to be. But that's another story...) >> I would recommend using equivalent environment variables to the >> parameters instead, for example SIGN_WITH for --sign-with and so >> on. This would, of course, require switching positional arguments to >> options but I already explain why that would be a good idea anyways >> earlier. > > environment variables won't work for arguments that can be supplied > multiple times, because then we have to invent a new delimiting scheme. > I definitely don't want to do that. True. That is a significant limitation. I am not sure how to work around that problem. >> File descriptors could be passable as distinct options, like >> --sign-with-fd for --sign-with. > > This is an interesting proposal, though i don't see how --sign-with=3D@FD= :3 > is much different from --sign-with-fd=3D3 -- i guess it lets you use > files that are literally named @FD:3 ? Is that important? It's less magic, more explicit, and correlates better with other commandline APIs I have encountered. > If you could open a merge request that proposes this change i'd be happy > to consider it. > > I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/14 to > keep track of it. I'll see what I can do. >> I've dealt with commandline applications that have special meanings with >> @, and in retrospect, it was a bad idea. In particular, Python's >> argparse module supports using a prefix argument to mean "read options >> from this file" and I've used it to implement crude configuration file >> support for monkeysign and other programs. It's confusing for users and >> does not work very well. > > afaict, this is not mandatory in argparse, and i wouldn't recommend it > for any sop implementation: > https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars > > I think the way it's specified in `sop` right now is pretty pincipled > and hard to screw up, but i could be wrong. > >> Specifically in this case, I would also worry about security >> vulnerabilities with untrusted filenames being passed to the program. > > can you explain this more? Say you think you are in a trusted directory with "CERTS" that you want to encrypt to. You call: sop encrypt * < /tmp/file > /tmp/file.pgp Except you made a mistake and the attacker has control of the current directory, and injects a file named (say) @ENV:SOMETHING. Assuming they have control over the SOMETHING environment, they can now add an encryption key to the message. Control of the environment is kind of a stretch, I must admit, but in certain environments (most notably web servers), a *lot* of stuff can end up there and it shouldn't be completely trusted this way. >>> CERTS {#certs} >>> ----- >>>=20 >>> One or more OpenPGP certificates (section 11.1 of {{I-D.ietf-openpgp-rf= c4880bis}}), aka "Transferable Public Key". >>> May be armored. >>>=20 >>> Although some existing workflows may prefer to use one `CERTS` object w= ith multiple certificates in it (a "keyring"), supplying exactly one certif= icate per `CERTS` input will make error reporting clearer and easier. >> >> This last bit is in contradiction with `extract-cert` command >> documentation which says it will "only contain one cert". Maybe we >> should just pick one and stick with it here? > > I have carefully considered this, and i do not think these are in > contradiction. > > extract-cert explicitly says it will contain only one cert *because* the > other places where `CERTS` might be supplied could contain more than > one. > > The fact is that people use "keyrings" today, including some that > contain hundreds or thousands of keys. If a distro, for example, wants > to use `sop` to verify that a package is signed by one of their > developers, i don't want the distro to need to put each developer's key > in a separate file. Understood. >> That last part doesn't *look* like "arbitrary text" to me. It looks like >> some explanatory message of the operation. If that's the case, we should >> make that explicit and say why the text is present at all. Calling it a >> "note" or "message" would already be an improvement. > > patches welcome, particularly for this kind of editorial cleanup :) https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/12 [...] >> It would also be great if we could explain where those magic numbers >> come from in the first place. I suspect they were chosen to not overlap >> with existing error codes, but that's just a guess. > > Justus picked 69 in his OpenPGP Interoperability Test Suite. I chose > the others as "reasonable-sized primes" just for fun. I don't think > this information belongs in this document, as it doesn't matter. I love this kind of information in text, it makes it less dull. :p [...] >>> FIXME: if an encrypted OpenPGP message arrives without metadata, it is = difficult to know which signers to consider when decrypting. >>> How do we do this efficiently without invoking `sop decrypt` twice, onc= e without `--verify-*` and again with the expected identity material? >>=20=20 >> Maybe we could use a "sop probe" command for this and other things? > > I don't understand what you think a "sop probe" command would do. if > you'd like to propose it as an MR, i'd consider it, though. `sop probe` would do the minimal amount of work required to determine which keys ("signers") to consider when decrypting, then call `decrypt` properly. `sop probe` could also do the general task of parsing OpenPGP messages into packets and stuff like that. >>> Compression {#compression} > [=E2=80=A6] >> How about decryption? Do we attempt decompression during decrypt? > > It will be interesting to see what implementers do! I've left `sop` > deliberately agnostic there, and i would like to learn from test suites > what the answer is. Should we make that decision clearer in the document? Thanks again! --=20 Antoine Beaupr=C3=A9 torproject.org system administration From nobody Wed Nov 13 16:18:22 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3351812004E for ; Wed, 13 Nov 2019 16:18:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=F8PO8Ijy; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=Z0xH3Eku Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RgL4_pGNd966 for ; Wed, 13 Nov 2019 16:18:17 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [IPv6:2001:470:1:116::7]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C2DF120045 for ; Wed, 13 Nov 2019 16:18:17 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1573690695; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=PAo7fZh0thdMuNwcbCQbaAMilcpOsi/DlEIu8qIgiTw=; b=F8PO8IjyvjeA5qHjkgYgE1X/pNYPXnRPLml2buuKeYgrPlJIdueJ30d7 FgccUbJQLIE/BNkDHHRkHyCcL7V5AQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1573690694; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=PAo7fZh0thdMuNwcbCQbaAMilcpOsi/DlEIu8qIgiTw=; b=Z0xH3EkuAsdVOcRD4ZwO6PCjo8SPqBwwXoThKb4DFn/e9ZoU9wRUpDuj oTyJugY4+wHVACxrj9vM7aynjWChqLAvkxtfRkOSumriOXp0bLEy8yMxHY pxeRHd3cORTiwFxL4qXmuUquYNem8ZTJOgf2B1tvHgMG9ZE16Yk2FWjzSs ZbB+wjkwSdF/L4fSjXnirP0G5QcR4OIYUHTcIWMKj7LfMx9TQ0W5kFUp76 Low1nygAHYlePeKE00yaXOPqFYVN+p5XF2+NkOgiULnLxJ+iuh7ek/T7yU VNmmVM9RyloZOB3/8zhNRfbh5Q7mfhX2YQZkLiponuqLFYORnIuF5Q== Received: from fifthhorseman.net (unknown [185.97.93.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 9BCD0F9A5; Wed, 13 Nov 2019 19:18:13 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id EC9F9203CC; Wed, 13 Nov 2019 19:18:07 -0500 (EST) From: Daniel Kahn Gillmor To: Antoine =?utf-8?Q?Beaupr=C3=A9?= , openpgp@ietf.org In-Reply-To: <87r22dm2fr.fsf@curie.anarc.at> References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> <87r22dm2fr.fsf@curie.anarc.at> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Thu, 14 Nov 2019 03:18:07 +0300 Message-ID: <87tv77nwe8.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Nov 2019 00:18:20 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I've incorporated some of your suggestions from this e-mail directly in the draft now. Thanks a lot for them! more discussion below=E2=80=A6 On Tue 2019-11-12 12:26:00 -0500, Antoine Beaupr=C3=A9 wrote: > I guess what I'm wondering is how I would make this work with my yubikey > at all. Or maybe I got this backwards and the yubikey interface is what > should implement sop directly? I have no idea about how to make it work with hardware tokens. I remain frankly unconvinced about the seurity tradeoffs for most uses of hardware tokens (something i guess i need to actually write down more formally), and i'm unlikely to spend a lot of time developing how to integrate them with `sop`. I would only assume that some `sop` implementation would choose to carve out `@MYHSM:xxx` as an input space for `KEY`-style inputs, where `xxx` is some form of addressing scheme that indicates which secret key on which device should be used to interact with the device. But sorting out the addressing scheme alone is problematic enough don't plan to incorporate any of that detail in this draft. A well-formed, thorough yet compact merge request might convince me otherwise, but color me skeptical at the moment. >>> I find those examples confusing. Multiple arguments, in particular, >>> seems ambiguous. Is it "CERT DATA"? or "CERT DATA"? >> >> ??? i think those are the same thing, but i'll just assume you meant >> "DATA CERTS" at the end. The answer is that there must be exactly one >> SIGNATURE object to verify and there may be multiple certs, so the only >> possible way to do it is SIGNATURE first, then CERTS. > > Ah, yes, sorry about this. I was specifically refering to: > > sop verify announcement.txt.asc alice.pgp < announcement.txt > > And `"CERT DATA" or "DATA CERT"`? > > And I guess where we differ is I am not sure it's that clear that the > first argument of a series can be different from the rest... well, the middle arguments of a series are *definitely* hard to distinguish, so the only plausible distinctions when you've got one-vs-many positional arguments is whether the "one" goes first or last. But let's follow up on that over at https://gitlab.com/dkg/openpgp-stateless-cli/issues/7 and in particular, on your ongoing merge request at: https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/13 If anyone else has strong feelings about this choice, please take a look over there and follow up, either here on list, or on those tickets. >>> How do we generate purpose-specific subkeys? >> >> With `sop`, you do not ;) > > Sad. I be >> If you want to do fancy OpenPGP certificate generation, you do that with >> your toolkit's own fancy features. >> >> I've opened https://gitlab.com/dkg/openpgp-stateless-cli/issues/2 to >> track that maybe we do want some rough guidance about what kinds of >> secret key capabilities we want any `sop` to be able to generate here >> though. > > Commented on that. Would still love to see a more decent way to handle > subkeys because that's a really hard thing to do in existing > implementations. > > At least creating split subkeys by default would be a great start, IMHO. This is exactly the sort of decision that i want to see implementers make, so we can document their choices. `sop` is not about fancy key management, nor should it be. As i wrote in https://gitlab.com/dkg/openpgp-stateless-cli/issues/2, i do not want `sop` to place any detailed constraints here, i just want the generated key to be functional for use with `sop`. >> We don't mandate UTF-8 unless the signer claims that the thing being >> signed is text. If so, it really does need to be UTF-8. I have no >> patience for non-UTF-8-encoded text in 2019. >> >> OpenPGP embeds UTF-8 explicitly in its User ID formatting. Any OpenPGP >> implementation must already handle UTF-8. >> >> if anyone thinks that dealing with different character encodings is a >> good idea, please consider that the character encoding is not recorded >> in the signature itself, leading charset-switching attacks like those in >> https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/ >> >> Do you think this information belongs in this document? > > Absolutely, otherwise it looks like an arbitrary decision. I've just added the following subsection with "Guidance for Implementers": Text is always UTF-8 {#utf8} -------------------- Various places in this specification require UTF-8 {{RFC3629}} when enc= oding text. `sop` implementations SHOULD NOT consider textual data in any o= ther character encoding. OpenPGP Implementations MUST already handle UTF-8, because various part= s of {{RFC4880}} require it, including: - User ID - Notation name - Reason for revocation - ASCII-armor Comment: header Dealing with messages in other charsets leads to weird security failure= s like {{Charset-Switching}}, especially when the charset indication is not= covered by any sort of cryptographic integrity check. Restricting textual data to `UTF-8` universally across the OpenPGP ecos= ystem eliminates any such risk without losing functionality, since `UTF-8` = can encode all known characters. If any thinks that's either wrong or insufficient, please send corrections/improvements! > I wish we didn't have to deal with this distinction, but if so, maybe we > should clarify the source of it here. Otherwise it comes as a surprise > to me, an experience OpenPGP user. As a user, you shouldn't ever need to see it. As an implementer, you do need to think about it. I've added the following text to the discussion of `sop sign`: `--as=3Dbinary` SHOULD result in an OpenPGP signature of type 0x00 ("Si= gnature of a binary document"). `--as=3Dtext` SHOULD result in an OpenPGP signature of type 0x01 ("Sign= ature of a canonical text document"). See section 5.2.1 of {{RFC4880}} for more details. And i've added a new subsection in "Guidance for Conumers": Choosing between `--as=3Dtext` and `--as=3Dbinary` ------------------------------------------------------ A program that invokes `sop` to generate an OpenPGP signature typically= needs to decide whether it is making a text or binary signature. By default, `sop` will make a binary signature. The caller of `sop sign` should choose `--as=3Dtext` only when it knows= that: - the data being signed is in fact textual, and encoded in `UTF-8`, and - the signed data might be transmitted to the recipient (the verifier = of the signature) over a channel that has the propensity to transform line-= endings. Examples of such channels include FTP ({{RFC959}}) and SMTP ({{RFC5321}= }). > What I'm saying is the `sop sign` example is error prone. Forget the `<` > and the mandated order and you might reverse the signing key and the > message. sure. if you screw up any API, you can screw up any API :) >>>> If `sop decrypt` fails for any reason and the identified `--session-ke= y-out` >>>> file already exists in the filesystem, the file will be unlinked. >>>=20=20 >>> This seems dangerous! Why do we delete a file we haven't created? >>> Explain. >> >> We don't want the user to run `sop`, and then inspect a file that was >> already in the filesystem thinking that it is `sop`s output. If you >> think that's a bad decision, please suggest what we should do >> differently. > > Maybe we should not overwrite existing files at all and fail earlier? I think you're proposing that if the `--sessionkey-out` file already exists in the filesystem, that should be an error in the first place. I'd be happy to entertain that idea, if anyone wants to provide text for it. If you decide to try to write it up, please think about how it works for the other scenarios where `sop` can produce output on more than stdout. it would be nice if these mechanisms all had the same behavior. >>>> [`--with-session-key`] enables decryption of the `CIPHERTEXT` using th= e session key directly against the `SEIPD` packet. >>>> This option can be used multiple times if several possible session key= s should be tried. >>> >>> What happens if both "in" and "out" are provided? I can venture a guess, >>> but it would be important to make that explicit as there can be horrible >>> bugs there. >> >> Please do venture a guess, in the form of proposed text! I'd also love >> to hear what the horrible bugs are. I don't see them. > > I would argue that both options should not be provided at once. One > implementation that could come up would be that the program attempts to > read the file as it's writing it, truncating the precious key before it > has time to read it. Ah, you're not talking about providing both options -- you're talking about providing both options pointing *at the same file*. i agree, that sounds like a bad idea, but it's a bad idea for *any* pair of input and output fields. > We can continue the discussion in issue #13, but the TL;DR: is that I > agree that stripping trailing control characters is a good idea, but > disagree about whitespace in general. I hope other folks will weigh in on #13. There's interesting discussion going on there about what properties it's reasonable to expect from a "well-formed" password. > I don't know how OpenPGP packets are built. Can't we show the signature > on the output of decrypt? Absolutely not. Mixing the cleartext output with the signature verification stream is a classic cause of failures. What if the cleartext data happens to "look like" a signature verification? how is the consumer supposed to distinguish between them? It is critical to keep them separate. >> But if the primary operation is decryption, i don't think we should fail >> on signature validity for reasons outlined above. > > But that assumes decryption is the primary operation. The subcommand is "sop decrypt". By definition, "decrypt" is the primary operation. > In the context where all my email traffic is encrypted with OpenPGP, > for example, decryption is not the primary operation anymore. I *do* > want to fail properly on signature validity, it becomes a primary > operation when encryption is "default"... You want a *failure* in the sense that you think that an MUA shouldn't show the user the cleartext of the message if no valid signature can be found? This is suprising to me, and i know of no MUA that does this. >>> File descriptors could be passable as distinct options, like >>> --sign-with-fd for --sign-with. >> >> This is an interesting proposal, though i don't see how --sign-with=3D@F= D:3 >> is much different from --sign-with-fd=3D3 -- i guess it lets you use >> files that are literally named @FD:3 ? Is that important? > > It's less magic, more explicit, and correlates better with other > commandline APIs I have encountered. it looks to me like it would make the description of the command line significantly more verbose, but i'm willing to consider it if someone wants to propose a specific textual change. > Say you think you are in a trusted directory with "CERTS" that you want > to encrypt to. You call: > > sop encrypt * < /tmp/file > /tmp/file.pgp > > Except you made a mistake and the attacker has control of the current > directory, and injects a file named (say) @ENV:SOMETHING. Assuming they > have control over the SOMETHING environment, they can now add an > encryption key to the message. if the attacker has control of the directory, they can inject an encryption key in the first place, right? i don't think @ENV makes this any worse... > Control of the environment is kind of a stretch, I must admit, but in > certain environments (most notably web servers), a *lot* of stuff can > end up there and it shouldn't be completely trusted this way. perhaps when an `@`-prefixed argument is supplied, if a file with a matching literal name exists, `sop` should fail with an error because of the ambiguity? This seems like an unlikely and unusual situation, but i can see how it might be worth thinking about it. Perhaps it would be interesting to contrast a MR that contains this guidance with a MR that switches over to the --with-$foo-fd=3D approach you've suggested. >> patches welcome, particularly for this kind of editorial cleanup :) > > https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/12 thanks, merged ;) >>> It would also be great if we could explain where those magic numbers >>> come from in the first place. I suspect they were chosen to not overlap >>> with existing error codes, but that's just a guess. >> >> Justus picked 69 in his OpenPGP Interoperability Test Suite. I chose >> the others as "reasonable-sized primes" just for fun. I don't think >> this information belongs in this document, as it doesn't matter. > > I love this kind of information in text, it makes it less dull. :p i think we have a difference of opinion here. maybe this is ok in an acknowlegements section, but i definitely don't want to read discursive stories when i'm trying to extract technical information. If you want to supply a patch for the acknowledgements section, i'd be willing to consider merging it. > `sop probe` would do the minimal amount of work required to determine > which keys ("signers") to consider when decrypting, then call `decrypt` > properly. I don't think this is "stateless", and i don't think it's well-specified. I also don't think it's particularly useful to know *that* a thing was signed (by some arbitrary certificates) if you haven't already made some determination that the certificate is meaningful in the current context. > `sop probe` could also do the general task of parsing OpenPGP messages > into packets and stuff like that. this doesn't sound like a subcommand at the same level of abstraction that `sop` is aiming for. So i'm still not convinced. But if you (or anyone) wants to make a merge request that proposes new subcommands, i'm definitely up for reviewing them. >>>> Compression {#compression} >> [=E2=80=A6] >>> How about decryption? Do we attempt decompression during decrypt? >> >> It will be interesting to see what implementers do! I've left `sop` >> deliberately agnostic there, and i would like to learn from test suites >> what the answer is. > > Should we make that decision clearer in the document? it's not really a decision :) Hopefully my description of how it would work as part of an interop suite will give a hint of this kind of approach. Thanks for the discussion! --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXcydPwAKCRB2GBllKa5f +PxpAQDE9eOxplQp9r2yhr1K8nSY7bB6ZfcVZC02NG+ag7T3pgEAlAfAcNUWT/jQ 57A3gM4A0QVwMyEjwtQ6BK0ZlI5f+QI= =wd2E -----END PGP SIGNATURE----- --=-=-=-- From nobody Thu Nov 14 11:52:26 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F75E1200A1 for ; Thu, 14 Nov 2019 11:52:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hBaIZdWE2Szy for ; Thu, 14 Nov 2019 11:52:22 -0800 (PST) Received: from marcos.anarc.at (marcos.anarc.at [206.248.172.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B24E212004F for ; Thu, 14 Nov 2019 11:52:22 -0800 (PST) Received: by marcos.anarc.at (Postfix, from userid 1000) id 9D2FD10E081; Thu, 14 Nov 2019 14:52:20 -0500 (EST) Received: (nullmailer pid 28717 invoked by uid 1000); Thu, 14 Nov 2019 19:52:20 -0000 From: =?utf-8?Q?Antoine_Beaupr=C3=A9?= To: Daniel Kahn Gillmor , openpgp@ietf.org In-Reply-To: <87tv77nwe8.fsf@fifthhorseman.net> Organization: Tor References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> <87r22dm2fr.fsf@curie.anarc.at> <87tv77nwe8.fsf@fifthhorseman.net> Date: Thu, 14 Nov 2019 14:52:20 -0500 Message-ID: <87pnhub5hn.fsf@angela.anarc.at> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Nov 2019 19:52:25 -0000 On 2019-11-14 03:18:07, Daniel Kahn Gillmor wrote: > I've incorporated some of your suggestions from this e-mail directly in > the draft now. Thanks a lot for them! more discussion below=E2=80=A6 > > On Tue 2019-11-12 12:26:00 -0500, Antoine Beaupr=C3=A9 wrote: >> I guess what I'm wondering is how I would make this work with my yubikey >> at all. Or maybe I got this backwards and the yubikey interface is what >> should implement sop directly? > > I have no idea about how to make it work with hardware tokens. I remain > frankly unconvinced about the seurity tradeoffs for most uses of > hardware tokens (something i guess i need to actually write down more > formally), and i'm unlikely to spend a lot of time developing how to > integrate them with `sop`. I'm not sure it would be productive for me to engage at that level here right now, given how long this thread is already. :) Suffice to say I believe there's enough of a use case of those devices that we might want to think about how this standards interoperates with them. I guess we don't have good answers for this in any case. At least I don't know how to materialize those things in standards right now anyways, but I would love for this to change in the future. [...] [all good stuff elided] [...] >> What I'm saying is the `sop sign` example is error prone. Forget the `<` >> and the mandated order and you might reverse the signing key and the >> message. > > sure. if you screw up any API, you can screw up any API :) Some APIs are harder to screw up than others. Naming things, specifically, forces people to think about meaning. Ordering them doesn't. >>>>> If `sop decrypt` fails for any reason and the identified `--session-k= ey-out` >>>>> file already exists in the filesystem, the file will be unlinked. >>>>=20=20 >>>> This seems dangerous! Why do we delete a file we haven't created? >>>> Explain. >>> >>> We don't want the user to run `sop`, and then inspect a file that was >>> already in the filesystem thinking that it is `sop`s output. If you >>> think that's a bad decision, please suggest what we should do >>> differently. >> >> Maybe we should not overwrite existing files at all and fail earlier? > > I think you're proposing that if the `--sessionkey-out` file already > exists in the filesystem, that should be an error in the first place. > I'd be happy to entertain that idea, if anyone wants to provide text for > it. Yes, i think that might be preferable. > If you decide to try to write it up, please think about how it works for > the other scenarios where `sop` can produce output on more than stdout. > it would be nice if these mechanisms all had the same behavior. Okay, I'll keep that in mind. >>>>> [`--with-session-key`] enables decryption of the `CIPHERTEXT` using t= he session key directly against the `SEIPD` packet. >>>>> This option can be used multiple times if several possible session ke= ys should be tried. >>>> >>>> What happens if both "in" and "out" are provided? I can venture a gues= s, >>>> but it would be important to make that explicit as there can be horrib= le >>>> bugs there. >>> >>> Please do venture a guess, in the form of proposed text! I'd also love >>> to hear what the horrible bugs are. I don't see them. >> >> I would argue that both options should not be provided at once. One >> implementation that could come up would be that the program attempts to >> read the file as it's writing it, truncating the precious key before it >> has time to read it. > > Ah, you're not talking about providing both options -- you're talking > about providing both options pointing *at the same file*. i agree, that > sounds like a bad idea, but it's a bad idea for *any* pair of input and > output fields. This is something I came up with recently while writing another program. I started by writing something that would read parameters from a config file, then realized I could also *save* parameters to that same file (or another). So I have two parameters, usually pointing at the same file (but not necessarily). The way "conflicts" are handled is simple: the file is first read, and closed. Then it is written to. As long as that's done in order and there's no parallelism, it works correctly. I was wondering if we might want to do such a suggestion specifically for the session key because it's this one case where you have a state that can be read and written. Sure, everything in sop read and writes to stuff, but in general they are different objects (cleartext vs encrypted) that won't be written to the same file unless the user does a mistake. [...] >> I don't know how OpenPGP packets are built. Can't we show the signature >> on the output of decrypt? > > Absolutely not. Mixing the cleartext output with the signature > verification stream is a classic cause of failures. What if the > cleartext data happens to "look like" a signature verification? how is > the consumer supposed to distinguish between them? > > It is critical to keep them separate. Understood. [...] >> In the context where all my email traffic is encrypted with OpenPGP, >> for example, decryption is not the primary operation anymore. I *do* >> want to fail properly on signature validity, it becomes a primary >> operation when encryption is "default"... > > You want a *failure* in the sense that you think that an MUA shouldn't > show the user the cleartext of the message if no valid signature can be > found? > > This is suprising to me, and i know of no MUA that does this. Didn't think of that pattern. The system you're proposing makes more sense in that context. [... possibility of making yet another patch to support -fd ...] >> Say you think you are in a trusted directory with "CERTS" that you want >> to encrypt to. You call: >> >> sop encrypt * < /tmp/file > /tmp/file.pgp >> >> Except you made a mistake and the attacker has control of the current >> directory, and injects a file named (say) @ENV:SOMETHING. Assuming they >> have control over the SOMETHING environment, they can now add an >> encryption key to the message. > > if the attacker has control of the directory, they can inject an > encryption key in the first place, right? i don't think @ENV makes this > any worse... I guess so. It still makes me iffy even if I can't put my finger on it. [...] >> `sop probe` would do the minimal amount of work required to determine >> which keys ("signers") to consider when decrypting, then call `decrypt` >> properly. > > I don't think this is "stateless", and i don't think it's > well-specified. Sure, it's stateless. The input is external and the output is purely informative. But of coruse, it's not well-specified. It was just an idea to work around the double-pass problem you identified. [...] > Thanks for the discussion! You're welcome! I'm not sure I will have much more time to provide patches and more work, but at least that was something. :) a. --=20 Antoine Beaupr=C3=A9 torproject.org system administration From nobody Fri Nov 15 03:17:56 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4191712080E for ; Fri, 15 Nov 2019 03:17:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FD4tWqxMgC-T for ; Fri, 15 Nov 2019 03:17:52 -0800 (PST) Received: from mail.dasr.de (mail.dasr.de [217.69.77.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 357351207FC for ; Fri, 15 Nov 2019 03:17:51 -0800 (PST) Received: from p54abdd01.dip0.t-ipconnect.de ([84.171.221.1] helo=forster.huenfield.org) by mail.dasr.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1iVZbp-000104-So for openpgp@ietf.org; Fri, 15 Nov 2019 11:17:49 +0000 Received: from grit.huenfield.org ([192.168.20.9] helo=grit.walfield.org) by forster.huenfield.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1iVZbp-00058R-7l for openpgp@ietf.org; Fri, 15 Nov 2019 12:17:49 +0100 Date: Fri, 15 Nov 2019 12:17:49 +0100 Message-ID: <87zhgxo0bm.wl-neal@walfield.org> From: "Neal H. Walfield" To: openpgp@ietf.org User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 EasyPG/1.0.0 Emacs/26 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 192.168.20.9 X-SA-Exim-Mail-From: neal@walfield.org X-SA-Exim-Scanned: No (on forster.huenfield.org); SAEximRunCond expanded to false Archived-At: Subject: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Nov 2019 11:17:54 -0000 Hi, Our OpenPGP library is being used in an application that does a real-time exchange. Specifically, two MUAs sometimes communicate via an imap folder. During testing, the application developers observed that the message exchange would sometimes fail, because a signature was considered invalid. Eventually, we figured out that the problem was due to clock skew: the sender would sign a message at, say, 9:00 and the receiver would correctly reject it because its clock said that the current time was 8:59, and 9:00, the time stored in the message's Signature Creation Time subpacket, is clearly in the future. It turns out that a second of clock skew is actually pretty good. The testing was done on a VM and the application developer reported that up to 20 minutes of clock skew are not unusual. Adding a tolerance seems like a reasonable way to deal with this problem. But, there are situations where a tolerance would cause other problems. Consider, for instance, an application that generates messages that contain its current state, and we want to know its state at some point in time. Specifically, consider messages, Ma and Mb, and a time T. If Ta < T < Tb (< current time), then we should use Ma. But, if T is significantly close to Tb, we may not reject Mb and use it instead! An obvious example of this type of message is self signatures. It is tempting to just not use tolerances with self-signatures, but if an application generates a key, and immediately sends it to another computer, the second computer may reject the key as being invalid, because it doesn't consider it to have any valid self-signatures! So, some tolerance is also needed for self signatures. Also, the example also applies to normal ("user data") messages, so specially handling self-signatures is not the right approach, anyway. Perhaps the best thing to do is to only use a tolerance when asking if a signature is valid "now". How do other implementations deal with this? Thoughts? Thanks! :) Neal From nobody Fri Nov 15 04:03:08 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F65A120824 for ; Fri, 15 Nov 2019 04:03:06 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.988 X-Spam-Level: X-Spam-Status: No, score=-1.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ihtfp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B-HBrS6ZqekR for ; Fri, 15 Nov 2019 04:03:04 -0800 (PST) Received: from mail2.ihtfp.org (MAIL2.IHTFP.ORG [204.107.200.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F0F65120823 for ; Fri, 15 Nov 2019 04:03:02 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mail2.ihtfp.org (Postfix) with ESMTP id 80C62E2040; Fri, 15 Nov 2019 06:38:13 -0500 (EST) Received: from mail2.ihtfp.org ([127.0.0.1]) by localhost (mail2.ihtfp.org [127.0.0.1]) (amavisd-maia, port 10024) with ESMTP id 10511-04; Fri, 15 Nov 2019 06:38:06 -0500 (EST) Received: by mail2.ihtfp.org (Postfix, from userid 48) id 1C05BE2044; Fri, 15 Nov 2019 06:38:06 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ihtfp.com; s=default; t=1573817886; bh=4hUH9krsRgR0/Vilyi7nYViMOU07lOHB149aNhUnS3Y=; h=In-Reply-To:References:Date:Subject:From:To:Cc; b=G3avkb8MjV3ZyeSFFLAJsdEKRxQVPgG5D+UbqjiqGP7iq0hF8CRTtqlkBfk+TfT/6 qO/jbzmHsmPsByWRjCoXcZzUCIxSQimGDIv7t+v/8fKpCgcvcH8datexXbSV9uzIAK ty1PU4LMwbMbO96W5q3L+mHjK6TLZmv/xu0RduBs= Received: from 99.46.190.172 (SquirrelMail authenticated user warlord) by mail2.ihtfp.org with HTTP; Fri, 15 Nov 2019 06:38:06 -0500 Message-ID: In-Reply-To: <87zhgxo0bm.wl-neal@walfield.org> References: <87zhgxo0bm.wl-neal@walfield.org> Date: Fri, 15 Nov 2019 06:38:06 -0500 From: "Derek Atkins" To: "Neal H. Walfield" Cc: openpgp@ietf.org User-Agent: SquirrelMail/1.4.22-14.fc20 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: Maia Mailguard 1.0.2a Archived-At: Subject: Re: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Nov 2019 12:03:06 -0000 Hi Neal, There is plenty of prior art on dealing with time skew. Kerberos, for example, has been dealing with it since the 1980s! In short, you have a sliding window of acceptance. Kerberos sets this as +/- 5 minutes. So if the timestamp is within that 10-minute range it is considered current (valid). If you are seeing 20-minute clock skew on a VM, then you should really set up ntp. As for your second issue, my questions to you are: 1) are the messages are guaranteed to be in order (or could they be reordered). 2) can the validator maintain per-sender state? If YES to both questions, then the easiest thing to do is maintain the last timestamp sent and reject if the next message has a timestamp prior to the last one seen. Note that this state must be per-sender, so if you have Alice talking to Bob and Charlie, Alice needs to maintain separate last-seen numbers for both Bob and Charlie, because their clocks can be skewed differently. Hope this helps, -derek On Fri, November 15, 2019 6:17 am, Neal H. Walfield wrote: > Hi, > > Our OpenPGP library is being used in an application that does a > real-time exchange. Specifically, two MUAs sometimes communicate via > an imap folder. During testing, the application developers observed > that the message exchange would sometimes fail, because a signature > was considered invalid. > > Eventually, we figured out that the problem was due to clock skew: the > sender would sign a message at, say, 9:00 and the receiver would > correctly reject it because its clock said that the current time was > 8:59, and 9:00, the time stored in the message's Signature Creation > Time subpacket, is clearly in the future. > > It turns out that a second of clock skew is actually pretty good. The > testing was done on a VM and the application developer reported that > up to 20 minutes of clock skew are not unusual. > > Adding a tolerance seems like a reasonable way to deal with this > problem. But, there are situations where a tolerance would cause > other problems. Consider, for instance, an application that generates > messages that contain its current state, and we want to know its state > at some point in time. Specifically, consider messages, Ma and Mb, > and a time T. If Ta < T < Tb (< current time), then we should use Ma. > But, if T is significantly close to Tb, we may not reject Mb and use > it instead! > > An obvious example of this type of message is self signatures. It is > tempting to just not use tolerances with self-signatures, but if an > application generates a key, and immediately sends it to another > computer, the second computer may reject the key as being invalid, > because it doesn't consider it to have any valid self-signatures! So, > some tolerance is also needed for self signatures. Also, the example > also applies to normal ("user data") messages, so specially handling > self-signatures is not the right approach, anyway. > > Perhaps the best thing to do is to only use a tolerance when asking if > a signature is valid "now". > > > How do other implementations deal with this? Thoughts? > > Thanks! > > :) Neal > > _______________________________________________ > openpgp mailing list > openpgp@ietf.org > https://www.ietf.org/mailman/listinfo/openpgp > -- Derek Atkins 617-623-3745 derek@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant From nobody Sat Nov 16 06:01:32 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6CB9C1200B4 for ; Sat, 16 Nov 2019 06:01:30 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FDeQNfGt5Zfs for ; Sat, 16 Nov 2019 06:01:28 -0800 (PST) Received: from elasmtp-mealy.atl.sa.earthlink.net (elasmtp-mealy.atl.sa.earthlink.net [209.86.89.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D14D912001A for ; Sat, 16 Nov 2019 06:01:28 -0800 (PST) Received: from [66.31.15.242] (helo=Williams-MacBook-Pro.local) by elasmtp-mealy.atl.sa.earthlink.net with esmtpa (Exim 4) (envelope-from ) id 1iVydj-000DV6-Em for openpgp@ietf.org; Sat, 16 Nov 2019 09:01:27 -0500 Date: Sat, 16 Nov 2019 09:01:27 -0500 From: Bill Frantz To: openpgp@ietf.org X-Priority: 3 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: Mailsmith 2.4.3 (480) X-ELNK-Trace: 3a5e54fa03f1b3e21aa676d7e74259b7b3291a7d08dfec797604d40e6e4822429c694a1ebfa7a94c350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 66.31.15.242 Archived-At: Subject: Re: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Nov 2019 14:01:31 -0000 On 11/15/19 at 6:38 AM, derek@ihtfp.com (Derek Atkins) wrote: >If you are seeing 20-minute clock skew on a VM, then you should really set >up ntp. Amateur radio operators have a significant time skew problem=20 when using certain digital modes (Key words: FT8, FT4, wsjt-x,=20 and wsjt). To successfully complete a decode, the sender and=20 receiver's clocks must be within 1/2 to 1 second of each other. =46or the majority of operators, NTP is "good enough". Some=20 operations, particularly those in locations without Internet=20 access, use atomic clocks or GPS time. Cheers - Bill ----------------------------------------------------------------------- Bill Frantz |Security, like correctness, is| Periwinkle (408)356-8506 |not an add-on feature. - Attr-| 16345=20 Englewood Ave www.pwpconsult.com |ibuted to Andrew Tanenbaum | Los Gatos,=20 CA 95032 From nobody Sat Nov 16 09:08:59 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A45412008C for ; Sat, 16 Nov 2019 09:08:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hcBcaie8ybHa for ; Sat, 16 Nov 2019 09:08:55 -0800 (PST) Received: from thumb.scru.org (thumb.scru.org [IPv6:2600:3c00:e000:227::1]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8F70912001A for ; Sat, 16 Nov 2019 09:08:55 -0800 (PST) Received: by thumb.scru.org (Postfix, from userid 1000) id ECD8541476; Sat, 16 Nov 2019 17:08:53 +0000 (UTC) Date: Sat, 16 Nov 2019 17:08:53 +0000 From: Clint Adams To: openpgp@ietf.org Message-ID: <20191116170853.GA10240@scru.org> References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87h83arpby.fsf@fifthhorseman.net> User-Agent: Mutt/1.10.1 (2018-07-13) Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Nov 2019 17:08:57 -0000 > >> armor: armor: Add ASCII Armor > >> ----------------------------- > > > > [...] > > > >> If the incoming data is already armored, and the `--allow-nested` flag > >> is not specified, the data MUST be output with no modifications. > >> Data is considered ASCII armored iff the first 14 bytes are exactly > >> `-----BEGIN PGP`. This operation is thus idempotent by default. > > > > Explain why we want idempotent and why we want to do this guessing game. > > I'm at a bit of a loss here, because these seem obvious to me. > > We want a guessing game because users who don't know what they want are > going to guess anyway, and they're likely to guess wrong. We might as > well guess on their behalf if they don't know. > > We want idempotent because "ensure this thing is armored" is a > reasonable semantic request -- indeed, it's probably the *only* > reasonable semantic request. Perhaps we could drop --allow-nested? As an implementer, I don't want this because this kind of run-time detection is annoying, though it would be more annoying if the check weren't explicit. As a user, I don't want this because it violates the POLA. Why would I be directing something to "ensure this thing is armored"? If I piped something to base64 and it came out unchanged I would be shocked and horrified. Along those lines, having some subcommands default to binary output and others default to ASCII Armor also seems confusing except in cases like `armor` and `dearmor` where the intent would seem obvious (modulo this whole idempotency thing). From nobody Sat Nov 16 09:40:23 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E56E120073 for ; Sat, 16 Nov 2019 09:40:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.699 X-Spam-Level: X-Spam-Status: No, score=-0.699 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ykNSe1a1Qfpu for ; Sat, 16 Nov 2019 09:40:20 -0800 (PST) Received: from dragon.pibit.ch (dragon.pibit.ch [94.231.81.244]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9CADB12001A for ; Sat, 16 Nov 2019 09:40:20 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by dragon.pibit.ch (Postfix) with ESMTP id 2ED88171C12C for ; Sat, 16 Nov 2019 18:40:18 +0100 (CET) Received: from dragon.pibit.ch ([127.0.0.1]) by localhost (dragon.pibit.ch [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 81vrU2TfHWD7 for ; Sat, 16 Nov 2019 18:40:18 +0100 (CET) Received: from [192.168.179.133] (212-51-143-241.fiber7.init7.net [212.51.143.241]) by dragon.pibit.ch (Postfix) with ESMTPSA id C89B4171C05E for ; Sat, 16 Nov 2019 18:40:17 +0100 (CET) To: openpgp@ietf.org References: <87zhgxo0bm.wl-neal@walfield.org> From: Claudio Luck Message-ID: Date: Sat, 16 Nov 2019 18:40:16 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <87zhgxo0bm.wl-neal@walfield.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="GrjVhzMLAz8xgW4Xsyz0ZXLEMxjGLHWld" Archived-At: Subject: Re: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Nov 2019 17:40:22 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --GrjVhzMLAz8xgW4Xsyz0ZXLEMxjGLHWld Content-Type: multipart/mixed; boundary="avpPvdRbFZC6zK478z2CLwMCHaXzBLFkQ" --avpPvdRbFZC6zK478z2CLwMCHaXzBLFkQ Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 15.11.19 12:17, Neal H. Walfield wrote: >=20 > How do other implementations deal with this? Thoughts? If we live in an asynchronous messaging world with no global time concept, then the sender is free to hand out back-dated signatures. The receiver can't tell the difference between in-transit delay and back-dating. This can be used on purpose by the sender to induce some tolerance at the receiver side. Cheers Claudio --avpPvdRbFZC6zK478z2CLwMCHaXzBLFkQ-- --GrjVhzMLAz8xgW4Xsyz0ZXLEMxjGLHWld Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE1Vmp61E/2W6ioFlY6OYP19jZiboFAl3QNIAACgkQ6OYP19jZ ibrQfxAApXxNZQCxZthcGeazbC3YJmLjwp3inIQ6YZRBbBQ85CpDj0yIxsWMC7cw wdBOKl092aJbWXHL/M6nyAcR49ll2VEGa1hOi2okyWhvm8Low5MiVkNInyayDZ9v JDnltCsdCnOqX4jBazYc1QvtakhKI+OtqmRt8d07xjh0q3jCOBsvCqM6Cud39Ot6 mdILYYTNOdEdnJtW6q1iWXyHdwrwB4XroWfcSO7xOdrbYXtmHgDAYWoitMftEhiO GNqYGUrP5uE6p+03VQzhSXUeWrj2Q/voGhIIeDvXOzCpSkpfgAfPqUY89AI4YJPY /za+sheUcqUQ83sD9r5/O1w6ZFsLhbVC3NdzBrvmNhIApl75h0QM8NYqELg5TZWG VzvmoMy+dhi3U5E2ud94NKU9wd2YXidGGJScxIEfyza3/CHSQ7qmoCPEpNVHc9XV dGFz6qtA7VMzw435Xy6sbshhbCPOvuCMAeKKrRtHYtCbwxtxR280IK4IpmH1nMYF lKEVqtSc+8JhnUzyA+o79Ddg3IOqTHzStUIwuJbnmUvZbfWsLdgVUNwQR0FjBZFp F3uM8KXc6OmfY0y2QMP+kiTGUg6kIiCYAylUJpoHmZwQEiwMxqNvc6Cv0dOZEh/Z 3t8vmU+n+kHOOeNpJPedK9RbK6RLKt0e7oEQBvq3bCHC8PAeiLY= =iyNr -----END PGP SIGNATURE----- --GrjVhzMLAz8xgW4Xsyz0ZXLEMxjGLHWld-- From nobody Sat Nov 16 15:06:19 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A5A412022E for ; Sat, 16 Nov 2019 15:06:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=icloud.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9EFUn4TvC8QV for ; Sat, 16 Nov 2019 15:06:15 -0800 (PST) Received: from pv50p00im-ztdg10012001.me.com (pv50p00im-ztdg10012001.me.com [17.58.6.51]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CBD6A12006F for ; Sat, 16 Nov 2019 15:06:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1573945574; bh=2dnGcdfWX5AnIwBBlHWkazzkOUjPciN4cm9zlquSuE0=; h=Content-Type:Subject:From:Date:Message-Id:To; b=CNsRrcRKY8I/9dBb5EW2P9L714Pu/DvC+d+IacQnJwLUGBUXcNsaiE1EUCdO5hJyF m0+EBVk85Sd30JPMPkyPpFTV4C3Evtch3xqPuvwQR75h/5gldcdbqD9LZB/tUgbtRB vvOMvMgMHMO28zP5eTjp5DZoj2cKOL/27OQIEkMjaQVwITEiFuXOXkxLrY1ZMBd0vx kphaW8Efh1cB9SYyTqSSVH3F4WDUub7HE6I3SQsNmHc5dX0KoV40xb6GAbCVkNs/VB WgN/QS1lRAXw8kHqtHbE2F/O/fGhpNrkuXNfSl66ZjpwDu/EhiBHvdTBs/zPeebYPC XToz9c7jLKW8Q== Received: from [192.168.7.69] (thing1.merrymeet.com [173.164.244.99]) by pv50p00im-ztdg10012001.me.com (Postfix) with ESMTPSA id 378412807B8; Sat, 16 Nov 2019 23:06:14 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) From: Jon Callas In-Reply-To: Date: Sat, 16 Nov 2019 15:06:13 -0800 Cc: Jon Callas , openpgp@ietf.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <87zhgxo0bm.wl-neal@walfield.org> To: Claudio Luck X-Mailer: Apple Mail (2.3445.104.11) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-11-16_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1911160216 Archived-At: Subject: Re: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Nov 2019 23:06:17 -0000 > On Nov 16, 2019, at 9:40 AM, Claudio Luck = wrote: >=20 > If we live in an asynchronous messaging world with no global time > concept, then the sender is free to hand out back-dated signatures. = The > receiver can't tell the difference between in-transit delay and > back-dating. This can be used on purpose by the sender to induce some > tolerance at the receiver side. We do live in that world; it's not an if. Newton argued for a notion of absolute time, but also conceded Galileo's = point that you can switch frames of reference and have a conversion = between them. Leibniz, on the other hand, argued that space is = meaningless except as relative distance and that time *only* makes sense = as an expression of relative motion. Ernst Mach also had a number of = pithy things to say about this, particularly that even if you have = absolute space or time, all math and physics works if you declare that = you're still and the universe is moving or ticking. I'm especially fond of Poincar=C3=A9's "The Measure of Time" as he hits = not only on the physical aspects but the experiential ones as well = (English translation here: = ). Of course, = Einstein tied a bow around all of this in his Special Relativity which = flat out declares what Leibniz gestures toward, that space and time are = linked and there is no preferred frame of reference. Of course, I can do the same thing towards Einstein that Leibniz and = Mach did and note that if there's no preferred position or clock, I can = just declare one and everything works fine. Operationally, this is what = we do with GPS/atomic/NTP time. We declare that to be our frame of = reference. However, even that breaks down for surprisingly short distances. Network = delays and the consequent "lag" means that you can't establish primacy = on most multiplayer games in a lot of circumstances. This is why lots of = them try to avoid situations where jumping around corners etc. are easy = to cheat at. This is exactly the same problem you're talking about. = There are a lot of interesting papers, but here's one that is precisely = trying to create an absolute frame of reference, "Lag Compensation for = First-Person Shooter Games in Cloud Gaming" = . Note = that they're not doing any mathematical security (like signatures) here, = this is all trusting the network. They have this problem because it is = inherent to any system that has space and time involved with it. In the general case, you can't consider a time measurement to be a = scalar, it has to be at the very least a complex number of the form = [time, skew]. As Derek noted, Kerberos used a skew of five minutes. = While Neal Walfield noted in his original post that he's seen skew of = 20min, I concur that that seems a bit long. My naive home set-up = commonly has alarms across devices being =C2=B12s or less, but that's = because they're all getting time from some combination of NTP and = cellular network time, which is ultimately GPS time (and of course, = skew). I think five minutes is likely reasonable, but *some* skew is = unavoidable. Moreover, anyone who's on satellite networks is seeing = latency of over a second and once you throw in normal exponential = backoff, five minutes seems about as short as is reasonable. Thus, you're absolutely right, if time is a scalar, then someone can = cheat. There are situations (like real-time internet games) where the = exact problem you mention, that someone can hide cheating in the skew is = a well-known, unsolvable problem. Jon From nobody Sun Nov 17 02:41:44 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D0011120059 for ; Sun, 17 Nov 2019 02:41:41 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=2o7jfctq; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=34LPZJnz Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NwHcWFCizCXR for ; Sun, 17 Nov 2019 02:41:40 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [IPv6:2001:470:1:116::7]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1714012004D for ; Sun, 17 Nov 2019 02:41:40 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1573987299; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=dhDjSldOuNTf8RgZ8uH8SjXa6VgAHw5D+DA9Tpq7toU=; b=2o7jfctqnXwZv+jhxX4avl0zQGEVQ2TK8W9d7FMNtW/46FQj+3Awy6F1 2dLHcMk/XmWLSc8etAau5k0YidPnDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1573987299; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=dhDjSldOuNTf8RgZ8uH8SjXa6VgAHw5D+DA9Tpq7toU=; b=34LPZJnz9gtKRrQtMQG1bzvikpFU8F7gEVGOV4ZuI9/6O55ES9RZS6nC DwXOdfXv2WJ7majxOuBcupsI0c1iKyr+Dqzue/F6Lpx6tZz7yjeDaV47q1 6FVmeu9RbtltD6edORk5LlN71zQvkkuPRqbUtac/m+jNbhNDhf2el9uhde mjXoXozoyWQBThyOKtUe004qRC0ZLQuAwY50c3ZTLXRorR+keun/mkjWLB CMw//My0u6t04Ow5kieR6+cxiX7KllHO6eh9/WkhuV0ya6Hst/krM2UiwB tGAZYeuAGLNFnHtdGxxAmMSoUY7MiMbVG9oNwR6qe6yt7bvlfs1n+A== Received: from fifthhorseman.net (unknown [103.137.210.129]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 35909F9A7; Sun, 17 Nov 2019 05:41:38 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id 2B3CB20615; Sun, 17 Nov 2019 00:27:15 +0200 (EET) From: Daniel Kahn Gillmor To: Antoine =?utf-8?Q?Beaupr=C3=A9?= , openpgp@ietf.org In-Reply-To: <87pnhub5hn.fsf@angela.anarc.at> References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> <87r22dm2fr.fsf@curie.anarc.at> <87tv77nwe8.fsf@fifthhorseman.net> <87pnhub5hn.fsf@angela.anarc.at> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Sun, 17 Nov 2019 01:27:14 +0300 Message-ID: <87v9rjmp8d.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Nov 2019 10:41:42 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Thu 2019-11-14 14:52:20 -0500, Antoine Beaupr=C3=A9 wrote: > [ dkg wrote: ] >> I think you're proposing that if the `--sessionkey-out` file already >> exists in the filesystem, that should be an error in the first place. >> I'd be happy to entertain that idea, if anyone wants to provide text for >> it. > > Yes, i think that might be preferable. i think you're right. thanks for that. I've committed that change in d027f6e3b29417387daf541d251a541e179898b1. >> If you decide to try to write it up, please think about how it works for >> the other scenarios where `sop` can produce output on more than stdout. >> it would be nice if these mechanisms all had the same behavior. > > Okay, I'll keep that in mind. On inspection, only `sop decrypt` has multiple outputs, so it wasn't as wide a change as i'd feared. > This is something I came up with recently while writing another > program. I started by writing something that would read parameters from > a config file, then realized I could also *save* parameters to that same > file (or another). So I have two parameters, usually pointing at the > same file (but not necessarily). > > The way "conflicts" are handled is simple: the file is first read, and > closed. Then it is written to. As long as that's done in order and > there's no parallelism, it works correctly. > > I was wondering if we might want to do such a suggestion specifically > for the session key because it's this one case where you have a state > that can be read and written. Sure, everything in sop read and writes to > stuff, but in general they are different objects (cleartext vs > encrypted) that won't be written to the same file unless the user does a > mistake. This is bizarre stuff, and i'm struggling to imagine any use case for it with `sop` as it is currently defined, just for supplying both =2D-with-session-key and --session-key-out at the same time (let alone pointing them both at the same fle). If a user invokes `sop decrypt` with a known session key, presumably it is because they already know the session key. Why, in that circumstance, would they ask `sop decrypt` for the session key? Are you imagining a situation where the user of `sop decrypt` is *guessing* at the session-key, but is unsure? For example, supplying multiple --with-session-key arguments, or --with-session-key alongside a KEY or a --with-password argument? I guess i can see it for some situation where "i've got a pile of session keys and a pile of messages, and i want to figure out which session key belongs to which message". Anyway, I've just added an explicit failure mode when an indirect input is pointed to a file that does not exist (in c7c05598a5532c203de28b0dc6e1bfb69eecf549). And given that it's now an error to point --with-session-key at an existing file, it should be impossible to point both options to the same file. so hopefully this concerns is now moot. --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXdB3wgAKCRB2GBllKa5f +JkWAP4/qMKNgKyK7wquDq7zF6wMHDuCQ8cH2B2Iuiip3cpWawEAz3WRklpMa0VN qx33vjLwPpWqq5d5bgfnKh9WEsmM6gc= =ktdn -----END PGP SIGNATURE----- --=-=-=-- From nobody Sun Nov 17 06:22:00 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C1F512010D for ; Sun, 17 Nov 2019 06:21:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (unsupported algorithm ed25519-sha256)" header.d=fifthhorseman.net header.b=afHgcmwD; dkim=pass (2048-bit key) header.d=fifthhorseman.net header.b=M9P9JQX+ Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iQeGUZ4V9NQT for ; Sun, 17 Nov 2019 06:21:57 -0800 (PST) Received: from che.mayfirst.org (che.mayfirst.org [IPv6:2001:470:1:116::7]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B5BFC12006D for ; Sun, 17 Nov 2019 06:21:57 -0800 (PST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1574000515; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=V/+bfCsdqaKsGaB/SqBbHRAzU7SWvQqN63FlgL1PMoA=; b=afHgcmwDsgXQ2YY/GBCVz9V+NWWjYj/SX+/evQ6sMnvyVxjkv5oyf/gp xEBPuT9bZ9hjWCLuwT29wAD0wDxtDg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1574000515; h=from : to : subject : in-reply-to : references : date : message-id : mime-version : content-type : from; bh=V/+bfCsdqaKsGaB/SqBbHRAzU7SWvQqN63FlgL1PMoA=; b=M9P9JQX+VdU+doF+8aNFuu6h1DzVMfJzEE9jb8ypMgAa6I/qSvOndB3q bfZBw58q++SfpDaLeW+1LKzZSQTSCq8p5EiUze1k0cRVjaw6+1kKzs1tGN SvBQKTJ2GhdT12C+Jzxh/l3OLc89kfjiytiSS35ysW58aQl5NxSRFWvGSo U6lqdkoQUqwFQI+Ab3B8lcy3mqTie/kfuck6ZfE8xW64WezeqsUbAqp+bj Eu4Z3ae1Y3lw64zTnqapJ/kjPP7EMD+ZwQPB+BIXymInW0MQ3n5/VWzvUr m7OtMgRFBmWz6LlMS83D+SGkYmxzsvvAaV+u6MCDQcV3vtqH9tfiuQ== Received: from fifthhorseman.net (unknown [182.55.86.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 88C79F9A9; Sun, 17 Nov 2019 09:21:55 -0500 (EST) Received: by fifthhorseman.net (Postfix, from userid 1000) id 048DF2058E; Sun, 17 Nov 2019 16:19:06 +0200 (EET) From: Daniel Kahn Gillmor To: Clint Adams , openpgp@ietf.org In-Reply-To: <20191116170853.GA10240@scru.org> References: <87mud28fds.fsf@curie.anarc.at> <87h83arpby.fsf@fifthhorseman.net> <20191116170853.GA10240@scru.org> Autocrypt: addr=dkg@fifthhorseman.net; prefer-encrypt=mutual; keydata= mDMEXEK/AhYJKwYBBAHaRw8BAQdAr/gSROcn+6m8ijTN0DV9AahoHGafy52RRkhCZVwxhEe0K0Rh bmllbCBLYWhuIEdpbGxtb3IgPGRrZ0BmaWZ0aGhvcnNlbWFuLm5ldD6ImQQTFggAQQIbAQUJA8Jn AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBMS8Lds4zOlkhevpwvIGkReQOOXGBQJcQsbzAhkB AAoJEPIGkReQOOXG4fkBAO1joRxqAZY57PjdzGieXLpluk9RkWa3ufkt3YUVEpH/AP9c+pgIxtyW +FwMQRjlqljuj8amdN4zuEqaCy4hhz/1DbgzBFxCv4sWCSsGAQQB2kcPAQEHQERSZxSPmgtdw6nN u7uxY7bzb9TnPrGAOp9kClBLRwGfiPUEGBYIACYWIQTEvC3bOMzpZIXr6cLyBpEXkDjlxgUCXEK/ iwIbAgUJAeEzgACBCRDyBpEXkDjlxnYgBBkWCAAdFiEEyQ5tNiAKG5IqFQnndhgZZSmuX/gFAlxC v4sACgkQdhgZZSmuX/iVWgD/fCU4ONzgy8w8UCHGmrmIZfDvdhg512NIBfx+Mz9ls5kA/Rq97vz4 z48MFuBdCuu0W/fVqVjnY7LN5n+CQJwGC0MIA7QA/RyY7Sz2gFIOcrns0RpoHr+3WI+won3xCD8+ sVXSHZvCAP98HCjDnw/b0lGuCR7coTXKLIM44/LFWgXAdZjm1wjODbg4BFxCv50SCisGAQQBl1UB BQEBB0BG4iXnHX/fs35NWKMWQTQoRI7oiAUt0wJHFFJbomxXbAMBCAeIfgQYFggAJhYhBMS8Lds4 zOlkhevpwvIGkReQOOXGBQJcQr+dAhsMBQkB4TOAAAoJEPIGkReQOOXGe/cBAPlek5d9xzcXUn/D kY6jKmxe26CTws3ZkbK6Aa5Ey/qKAP0VuPQSCRxA7RKfcB/XrEphfUFkraL06Xn/xGwJ+D0hCw== Date: Sun, 17 Nov 2019 22:19:05 +0800 Message-ID: <87mucumvqe.fsf@fifthhorseman.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Archived-At: Subject: Re: [openpgp] review of the SOP draft X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Nov 2019 14:22:00 -0000 --=-=-= Content-Type: text/plain Hi Clint-- Thanks for your feedback! On Sat 2019-11-16 17:08:53 +0000, Clint Adams wrote: > As an implementer, I don't want this because this kind of run-time > detection is annoying, though it would be more annoying if the check > weren't explicit. I understand that you'd like to implement something more minimal, but sop is trying to supply plausible interfaces for use cases for a developer who doesn't know all the grubby details of OpenPGP but still wants to work with confidential data as framed by OpenPGP. > As a user, I don't want this because it violates the POLA. Why would > I be directing something to "ensure this thing is armored"? If I piped > something to base64 and it came out unchanged I would be shocked and > horrified. The only use case i'm currently imagining for using `sop armor` is that the user has come across some OpenPGP content in some environment, and they want to transmit it over a channel that is not 8-bit clean (or perhaps they want to put it into a text-based revision control system that would rather not host binary blobs). They have two choices: - detect whether it is already armored, and if not, invoke `sop armor` - unconditionally invoke `sop armor`, to ensure that it is armored. The latter sounds simpler and easier, and it really isn't surprising or astonishing, so i'm not sure it really violates POLA. Vincent Breitmoser proposed the idempotency requirement over in https://gitlab.com/dkg/openpgp-stateless-cli/merge_requests/3 and i merged it because it seemed plausible to me. I'm currenly thinking of dropping --allow-nested entirely, but i see you've given that a thumbs-down on https://gitlab.com/dkg/openpgp-stateless-cli/issues/15 if you want me to consider that more, a clearer use for nested armoring would be welcome. Do you have another use case that we should consider? > Along those lines, having some subcommands default to binary output > and others default to ASCII Armor also seems confusing except in cases > like `armor` and `dearmor` where the intent would seem obvious > (modulo this whole idempotency thing). I believe all the subcommands default to armored output (with the exception of "sop dearmor"). Are you seeing somewhere where that's not the case? --dkg --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTJDm02IAobkioVCed2GBllKa5f+AUCXdFW2QAKCRB2GBllKa5f +OzSAP9QmvatwaUFJ1HuIA3uvqC8t92lUhYmHK51l3ifLOY9NQD+PbIGDh1tsLK0 hVPGBUQ4aIzmUI4l++G9hRDhafkiuQ4= =2SWs -----END PGP SIGNATURE----- --=-=-=-- From nobody Mon Nov 18 01:50:16 2019 Return-Path: X-Original-To: openpgp@ietfa.amsl.com Delivered-To: openpgp@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CB9E1208A9 for ; Mon, 18 Nov 2019 01:50:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A6JSWObUphki for ; Mon, 18 Nov 2019 01:50:13 -0800 (PST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 008091208FA for ; Mon, 18 Nov 2019 01:50:12 -0800 (PST) Received: from mit.edu ([24.16.140.251]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id xAI9o1Zg009631 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Nov 2019 04:50:04 -0500 Date: Mon, 18 Nov 2019 01:50:01 -0800 From: Benjamin Kaduk To: Jon Callas Cc: Claudio Luck , openpgp@ietf.org, Jon Callas Message-ID: <20191118095001.GM32847@mit.edu> References: <87zhgxo0bm.wl-neal@walfield.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) Archived-At: Subject: Re: [openpgp] Dealing with clock skew X-BeenThere: openpgp@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Ongoing discussion of OpenPGP issues." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2019 09:50:14 -0000 On Sat, Nov 16, 2019 at 03:06:13PM -0800, Jon Callas wrote: > > > In the general case, you can't consider a time measurement to be a scalar, it has to be at the very least a complex number of the form [time, skew]. As Derek noted, Kerberos used a skew of five minutes. While Neal Walfield noted in his original post that he's seen skew of 20min, I concur that that seems a bit long. My naive home set-up commonly has alarms across devices being ±2s or less, but that's because they're all getting time from some combination of NTP and cellular network time, which is ultimately GPS time (and of course, skew). I think five minutes is likely reasonable, but *some* skew is unavoidable. Moreover, anyone who's on satellite networks is seeing latency of over a second and once you throw in normal exponential backoff, five minutes seems about as short as is reasonable. I believe that if Kerberos was starting over now, the 5 minutes would be seen as excessively long, FWIW. -Ben