Subject: Re: armour pierced with PGP 8 arrow

From: [email-masked] (Peter Gutmann)
Date: Wed, 10 Dec 2003 19:47:10 +1300

Ian Grigg <[email-masked]> writes:

>It appears that PGP 8 is breaking the spirit and intent of the ascii
>armouring format, if not the "letter of the law."
>
>What it is doing is in essence putting in a Version that is too long for some
>mailers' line slicing paramaters. The result is that people receive this:
>
>Version: PGP 8.0.2 - not licensed for commercial use:
>www.pgp.com

Is it really a line-length issue, or something else like the presence of the
second colon in the line for something that's scanning for <string>:<string>?
For a mailer to be unable to handle a 66-char line sounds pretty... broken.

Peter.

---

From: Ian Grigg <[email-masked]>
Date: Wed, 10 Dec 2003 12:06:37 -0500

> second colon in the line for something that's scanning for <string>:<string>?

Well, the slicing that is occuring is in the
transmission process, in this case, so I am
theorising that this is a line length issue.
(I haven't been able to check this, because
the person sending the dodgy email doesn't
understand what I am asking...  That's the
way it is in userland.)

But you raise a point I hadn't seen, the line
above is badly formatted in a second way, in that
it includes two ": " separators.

Examination of the para in section 6.2, page 50:

    The format of an Armor Header is that of a key-value pair.  A colon
    (':' 0x38) and a single space (0x20) separate the key and value.
    OpenPGP should consider improperly formatted Armor Headers to be
    corruption of the ASCII Armor.  Unknown keys should be reported to
    the user, but OpenPGP should continue to process the message.

indicates that this is not explicitly ruled
against, but I'd say it is implicit stated
that only one separator is permitted.

Or, in the alternate, if two separators are
permitted, then the text should be clarified,
as different parsing strategies will result
in confusion.

Should there be only one ": " or are additionals
permitted in the value and thereafter ignored?

As the ID says "OpenPGP should consider improperly
formatted Armor Headers to be corruption of the
ASCII Armor," I suggest this be explicitly
clarified to improve interworkability.

> For a mailer to be unable to handle a 66-char line sounds pretty... broken.

My view is that when it comes to lines, all
mailers seem broken.  But what do I know?

Mostly, this seems to occur when going from
one distinct community to another.  Within
a particular flavour of mailers, they all
understand each other, but across distinct
communities, the mailers have different
ways of dealing with things.

I did a quick straw poll today, and found

   76,72,76,58,72,56,72,72

A few points:

* In communities where they always reply
with full interpolation, and full (audit)
chains of replies, sometimes people set
the width to be very narrow, because they
end up with scores (i.e., 20) series of
replies, which scan across to the right
hand columns.

* from the *same* email, 50-72!  Which
makes me think that this guy had his mailer
set up for proportional text.  Another couple
of emails I wasn't able to work out what was
going on, but inserted lines seemed to be
part of the bug set.

* I've just noticed another thing about
the community stuff - Macs tend to interpret
a trailing space as a continuation character,
and that sometimes works and sometimes doesn't
(I suppose it always works on Macs).

* If you look at my original mail, you will see
that the sliced line I presented was broken,
but in Will Price's reply, it had been repaired,
presumably by his mailer, so in his view, there
was apparently nothing wrong!

iang

---

From: Ian Grigg <[email-masked]>
Date: Thu, 11 Dec 2003 12:17:20 -0500

> > second colon in the line for something that's scanning for <string>:<string>?

Peter brought up the issue of the additional
": " separators and I opined that the draft
should be clearer on this issue.

On reflection, I think it should not be permitted.

The reason for this is that when you combine
it with the line slicing behaviour, then some
games are possible:

Version: 1.0.0 non-commercial, upgrade to Version: 2.0.0-commercial

Could result in an embarressing split.  Now, that's
a superficial and ignorable result, and only presented
for the sake of showing what might happen.

I can see no good reason to leave multiple separators
as permitted in the ID, so I'd suggest adding a simple
restriction such as "Only one separator is permitted."

As another observation, the use of the term "Armour
Headers" appears overloaded.  Could this be clarified
by changing the current usage into:

   Armour Headers:  -----.*PGP.*-----
   Optional Headers:  Version: ////

Or even Comment Headers, or Optional Armour Headers,
or Optional Comments?

iang

---

From: David Shaw <[email-masked]>
Date: Thu, 11 Dec 2003 13:04:02 -0500

On Thu, Dec 11, 2003 at 12:17:20PM -0500, Ian Grigg wrote:
> 
> Ian Grigg wrote:
> > 
> > Peter Gutmann wrote:
> 
> > > Is it really a line-length issue, or something else like the
> > > presence of the second colon in the line for something that's
> > > scanning for <string>:<string>?
> 
> Could result in an embarressing split.  Now, that's a superficial
> and ignorable result, and only presented for the sake of showing
> what might happen.
> 
> I can see no good reason to leave multiple separators as permitted
> in the ID, so I'd suggest adding a simple restriction such as "Only
> one separator is permitted."

I disagree we need to change anything here.  There is already only one
separator permitted.  Using your example:

The second instance of colon-space is NOT a separator.  It's just part
of the value.

This isn't very complicated.  I'd be quite surprised to hear of any
parser that didn't do:

a) Find the leftmost colon-space.
b) The string to the left is the key.
c) The string to the right is the value.

That's how email works (note the subject line of this message has two
colons!), how news works, and how OpenPGP works.

David

---

From: Ian Grigg <[email-masked]>
Date: Thu, 11 Dec 2003 14:01:29 -0500

David Shaw wrote:
...

Sorry, I should have spelt this out:

> > Could result in an embarressing split.

Such as:

Version: 1.0.0 non-commercial, upgrade to
Version: 2.0.0-commercial

When the line slicing behaviour is set to (about) column 42.

> c) The string to the right is the value.

How do parsers handle the above case?  Not that it's
important.  The thing to realise here is that the
additional separator's presence, coupled with the
ability of line slicing by mailers, introduces an
exceptional case that's simply better off made
illegal.

The spirit of ascii armouring is to get the message
through in the face of aggressive and unpredictable
actions of many users' mailers.  In this case, I
think this spirit is preserved by making the optional
Armor Headings as simple as possible, and encouraging
strict legality.

iang

PS: actually, the above case is "legal".  If there
was no separator in the second split line, then
the armouring is broken ("OpenPGP should consider
improperly formatted Armor Headers to be corruption
of the ASCII Armor."), and the message should not be
accepted.  But that's hardly a reliable strategy.

---

From: [email-masked]
Date: Fri, 12 Dec 2003 04:56:22 +0000

> When the line slicing behaviour is set to (about) column 42.

If I have to set my line width to 42, your mail system is so broken
that I don't need to talk to it. 72 is reasonable, and and one should 
expect lines exceeding 80 characters to be wrapped. Anything under 68 is
unreasonably small. I, for example, am using 72 character lines in my
editor (vim).

> How do parsers handle the above case?  Not that it's

Do a) with strchr(3) or memchr(3) by finding a colon and then check that
the next character is a space. If it's not, abort the message. The
code is:

if (!(ptr=(uint8_t *)memchr(line, ':', sizeof(line))) || ptr[1]!=' ')
	exit(1);

> important.

-- 
Brian M. Carlson <[email-masked]> 0x560553e7

---

From: Ian Grigg <[email-masked]>
Date: Fri, 12 Dec 2003 10:43:46 -0500

> unreasonably small. I, for example, am using 72 character lines in my

It does appear to be the consensus that there
is no need or desire to change the ID on this
issue.  However, I'm not seeing a lot of real
consideration of the real points here.  So  I'm
unclear here whether to keep kicking this dying
horse, or perhaps Derik could rule on closing
it?

?  I disagree with the above logic on the
following grounds:

   OpenPGP is not only used for mail, and in
   fact takes great care to be more universally
   useful (scan the ID for the word "mail", I
   just did!).  We here should also strive to
   move from the particular to the general, when
   looking at examples, and back again.

   Security standards are not built on such
   considerations of "yours is broken because
   I don't like your numbers."  Bad software
   is certainly built with those sorts of
   criteria, but bad software is not what we
   are about here.  This is about security,
   so detail is important.

   The offending number chosen was related to
   the example;  the point remains with an
   example of a different number.  You pick.

   Notwithstanding many peoples' desires to
   impose their views of how mail works on
   the rest of the world, there are other
   mail considerations in the future:  PDAs
   and phones have smaller screens, for example.
   Web mailers and hush-style mailers can be
   very unkind to formatters.  Chat is replacing
   email....  the list goes on - if OpenPGP is
   to be based on the email view of the past,
   then, we should all stop work on it right
   now.

The greatest success comes when multiple
competing implementations communicate with
ease, and without games.  Here we have a case
where it is apparently easy to legally create
a correct message that gets turned into a
recipient message of indeterminate legality
by transport.

Fixing this costs nobody much at all [1].

> if (!(ptr=(uint8_t *)memchr(line, ':', sizeof(line))) || ptr[1]!=' ')

Please make sure you read the entire example.

That doesn't decide which line of the two lines
in the previous text, post-slicing, goes on to
become the dominating one.

The game to play in security programming is to see
if a) we can confuse the other party, and b) we can
create a security breach out of a confusion.

We've shown a).  I grant b) is a little harder, as
optional headers aren't supposed to be "used" for
such things.  But, who knows what adventurous souls
have in their minds for the future... [2]

iang

[1] I just thought late last night how to breach
b)...  Imagine you are a company selling a device
that filters on the headers.  Imagine that you
make commercial decisions on the basis of those
headers.  (Because that's all you can read.)  Then,
an enterprising young chap realises that all he
has to do is to slice the headers, and they are
still human-readable, but they bypass the filtering
of the proxy that handles incoming messages.

marketing reference: NY AG v. WS == 1.3 bn.

[2]  the guys
over at PGP Inc might want to rewrite their
comment to be a bit shorter than the ASCII-
Armoured data lines - rule of thumb - and to
change the extra ": " to some other symbol.

If they haven't already done these steps,
I'd be very surprised!  Also, they should
reject sliced headers, or suggest we change
the ID on that point.  But, no real cost.

GPG might want to detect and warn that the
headers appear corrupted and can be fixed
easily with an editor, rather than silently
bailing on ID strictness.

---

From: Konrad Rosenbaum <[email-masked]>
Date: Fri, 12 Dec 2003 18:44:26 +0100

> The second instance of colon-space is NOT a separator.  It's just part

As long as the line is not split, you are correct.

> colons!), how news works, and how OpenPGP works.

With one notable difference: in the case any MTA splits headerlines, it will 
add space(s) in front of the next line, which is defined to continue the line 
in RFC822. When a MUA or MTA splits body lines it will not add anything 
(maybe except for some ">" characters). So the receiving system will not be 
able to tell the difference between a new header line an a continuation.

I'm a newbie to implementing OpenPGP - so please forgive me, if my question is 
stupid: why consider the additional headers at all? They seem to me to only 
have any value to human readers and none at all to OpenPGP itself, since all 
important information is already encoded into the binary message blocks (once 
you got them out of their armor). Is there any technical reason for the RFC 
not saying "Implementations shall ignore header lines."?

        Konrad

---

From: Ian Grigg <[email-masked]>
Date: Fri, 12 Dec 2003 15:38:53 -0500

> As long as the line is not split, you are correct.

Knowing how coders make different assumptions, and have
different scanning tools available to them, even this could
cause some surprises.  For e.g., in list based languages,
popping is more natural than parsing left-to-right.

Here's some perl:

perl -e ' $x = "Version: 1.0.0 non-commercial, upgrade to Version: 2.0.0-commercial"; @a = (split(": ", $x)); $_ = shift(@a); print "$_\n"; /Version/ && print pop(@a), "\n"; '

...
> not saying "Implementations shall ignore header lines."?

This is meta-part of the issue.  Currently, the ID says:

   "OpenPGP should consider improperly
    formatted Armor Headers to be
    corruption of the ASCII Armor."

I wonder whether that is too strong.  It seems reasonable
to say that implementations may attempt to parse beyond and
ignore improperly formatted headers.  Perhaps with warnings.

As we have not gone to any great extent to define why the
optional ascii armour headers are important, it seems odd
to treat them as if they are important enough to be capable
of breaking the armoured message.

iang