Subject: draft-ietf-usefor-article-04.txt: 6.21.3 considered harmful.

From: Thomas Roessler <[email-masked]>
Date: Sat, 14 Apr 2001 11:23:12 +0200

I'm writing as one of the co-authors of
draft-ietf-openpgp-mime-06, which is hoped to become the
successor of RFC 2015 (PGP/MIME) soon.

In section 6.21.3 of draft-*-article-04, you write the following:

   In the same way, Content-Types requiring special processing for
   their display, such as "application", "image", "audio", "video"
   and "multipart/related" are discouraged except in groups
   specifically intended (by policy or custom) to include them.
   Exceptionally, those application types defined in [RFC 1847] and
   [RFC 2015] for use within "multipart/signed" articles, and the
   type "application/pgp-keys" (or other similar types containing
   digital certificates) may be used freely but, contrary to [RFC
   2015] and unless the article is intended to be sent by mail also,
   the Content-Transfer-Encoding SHOULD be left as "8bit" (or "7bit"
   as appropriate).

I consider this section harmful for various reasons.

First, it should be noted that problems with transmitting 8bit
content are not the only reason for which an agent implementing
PGP/MIME may elect to use quoted-printable or base64 as a body
part's encoding.

In particular, draft-ietf-openpgp-mime-06 specifies that the signed
material in a PGP/MIME multipart/signed entity MUST NOT include any
trailing whitespace.  This may either be achieved by cutting off
trailing whitespace, or by protecting it with an appropriate
transfer encoding.

The reason for this lies in the definition of a text-mode signature
in RFC 2440 [OpenPGP], which is incompatible with the definition
used by traditional implementations (aka pgp version 2).

More precisely, a text-mode signature traditionally meant that any
line feed has to be canonicalized to CR LF before hashing.  With
OpenPGP, trailing whitespace is additionally snipped.  (That's the
kind of processing which has traditionally been applied to
clearsigned messages.)

Now, the processing rules for multipart/signed make sure that line
feeds are canonical before they hit the signing engine.  With
traditional PGP, that meant that text-mode and binary-mode hashes of
the material would be identical.  This is no longer valid with
OpenPGP, which has two consequences:  (1) In order to determine what
hash is to be computed, a receiving agent would have to look at the
signature.  This would break the possibility to process
multipart/signed in one pass.   (2) In the presence of trailing
whitespace, text-mode signatures wouldn't be interoperable between
implementations using PGP 2 and OpenPGP implementations.

The best way to fix both problems is to make sure that no trailing
whitespace is present, by applying appropriate encodings.  (With
text/plain; format=flowed, this will be something which is
encountered quite frequently.)

For this reason alone, mandating transparent encodings may not be
the best idea at this point.

It should also be noted it's impossible to convert a
multipart/signed entity which contains 8bit body parts to any 7bit
format without breaking the signature.  (Remember that MIME
explicitly forbids nested MIME encodings: Recoding has to happen on
the leaf level of a nested MIME structure.)

This implies, that a multipart/signed entity wihich contains 8bit
body parts cannot be legally transported via Internet e-mail without
ruining the signature - be it as part of message/rfc822
encapsulation, be it as a direct attachment.

Finally, I'd like to warn you that signature generation and
verification interoperability may seriously be endangered by
allowing 8bit body parts in a multipart/signed.

More precisely, a multipart/signed body part may generally contain
textual data in various character sets - it may even contain a
nested multipart/mixed with text/plain subparts which bear different
character sets.

However, PGP signature generators and verifiers have traditionally
been expecting that they are presented with data in a local
character set when preparing text-mode signatures.  For this reason,
signature generators and verifiers may attempt to recode data they
are presented with to whatever character set is considered to be
canonical (utf-8 with OpenPGP, iso-latin-1 or koi-8 with pgp 2).
When applying such (unnecessary) transformations to 8bit messages in
on-the-wire-format (which would inevitably happen when a
multipart/signed is verified or generated), interoperability will,
of course, be ruined.

Restricting the signed material to 7bit data also helps to make
signature generation and verification more robust against such
problems, and implementations considerably simpler.

For all these reasons, I urge the Usenet format working group to
remove the reference to multipart/signed from section 6.21.3.

-- 
Thomas Roessler			    <[email-masked]>

---

From: [email-masked] (Erland Sommarskog)
Date: 16 Apr 2001 12:51:51 -0000

> the leaf level of a nested MIME structure.)

Since I know very little about this signing business, I'll have to
ask some really stupid questions.

What is in multipart/signed? Is it just the signature? Or is it any part
of the message that a human is supposed to read?

If it's only the signature, I don't have any issue with it. If it is text
intended to be read by humans, is the problem both trailing white-space
and octets >= 128? If only trailing white-space is the problem, wouldn't
be better to define an encoding that took care of those and nothing else?

What I care about in the end is that text which is supposed to be read
by humans are not mutilated.
--
Erland Sommarskog, Stockholm, [email-masked]

---

From: Florian Weimer <[email-masked]>
Date: 16 Apr 2001 20:01:04 +0200

> ask some really stupid questions.

I'm sorry but I've already replied on the USEFOR list only.  The
discussion will be continued over there. :-/

-- 
Florian Weimer 	                  [email-masked]
University of Stuttgart           http://cert.uni-stuttgart.de/
RUS-CERT                          +49-711-685-5973/fax +49-711-685-5898

---

From: Brad Templeton <[email-masked]>
Date: Mon, 16 Apr 2001 12:51:55 -0700

> of the message that a human is supposed to read?

Mulitpart/signed, while probably the best of the singature forms, is not
suitable to USENET.  It only signs the body.   Signing the body is not
just the least interesting thing we can do in USENET (99.9% of all problems
come from forged headers, not modification of bodies) it can actually
have negative value, if it leads people to think the article is "signed"
and thus can be trusted in ways that it actually can't.

---

From: Russ Allbery <[email-masked]>
Date: 16 Apr 2001 14:07:07 -0700

Brad Templeton <[email-masked]> writes:

> just the least interesting thing we can do in USENET (99.9% of all
> problems come from forged headers, not modification of bodies) it can
> actually have negative value, if it leads people to think the article is
> "signed" and thus can be trusted in ways that it actually can't.

This depends rather heavily on the context.  What you say is true for some
applications (like control messages) and not true for others (like
official announcements from some entity or another that are posted to
Usenet).  At the worst, what it means that one has to include the
important information that should be authenticated by the signature, such
as the date and the author, in the body of the message.  Quite frequently
for things like announcements this is done as a matter of course anyway.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Brad Templeton <[email-masked]>
Date: Mon, 16 Apr 2001 14:23:57 -0700

> for things like announcements this is done as a matter of course anyway.

Yes, but manual verification of signatures is of dubious value.  For
digital signature authentication to work properly, it is unfortunately
necessary that all messages in a class be signed, and that the presence
of an unsigned or improperly signed message be a major anomaly which gets
a fair bit of attention, or which is in fact forbidden.

If newsgroups allow both signed and unsigned messages, then it is trivial
to cause trouble by simply not signing your unsigned messages.  If a given
user signs all his messages, it is not even sufficient for the software to
be able to warn that a user who normally signs all his messages has not
signed one, because the forger can play any number of tricks, including
simply posting a message "From: [email-masked]" which looks like you to
the non-careful eye, or worse "From: [email-masked]" which *is*
you but the software is unlikley to trap the warning that all messages
from this address should be signed.  Or [email-masked]
or many other forms which point to you.

Of course you're even assuming something outside the multipart/signed spec,
that the software is checking the From line at all, based on parsing it
somehow out of the body.

People have been able to sign article bodies for some time.  It's not widely
used and it's really not very useful.   It would be a poor intermediate
step because it's the wrong problem, and can lead to problems as well
as limited benefit.

All signing bodies does, without other checks, is allow me to prove after
the fact that I really sent a message.  It doesn't prove that I _didn't_
send it, for I am free to post unsigned messages which I can later disclaim.
It also allows those who know my key to verify I sent a message, if they
check.

I've never been called upon to prove I sent a message.  I don't recall ever
seeing this for anybody else either.   I have seen cases where people wonder
if a person really sent a message, of course, but I've also seen a lot of
forgeries that were so blatantly obvious that nobody of the sort who would
check signatures would ever be fooled, and yet people are fooled.

What you need is a system that shows that messages are from who they say
they are from, and makes a big deal when they aren't.  If 90% of messages
start having a "The sender of this message could not be authenticated"
then people simply start ignoring the warning, and it becomes of limited value.

---

From: Russ Allbery <[email-masked]>
Date: 16 Apr 2001 14:42:51 -0700

>> This depends rather heavily on the context.  What you say is true for
>> some applications (like control messages) and not true for others (like
>> important information that should be authenticated by the signature,
>> such as the date and the author, in the body of the message.  Quite
>> frequently for things like announcements this is done as a matter of
>> course anyway.

> Yes, but manual verification of signatures is of dubious value.

That seems to be a non-sequitur.  It's certainly possible to automatically
verify a multipart/signed message.  Some news readers and many mail
readers already do this.

> For digital signature authentication to work properly, it is
> unfortunately necessary that all messages in a class be signed, and that
> the presence of an unsigned or improperly signed message be a major
> anomaly which gets a fair bit of attention, or which is in fact
> forbidden.

You're solving a different problem than I'm talking about.  For occasional
use for important announcements, the multipart/signed protocol works just
fine.  I understand that you're trying to solve the problem of fully
authenticated message streams, which I agree is an interesting theoretical
problem.  I don't expect to see such a thing show up on Usenet soon,
however, since most people currently using Usenet don't really have much
use for it.

*shrug*  I know you care more about it than I do, and I may be
underestimating the demand.

> People have been able to sign article bodies for some time.  It's not
> widely used and it's really not very useful.

It does, however, have the advantage of actually being deployed, working,
and standardized.  The sort of signature system that you're describing
would be more useful for some applications than multipart/signed, I agree,
should someone eventually write it and standardize it.

Making it work well within MIME would be quite difficult, as we've already
established in some of the previous rounds of discussion on that topic.

---

From: Brad Templeton <[email-masked]>
Date: Mon, 16 Apr 2001 16:20:37 -0700

> verify a multipart/signed message.  Some news readers and many mail

If all the software does is include a string about the verification, that
is then manually checked, it's manual verification.    Humans ignore
warnings which happen frequently.  Thus if most (or even just a lot, like
a few percent) of otherwise valid messages have the warning, people learn
to ignore the warning.

> authenticated message streams, which I agree is an interesting theoretical

What are these "important announcements" though?  As I indicated, I mean
that all the messages in a "class" have to be signed, so that it becomes
worthy of note when a message in a class is not signed, and this can cause
a warning that will be paid attention to.

So what is this class called "important annoucements" and how is it defined,
and how does the software detect it, and know when an important annoucement
comes that is not validly signed, so that it can warn me about it?

If you just mean that "before you say something important" be sure to sign
it, again the value of that is limited, though not zero.  It means that
people who are in the know, after reading something of some importance
(whatever that is) would then look to see if it is signed, or have their
brain pay attention to the "this is not signed" warning which appears on
almost all messages.

I would rather solve the real problems the net faces with security,
which are unauthorized control messages (and the resulting lack of trust
in authorized control messages), most of all cancel,  and large scale spam.

Don't get me wrong, I think bodies should be authenticated, but the value
from it, at least in today's net, is minimal, and the backlash over it,
namely a lot of mime stuff most readers don't know how to handle tacked on
the end of messages, far worse than the benefit.

We will eventually need to put all that mime stuff in, and it will annoy
a lot of users (we've all seen this) but we should do it for something
really useful, not mostly ignored signature of bodies.

---

From: Russ Allbery <[email-masked]>
Date: 16 Apr 2001 16:50:48 -0700

>> That seems to be a non-sequitur.  It's certainly possible to
>> automatically verify a multipart/signed message.  Some news readers and
>> many mail readers already do this.

> If all the software does is include a string about the verification,
> that is then manually checked, it's manual verification.  Humans ignore
> warnings which happen frequently.  Thus if most (or even just a lot,
> like a few percent) of otherwise valid messages have the warning, people
> learn to ignore the warning.

It seems reasonably easy to warn if the identity associated with the
signature doesn't match the displayed identity derived from the From
header, but I haven't thought a great deal about the user interface
portion of this.

> worthy of note when a message in a class is not signed, and this can
> cause a warning that will be paid attention to.

There are some announce groups where all the content is signed.
comp.os.linux.announce, for example.  There are some people who always
sign all of their posts.

> in authorized control messages), most of all cancel, and large scale

Yes, I agree.  However, no one is asking you to solve any of these other
problems.  This thread started with the OpenPGP folks asking us to please
not arbitrarily mangle their existing standard.  That seems rather
reasonable to me.  We don't have to do any work at all beyond not
interfering with their (existing!) standard.

> really useful, not mostly ignored signature of bodies.

So it's your position that this standard should *not* adopt MIME?

To repeat myself (again), I feel very strongly that adopting MIME is a
binary switch.  Either we adopt MIME or we don't.  The time to try to
fiddle with MIME was when MIME was being developed, and should have been
done as part of a different working group than this one; at this point,
it's just Not Invented Here syndrome.  Either netnews should continue to
track the mail standards (which I would argue should be the *default*
since that's been the policy behind netnews from the beginning), or we
should explicitly say that we're not adopting MIME, but trying to grab
bits and pieces creates exactly the kinds of internal inconsistencies that
are being pointed out in this thread.

If we want multipart messages, well-labelled character sets, the ability
to transfer non-text messages without unlabelled structural content like
uuencode, and the ability to take advantage of all of the work that the
(quite a bit larger, quite a bit better-funded, and quite a bit better at
developing and deploying new technology) mail community is doing, then I
think we should simply say that the MIME standard also applies to netnews
messages and then pick up the pieces, if any, within the framework already
laid out for extending MIME.  The current draft leans more in that
direction than in the direction of not adopting MIME, but currently in
some places is still trying to balance on the top of non-existent fences.

And lest people worry too much about some portions of that, let me point
out that (a) Usenet is not the only thing that netnews is used for and the
standard should be usable for things other than Usenet, and (b) it's still
perfectly reasonable for individual hierarchies to have their own policies
about acceptable message body formats above and beyond what the standard
*allows* for.  If they want to try to figure out how to get message/signed
working without QP, that's their lookout; it's a heck of a lot easier to
change hierarchy policies than it is to change an RFC.

---

From: Brad Templeton <[email-masked]>
Date: Mon, 16 Apr 2001 17:16:00 -0700

> header, but I haven't thought a great deal about the user interface

Indeed that's good.  That's header verification, which is what I'm
pushing for, though you should verify more than just the From header.

There are a variety of ways to do this.  The right "mime" way is ugly,
namely to embed another multipart in the 1st half of the mult/signed,
which in turn has the body and a representation of the header.

I have advocated (and the multipart/signed people I have talked to have
agreed, saying they should have done this in the first place) that the
multipart/signed form allow more than 2 parts, with semantics in the final
signature part controlling just how the signature is checked on parts
1...N-1.   USENET could be the first to include this extension.

If there were a body issuing certificates for email addresses, that could
be used to verify just the from line or other email lines, but again,
why stop there.

Somehow, somewhere in the body, a hash of the valid header needs to be
provided.   Putting it in the text is ugly, but is the only way the
existing multipart/signed spec would verify it.

The best option is probably to extend the spec.

> comp.os.linux.announce, for example.  There are some people who always

A group where everything is signed can be secured, if the software is
then programmed to reject or give warnings on the rare unsigned messages.
(ideally relays should just not carry them in such a group, just as they
in theory reject unauthorized postings in moderated groups.)

However, support for "people who signs all their posts" is the hard issue.
You need to spec a lot of extra stuff to make this work, somehow building
databases of people who are in this state, and getting software to use those
databases.

You must have a certificate system, because you can't just put this flag on
a user once they sign their first post, as the first one might be forged,
thus locking the user out from posting under his own email.

But even then, as I have indicated, it's easy for me to post as
"[email-masked]" even if you are the only one who can post
as "[email-masked]".   It would be tough for you to figure out every
form of your name and reserve it.

> So it's your position that this standard should *not* adopt MIME?

It should absolutely adopt MIME, and should make it a SHOULD, possibly
even a MUST.  At the same time, non-conformant readers will exist for
quite some time, so we must for some years be aware of the older readers,
and decide how we will _use_ MIME, comparing the tradeoff for those users
with the benefit of it.

Securing headers is well worth the ugly display to users of old newsreaders.
Signing bodies is less convincing a sell.

> fiddle with MIME was when MIME was being developed, and should have been

MIME is meant to be extended, and major users of it are free to develop
extensions and try to get them adopted outside their space.

I fully agree we should throw that binary switch, but I also face reality.
I've seen newsgroups where people post with wasteful stuff, for no benefit
to most of the readers, and it gets a lot of antipathy.

Mime multipart/alternative was created so MIME systems could go through
evolutions, but it's a very bulky system, more suited to mail than news,
and so when it first started being used by Netscape, people were screamed
at to go away.  It's a little more accepted now but it took a lot.

That was, in part, why I pushed the concept of out of band markup, which
is a better migration to rich text than multipart/alternative. 

Note that at this time, the vast majority of USENET users have a reader
that handles not just MIME, but even HTML.

Still, some things are worth it and some are not.   Non-english
character sets, hyperlinks, structure, and securing headers are all worth
it. 

Signing bodies is something we've seen for a while, but I have yet to
see it actually be useful to anybody.

---

From: Derek Atkins <[email-masked]>
Date: 16 Apr 2001 21:29:55 -0400

You might want to take a look at draft-ietf-impp-msgfmt-??.txt (please
forgive me for not knowing the current revision number).  This draft
tries to describe a MIME format for encapsulating a MIME message with
it's associated meta-content (header information).  The purpose of
this draft was to allow a sender to sign a message and associated
message headers, and then allow a transport system to send the message
from the sender to the recipient(s).

Using MSGFMT does imply some duplication of headers if your transport
system requires them (e.g. IMPP, SMTP, etc).  But if you don't need
the transport-layer headers, you can stuff them into the MSGFMT part
and sign them.  Unfortunately this would be an incompatible change
with the current netnews protocols.

But I figured I'd point out that there are efforts to try to unify the
message/message-meta-data dichotomy.

Enjoy!

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       [email-masked]                        PGP key available

---

From: Per Abrahamsen <[email-masked]>
Date: 19 Apr 2001 16:02:04 +0200

This discussion seem to have degenerated into

---

From: Per Abrahamsen <[email-masked]>
Date: 19 Apr 2001 16:12:50 +0200

> This discussion seem to have degenerated into 

Oops, I hit the send key too soon.  

Basically, I agree USEFOR should not attempt to overrule RFC 2015.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 23 Apr 2001 10:18:30 +0100

> Basically, I agree USEFOR should not attempt to overrule RFC 2015. 

Having thought about this, I agree.

We should not be modifying MIME in any shape, size, or form. Just using it.

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:  +44 20 8371 1138
Internet Expert     | Home:  <[email-masked]>  | Fax:  +44 20 8371 1037
Demon Internet      | WWW: http://www.davros.org | DFax: +44 20 8371 4037
Thus plc            |                            | Mobile: +44 7973 377646