Subject: Semantic header (field) Content

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 6 Jul 2005 19:07:15 GMT

I am now reviewing the definition of semantic content that I proposed when
revising the mechanisms for constructing References headers. I want to
make the definition as simple as possible, consistent with its achieving
the intended effect. I have come across two cases where I need some
guidance.

1. There are some discouraging remarks concerning the ancient practice of
using a Subject beginning with "cmsg" to be interpreted as a control
message (where there is not a genuine Control header as well, of course).

For this purpose, does the discouragement need to include the case where
the "cmsg" is hidden inside an encoded-word? I would presume not, since I
doubt any existing software which tries to interpret "cmsg" in that way
was smart enough to detect it when inside an encoded-word.

2. A followup agent is permitted (MAY) to prepend a "Re: " to the Subject
if there is not one there already. Is it to be permitted to enclose that
"Re: " within an encoded-word (I would hope not, because there is no
benefit in doing so, and it is not done in current practice AFAIK). More
to the point, however, is whether it is expected to decode an initial
encoded-word before detecting whether an initial "Re: " is there already
(I would not expect that to happen often - has anyone ever seen such a
thing - but you never know).

Essentially, I can get either interpretation by saying that the "semantic
content" is what you get after/before decoding any encoded-words.

Note that USEAGE says you SHOULD decode encoded-words before trying to do
anything clever with a Subject header, but that is just a matter of being
prudent - we don't necessarily want to write that into USEPRO.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Wed, 06 Jul 2005 12:21:33 -0700

Charles Lindsey <[email-masked]> writes:

> 1. There are some discouraging remarks concerning the ancient practice
> of using a Subject beginning with "cmsg" to be interpreted as a control

> the "cmsg" is hidden inside an encoded-word?

No.

> 2. A followup agent is permitted (MAY) to prepend a "Re: " to the
> Subject if there is not one there already. Is it to be permitted to
> enclose that "Re: " within an encoded-word (I would hope not, because
> there is no benefit in doing so, and it is not done in current practice
> AFAIK).

My reading of RFC 2822 indicates no.

> More to the point, however, is whether it is expected to decode an
> initial encoded-word before detecting whether an initial "Re: " is there
> already (I would not expect that to happen often - has anyone ever seen
> such a thing - but you never know).

I don't think it should be required, although an implementation should be
allowed to do so if it wishes to, I think.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 07 Jul 2005 04:12:27 +0200

Charles Lindsey wrote:

> I am now reviewing the definition of semantic content that
> I proposed when revising the mechanisms for constructing
> References headers.

Better stay away from this idea, all you need is CFWS => FWS
in this case.

> does the discouragement need to include the case where
> the "cmsg" is hidden inside an encoded-word? I would presume
> not

ACK, RfC 1036 is much older than RfC 2047 and its predecessors.
Dito the "Re: " business.  In s-o-1036 it's IMO obvious that
"Re: " and "cmsg " are meant as is, not in MIME-encoded forms.

> Essentially, I can get either interpretation by saying that
> the "semantic content" is what you get after/before decoding
> any encoded-words.

Any semantics important for the transport / injection normally
works without decoding MIME-encoded words.  That's AFAIK the
point of MIME, it still works if all agents between the end-
points have no idea what it means.  So they also don't decode
or encode header fields, only a gateway could pull this stunt.

                         Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 7 Jul 2005 11:54:03 GMT

>> enclose that "Re: " within an encoded-word ...

>allowed to do so if it wishes to, I think.

OK, I agree with all that (and AFAIR that was our position when we last
discussed it - i.e. we agreed that the only mention/advice should go into
USEAGE). Frank seems to agree.

So that is what I shall do unless I hear loud screams otherwise.

---

From: Bruce Lilly <[email-masked]>
Date: Thu, 7 Jul 2005 13:31:36 -0400

> was smart enough to detect it when inside an encoded-word.

1. "[RFC4096]" covers most of the issues
2. A remaining issue is that the current specification (RFC 1036)
  *requires* interpretation as a control message.  We ought to point
  out that since 1036, that has been found to be problematic (citing
  specifics) and that best *current* (but unpublished) practice has for
  some time been to ignore it, and that the document specification
  continues that practice.

> benefit in doing so, and it is not done in current practice AFAIK).

Another falsehood:
1. You have maintained that it is Latin, and best current practice
   (BCP 18, RFC 2277 for Frank) is to indicate language, and RFC 2047
   is the standard method for doing so.  Particularly where the remainder
   of the subject field is *not* Latin, which is the vast majority of
   cases.  That is a clear benefit.  Of course, if you would like to
   retract the "it's Latin" assertion, we can dicuss the implications of
   that change in your position.
2. It has in fact happened in messages here, and you know that to be
   the case because it has been discussed here.
3. Treating some string in an unstructured field as if it were a keyword
   imposes structure on that field; we have had that discussion and the
   WG consensus was that the field is strictly unstructured.
4. There is not any reason to restrict what is put in an "only human-
   readable", "unstructured" field;
   Subject: =?us*la?q?Re:_?= Re: =?us*en?q?_the_note_between_?= Do & Mi
   Subject: =?us*la?q?Re:_?= Re: =?us*en?q?_element_number_?= 75
   Subject: =?us*la?q?Re:_?= RE: Re: =?us*en?q?_The_note_between_?= Do & Mi
     =?_according_to_?= Raoul Edwards
   Subject: cmsg was a really bad idea
   Subject: =?us*en?q?cmsg_was_a_really_bad_idea?=
   Subject: =?us*la?q?Re:_?= =?us*en?q?cmsg_was_a_really_bad_idea?=
   Subject: =?us*la?q?Re:_?= RE: Raoul Edwards
     =?us*en?q?_says_cmsg_was_a_really_bad_idea?=
   are perfectly valid and reasonable.
 
> More 

As already mentioned, it has happened here and has been discussed here.
Also:
5. The only reason to "detect" anything in an unstructured, only
   human-readable field is to detect an encoded-word, and the only
   reasons to do so are for display and for non-protocol, out-of-scope
   issues where treatment as displayed might be appropriate such as
   killfile processing and sorting by subject.

> content" is what you get after/before decoding any encoded-words.

In neither case do we need any such term.
  
> prudent - we don't necessarily want to write that into USEPRO.

Since by definition ("only human-readable content") unstructured fields
play no role in protocol (as we have agreed by consensus), not only do we
not want to write any such thing in USEPRO, we cannot do so because that
is a protocol document, and there is no protocol issue.

---

From: Seth Breidbart <[email-masked]>
Date: Thu, 7 Jul 2005 14:08:37 -0400 (EDT)

Bruce Lilly <[email-masked]> wrote:

["Re"]

> 1. You have maintained that it is Latin,

It is Latin.  It has been borrowed into English and some other
languages as well.

Seth

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 8 Jul 2005 11:26:10 GMT

>So that is what I shall do unless I hear loud screams otherwise.

I asked for a small clarification regarding the handling of "Re: ". We
discussed the substantive issue ad nauseam last summer and reached a
compromise (Seth's 4 points) that all the participants bar two signed up
to. I therefore regard that whole matter as now closed, and do not intend
to engage in further discussion of it.

So, as regards that one clarification I asked for, the text I now have is
as follows:

   The "semantic content" (often abbreviated to just "content" when the
   context is clear) of a header field is its semantic interpretation;
   i.e. what remains after unfolding it and removing its field name with
   its colon and any leading and trailing whitespace and, in the case of
   structured headers only, ignoring comments and other semantically
   invisible items and replacing white space by a single SP.

That definitions works fine, with a few grammatical tweaks, at all places
in USEPRO where the term "header content" currently appears.