Subject: Suggested References texts

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 26 Apr 2005 15:12:10 GMT

I think we are now sufficiently agreed about the References header (except
perhaps for John) to write some text.

Here is what I suggest. Note that, in addition to the USEFOR text written
by Ken, I have added some preliminary wording to outline the purpose of
the References header, and some semantics text to explain the "meaning"
of the list of message-ids.

I have also fixed the syntax error (the CFWS between the <msg-id>s is
obligatory), and have continued the liberty of using <msg-id> rather than
<msg-id-core>.

There is also a tentative NOTE at the end regarding the use of the
References header in message/partial. This is a place where what RFC 2046
suggests is woefully inadequate, and that needs to be pointed out
somewhere. But maybe this is not the right place.

Share and Enjoy!

Definitions (for inclusion in USEFOR or USEPRO as appropriate).

   A "followup" is an article containing a response to the contents of
   an earlier article. It will always include a References header
   pointing to that earlier article and any other "precursors".

   An article is a "precursor" of some later article which is a followup
   to it, or which is otherwise intended to be grouped with it for
   purposes of display (e.g. as a part of a multipart posting such as a
   FAQ).

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of a followup.

Within USEFOR:

3.2.1  References 

|  The References header is used in followups, and other articles with
|  related precursors (e.g. multipart FAQs and the fragments of a
|  message/partial), to facilitate the display or retrieval, by reading
|  and other agents, of threads of related articles. It is the same as
|  that specified in Section 3.6.4 of [RFC2822] with the added
|  restrictions detailed in Section 2.2 and those listed below:

   o  The updated <msg-id-core> construct defined in Section 3.1.3 MUST
      be used.

   o  Message identifiers MUST be separated with CFWS.

   o  Comments in CFWS between message identifiers can cause
      interoperability problems, so comments SHOULD NOT be generated,
      but MUST be accepted.

|  references      =  "References:" SP [CFWS] 1*( msg-id [CFWS] )
|                     CRLF

|  The list is composed of message identifiers of precursors of the
|  current article, sorted so that no article precedes any of its own
|  precursors. It SHOULD include both the earliest and the immediate
|  precursors on the current article, even if some of the intermediate
|  ones are omitted. A given message identifier MUST NOT appear more
|  than once.
[That actually defines a partial list. Even though USEPRO makes no
provsision for followups to several articles, people are bound to try to
do it. That is a minimal wording which reduces to the usual linear list
in the case of followups to a single precursor.]
|
|  The process of generating a References header by a followup agent is
|  set out in [USEPRO].
|
|    NOTE: The suggestion in [RFC 2046] that each fragment of a
|    message/partial should contain a References header referring only
|    to the immediatelyt preceding fragment does not fulfil the
|    recommendation above.  It would therefore be better for each
|    fragment to refer to all of the preceding ones.

Within USEPRO:

7.6.  Duties of a Followup Agent

.......

   4. If the precursor did not have a References header (F-3.2.1), the
      content of the followup's References header MUST be inherited from
      that of the Message-ID header of the precursor. A followup to an
      article which already had a References header MUST have a
      References header comprising the precursor's References header
      (subject to trimming as described below) followed by CFWS and the
      Message-ID header content of the precursor.
 
|       NOTE: The two "MUST"s above ensure compliance with the
|       definition of the term "followup". They are to be contrasted
|       with the weaker recomendation using "SHOULD" applied, in [RFC
|       2822], to the generation of "replies" in email. Moreover, in
|       Netnews, there is no expectation of any In-Reply-To header in a
|       followup.

      If the resulting References header is excessively long, it MAY be
      trimmed, but the first and the last two message identifiers MUST
      NOT be removed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Tue, 26 Apr 2005 12:37:58 -0400

> perhaps for John) to write some text.

Point of order.  The chair must declare consensus.

There are problems with what you wrote, and I think that is precisely because
you misunderstand the consensus.....

> of the list of message-ids.

If USEFOR is going to be syntax only, then there is no place for "purpose of
the References" header that can be vanilla enough to be in USEFOR.

> |  restrictions detailed in Section 2.2 and those listed below:

This isn't acceptable wording at all, especially not for USEFOR.  A
news client must be able to determine "followup" by looking at an article,
or the term is meaningless and should not appear in any standards document.

---

From: Russ Allbery <[email-masked]>
Date: Tue, 26 Apr 2005 12:08:24 -0700

Forrest J Cavalier <[email-masked]> writes:

> This isn't acceptable wording at all, especially not for USEFOR.  A news
> client must be able to determine "followup" by looking at an article, or
> the term is meaningless and should not appear in any standards document.

Well, John's right about this.  If this is a requirement, then we have to
define everything that contains a References header as a followup, whether
somewhat artificially or not, since there isn't any other way to detect a
followup.

I'm not sure where this requirement comes from, though.  It seems to me
that certain concepts that cannot be detected in software are still useful
to talk about in a standard when they affect what software does in
response to common user interface commands (like "post followup").  While
I can't tell in software if a random article on the wire is a followup or
is some other case where References is useful, I *can* tell if a
particular news client does the right thing by looking at it, which to me
makes it standards territory.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 27 Apr 2005 09:50:20 +0200

Charles Lindsey wrote:

> I have also fixed the syntax error (the CFWS between the
> <msg-id>s is obligatory), and have continued the liberty of
> using <msg-id> rather than <msg-id-core>.

Good (but see below).

> what RFC 2046 suggests is woefully inadequate

> A "followup" is an article containing a response to the
> contents of an earlier article. It will always include a
> References header pointing to that earlier article and any
> other "precursors".

Strong NAK to any change of <[email-masked]>

We know that References do _not_ always include 'any other
"precursors"', in fact there's a complete chapter explaining
how to trim the References.

> An article is a "precursor" of some later article which is a
> followup to it

Strong NAK to any change of <[email-masked]>

> or which is otherwise intended to be grouped with it for
> purposes of display (e.g. as a part of a multipart posting
> such as a FAQ).

Strong NAK, the different parts of a multipart FAQ might be
related by References, but this relation is not necessarily
that of a "precursor".

A "precursor" is simply the opposite of a "followup", so if
A is the precursor of B, then B is a followup of A.  There's
no case where A is the precursor of B, but B is no followup
of A, that would be madness.

> Within USEFOR:

Strong NAK, after <[email-masked]> this issue is
settled in Usefor, we only have to fix the syntax.

>| references = "References:" SP [CFWS] 1*(msg-id [CFWS]) CRLF

Strong NAK to anything below this line.  And the syntax is not
correct, we wanted at least one WSP to separate the msg-id.

   references  = "References:" SP msg-id-list CRLF
   msg-id-list = [CFWS] msg-id *( CFWS msg-id ) [CFWS]

That's the simple variant where a comment is enough to separate
the Message-IDs as in <a@b>(c)<d@e> but adjacent Message-IDs as
in <a@b><d@e> aren't allowed.

 [Usepro]
> If the resulting References header is excessively long, it
> MAY be trimmed, but the first and the last two message
> identifiers MUST NOT be removed.

I'd go for a SHOULD here, because some news servers and tools
really hate it if the References are too long, I've seen cases
where the References were truncated <shudder />  Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 27 Apr 2005 12:04:07 GMT

>Point of order.  The chair must declare consensus.

Since when has the Chair's permission been required before a member of
this WG can suggest some text?

>the References" header that can be vanilla enough to be in USEFOR.

There are two issues here.

A. Should USEFOR including brief explanatory wording for each header to
say what its purpose is? Remember that USEFOR is semantics as well as
syntax.

B. If so, then what should be said when introducing the References header?

Dealing with A first, RFC 2822 regularly provides such explanatory
wording.

   The "Subject:" field ... contains a short string identifying the topic
   of the message.

   These three fields are intended to have only human-readable content ...

   The originator fields indicate the mailbox(es) of the source of the
   message.  The "From:" field specifies the author(s) of the message, ...

And for the References field it says:

   The "In-Reply-To:" and "References:" fields are used when creating a
   reply to a message.

   Note: Some implementations parse the "References:" field to display
   the "thread of the discussion".  ...

And USEFOR also contains similar descriptive material (though often it
simply inherits what RFC 2822 had already said):

   The Path header indicates the route taken by an article since its
   injection into the Netnews system.

   The Injection-Date header contains ...  Its purpose is to prevent the
   reinjection into the news stream of "stale" articles which have
   already expired by the time they arrive at some relaying or serving
   agent.

So there is plenty of precedent for incorporating such material.

So now to issue B:

>> |  and other agents, of threads of related articles. ...

>or the term is meaningless and should not appear in any standards document.

Let us recap the history here. It is accepted, I believe, (using Frank
Ellermann's notation)
   followup -> references header
   no references header -> not a followup
and at one time we had
   references header -> followup

But then, some months back, it was pointed out that some writers of multipart
FAQs regularly used References headers to join the later parts to the
earlier ones, and RFC 2046 suggests the use of References headers in
message/partial. And there were other examples too.

Did we want to forbid these practices, and would they actually disappear
even if we did?

Far from forbidding them, the view expressed (without dissent, so far as I
recall) was that these practices were useful, and should perhaps even be
encouraged. Essentially, anyone incorporating a References header for such
reasons was in effect inviting reading agents (those that did threading
anyway) to present the various parts of the FAQ together and in the
correct order (irrespective of their order of arrival, differences in
Subject, or order of their Date headers).

Inevitably, this changes the information that a reading agent can deduce
when it sees a References header. It can no longer deduce "this is a
followup". It can only deduce "the poster wants you to treat this like a
followup" (by threading it, or by letting you retrieve the precursors by
clicking on them, or whatever other bells and whistles you provide).

So you can no longer assert "references header -> followup". The question
is 'Does this matter?'.  I don't see why it should. Why is is necessary
for a reading agent to be able to distinguish between (conventional)
followups and multipart FAQs (note that humans can still distinguish them
easily)? In any case, if this practice continues (as it surely will),
there is nothing that can be done about it, since the References header is
the only tool available.

Now there have been suggestions to "clean it up" by artificially redefining
the term "followup" to include any article that has a References header.
But that is all "smoke and mirrors" - it makes it appear that you can
detect "followups", but in reality it makes no difference at all as to
what reading agents can or cannot actually do.

Yes, if the WG wants that revised definition, it can be done, but I do not
see the point since it would make no technical difference to what actually
happens.

---

From: "Forrest J. Cavalier III" <[email-masked]>
Date: Wed, 27 Apr 2005 13:23:36 -0400

> clicking on them, or whatever other bells and whistles you provide).

Do you agree we should be concentrating on how the reading agent can
process articles?  I think you do.  And the text you suggested is flawed
because it declares nothing of use for reading agents.

(Here is the text:)

 >> |  and other agents, of threads of related articles. ...

What does "facilitate" mean?  What does "other articles" mean?  Since "e.g."
is not "i.e.", what other kinds of articles can have References?  What stops
someone from using References for other kinds of facilitation of display or
retrieval?  (Style sheets!)

That text is useless.  It doesn't even guide posters, much less readers.

If people really want to allow FAQs and multi-part to get threaded,
I wonder if the USEFOR text should be the following single sentence, (which cleverly
does not mention followup or FAQ or multi-part.)

     A References header indicates the previously posted context in which
     the article was created.

(That "the" means that you can only have at most one context per article.)

I would then say to leave "followup" out of USEFOR, (and maybe out of USEPRO too,
appearing in USEAGE, but I am not sure how all the documents are supposed
to share duties.)

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Apr 2005 13:51:26 GMT

>because it declares nothing of use for reading agents.

Then what else can be written? If References headers are to be allowed for
"pseudo-followups" (and there seems to be agreement on that), then what
the reading agent can or cannot deduce is fixed by the structure that we
have allowed. Playing around with the definitions cannot alter that.

>retrieval?  (Style sheets!)

"Facilitate" means that they can use the information (whose meaning is
defined by the semantics further down) in any way they choose in order to
provide threading, retrieval, etc. In fact, there are well-known
algorithms for threading that work pretty well, but implementors are free
to experiment with supposedly better ones, and to some extent different
users prefer different presentations. Yes, why not "style sheets"? The
important thing is that they need to rely on the References header being
present, and being constructed according to the given semantics.

>That text is useless.  It doesn't even guide posters, much less readers.

No, the guidance comes in the semantics paragraph further down, which is
supposed to tell them what they can rely on.

>     A References header indicates the previously posted context in which

Yes, alternative texts to the one I suggested can be considered.

>(That "the" means that you can only have at most one context per article.)

But people are going to try and followup to more than one article, whether
we like it or not. Someday there may even be an agreed way to do it. Hence
my semantics was as minimal and future proof as I could make it.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Apr 2005 13:30:42 GMT

>how to trim the References.

OK. Consider the word "any" struck.

>Strong NAK, ...

>of A, that would be madness.

Whilst I do not want the term "followup" to include those
"pseudo-followups" (if I can use that term without precipitating another
rant from John :-) ), I defined "precursor" to include the
"pseudo-precursors" because it simplified the wording of the semantics
further down. If you want me to limit "precursor" as you suggest, then
please suggest alternative wording for those semantics.

The term "followup" is widely used within the Usenet community, and it
would be artificial, misleading and unwise to define it in any other way.
But "precursor" is not so widely used and would not mislead in the same
way.

>>| references = "References:" SP [CFWS] 1*(msg-id [CFWS]) CRLF

Oops! That is still wrong.

>   msg-id-list = [CFWS] msg-id *( CFWS msg-id ) [CFWS]

or, following the style in draft-13,

   references      =  "References:" SP [CFWS] msg-id 1*(CFWS msg-id )
                      [CFWS] CRLF

>in <a@b><d@e> aren't allowed.

Agreed. Those <comment>s are "MUST accept, don't generate yet", so the
issue does not arise until "yet" happens, by which time most agents will
probably be able to manage without even a WSP.

>Strong NAK to anything below this line.

So you want me to omit

But that is the semantics (meaning) of the header, which is a proper matter
for USEFOR. It tells reading agents just what they can deduce from the
header, and it guides those who generate References for
"pseudo-followups".

RFC 2822 and our earlier ARTICLE drafts got away by just providing an
algorithm for generating this header, but that is not enough if the header
is not restricted to followups/replies.

>where the References were truncated <shudder />

I think we agreed earlier that the actual trimming details belonged in
USEAGE. But please see my remark in the latest USEAGE where I would like
to keep the whole header within 998 octets (rather than 21 entries),
because some followup/reply agents unfold the old references header and
then conveniently "forget" to refold the new.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 29 Apr 2005 01:05:14 +0200

Charles Lindsey wrote:

> If you want me to limit "precursor" as you suggest, then

I just don't want any additional wording in usefor-04, it's
one of these cases where each word is too much, unless you
plan to integrate the 2004 discussions with the complete
history of this header field in RfCs, news, mail, and UAs
as told by $number_of_USEFOR_participants_since_99.  <sigh<

It's like an attempt to define spam with all exceptions and
details, it's hopeless.  You really tried.  Now let's please
just get away with the bare minimum in usefor-04, for the
technical details usepro-03 should be good enough.

> references = "References:" SP [CFWS] msg-id 1*(CFWS msg-id )

ACK, one production less, OTOH a rather long rule.  I thought
that you could use the msg-id-list also in Control: cancel or
Supersedes:, but that's not the case, our Supersedes: has only
one msg-id, and Control: has no CFWS.

> So you want me to omit
[... some text about precursors in usefor-04 ...]

Yes, that's what I didn't want because it's essentially the
same as in 2822, but now that you ask I'm not so sure about:

>| A given message identifier MUST NOT appear more than once.

Is that a problem anywhere, bad enough for a MUST NOT ?  It's
obviously stupid, but is it potentially harmful ?

> that is the semantics (meaning) of the header, which is a
> proper matter for USEFOR.

Not if it's essentially the same as in 2822.  Or that's what I
thought the plan is:  Anything in RfC 2822 (excl. obs-stuff)
that's not explicity changed in usefor-03/04 is implicitly a
part of usefor-03/04.

That's why we don't explain mailbox, addr-spec, From:, Sender:,
etc.  We only explain what's different, e.g. the msg-id syntax
or something with the Date: timezone.  And of course all header
fields found only in news, among others Followup-To:.

Unless you start again with message/partial and other oddities
I don't see where the semantics of "normal" References: is so
different from RfC 2822 that it needs an explanation.

Sure, we don't have In-Reply-To, that's a minor difference, UAs
aren't expected to derive missing References: from In-Reply-To:
But that's an issue for usepro, not usefor, like trimming etc.

> It tells reading agents just what they can deduce from the
> header

If developers of reading agents need more info they can find it
where deelopers of followup-agents find it, in usepro.  What's
so special for news reading-agents, in relation to RfC 2822 ?

Or do you want some "security considerations" about References:
"Beware, the References in news can be wild and wonderful, and
 for mail it's the same problem.  But we wanted to document it
 explicitly. so take care."  Seriously, I don't get it.

> I think we agreed earlier that the actual trimming details
> belonged in USEAGE.

Quite possible, that's something between you and Alexey, this
split still confuses me.  But at least I got the idea what's
usefor-xx and what not:  usefor-xx is only the article format.

Not all why-s, just the minimum to describe what's actually
transmitted.  Form but not function.  Plus basic definitions.

                       Bye, Frank

---

From: Russ Allbery <[email-masked]>
Date: Thu, 28 Apr 2005 16:36:49 -0700

> as told by $number_of_USEFOR_participants_since_99.  <sigh<

Yup.

> But that's an issue for usepro, not usefor, like trimming etc.

The only thing that I can see mentioning, given this whole discussion, is
saying that References is required for followups, although I don't think
we should say much more than RFC 1036 does in that area.  For the rest, I
agree with you wholeheartedly.

I *don't* want to give the impression, even in USEFOR, that all the rules
in RFC 2822 for replies apply unmodified to followups.  Replies and
followups *aren't* the same thing, and I think it's worth making that
clear.  Among other things, correctly constructing a followup requires
dealing with fields that aren't in RFC 2822 at all.

>> I think we agreed earlier that the actual trimming details

If we want to strongly encourage (SHOULD) a particular trimming strategy,
that goes into USEPRO as it's part of the protocol.  Given the number of
bad trimming strategies that people use that cause interoperability
problems, I think this is probably a good idea.

> transmitted.  Form but not function.  Plus basic definitions.

Yup.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Apr 2005 15:14:24 GMT

>technical details usepro-03 should be good enough.

But usepro-03 doesn't provide those technical details.

It tells you how to construct a followup to an article that is already in
front of you (and RFC 2822 does the same for email replies, but is a bit
more complicated because it has to allow for cases where References
headers were not provided - that "SHOULD" - and it deals with In-Reply-To
as well).

But suppose you are a multipart FAQ writer trying to use References to
refer back to the earlier parts, how do you know what you can do? Or you
are writing a threading agent. How do you know what you can rely on in a
properly formed References header?

If all you are given is the bare syntax (which is what you seem to
suggest), then what is to stop you including <msg-id>s of articles that
never existed? What is to stop you listing <msg-id>s in the wrong order?
Where is the wording to say these things are wrong?

Now indeed, most FAQ writers are clueful enough to construct sensible
References headers, but in fact they are merely relying on "folklore" -
and the whole purpose of standards is to replace folklore by somethng with
more precision. Or else they have to "reverse engineer" the algorithm
given for followups in USEPRO, and deduce the underlying semantics from
that (with the danger of reading more into the algorithm than is strictly
justified). Likewise for those who write threading agents.

>obviously stupid, but is it potentially harmful ?

Yes, that sentence might go.

>that's not explicity changed in usefor-03/04 is implicitly a

But RFC 2888 doesn't explain the _meaning_ of the References header. It
just tells you how to contruct one when creating a Reply, which is not
what we want to use it for.

>etc.

Indeed, because RFC 2822 explains all that is needed for those.

>different from RfC 2822 that it needs an explanation.

Exactly. It is the "other oddities" which necessitate some extra
explanation.

>so special for news reading-agents, in relation to RfC 2822 ?

They are going to encounter References headers in articles that are not
followups (or replies in the 2822 sense). So there is nothing in USEPRO to
help them (except guesswork based on what followup agents do).

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Apr 2005 15:18:12 GMT

>dealing with fields that aren't in RFC 2822 at all.

Eh? Which fields might those be?

---

From: Russ Allbery <[email-masked]>
Date: Fri, 29 Apr 2005 09:28:10 -0700

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I *don't* want to give the impression, even in USEFOR, that all the
>> rules in RFC 2822 for replies apply unmodified to followups.  Replies
>> and followups *aren't* the same thing, and I think it's worth making
>> that clear.  Among other things, correctly constructing a followup
>> requires dealing with fields that aren't in RFC 2822 at all.

> Eh? Which fields might those be?

Followup-To and Newsgroups.

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 30 Apr 2005 02:44:16 +0200

Charles Lindsey wrote:

> suppose you are a multipart FAQ writer trying to use
> References to refer back to the earlier parts, how do
> you know what you can do?

There are many possible and correct ways to do this, and
they are all obvious.  It's 2005.  If somebody posts a
multipart FAQ in an NG where I don't like it I'm tempted
to report it as net abuse.

Please no UUCP arguments, and of course there's nothing
wrong with reading news offline, but URLs really exist
in this millennium.  Too many "FAQs" are in fact only
auto-posted propagada from self-appointed authorities.

> Or you are writing a threading agent. How do you know
> what you can rely on in a properly formed References
> header?

There will be two documented ways to create it, 2822 and
Usepro-xx, so I'd assume that that's "properly formed".

I'd test this assumption with some real References, find
that it's generally true, and use it.  Minimal testing
would convince me that a reading agent does not always
"see" all referenced articles.

Maybe I'd try a RTL strategy to find a visible precursor.

If that doesn't work I'd try to attach the article to an
existing thread with another strategy, this could be a
set operation on known Message-IDs.  If it still doesn't
work it's a new thread, e.g. the result of a Followup-To
from an invisible NG.

That takes care of anything with References.  The other
case, no References, is also simple, some regexp on the
Subject (Re: etc.) with a plausibility test on the Date.

If it's an UA also working for mail I'd add In-Reply-To
as another source for this algorithm (details specified
in 2822), or maybe I'd do this anyway.

There are several variants like allowing threads to be
broken by new subjects, but I'd stay away from it unless
my algorithm to compare Subjects supports 2231 / 2047.

> what is to stop you including <msg-id>s of articles that
> never existed?

Certainly irrelevant for a reading agent, it cannot know
whether referenced articles are invisible or fictitious.

For the author of a multipart FAQ it could be an idea to
base his parts on a fictitious root.  And after the tenth
question about an apparently lost root he knows that it's
a bad idea.  But in theory it's "allowed", a reading agent
crashing in this situation is broken.

> What is to stop you listing <msg-id>s in the wrong
> order?

Solved by induction, there's essentially one documented
way to create a 2822-reply or news-followup, the result
is "properly formed".  Even if the older References were
garbage, the last could be enough to find the precursor.

If not it's a case of garbage in garbage out, and there's
nothing Usepro-xx or one of its many agents can do about
it.  Okay, they should remove / ignore syntactical garbage.

> most FAQ writers are clueful enough to construct sensible
> References headers, but in fact they are merely relying
> on "folklore"

What's so difficult with these sets and trees ?  If I cannot
guarantee date( part( n+1 )) > date( part( n )) then I should
simply add part( n ) to the References in part( n+1 ).

That's no urban legend or rocket science, it's common sense.
And Usenet would survive it if I get wrong and let all parts
only reference part( 1 ) or other dubious ideas.

> they have to "reverse engineer" the algorithm given for
> followups in USEPRO, and deduce the underlying semantics
> from that

There are several more or less working algorithms.  What
you're proposing is to define THE algorithm.  But we know
that there is no single valid algorithm, or rather that we
are unable to agree on whatever single algorithm.

> RFC 2888 doesn't explain the _meaning_ of the References
> header.

No surprise if I look into the 2822 credits.  Was it exactly
the same stalemate as here with the same combattants ?  At
least 2822 mumbles something about "threads", good enough.

And it also doesn't forbid a more literal interpretation of
"reference".

But OTOH it doesn't mention this, if that's a clue I got it.

                            Bye, Frank

---

From: "Ruud H.G. van Tol" <[email-masked]>
Date: Sat, 30 Apr 2005 09:18:29 +0200

>    references      =  "References:" SP [CFWS] msg-id 1*(CFWS msg-id )

I suppose that was meant as

   references      =  "References:" SP [CFWS] msg-id *( CFWS msg-id )
                      [CFWS] CRLF

(changed the '1*' into '*'; inserted a space)

-- 
Grtz, Ruud

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Cc: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Sat, 30 Apr 2005 09:18:29 +0200
Organization: Chaos rules.
Sender: [email-masked]

>    references      =  "References:" SP [CFWS] msg-id 1*(CFWS msg-id )

I suppose that was meant as

   references      =  "References:" SP [CFWS] msg-id *( CFWS msg-id )
                      [CFWS] CRLF

(changed the '1*' into '*'; inserted a space)

-- 
Grtz, Ruud

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  2045 ABNF (was: Again broken)
Date:  Sat, 30 Apr 2005 05:14:59 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 71
Sender: [email-masked]

Charles Lindsey wrote:

>> So far I didn't know that usefor-03 implicitly includes
>> all 2045 terms.  I only knew that it includes 2822 minus
>> obs, and of course 2234bis.

> It needs to be stated somewhere.

It is, I was blind.  But we cannot use this pseudo-ABNF, it has
to be fixed for the three (?) 2045 terms needed in usefor-xx:

value, token, parameter, anything else ?

> It even appeared (but not normatively) in my old draft-13.

Yes, the "collected ABNF" was nice, but I never checked all
the details in 2003 for draft 7 and its "last last last call".

Fortunately Bruce was an inspiration how to get these details
right, but that was later from my POV.

1) value = token / quoted-string

No "value =" in 2822, no "value :=" in 2231, so that should
be no problem as soon as you have a valid "token" elsewhere.

2) parameter = attribute "=" value

Trouble, "parameter" is redefined in 2231.  While I'd love to
nullify 2231 that's not excatly realistic.  Maybe we could
nominate 2231 by a joint agreement of LTRU and USEFOR as the
most painful RfC. :-(  Better remove the one "archive-param".

2a) attribute = token

Same problem as with "parameter", but you only (?) need it
indirectly for "archive-param".  In theory it's very easy to
translate the 2231 gibberish to proper ABNF:

  attribute      = 1*attribute-char
  attribute-char = <any (US-ASCII) CHAR except SPACE, CTLs,
                     "*", "'", "%", or tspecials>

A proper subset of US-ASCII, "tspecials".defined in 2045, and
2045 took it from STD 11.  But not the same as the "specials"
in 2822.  I need some ASCII-subset tools. <sigh>

3) token = 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                 or tspecials>

Okay, that's the same as attribute-char plus "*", "'", "%".

> parameter = [CFWS] token [CFWS] "=" value
> value     = [CFWS] token [CFWS] / quoted-string

Where have you found these [CFWS] ?  Because 2045 mentions 822
and its comments, and 2822 claims to update 822 ?

> But I was forbidden from having that stuff incorporated into
> USEFOR.

Do you have a Message-ID for this obscure decree ?  I see the
potential problem with the optional CFWS, but otherwise your
"value" is just a copy from 2045.  Okay, "parameter" is a bad
case of 2231, see above.

> as a result, USEFOR does not adequately explain where [CFWS]
> can appear in Injection-Info.

Okay, then let's reinvent the wheel, no 2231 and no CFWS.  Bye

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  CFWS (was: Again broken)
Date:  Sat, 30 Apr 2005 03:11:26 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Charles Lindsey wrote:

> it ought to be discussed

Kill all CFWS in pre-2045 news-only-headers where it doesn't
contradict 2045 / 2822.  For new headers like User-Agent or
Injection-Info adopt common practice or 2045 / 2822 style.

I know that it's not what the WG once wanted, but you asked.

                     =80 0.02, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Again broken
Date:  Sat, 30 Apr 2005 03:32:34 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Charles Lindsey wrote:

> OK, if you don't want it fixed, let it go.

I've proposed many ways to fix it, e.g. by clear names for
the msg-id productions, or the last proposal was this note:

| Note: id-right is a domain, and no-fold-literal is the
| special case of a domain-literal (a.k.a. address-literal,
| see RfC 2821)

Plus a general note about avoiding domain-literal whereever
possible, i.e. the same idea as in STD 11.

> But just using <domain> in place of <id-right> will not
> fix it either, without further verbiage.

IBTD.  If the msg-id is broken in relation to 1036 / s-o-1036
usefor-xx MUST NOT fly as far as I'm concerned.  Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Suggested References texts
Date:  Sat, 30 Apr 2005 02:44:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 107
Sender: [email-masked]

                            Bye, Frank

Date: Fri, 29 Apr 2005 12:47:38 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Again broken
Sender: [email-masked]

Charles Lindsey wrote:
> If the WG wants this change, then it ought to be discussed first.

There you go again.

Charles, the only meaning of "change" that makes sense is a change
from current published standards and practice. I believe you have once
again used it to mean "an edit to the draft."  Please stop it.

If something appeared in the draft that was never discussed in the WG,
it ought to be removed.

Removing such things do not need to be discussed first, because removing
them is not a change, having them in is a change.

The draft is fluid, it isn't a big deal to delete and edit words.

Modifying servers to parse comments in control headers is a big
deal.

No matter what ideas of future scenarios anyone can conjure up, unless it
is a current request, or existing practice, or a forseeable need (not
a "would be nice.") it ought to be cut from the drafts.  It is outside
the charter.

To: [email-masked]
Subject: Re: Obsolete vs. obsolescent
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 29 Apr 2005 09:32:28 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> But evidently Frank doesn't want that to be said (he is complaining
> about current systems that object when you try to do it), and I don't
> think I want to do it either. Better just to let the usage die away
> (which USEPRO requires).

That isn't going to actually happen in any sort of reasonable time frame.

> Otherwise we shall find new software coming on stream that objects to
> 'cmsg' in Subjects (just to comply with the new USEFOR) just as the last
> remaining servers that might have interpreted that 'cmsg' are being
> withdrawn.

I wonder if any of us will live long enough to see the last remaining
servers that interpreted cmsg be withdrawn.

> How likely is it that people will 'accidentally' write Subject headers
> starting with 'cmsg'?

This doesn't seem like a relevant question to me.  Standards aren't
concerned only with things that can happen accidentally.

> How much of a problem are such headers causing at the present time?

They aren't propagated.  They have huge interoperability problems at the
present time.

> And if they do write such a header (without any intent to cause a
> Control action), how likely is it that it will be meaningful enough to
> cause some untoward effect?

That's not the breakage that I'm worried about (control messages should be
authenticated anyway instead of cancels, and it's pretty hard to write a
cancel control message accidentally).  The article will mysteriously
disappear, which is a different kind of breakage.

> If it ain't actually broke at the present time, then don't waste time
> trying to fix it.

It's very broken at the present time.

To: [email-masked]
Subject: Re: Suggested References texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 29 Apr 2005 09:28:10 -0700
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:20786
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Fri, 29 Apr 2005 15:14:24 GMT
Lines: 100
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:20788
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Obsolete vs. obsolescent
Date: Fri, 29 Apr 2005 15:33:26 GMT
Lines: 53
Sender: [email-masked]

>> Okay, but it's not nice when it has serious problems with the 
>> propagation.  An error message "administrative restriction"
>> is also not exactly clear.  Or if we'd expect gateways to do
>> something special, or better nothing unless they are forced
>> to bypass an "administrative restriction" we have to say so
>> somewhere.  Not in usefor-x, that was probably a bad idea.

>A subject starting with "cmsg " needs to be prohibited in anything that
>isn't a control message.  I don't feel strongly about whether that should
>be in USEFOR or USEPRO, although to me it feels like a format issue, but
>it needs to be in there somewhere.

If you want to say that, then USEFOR is clearly the right place to say it.

But evidently Frank doesn't want that to be said (he is complaining about
current systems that object when you try to do it), and I don't think I
want to do it either. Better just to let the usage die away (which USEPRO
requires). Otherwise we shall find new software coming on stream that
objects to 'cmsg' in Subjects (just to comply with the new USEFOR) just as
the last remaining servers that might have interpreted that 'cmsg' are
being withdrawn. Eventually, we shall have wide deployment of a mandatory
requirement for Subject header which is there to solve a problem that no
longer exists.

>  It's premature to assume that cmsg no
>longer receives special handling; we're not done with that transition.  We
>might never be, frankly.  Using cmsg in the Subject header of regular
>posts will cause significant interoperability problems.

How likely is it that people will 'accidentally' write Subject headers
starting with 'cmsg'? How much of a problem are such headers causing at
the present time? And if they do write such a header (without any intent
to cause a Control action), how likely is it that it will be meaningful
enough to cause some untoward effect? If it ain't actually broke at the
present time, then don't waste time trying to fix it.

But if the WG really wants this prohibition including, then let it say so,
in which case USEFOR is the place to do it.

To: [email-masked]
Xref: clerew local.usefor:20790
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Again broken
Date: Fri, 29 Apr 2005 16:08:38 GMT
Lines: 56
Sender: [email-masked]

>> <value>, <token> and <parameter> are all defined in RFC 2045

>Yes.  So far I didn't know that usefor-03 implicitly includes
>all 2045 terms.  I only knew that it includes 2822 minus obs,
>and of course 2234bis.

It needs to be stated somewhere.

>2045 is older than 2234.  IMHO it's better to copy (and check)
>what you need, instead of implicitly importing it.  Checking:

>     token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
>                or tspecials>

>Whatever that is, it's no proper ABNF.  So why did Keith do it
>in 3834,

Because that is exactly what RFC 2045 says. It even appeared (but not
normatively) in my old draft-13.

But the real problems are <parameter> and <value>. In draft-13, I actually
had (simplifying slightly, because there were also RFC 2231
complications):

parameter = [CFWS] token [CFWS] "=" value
value     = [CFWS] token [CFWS] / quoted-string

(again, not normatively)

But I was forbidden from having that stuff incorporated into USEFOR. As a
result, USEFOR does not adequately explain where [CFWS] can appear in
Injection-Info.

>> There is a problem in RFC 2822 (which you yourself spotted)

>A _real_ problem in 2822 ?  So far I tended to hallucinate
>problems, but the worst I'm sure of was an "I don't like it".

OK, if you don't want it fixed, let it go. But just using <domain> in
place of <id-right> will not fix it either, without further verbiage.

To: [email-masked]
Xref: clerew local.usefor:20787
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Fri, 29 Apr 2005 15:18:12 GMT
Lines: 20
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:20789
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Again broken
Date: Fri, 29 Apr 2005 15:48:22 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>RFC 2822 doesn't say anything about Control headers, for obvious reasons.
>What are we gaining by adding comments to the header and breaking existing
>software?

Consistency with conventions for headers elsewhere. Users don't want to
have to remember which headers allow comments and which don't (except
where there are strong and permanent reasons to the contrary, such as
performance issues that are never going to go away).

Sooner or later, some enterprising person is going to write

Control: cancel <[email-masked]> (unapproved article in moderated group)

and will be most miffed if it is rejected or doesn't work.

Yes, it may not work with many current servers, but there is no reason why
it should not work long term. It is not rocket science to preprocess all
headers to remove all comments and foldings before acting upon them.

Anyway, the WG has never asked for this particular comment to be forbidden
(and it _did_ decide very early on that comments were normally to be
accepted everywhere, but maybe not generated "yet"). It was not forbidden
in any of the earlier drafts, so it is premature for USEFOR to have
removed it.

If the WG wants this change, then it ought to be discussed first.

To: [email-masked]
Subject: Re: Again broken
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 28 Apr 2005 19:01:36 -0700
Sender: [email-masked]

>> The WG agreed early on that comments should be acceptable
>> in all places allowed by RFC 2822

> Especially where stupid users do stupid things, but Control ?

>> Control headers are so rare that performance issues can be
>> neglected.

> Okay, if you say so.  Convince Russ that it's a good idea.

RFC 2822 doesn't say anything about Control headers, for obvious reasons.
What are we gaining by adding comments to the header and breaking existing
software?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Again broken
Date:  Fri, 29 Apr 2005 03:49:55 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 112
Sender: [email-masked]

Charles Lindsey wrote:

>> Is that as you want it ?
> Yes. Do you see any technical error in it?

No, I just wanted to be sure that I can quote it as is.

> I will be trying to persuade them into adopting our
> syntax, or preferebaly some less bizarre subset of it.

The excluded ">" is bizarre, but allowing it elsewhere would be
a pain for gateways.  This syntax _is_ the Message-ID for mail
and news, bizarre or not.

I thought about putting it into an I-D, "updates 2822", ready,
problem solved.

>> Control: cancel<a@b><c@d>

> Hold on! You aren't allowed to cancel more than one
> article at a time.

Sorry, I wrote it after the very similar References issue.

> A. The FWS should be CFWS in all places.

I thought the server folks don't want CFWS in their headers,
because they don't want to waste time to parse comments.

> The WG agreed early on that comments should be acceptable
> in all places allowed by RFC 2822

Especially where stupid users do stupid things, but Control ?

> Control headers are so rare that performance issues can be
> neglected.

Okay, if you say so.  Convince Russ that it's a good idea.

> control   =  "Control:" SP [CFWS] control-command [CFWS] CRLF

Often you have the leading / trailing [CFWS] in the "inner"
rule with shorter lines, i.e.

  control         =  "Control:" SP control-command CRLF
  control-command =  [CFWS] verb *( CFWS argument ) [CFWS}

Of course irrelevant; I only see it when I try to quote it.

> On top of that, a "do not generate comments yet" wording
> would be in order, as for the References header

If you think that comments are stupid, and they are, and only
needed for users who trust that the rules are everywhere the
same, then using [FWS] and FWS without any additional wording
is much clearer than [CFWS] and CFWS with a rule to pray not
yet use comments in an "administrative" header like Control.

> I would prefer a generic statement to that effect covering
> all headers inherited from RFC 1036.

That's a trend, you love to say A in syntax, and then to add
long qualifications which are near to NOT A.  I'd just say
NOT A in syntax without qualification, and be done with it.

> <value>, <token> and <parameter> are all defined in RFC 2045

Yes.  So far I didn't know that usefor-03 implicitly includes
all 2045 terms.  I only knew that it includes 2822 minus obs,
and of course 2234bis.

2045 is older than 2234.  IMHO it's better to copy (and check)
what you need, instead of implicitly importing it.  Checking:

     token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                or tspecials>

Whatever that is, it's no proper ABNF.  So why did Keith do it
in 3834, Bruce must have seen this, I'm stunned.  2231 doesn't
fix it.  We shouldn't import this quasi-ABNF, even if Bruce
promises to let it pass.

> There is a problem in RFC 2822 (which you yourself spotted)

A _real_ problem in 2822 ?  So far I tended to hallucinate
problems, but the worst I'm sure of was an "I don't like it".

> If you still want this hole plugged, then please propose an
> alternative text.

That's simple, just say "id-domain" and "id-literal" _without_
additional wording.  Or say "address-literal".  Not the unclear
"id-right" and "no-fold-literal" stuff, where nobody knows what
it's supposed to mean.  Or add a note with this effect:

Note: id-right is a domain, and no-fold-literal is the special
case of a domain-literal (a.k.a. address-literal, see RfC 2821)

>>| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

> Yes, but there is more to say than that, because even a
> <domain-literal> is discouraged (does RFC 2822 still say
> that?) you want even more discouragement for other than IP
> addresses inside them, and you still need to extend it to
> <id-right>s.

More to say ?  That statement is perfectly clear, and we have
serious technical reasons why we want it.  No, AFAIK RfC 2882
does not say so, otherwise we wouldn't need it in usefor-xx.

                           Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Fixed
Date:  Fri, 29 Apr 2005 02:01:51 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Charles Lindsey wrote:

> But you asked for someone to come along and provide a
> casting vote

True.  I wasn't happy with how Alexey did it, no indication
that he understood why we agree on a syntax but disagree on
the stupid names of the productions.  As chair he's supposed
to handle such obscure problems with tact.

> You are the only person wanting <unique>@<domain>.

That's not the point, I'm the person who seriously does not
like "id-right", I think that your idea of "id-right" is not
the "id-domain" in my last proposal.

If you like to reopen negotiations, you can have "id-left".

The critical part for Usenet is the RHS with a domain.  The
LHS is then only a problem of *the* domain owner.  For your
"id-right" that's very unclear.
                               Bye, Frank

To: [email-masked]
Subject: Re: Obsolete vs. obsolescent
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 28 Apr 2005 16:45:55 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Okay, but it's not nice when it has serious problems with the 
> propagation.  An error message "administrative restriction"
> is also not exactly clear.  Or if we'd expect gateways to do
> something special, or better nothing unless they are forced
> to bypass an "administrative restriction" we have to say so
> somewhere.  Not in usefor-x, that was probably a bad idea.

A subject starting with "cmsg " needs to be prohibited in anything that
isn't a control message.  I don't feel strongly about whether that should
be in USEFOR or USEPRO, although to me it feels like a format issue, but
it needs to be in there somewhere.  It's premature to assume that cmsg no
longer receives special handling; we're not done with that transition.  We
might never be, frankly.  Using cmsg in the Subject header of regular
posts will cause significant interoperability problems.

> And they won't stop this immediately only because a new RfC
> wants it.  Let's forget Also-Control unless you're sure that
> it could be a similar problem.

Also-Control simply SHOULD NOT be used, but that's more USEPRO.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Obsolete vs. obsolescent
Date:  Fri, 29 Apr 2005 01:37:12 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Russ Allbery wrote:

 [cmsg]
> I don't feel too strongly about it, though.

Okay, but it's not nice when it has serious problems with the 
propagation.  An error message "administrative restriction"
is also not exactly clear.  Or if we'd expect gateways to do
something special, or better nothing unless they are forced
to bypass an "administrative restriction" we have to say so
somewhere.  Not in usefor-x, that was probably a bad idea.

But the usepro-03 text pretends that there's actually no
problem at all with Subject: cmsg.  And that's not the case,

The attempts to avoid obsolete problems are the problem -
not some old servers trying to act on Subject: cmsg, but
the many servers trying to protect old servers are a pain.

And they won't stop this immediately only because a new RfC
wants it.  Let's forget Also-Control unless you're sure that
it could be a similar problem.

                      Bye, Frank

To: [email-masked]
Subject: Re: Suggested References texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 28 Apr 2005 16:36:49 -0700
Sender: [email-masked]

Yup.

Yup.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Suggested References texts
Date:  Fri, 29 Apr 2005 01:05:14 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 85
Sender: [email-masked]

                       Bye, Frank

To: [email-masked]
Subject: Re: Obsolete vs. obsolescent
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 28 Apr 2005 10:06:04 -0700
Sender: [email-masked]

>> I think USEFOR does need an explicit mention that subjects starting
>> with cmsg should not be considered control messages, to document the
>> change from RFC 1036.

> No, I think it is a protocol issue, and the mention in USEPRO is
> sufficient.

The transformation of a message from a regular news message to a control
message because of the presence of five characters at the beginning of the
Subject line sounds like a syntax and semantics issue to me, even if we're
just warning that it used to be interpreted that way.  cmsg changes the
semantic meaning of the message entirely, if honored.

I don't feel too strongly about it, though.

To: [email-masked]
Xref: clerew local.usefor:20771
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Again broken (was: Fixed)
Date: Thu, 28 Apr 2005 15:03:19 GMT
Lines: 155
Sender: [email-masked]

>~~~ begin quoted ABNF ~~~
>message-id      =  "Message-ID:" SP [FWS] msg-id [FWS] CRLF

...........

>~~~ end of quoted sytax ~~~

>Is that as you want it ?

Yes. Do you see any technical error in it?

>  I'm not sure about Alexey's external
>expert for 2822bis, so I think the most simple solution to get
>an expert review for the two proposals is to post them on the
>rfc-822 mailing list.

I don't think there is current activity on 2822bis. As soon as there is, I
will be trying to persuade them into adopting our syntax, or preferebaly
some less bizarre subset of it. Their present msg-id definition should
just go into their obs-syntax. But that is a battle for another day and
another place.

>> The msg-id-core MUST NOT be more than 250 octets in length.

>s/msg-id-core/msg-id/g

Oops!

>>|  Also note that this updated ABNF applies wherever <msg-id> is used,
>>|  including the References header discussed in Section 3.2.1 and the
>>|  Supersedes header discussed in Section 3.2.5.

>IMHO that's obvious and needs no "note",

It's in the present text. I would have no worries if Ken decided to take
it out.

>BTW, in Control: you have the same syntax problem as in the
>References:, you separate the arguments by an _optional_ [FWS]:

>Control: cancel<a@b><c@d>

Hold on! You aren't allowed to cancel more than one article at a time.
Yes, I know CNews does it, but this WG decided long ago not to allow it,
since most other serving agents don't accept it.

>Control: newgroupthis.is.new
>Control: rmgroupthis.is.old

But yes, you have spotted a real bug there (but fortunately the error is
not repeated in the syntax of the actual <control-command>s in USEPRO).

But whilst we are here, there are two other errors in the Control syntax.

A. The FWS should be CFWS in all places.

The WG agreed early on that comments should be acceptable in all places
allowed by RFC 2822, but on a "MUST accept, do not generate yet" basis.
Later on, we agreed that they should be disallowed wherevoer there was a
severe performance penalty, so they were removed from Newsgroups,
Message-ID, Path, Followup-To and Distribution (these were intended to be
permanent exclusions, not just until "yet" happened).

But it was never agreed for Control, because Control headers are so rare
that performance issues can be neglected.

B. s/control-message/control-command/

This is to accord with a change I made in USEPRO. The problem is that the
term "control message" is already defined to mean the whole article
containing a Control header, and to have the syntactic element
<control-message> meaning something different seemed a bad idea.

So I believe the complete syntax for Control headers should be:

   control         =  "Control:" SP [CFWS] control-command [CFWS] CRLF

   control-command =  verb *( CFWS argument )

   verb            =  token

   argument        =  value

On top of that, a "do not generate comments yet" wording would be in
order, as for the References header, but I would prefer a generic
statement to that effect covering all headers inherited from RFC 1036.

>...  "value" is apparently not
>defined in usefor-03 or 2822.  "token" is apparently also not
>defined in usefor-03 or 2822.

<value>, <token> and <parameter> are all defined in RFC 2045, and used at
various places in USEFOR. This needs to be stated somewhere, but there is
more to it than that because RFC 2045, being RFC 822 based, does not tell
you where to put the [CFWS]s in them, so we need some wording to cover
that.

Also, in Injection-Info, where it says

   post-host-param =  "posting-host" "=" host-value

it actually means

   post-host-param =  [CFWS] "posting-host" [CFWS] "=" [CFWS] host-value [CFWS]

and similarly for the other <*-param>s.

So that needs to be covered by explicit syntax or suitable verbiage.

>>| NOTE: It in RECOMMENDED in [RFC 2822] that, for ensuring
>>| global uniqueness, the <id-right> be some domain identifier

>It's unnecessary to quote that RECOMMENDATION, least of all the
>"some domain identifier".  Out of its original very convoluted
>RfC 2822 context this quote is in fact _worse_ than RfC 2822.

There is a problem in RFC 2822 (which you yourself spotted) in that there
is wording in 3.4.1 to the effect that a <domain-literal> is meant to be
an IP address (whether IPv4 or IPv6 or any other IPvxx), in spite of a
syntax apparently allowing much more, but they omitted to say the same
thing about a <no-fold-literal>.

The text I wrote was intended to plug this whole, but it was only a NOTE,
so had no normative effect. If you still want this hole plugged, then
please propose an alternative text.

>IIRC we had a "pseudo-consensus" (= you and me) about domain-
>literals in addresses, Message-IDs, and Path-identifiers, i.e.
>say something like STD 11 without shouting:

>| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

Yes, but there is more to say than that, because even a <domain-literal>
is discouraged (does RFC 2822 still say that?) you want even more
discouragement for other than IP addresses inside them, and you still need
to extend it to <id-right>s.

To: [email-masked]
Xref: clerew local.usefor:20768
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Thu, 28 Apr 2005 13:30:42 GMT
Lines: 110
Sender: [email-masked]

So you want me to omit

To: [email-masked]
Xref: clerew local.usefor:20769
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Thu, 28 Apr 2005 13:51:26 GMT
Lines: 68
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:20765
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Thu, 28 Apr 2005 11:29:40 GMT
Lines: 70
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>I think it is a requirement that a news client be able to determine
>"followup" by inspecting the article headers.

OK. Then you need to tell us exactly WHY that requirement is needed.

As opposed to a requirement "that a news client be able to determine
articles that the poster intends to be treated the same way as followups
...".

>If it is not a requirement, then the term "followup" is meaningless
>and should not appear in any document.

The term "followup" has to appear in our standards, if only because we
have to explain the purpose of the Followup-To header. And we have to
explain the duties of a followup agent. And USEAGE (which is not
standards-track) also explains how to contruct attributions and quotations
in followups.

>In support of this, I point that the References header is for
>the use of the READER, not the POSTER.  The poster already knows how
>all the articles are related.

Indeed, but the POSTER gets some say in how the READER gets to view his
articles by including (or not including) a References header.

>Frank, you allowed that followup => references.  If you don't
>hold that references => followup, then does the standard allow an
>article to reference more than one thread?  I think that
>allowing References to appear in an article which is not a reply
>to a single article (which I thought was the term "followup"),
>leads to undecidable actions about threading. What is the READER
>agent supposed to do in that situation?

Both USEPRO and RFC 2822 state explicitly that they make no provision for
followups/replies to multiple precursors, though they do not explicitly
forbid them. Some future extension to either or both documents may do so -
various suggestions have been made as to how to do it, but there is no
agreed method yet.

Nevertheless, people will and do do it, and try to construct a References
header to suit. In fact, if this is done sensibly, existing threading
agents will produce a tolerable behaviour (by which I mean that they will
not list an article as available for reading earlier than any of its
precursors).

In my suggested text for the semantics of the References header, I tried
to word it so as to afford the maximum flexibility for such future
extensions, whilst always ensuring correct behaviour in the single
precursor case.

>Is there a term for articles that can have references but are
>not followups?  If not, why the special term "followup"?  What
>does the apperance of References header mean, from a READER perspective?

I once coined the term "pseudo-followup" just for use in our discussions
in this WG (not with any intent of using that term in any document). I was
rewarded with the usual multi-page rant from John :-( .

To: [email-masked]
Xref: clerew local.usefor:20770
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Obsolete vs. obsolescent
Date: Thu, 28 Apr 2005 14:09:23 GMT
Lines: 55
Sender: [email-masked]

>> IMHO all that should be done is to update this statement:

>> | As a result, this header is to be regarded as obsolete, and
>> | it will likely be removed entirely in a future version of
>> | this standard.

>>   As a result, this header is to be regarded as obsolescent.
>>   Servers and UAs MUST accept but SHOULD ignore it, and SHOULD
>>   NOT generate it.

>I dislike saying that servers MUST accept a header, since servers are not
>required to accept *any* article.  I would also leave in the bit about
>removing it in a future version of the standard.

I agree (though there is a general caveat in USEPRO that the right of any
server to drop any article on the floor overrides any "MUST" requirement).

But I am happy with the s/obsolete/obsolescent/.

>> Subject: cmsg is not mentioned anywhere in usefor-03, that's a
>> serious bug.  It's mentioned in usepro-03, but it only says
>> that this is now obsolete.  That's of course FUBAR, it's a
>> severe security violation.  We cannot jump from RfC 1036 to a
>> completely different procedure at date X.  AFAIK RfCs must not
>> have a flag day.

>I think USEFOR does need an explicit mention that subjects starting with
>cmsg should not be considered control messages, to document the change
>from RFC 1036.

No, I think it is a protocol issue, and the mention in USEPRO is
sufficient.

Any I do not share Frank's worries about a flag day. That usage is already
ignored by most servers, so nobody should be relying on its working
anymore. If the few remaining servers that honour it suddenly become
non-compliant the day our RFC gets published, then so be it. The sky will
not immediately fall in, but anyone continuing to use or accept the
practice will be severaly sat upon. In any case, I have not seen or heard
of anybody actually trying to use the feature in earnest for many years.

To: [email-masked]
Xref: clerew local.usefor:20766
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Thu, 28 Apr 2005 11:40:18 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Alexey Melnikov <[email-masked]> writes:

>Alexey Melnikov wrote:

>> I do, but the devil is in details: what exactly should be said and where.

>The USEFOR document can say that certain types of messages MAY require 
>the References header. But I just think this should be obvious.

I think we say the minimum necessary to prevent other readers from
misinterpreting what the document says in the way that John has been
misinterpreting it.

A definition of "followup" that clearly distinguishes them from RFC 2822
"replies" is a good start (and I think we already agreed to that). Also,
if we intend that References may sometimes be used with non-followups, it
would be useful to say so.

To: [email-masked]
Xref: clerew local.usefor:20773
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Date: Thu, 28 Apr 2005 15:36:33 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>So it seems to me that this is a genuine inter-operability issue -- not
>at the protocol levels : the articles get to the far end just fine and
>the servers will serve them perfectly well. The issue occurs at the
>level at which the readers view the newsgroups.  For that reason I think
>that a standardised way of expressing which pile an article belongs in
>is of very significant value. Yes at present some software is a bit
>flaky, but that doesn't mean we should fail to say what should be done.

I agree entirely. RFC 2119 is very badly written, and is wide open to
those who claim that "interoperability" means only that "it must be
delivered correctly at the far end". The WG has long striven to use "MUST"
wording for cases where "it does not behave as it is intended to behave if
you don't say 'MUST'".

Anyway, to deal with this present case, you will find some wording in
USEPRO 7.7 entitled "Duties of a Reading Agent" where it says, in effect,
that the agent MAY take note of the References, Subject and Date headers
in order to display list of articles sensibly. This makes such displaying
an optional part of the protocol, which in turn means that reading agents
are entitled to rely upon the presence and correctness of those headers.

Yes, it is pretty meaningless, and it is only there to satisfy the pedants
who object to the use of MUST and SHOULD when dealing with the behaviour
of followup agents. But if that is the price we have to pay to get this
wording in, ..........

>sounds like a cock-up :(

It WAS a cock-up.

>so best fix it then!  ASAP!!

Exactly.

>Signature by unknown keyid: 0xD8D683E2

Eh? I though I knew all your public keys.

To: [email-masked]
Xref: clerew local.usefor:20767
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: News headers (was References)
Date: Thu, 28 Apr 2005 12:06:46 GMT
Lines: 58
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>RFC2822 does not say "email replies". Since USEFOR defers to RFC2822, we 
>have chosen to expand any "email" limitation of RFC2822 into USENET. If we 
>are so pedantic that we say "RFC2822 is EMAIL only" even in the face of 
>explicit references to RFC2822 by draft news standards, then exactly what 
>headers DOES news have and how are they defined? Many of them are defined 
>as "as found in RFC2822". But since those are EMAIL headers, then they 
>don't apply to news. Sigh.

You do have a point there regarding which headers are officially part of
"News".

RFC 2822 defines "Optional fields", so that the following header is
perfectly legal in any email:

      Foobar: bar foo

However, the clear intent of that field was to allow other standards to
define further email headers, such as

      Content-type: text/plain

and have them automatically be acceptable to RFC 2822. It was also
intended to allow any header beginning with "X-". But those intents were
not clearly stated, which is a pity IMO.

The situation was somewhat restored by RFC 3864, which establishes an IANA
registry for email (and also news and HTTP) headers.

What I would like to see in USEFOR is a clear statement that the headers
it explicitly mentions (From, Subject, Newsgroups, etc) are "official
Netnews headers" and that ones it does not mention are "passengers" which
are generally to be ignored by news software, but which should
nevertheless be passed on untouched by transports. Not that I am proposing
to define "passenger" as a technical term for the final document.

Examples of such "passengers" are the Received and In-Reply-To headers,
which are not, and never have been, a part of Netnews, but which do show
up occasionally, especially in articles gatewayed from email.

Indeed, I would go further, and say that any header appearing in a News
article SHOULD either be one blessed by the RFC 3864 registry, or else
should begin with "X-".

In fact, the old draft-13 contained wording of this nature, but it seems
not to have made its way into USEFOR yet.

To: [email-masked]
Xref: clerew local.usefor:20772
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fixed
Date: Thu, 28 Apr 2005 15:18:48 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Alexey Melnikov wrote:

>>> the general principle that we do not change anything in RFC
>>> 2822 that we do not need to change.

>> I concur with your preference.

>I don't.

But you asked for someone to come along and provide a casting vote to
settle the difference between you (wanting <unique>@<domain>) and me (wanting
to retain <id-left> and <id-right>). Now you have it, and it puts you in a
minority (even if it does not amount to a ruling from the Chair). You are
the only person wanting <unique>@<domain>.

To: [email-masked]
Xref: clerew local.usefor:20774
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Thu, 28 Apr 2005 15:42:13 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Either we define followups by poster intent and some articles with
>Reference headers aren't followups, or we define followups by the presence
>of a Reference header and we can't make a strong statement about intent.
>We can't have it both ways; Reference headers are used in practice for
>things other than followups, and I don't expect that practice to change
>(nor do I think we really *want* it to change).

Exactly. But still John believes that the real world can be changed by
smoke and mirrors.

I have explained it to him.
Seth has explained it to him.
Shmuel has explained it to him.
And now you have explained it to him.

But he still does not see it. I should not waste any more time on it.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Obsolete vs. obsolescent
Date:  Thu, 28 Apr 2005 14:59:39 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 34
Sender: [email-masked]

Russ Allbery wrote:
 
> I dislike saying that servers MUST accept a header

Yes, we have some seriouss case of "Deja-Vu" on this
list, I'm not immune, sorry.  What I really wanted to
say, there's a difference between the "Lines:" and
other obsolete header fields in 3.3.

"Lines:" is deprecated _now_ by the future Usefor RfC.
It was optional and unreliable in s-o-1036.  

Some other headers mentioned in 3.3 are something I've
never seen or heard of:  Relay-Version, Posting-Version,
Date-Received.  

s-o-1036 says "thoroughly obsolete" about these three
header fields listed in its appendix A.3, that was 1994.

Finally there are some headers introduced by s-o-1036
that didn't make it (unlike Supersedes):  See-Also,
Article-Names, Article-Replacing, and Also-Control.

Strange, my copy of s-o-1036 (Jan 17 1994) says
"Article-Replacing", Usefor-03 says "Article-Updates".

Nothing of this is critical, but Also-Control could be
a special case like Subject: cmsg cancel.  It would be
really easier to build on s-o-1036, pro forma.  Even if
the future Usefor RfC is published only one second after
s-o-1036 it could eliminate some weird logical problems.

                            Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: References
Date:  Thu, 28 Apr 2005 14:04:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 54
Sender: [email-masked]

Forrest J. Cavalier III wrote:
 
> I think it is a requirement that a news client be able to
> determine "followup" by inspecting the article headers.

Maybe you like to join Bruce and some others including me
in our time travel excursion to fix some RfCs, you could
be the advisor to Dave Crocker when he fixes STD 11 4.6.3.
We could do radical things like publishing s-o-1036 as RfC.

Seriously, this "requirement" is wishful thinking, you find
it as See-Also in s-o-1036.  All we can do today is to say
how a followup is supposed to be created and how References
are trimmed.

We cannot guarantee that all References were built this way.
The very best we can do is to be silent about more creative
ways to generate References.
 
> If it is not a requirement, then the term "followup" is
> meaningless and should not appear in any document.

"Followup" has a perfect meaning defined by Followup-To (or
Newsgroups).

> What is the READER agent supposed to do in that situation?

It's supposed to expect the worst, but to work for the 99.9%
ordinary followups.  In the same way as it did for the last
decade.  

BTW, I've tested the effect of an article referencing itself
on this list some moths ago.  Let's say that I won't test
this again.

> Is there a term for articles that can have references but
> are not followups?

How about "literal Reference" ?  "STD 11 4.6.3 Reference" ?
"Multi-2822 quasi-In-Reply-To" ?  Or good old "See-Also" ?

> why the special term "followup"?

Maybe ask Henry or Bruce, it must be very old, "Followup-To"
is very old.  For me nothing before s-o-1036 is real, all I
knew about Usenet before s-o-1036 was "Emily Postnews", and
the fascinating differences from FidoNet, because I used to
hang out in a gateways NG ("echo" from my POV) for some time.

That's also the reason why I consider attempts to devalue
Usenet Message-IDs to some <id-left@id-right> as blasphemy.

                      Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: References
Date:  Thu, 28 Apr 2005 13:12:15 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 17
Sender: [email-masked]

Alexey Melnikov wrote:

> I just think this should be obvious.

Sure.  It was discussed here for the major part of 2004.

News is not the same as mail and 2822.  There's a note
in s-o-1036 explaining why some References should be a
See-Also, but we know that See-Also didn't fly.

So we have References used for followups as defined in
Usepro, and we have References as defined in STD 11 4.6,
and unfortunately that's not always identical.

What else is new ?  Is Usefor supposed to document common
practice or wishful thinking based on 2822 ?  Bye, Frank

Date: Wed, 27 Apr 2005 16:35:06 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Alexey Melnikov <alexey.melnikov-usefor@xxxxxxxxx>:

>The USEFOR document can say that certain types of messages MAY require the 
>References header. But I just think this should be obvious.

It is interesting to note that just this morning in the office we had a 
discussion about the use of the terms "obvious" and "clearly", and that 
when someone says something is either "obvious" or "clearly", it usually 
isn't.

In this case, USEFOR defers to RFC2822, which clearly and obviously says
that References is optional -- specifically, it uses the RFC2119 term
"SHOULD", and in some places only "may", and further talks about how to
process replies that do not contain a References header. It is hard to 
imagine that this is a requirement.

Saying that USEFOR "obviously" says that some articles require a
References header is, well, contrary to what USEFOR actually says. That's 
fine, it has been changed and we can live with that, as long as the change 
is propogated to all relevant documents, which includes USEPRO.

We could change it back, but so far only a very small handful of people 
have been arguing that RFC2119 language is justified, and even then they 
haven't been providing any examples of interoperability issues, just 
things like "ensure that definitions are obeyed" and they think it should 
say "MUST".

Date: Wed, 27 Apr 2005 14:37:25 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

Me:  Yes. It is pretty clear that all replies are followups.

>What about, er, replies? In the trn sense.  An e-mail reply to a Usenet
>message. Those are what RFC 2822 more obviously applies to; 

RFC2822 does not say "email replies". Since USEFOR defers to RFC2822, we 
have chosen to expand any "email" limitation of RFC2822 into USENET. If we 
are so pedantic that we say "RFC2822 is EMAIL only" even in the face of 
explicit references to RFC2822 by draft news standards, then exactly what 
headers DOES news have and how are they defined? Many of them are defined 
as "as found in RFC2822". But since those are EMAIL headers, then they 
don't apply to news. Sigh. 

>This thread is convincing me that it's a bit confusing to defer to RFC
>2822 entirely for References when RFC 2822 only talks about replies and
>clearly is intended to focus on mail messages.  Usenet followups are a bit
>of a different beast with historically stronger restrictions on the
>presence of the References header that e-mail is only now starting to
>adopt. 

"Historically stronger" is not found in RFC2119 as a reason for mandates, 
and "historically stronger" is apparently not strong enough to have 
resulted in the mandates being continued into draft-03 of USEFOR.

>I still feel like this could be dealt with definitionally, but
>followups in the Usenet sense have References headers, period.  It's not
>just a SHOULD. 

Currently, it is just a SHOULD. If we choose to return it to MUST, we can 
do so, but what do we use to justify that change, and then, do we have the 
nerve to do it right or do we continue to contradict ourselves?

>Either we define followups by poster intent and some articles with
>Reference headers aren't followups,

The latter does not follow from the former. We can easily (and have, in 
the past) prohibit References headers in non-followups. 

> or we define followups by the presence
>of a Reference header and we can't make a strong statement about intent.

So I can change an article from being a followup to not being a followup
simply by removing one header? It seems odd that I can change the
origination of an article by simply removing a header when his article
reaches me halfway around the world from where he posted it.

>We can't have it both ways; Reference headers are used in practice for
>things other than followups, and I don't expect that practice to change
>(nor do I think we really *want* it to change).

We can solve this problem by simply expanding trivially the definition of 
followup. We don't have to dick with the mandates for when a References 
header is or is not to appear in an article. But "we" chose the latter 
course, to the point that they are now optional and followups are 
indetectable.

Ken Murchison <ken@xxxxxxxxxx>:

>...USEFOR discusses semantics elsewhere so we can and probably should 
>discuss the use of the References header in a more Usenet-centric way.

And exactly what shall our excuse be for ignoring RFC2119 after removing 
mandates regarding a header whose mandates have always been a point of 
debate? 

Date: Wed, 27 Apr 2005 14:17:21 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Richard Clayton <richard@xxxxxxxxxxxxxx>:

> one article they will cope with --

Many articles they will cope with, I hope. Many articles have no 
References header.

>-- but if no articles
>have References then they will fail to function as their users currently
>expect. I call that broken, and so will they.

Once again we are stuck with the sometimes unusual expectations of USENET 
users. And how did we get to "no articles have References headers?" Are 
you imagining some prohibition on the use of References headers? 

>I believe that References:  should be compulsory, 

RFC2119 defines when MUST is appropriate. This is not such a time.

>however,
>now it's clear that there's the prospect of (a) the effort finishing
>(one way or another) and (b) posters being permitted to break
>"threading" (and if there is no language to compel them, then they will)
>then it is time to speak up

Posters are always permitted to "break threading". They will always be 
permitted to "break threading".

Seth Breidbart <sethb@xxxxxxxxx>:

>Define "break".  If you mean "crash and burn and let the magic smoke
>out of the computer" then no. 

Well, let's see. "Break" as in "fail to process articles", ok? No, since 
we are talking about interoperability, that would have to be "fail to 
accept articles or to pass them to other news system software." 

I read news today. I found articles (gasp!) without References headers. My 
newsreader did not break. It simply did not care.

>If you mean "Do the wrong thing (from
>the viewpoints of both the poster and reader)" then yes.

Since this is a display issue, what is "the wrong thing" and how do you 
know when it happens? "Wrong" is completely subjective. Subjective is not 
how one writes standards. In this specific case, some people think that 
"right" is "sort by subject content". Some think that "sort by precursor" 
is "right" and "sort by subject" is wrong. Who is "right" and who is 
"wrong"? 

>If a random set of Subject header content was changed to "3" the
>system wouldn't collapse, either; so why is specifying that some
>headers MUST NOT be changed by transit servers an interoperability
>issue?

Because transport agents are DEFINED not to change the message they are 
transporing, and the header you are referring to is a USER controlled 
header, not a news system controlled one. 

>Answer: That's doing _the wrong thing_.

That's your opinion. I could imagine a system where the Subject header IS 
changed in transport that is quite correct. In fact, some email spam 
systems do exactly that.

Date: Wed, 27 Apr 2005 20:30:06 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Alexey Melnikov wrote:

> Ken Murchison wrote:
>
>> Russ Allbery wrote:
>>
>>> This thread is convincing me that it's a bit confusing to defer to RFC
>>> 2822 entirely for References when RFC 2822 only talks about replies and
>>> clearly is intended to focus on mail messages.  Usenet followups are 
>>> a bit
>>> of a different beast with historically stronger restrictions on the
>>> presence of the References header that e-mail is only now starting to
>>> adopt.  I still feel like this could be dealt with definitionally, but
>>> followups in the Usenet sense have References headers, period.  It's 
>>> not
>>> just a SHOULD.  And I don't feel like that's a divergence from RFC 
>>> 2822,
>>> since RFC 2822 doesn't deal with Usenet followups and the semantics 
>>> of the
>>> References header haven't changed.
>>
>> I'm leaning towards the same conclusion.  Deferring to RFC 2822 in 
>> draft-03 is looking like bad idea.  As Charles stated, USEFOR 
>> discusses semantics elsewhere so we can and probably should discuss 
>> the use of the References header in a more Usenet-centric way.
>>
>> Does our Chair agree?
>
> I do, but the devil is in details: what exactly should be said and where.

The USEFOR document can say that certain types of messages MAY require 
the References header. But I just think this should be obvious.

Date: Wed, 27 Apr 2005 20:20:11 +0100
From: Alexey Melnikov <[email-masked]>
To: Ken Murchison <[email-masked]>
CC: [email-masked]
Subject: Re: References
Sender: [email-masked]

Ken Murchison wrote:

> Russ Allbery wrote:
>
>> This thread is convincing me that it's a bit confusing to defer to RFC
>> 2822 entirely for References when RFC 2822 only talks about replies and
>> clearly is intended to focus on mail messages.  Usenet followups are 
>> a bit
>> of a different beast with historically stronger restrictions on the
>> presence of the References header that e-mail is only now starting to
>> adopt.  I still feel like this could be dealt with definitionally, but
>> followups in the Usenet sense have References headers, period.  It's not
>> just a SHOULD.  And I don't feel like that's a divergence from RFC 2822,
>> since RFC 2822 doesn't deal with Usenet followups and the semantics 
>> of the
>> References header haven't changed.
>
> I'm leaning towards the same conclusion.  Deferring to RFC 2822 in 
> draft-03 is looking like bad idea.  As Charles stated, USEFOR 
> discusses semantics elsewhere so we can and probably should discuss 
> the use of the References header in a more Usenet-centric way.
>
> Does our Chair agree?

I do, but the devil is in details: what exactly should be said and where.

Date: Wed, 27 Apr 2005 14:42:17 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
CC: Alexey Melnikov <[email-masked]>
Subject: Re: References
Sender: [email-masked]

Russ Allbery wrote:

> This thread is convincing me that it's a bit confusing to defer to RFC
> 2822 entirely for References when RFC 2822 only talks about replies and
> clearly is intended to focus on mail messages.  Usenet followups are a bit
> of a different beast with historically stronger restrictions on the
> presence of the References header that e-mail is only now starting to
> adopt.  I still feel like this could be dealt with definitionally, but
> followups in the Usenet sense have References headers, period.  It's not
> just a SHOULD.  And I don't feel like that's a divergence from RFC 2822,
> since RFC 2822 doesn't deal with Usenet followups and the semantics of the
> References header haven't changed.

I'm leaning towards the same conclusion.  Deferring to RFC 2822 in 
draft-03 is looking like bad idea.  As Charles stated, USEFOR discusses 
semantics elsewhere so we can and probably should discuss the use of the 
References header in a more Usenet-centric way.

Does our Chair agree?

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

To: [email-masked]
Subject: Re: Suggested References texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 27 Apr 2005 11:08:49 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:
> Russ Allbery <rra@xxxxxxxxxxxx>:

>> If this is a requirement, then we have to define everything that
>> contains a References header as a followup,

> You have it backwards. We define a followup by the poster's intent; we
> DETECT followups with the References header. There is a difference. Your
> definition leaves us with a circle: a followup contains a References
> header, and a References header must be used in a followup. Ok, what's a
> followup?

Either we define followups by poster intent and some articles with
Reference headers aren't followups, or we define followups by the presence
of a Reference header and we can't make a strong statement about intent.
We can't have it both ways; Reference headers are used in practice for
things other than followups, and I don't expect that practice to change
(nor do I think we really *want* it to change).

To: [email-masked]
Subject: Re: References
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 27 Apr 2005 11:04:34 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:
> "Forrest J. Cavalier III" <mibsoft@xxxxxxxx>:

>> Did anyone ever advocate that 2822-reply => followup?

> Yes. It is pretty clear that all replies are followups.

What about, er, replies?  In the trn sense.  An e-mail reply to a Usenet
message.  Those are what RFC 2822 more obviously applies to; followups in
the Usenet sense have additional restrictions that RFC 2822 has no comment
on and aren't really something RFC 2822 was trying to deal with.

This thread is convincing me that it's a bit confusing to defer to RFC
2822 entirely for References when RFC 2822 only talks about replies and
clearly is intended to focus on mail messages.  Usenet followups are a bit
of a different beast with historically stronger restrictions on the
presence of the References header that e-mail is only now starting to
adopt.  I still feel like this could be dealt with definitionally, but
followups in the Usenet sense have References headers, period.  It's not
just a SHOULD.  And I don't feel like that's a divergence from RFC 2822,
since RFC 2822 doesn't deal with Usenet followups and the semantics of the
References header haven't changed.

Date: Wed, 27 Apr 2005 11:00:33 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Suggested References texts
Sender: [email-masked]

>Let us recap the history here. It is accepted, I believe, (using Frank

Frank's notation is confusing. Does '=>' mean "is a member of", "implies", 
or something else? Simple english works so much better here, I suggest it 
be used.

>   followup -> references header
 
A followup requires a References header? (Yes under RFC1036, no under 
RFC2822). A followup contains a References header?  A followup is defined 
by a References header? What?

>   no references header -> not a followup
 
Ahhh, perhaps '=>' means "means"? (Is '->' the same as '=>'?) So, no 
References means an article is not a followup? Well, yes, according to 
RFC1036, not according to USEFOR draft 03 and RFC2822. 

>   references header -> followup

And then you changed it.

>message/partial. And there were other examples too.

None of which required the change you made.

>Did we want to forbid these practices, and would they actually disappear

So the choice was to evicerate the References header instead of expand the 
definition of followup trivially.

>when it sees a References header.

Wrong. Making References legal in non-followups changed the information 
that could be deduced. "Is this a followup" cannot be answered using your 
changes. It could before you made them.

>It can no longer deduce "this is a followup".

I keep telling you this, and yet, you keep telling me that this is not a
change.

>is 'Does this matter?'.  I don't see why it should.

That is your value judgement, not a fact. In fact, many people do think it 
matters, and many people want that ability. Richard Clayton was just here 
arguing that it is important, and that he even considers it an 
interoperability issue. Do you understand the difference between saying "I 
don't think it is important and thus shouldn't be possible" and "I don't 
think it is important enough to merit RFC2119 mandates?"

But now that it is gone, we need to unify on one position. Either it is or 
it isn't important. One document written as if it isn't and one written as 
if it is is just ridiculous. Pick one and stick with it for more than a 
week.

To: [email-masked]
Subject: Re: Obsolete vs. obsolescent
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 27 Apr 2005 10:57:51 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> IMHO all that should be done is to update this statement:

> | As a result, this header is to be regarded as obsolete, and
> | it will likely be removed entirely in a future version of
> | this standard.

>   As a result, this header is to be regarded as obsolescent.
>   Servers and UAs MUST accept but SHOULD ignore it, and SHOULD
>   NOT generate it.

I dislike saying that servers MUST accept a header, since servers are not
required to accept *any* article.  I would also leave in the bit about
removing it in a future version of the standard.

> Subject: cmsg is not mentioned anywhere in usefor-03, that's a
> serious bug.  It's mentioned in usepro-03, but it only says
> that this is now obsolete.  That's of course FUBAR, it's a
> severe security violation.  We cannot jump from RfC 1036 to a
> completely different procedure at date X.  AFAIK RfCs must not
> have a flag day.

I think USEFOR does need an explicit mention that subjects starting with
cmsg should not be considered control messages, to document the change
from RFC 1036.

Date: Wed, 27 Apr 2005 13:50:27 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

John Stanley <[email-masked]> wrote:

> However, if you are trying to argue that "followup" is an important
> concept, and detecting such is an important action, then ok, maybe
> it is.  Does it reach the level of importance that it is an
> interoperability issue? Will newsreaders break if they get an
> article without a References header? Does it truly merit a MUST, or
> is it only worth a SHOULD?

Define "break".  If you mean "crash and burn and let the magic smoke
out of the computer" then no.  If you mean "Do the wrong thing (from
the viewpoints of both the poster and reader)" then yes.

I'd prefer a MUST but would settle for a SHOULD.

> And yet, not a single thing you mentioned is interoperability. In
> fact, right now, news readers MUST deal with articles that do not
> have a References header. An article without a References header
> CANNOT be an interoperability issue, since the system would be
> collapsing today.

If the system continues to act, but sometimes acts wrongly, it isn't
collapsing.  But whatever is making it act wrongly _is_ an
interoperability issue.

If a random set of Subject header content was changed to "3" the
system wouldn't collapse, either; so why is specifying that some
headers MUST NOT be changed by transit servers an interoperability
issue?  Answer: That's doing _the wrong thing_.

Seth

Date: Wed, 27 Apr 2005 18:46:37 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

In message <[email-masked]>, John
Stanley <[email-masked]> writes

>However, if you are trying to argue that "followup" is an important 
>concept, and detecting such is an important action, then ok, maybe it is. 
>Does it reach the level of importance that it is an interoperability 
>issue? 

in my view yes

>Will newsreaders break if they get an article without a References 
>header? 

one article they will cope with -- and their heuristics may mean that
they function as if the header had been present -- but if no articles
have References then they will fail to function as their users currently
expect. I call that broken, and so will they.

>Does it truly merit a MUST, or is it only worth a SHOULD?

it should be a MUST

>>So it seems to me that this is a genuine inter-operability issue --
>
>And yet, not a single thing you mentioned is interoperability. 

I suspect that's because you're thinking about servers and not about the
people who read Usenet and the software that they use for that. I gave
some examples of how that causes problems earlier

>We already say what should be done. What change do you want made?

I believe that References:  should be compulsory, and that all the other
clues we currently have like "Re:", the unmangled subject, a later
datestamp, and "In article xxx so-and-so said" can be made optional (or
in the interests of tidyness abandoned altogether)

>>sounds like a cock-up :(
>
>Sounds like a decision and the natural consequences of that decision.
>
>>so best fix it then!  ASAP!!
>
>Best determine that it is broken before demanding it be fixed. This group 
>has been working for a long time; have you read the history of this 
>discussion before telling us something we have done is broken?

oh yes, I've read a lot ... all the way down the years :( ...  however,
now it's clear that there's the prospect of (a) the effort finishing
(one way or another) and (b) posters being permitted to break
"threading" (and if there is no language to compel them, then they will)
then it is time to speak up

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Wed, 27 Apr 2005 10:42:19 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

"Forrest J. Cavalier III" <mibsoft@xxxxxxxx>:

>Did anyone ever advocate that 2822-reply => followup?

Yes. It is pretty clear that all replies are followups. Look at the 
definitions we have come up with for "followup", and each one has included 
the set of articles that are "a response to another". Sometimes the 
definition also includes other relationships, but "response to another" 
has always been included. And that is a "reply".

Date: Wed, 27 Apr 2005 10:35:02 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Richard Clayton <richard@xxxxxxxxxxxxxx>:

>the readers of Usenet believe in things like threads and followups; they
>like the ability to split a series of rambling conversations into
>disparate piles of articles and understand each of those piles one at a
>time...

If you read USENET for any length of time, you will find that readers (at 
least posters) of USENET believe in all kinds of oddball things. Does 
their belief in something elevate that thing into an interoperability 
issue, or even something that we need to deal with? 

>However, there seems to be an argument being presented that when you
>look deeply at the specifications the notion of followups and threads is
>mainly an illusion fostered by the reading tools and one can lawfully
>miss out all the headers that allow the illusion to be constructed.

I'm trying to understand what you are saying here, but it is couched in
such odd language that I'm not certain. "Lawfully miss out?" If I get what 
you are saying, then yes, of course we can refuse to mandate a header that 
allows illusions to be propogated. In fact, propogating illusions is not 
the job of a technical standard.

However, if you are trying to argue that "followup" is an important 
concept, and detecting such is an important action, then ok, maybe it is. 
Does it reach the level of importance that it is an interoperability 
issue? Will newsreaders break if they get an article without a References 
header? Does it truly merit a MUST, or is it only worth a SHOULD?

>So it seems to me that this is a genuine inter-operability issue --

And yet, not a single thing you mentioned is interoperability. In fact, 
right now, news readers MUST deal with articles that do not have a 
References header. An article without a References header CANNOT be an 
interoperability issue, since the system would be collapsing today.

>The issue occurs at the level at which the readers view the newsgroups. 

One newsreader dealing with one article for one reader is hardly an 
interoperability issue. What you've just described is a display issue, and 
we do not deal with display issues. Readers are free to display articles 
as they see fit.

>For that reason I think
>that a standardised way of expressing which pile an article belongs in
>is of very significant value.

That may be true. Since we are not undefining the References header, 
we are not removing the 'standardized way of expressing' anything. It's 
just been made optional. 

>Yes at present some software is a bit
>flaky, but that doesn't mean we should fail to say what should be done.

We already say what should be done. What change do you want made?

>sounds like a cock-up :(

Sounds like a decision and the natural consequences of that decision.

>so best fix it then!  ASAP!!

Best determine that it is broken before demanding it be fixed. This group 
has been working for a long time; have you read the history of this 
discussion before telling us something we have done is broken?

>ah....   perhaps I could find it earlier in the "thread" ? 

It isn't hard. It doesn't even require a References header, which is, by 
the way, optional in mail, and requires knowledge of the precursor message 
ids in any case. When the precursor message id is not known, it is 
improper to fabricate one.

Date: Wed, 27 Apr 2005 13:23:36 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Suggested References texts
Sender: [email-masked]

(Here is the text:)

Date: Wed, 27 Apr 2005 10:12:14 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

I said:
> It cannot be mandatory in USEPRO when it is already defined to be 
> optional in both USEFOR and RFC2822.

Alexey Melnikov <alexey.melnikov-usefor@xxxxxxxxx>:

>Surely it can, because USEFOR equally applies to *all* types of USEFOR 
>articles and USEPRO may add additional restrictions for particular types 
>of articles.

While USEFOR applies to all types of articles, the section of USEFOR being 
discussed applies to the References header, and the section of RFC2822 
that USEFOR defers to talks specifically about "replies". USEFOR says "see 
RFC2822", and RFC2822 says References is optional in replies. 

In addition, since USEFOR applies to ALL articles, a definition therein
that makes a header optional makes it optional for ALL articles.

>See my comment above.

Stop being insulting, as if I would ignore your "comment above" until I 
saw your command to "see it" here.

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>One WG member apparently wants an equivalences in the form
>of followup <=> references and followup <=> 2822-reply, but
>that's obviously wrong, 

Yes, that's obviously wrong. Can you tell us who it is who has said this? 

>and the proposed alternative of no
>relation at all between followup and references is also wrong.

It is not wrong if that is how the standards make it, and that is the 
direction that we are heading by making References optional.

Date: Wed, 27 Apr 2005 09:59:15 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Suggested References texts
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

>If this is a requirement, then we have to
>define everything that contains a References header as a followup,

You have it backwards. We define a followup by the poster's intent; we 
DETECT followups with the References header. There is a difference. Your 
definition leaves us with a circle: a followup contains a References 
header, and a References header must be used in a followup. Ok, what's a 
followup?

To: [email-masked]
Xref: clerew local.usefor:20740
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Wed, 27 Apr 2005 13:04:25 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Ken Murchison <[email-masked]> writes:

>I disagree.  It is my opinion (and I believe our Chair's) that USEFOR 
>only document the syntax of an article.  The use or non-use of any 
>particular "optional" header such as References belongs in either USEPRO 
>  or USEAGE.

It also documents "semantics" - that is material that explains the
"meaning" of the header contents, and other explanatory/introductory
material, as in RFC 2822. See my reply to Forrest.

For sure, USEPRO tells you how to construct a "followup", but if there is
to be mention of "other" uses of References, then there is no obvious
place in USEPRO to mention them (unless you can suggest one).

The essential semantics of the References header, of course, is that it
contains a list of "precursors", and the special significance of the order
in which they are listed. But that in turn relies on a careful definition
of "precursor" (which is another possible place to point out that they
apply to more than "followups").

To: [email-masked]
Xref: clerew local.usefor:20739
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Suggested References texts
Date: Wed, 27 Apr 2005 12:04:07 GMT
Lines: 125
Sender: [email-masked]

To: [email-masked]
Date: Wed, 27 Apr 2005 11:53:11 +0200
From: Frank Ellermann <[email-masked]>
Organization: <URL:http://purl.net/xyzzy>
Subject: Again broken (was: Fixed)
Sender: [email-masked]

Charles Lindsey wrote:

~~~ begin quoted ABNF ~~~
message-id      =  "Message-ID:" SP [FWS] msg-id [FWS] CRLF

msg-id          =  "<" id-left "@" id-right ">"
                   ; maximum length is 250 octets

id-left         =  dot-atom-text / no-fold-quote

id-right        =  dot-atom-text / no-fold-literal

no-fold-quote   =  DQUOTE
                      (  "." *mqtext /
                         *mqtext "." /
                         *mqtext mqspecial *mqtext )
                      DQUOTE

mqtext          =  atext / "." / mqspecial

mqspecial       =  "(" / ")" /      ; same as specials except
                   "<" /            ; "\" and DQUOTE quoted
                   "[" / "]" /      ; "." doubled and ">" omitted
                   ":" / ";" /
                   "@" / "," /
                   ".." / "\\" / "\" DQUOTE

no-fold-literal =  "[" *( mdtext / "\[" / "\]" / "\\" ) "]"

mdtext          =  %d33-61 /        ; The rest of the US-ASCII
                   %d63-90 /        ; characters not including
                   %d94-126         ; ">", "[", "]", or "\"
~~~ end of quoted sytax ~~~

Is that as you want it ?  I'm not sure about Alexey's external
expert for 2822bis, so I think the most simple solution to get
an expert review for the two proposals is to post them on the
rfc-822 mailing list.

Further proposed text:

> The msg-id-core MUST NOT be more than 250 octets in length.

s/msg-id-core/msg-id/g

>|  Also note that this updated ABNF applies wherever <msg-id> is used,
>|  including the References header discussed in Section 3.2.1 and the
>|  Supersedes header discussed in Section 3.2.5.

IMHO that's obvious and needs no "note", excl. References: the
two other cases (Supersedes: and Control: cancel) don't exist
in mail.

BTW, in Control: you have the same syntax problem as in the
References:, you separate the arguments by an _optional_ [FWS]:

Control: cancel<a@b><c@d>
Control: newgroupthis.is.new
Control: rmgroupthis.is.old

You need a mandatory FWS in 3.2.4.  "value" is apparently not
defined in usefor-03 or 2822.  "token" is apparently also not
defined in usefor-03 or 2822.

Did anybody try an ABNF checker on usefor-03 ?  (I didn't)

>| NOTE: It in RECOMMENDED in [RFC 2822] that, for ensuring
>| global uniqueness, the <id-right> be some domain identifier

It's unnecessary to quote that RECOMMENDATION, least of all the
"some domain identifier".  Out of its original very convoluted
RfC 2822 context this quote is in fact _worse_ than RfC 2822.

RfC 1036 or s-o-1036 is _much_ better than this sad parody of
a Message-ID.  Usenet is better off without this idea, it is
harmful, it should never become an RfC.

IIRC we had a "pseudo-consensus" (= you and me) about domain-
literals in addresses, Message-IDs, and Path-identifiers, i.e.
say something like STD 11 without shouting:

| Note:  THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

Apparently that didn't make it yet into usefor-03 or usepro-03,
it's roughly equivalent to a SHOULD NOT mentioning the "domain"
in all "addr-spec" found in any From: / Sender: / Reply-To: /
Approved:, the "id-right" (your 2822-terminology) of a msg-id,
the "path-identity", do we need it anywhere else ?  Bye, Frank

Date: Wed, 27 Apr 2005 07:14:25 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Frank Ellermann wrote:
> One WG member apparently wants an equivalences in the form
> of followup <=> references and followup <=> 2822-reply, but
> that's obviously wrong, and the proposed alternative of no
> relation at all between followup and references is also wrong.

Did anyone ever advocate that 2822-reply => followup?

It isn't one WG member that wants this:
followup <=> references

I think it is a requirement that a news client be able to determine
"followup" by inspecting the article headers.

If it is not a requirement, then the term "followup" is meaningless
and should not appear in any document.

In support of this, I point that the References header is for
the use of the READER, not the POSTER.  The poster already knows how
all the articles are related.

Also, if not mandating anything required by the READER for
interoperability, how can a document merit anything stronger than a
"MAY add a References header"?  (And if it is this weak, what's the
point?)

Frank, you allowed that followup => references.  If you don't
hold that references => followup, then does the standard allow an
article to reference more than one thread?  I think that
allowing References to appear in an article which is not a reply
to a single article (which I thought was the term "followup"),
leads to undecidable actions about threading. What is the READER
agent supposed to do in that situation?

Is there a term for articles that can have references but are
not followups?  If not, why the special term "followup"?  What
does the apperance of References header mean, from a READER perspective?

Sure it is POSSIBLE to define a standard which describes unambiguous
construction of an article, but not unambiguous interpretation of that
article.  But I think that's ludicrous.  Is there any other IETF standard
that does this?

Date: Wed, 27 Apr 2005 11:43:33 +0100
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

In message <[email-masked]>, John
Stanley <[email-masked]> writes

>We have two (sane) options:

I come at this as a newsreading client designer, author and implementer

>1. RFC1036 was correct. Being able to identify "followups" is so important 
>that the header that allows this merits RFC2119 mandates.

the readers of Usenet believe in things like threads and followups; they
like the ability to split a series of rambling conversations into
disparate piles of articles and understand each of those piles one at a
time...

   of course we normally call the piles "threads" but a thread is always
   an artificial construct because it cannot be reliably constructed
   without a complete set of articles to hand and Usenet never
   guarantees that you have this -- so "piles" is rather closer to
   reality :)

there are flaws with a divide and conquer approach to newsgroup reading,
not least if you check a newsgroup for new material faster than new
articles flow in -- but in general it has been found to work; and we can
see the concept being copied into other media such as web-based forums.
viz: it may be flawed, but it's the best UI model we currently have

However, there seems to be an argument being presented that when you
look deeply at the specifications the notion of followups and threads is
mainly an illusion fostered by the reading tools and one can lawfully
miss out all the headers that allow the illusion to be constructed. But
even if the illusion is built on sand (and my experience tells me how
many heuristics you need to stabilise the sand), it still has
considerable practical value

>2. RFC1036 was incorrect. Being able to identify "followups" is not
>important enough to merit RFC2119 language and the header that allows this
>is thus optional. 

if there is no standardised way of making the piles of articles in a
consistent manner then as time goes on and posting software diverges
from traditional working (and there is no document to guide them
otherwise) those heuristics I mentioned above will have less and less to
work with.  ie: the piles will be different in different software.

This will lead to considerable confusion amongst the readers of the
newsgroup. Some will believe that a particular article belongs in one
pile, some in another. Some will miss the article altogether because
they have used "kill" features of their software to exclude some of the
piles as uninteresting. They will express this confusion when they write
further articles. Most know not to say "the thread above this one" to
express article relationships -- but they often say "earlier in this
thread" to say where to find other material...

So it seems to me that this is a genuine inter-operability issue -- not
at the protocol levels : the articles get to the far end just fine and
the servers will serve them perfectly well. The issue occurs at the
level at which the readers view the newsgroups.  For that reason I think
that a standardised way of expressing which pile an article belongs in
is of very significant value. Yes at present some software is a bit
flaky, but that doesn't mean we should fail to say what should be done.

>Unfortunately for position 1, we've already removed the mandate.

sounds like a cock-up :(

so best fix it then!  ASAP!!

>Other people can read it the first time I send it; you 
>ought to be able to, too.

ah....   perhaps I could find it earlier in the "thread" ?  :)

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Fixed
Date:  Wed, 27 Apr 2005 12:34:48 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Alexey Melnikov wrote:

>> the general principle that we do not change anything in RFC
>> 2822 that we do not need to change.

> I concur with your preference.

I don't.  The RfC 2822 concept of Message-IDs is "wrong", and
the RfC 1036 concept <unique@domain> is better.  Of course at
least I believe that the RfC 2822 concept is in fact still the
same.  But as far as I can tell it at least Charles thinks that
it's not.  I'm not sure what he believes what an "id-right" is.

> I suggest they talk to Pete Resnick first. As far as I
> remember he has already agreed to publish 2822bis.

I don't recall any discussion about an 2822bis in the past 12
months on the 822-list, but I could post the two proposals for
the _equivalent_ msg-id syntax there (and the reason for this
controversy, "id-right" vs. "id-domain").

I've added an "id-fication" of my proposal below, bye, Frank

~~~ begin ~~~
msg-id       =  "<" id-local "@" id-domain ">"

id-local     = dot-atom-text / ( DQUOTE id-quote DQUOTE )
id-quote     = ( "." [id-text] ) /
               ( [id-text] "." ) /
               ( [id-text] id-special [id-text] )

id-text      = 1*( atext / "." / id-special )
id-special   = "(" / ")" / "," /       ; all specials, minus ">",
               "[" / "]" / "@" /       ; minus DQUOTE, minus "\",
               ":" / ";" / "<" /       ; minus single ".", plus:
               ".." / "\\" / ( "\" DQUOTE )

id-domain    = dot-atom-text / ("[" id-literal "]")
id-literal   = 1*( %d33-61 /           ; printable ASCII minus
                   %d63-90 /           ; ">", "[", "\", "]"
                   %d94-126 /          ; plus "\[", "\\, "\]"
                   "\[" / "\\" / "\]" )
~~~ end ~~~

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Obsolete vs. obsolescent
Date:  Wed, 27 Apr 2005 10:38:20 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 46
Sender: [email-masked]

Ken Murchison wrote:
 
> Care to craft some text and tell me where you'd like to see
> it?

IMHO all that should be done is to update this statement:

| As a result, this header is to be regarded as obsolete, and
| it will likely be removed entirely in a future version of
| this standard.

  As a result, this header is to be regarded as obsolescent.
  Servers and UAs MUST accept but SHOULD ignore it, and SHOULD
  NOT generate it.

For the obsolete headers in 3.3 you already have a definition:

| Articles containing these headers MUST NOT be generated.
| Persons writing new agents SHOULD ignore any former meanings
| these headers.

That should be "of these headers", but the sentence makes no
sense, the handling depends on the header field.  Probably new
agents MUST ignore the former meaning of the obsolete header
fields.  But for Also-Control I'd opt for the same procedure
as for a Subject: cmsg without a corresponding Control:

Oops, "new" problem:

Subject: cmsg is not mentioned anywhere in usefor-03, that's a
serious bug.  It's mentioned in usepro-03, but it only says
that this is now obsolete.  That's of course FUBAR, it's a
severe security violation.  We cannot jump from RfC 1036 to a
completely different procedure at date X.  AFAIK RfCs must not
have a flag day.
  
Even if it's only in theory (which it's not, Subject: cmsg has
serious difficulties to propagate, the last time I checked it
Google ignored a Subject: Re: cmsg, and that's not the only
oddity with a Subject: cmsg)

Maybe publishing Henry's s-o-1036 first could formally save
this situation, otherwise Subject: cmsg is IMO a showstopper.

                          Bye, Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Suggested References texts
Date:  Wed, 27 Apr 2005 09:50:20 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 65
Sender: [email-masked]

Good (but see below).

 [Usepro]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: References
Date:  Wed, 27 Apr 2005 09:11:16 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Ken Murchison wrote:
 
> I don't see a contradiction here

There's no contradiction.  We have followup => references,
and after you've modified the definition as indicated in...

 <[email-masked]>
| Done.

...this issue is settled for Usefor-04 and the future RfC,
as far as I'm concerned.

One WG member apparently wants an equivalences in the form
of followup <=> references and followup <=> 2822-reply, but
that's obviously wrong, and the proposed alternative of no
relation at all between followup and references is also wrong.

                        Bye, Frank

Date: Tue, 26 Apr 2005 13:28:08 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Ken Murchison <ken@xxxxxxxxxx>:

>You have misquoted me here. The above was written by Charles.

You are correct. My mistake, I'm sorry. I'm used to the standard methods
of quoting material (using '>') and not this blue-line crap that doesn't
cut-and-paste. I'm also used to people eliding things they are NOT
responding to and not including large volumes of text just to add one line
at the bottom.

> Thank you for your pleasant and constructive tone.

You are welcome.  I'm sorry you are unhappy that you got caught trying to
reword the problem to make it go away. "Should/should not" is not the
issue and pretending it is is pretty insulting to the person you tried
pulling that on. Excuse me for pointing out that I caught you.

>Is your argument that References shouldn't be required for followups 
>(regardless of what USEFOR/USEPRO currently says) or that the current text 
>is incorrect/ambiguous/contradictory/shitty?

My argument is exactly as I stated: you cannot say "this is optional" and
"this is mandatory" at the same time. This is a problem; at least, I think
most people view contradictions in technical standards as a problem.
Perhaps you do not. The solution is as I stated: pick one and stick with
it, including all the implications that go with it.

>If the former, please state so. If the latter, please suggest some text 
>for one or more of the documents that you feel is technically sound.

Before we can write text we must make the choice. To do otherwise is a
complete waste of time. Writing text and THEN justifying it is backwards,
and it is why we are spending so much time now. The mandate has been 
removed from the draft and people are busy trying to pretend it is still 
there or should simply be reinstated without justification. And yet, there 
must have been a reason it was removed.

We have two (sane) options:

1. RFC1036 was correct. Being able to identify "followups" is so important 
that the header that allows this merits RFC2119 mandates. The implication 
from this is that References MUST appear in followups and MUST NOT appear 
elsewhere, since any other combination of RFC2119 language would not 
accomplish the goal of allowing identification of followups. A further 
implication of this choice is that we MUST document the difference from 
RFC2822 where the header and article format are defined. That's in USEFOR.
Another implication is that we have to define followups. That's not really 
that hard; I've provided a proposed definition that meets current 
practice.  

2. RFC1036 was incorrect. Being able to identify "followups" is not
important enough to merit RFC2119 language and the header that allows this
is thus optional. Since the only method of identifying a followup has been
made optional, the concept of followup is thus not worth defining since
nobody can detect them when they appear anyway. At least, not detect them
with any certainty. (Do you have a mole on your left shoulder? Your
messages don't contain any way of telling; fortunately, I don't care,
although someone else might.)

Once we have chosen the path we wish to take, we can look at how to 
accomplish it.

Unfortunately for position 1, we've already removed the mandate. USEFOR
defers to RFC2822, RFC2822 says "SHOULD" and "may contain". Putting a
mandate BACK after deciding to remove it should, in a correctly operating
system, require justification.  We were getting by with "historical
reasons" when it was a mandate copied from RFC1036; if that wasn't
sufficient to keep it in place, then it won't be enough to put it back now
that it is gone. I've seen no interoperability issues; do you have 
something to report?

So, looking at RFC2119, it appears the only remaining option is 2. USEFOR 
already makes the header optional. The best that USEPRO can do is say "IF 
you include the header, it must contain X". Most of the text Charles just 
proposed meets that criterion. Remove the parts that refer to the specious 
mandate. 

How do we deal with "duties of a followup agent"? Well, RFC2822 talks 
about "replies". That's a pretty common english word, why is it 
unavailable to us? USEPRO can simply talk about what headers are 
available for a reply and what the headers that DO get used in the reply 
contain. E.g., "If a References header is included, it MUST contain ...". 
That's fine. "You MUST include" this optional header is not. And when you 
talk about headers like Newsgroups and Path, you can say "This MUST be 
present and MUST contain..." because USEFOR already says they are 
mandatory.

Now, I've said it all again. Anyone up for a third pass?

And, if you are keeping score on such things, your demand that I repeat 
myself or provide text, with the implications of that demand, is pretty 
insulting, too. Other people can read it the first time I send it; you 
ought to be able to, too.

Date: Tue, 26 Apr 2005 21:11:50 +0100
From: Alexey Melnikov <[email-masked]>
To: John Stanley <[email-masked]>
CC: [email-masked]
Subject: Re: References
Sender: [email-masked]

John Stanley wrote:

>Ken Murchison <ken@xxxxxxxxxx>:
>  
>
>>It is mandatory as stated in USEPRO.
>>    
>>
>It cannot be mandatory in USEPRO when it is already defined to be optional 
>in both USEFOR and RFC2822.
>
Surely it can, because USEFOR equally applies to *all* types of USEFOR 
articles and USEPRO may add additional restrictions for particular types 
of articles.

>>I'll reiterate what I stated elsewhere in the thread.
>>    
>>
>>USEFOR documents the proper syntax of the set of headers typically used in 
>>news articles.
>>    
>>
>That is not what you said. You said:
>  
>
>>It is my opinion (and I believe our Chair's) that USEFOR only document the 
>>syntax of an article.
>>    
>>
>
>The "syntax of an article" includes what headers it is required to 
>contain. The "syntax of a header" defines the contents of a header; the 
>syntax of an article defines the contents of an article. USEFOR is USENET 
>Format -- which is clearly not just the headers but the contents of 
>articles that are valid USENET articles. 
>
>USEFOR does not just tell us the correct syntax of the From header, it
>tells us that the From header is MANDATORY. It does the same for several
>other headers. Thus, it is clear that USEFOR is not just "syntax of
>headers", it includes "syntax of articles".
>
>In that job of "syntax of articles", USEFOR tells us that References is
>optional. References is optional.
>  
>
See my comment above.

Date: Tue, 26 Apr 2005 12:28:51 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Ken Murchison <ken@xxxxxxxxxx>:

> It is mandatory as stated in USEPRO.

It cannot be mandatory in USEPRO when it is already defined to be optional 
in both USEFOR and RFC2822. 

> I'll reiterate what I stated elsewhere in the thread.

> USEFOR documents the proper syntax of the set of headers typically used in 
> news articles.

That is not what you said. You said:

> It is my opinion (and I believe our Chair's) that USEFOR only document the 
> syntax of an article.

The "syntax of an article" includes what headers it is required to 
contain. The "syntax of a header" defines the contents of a header; the 
syntax of an article defines the contents of an article. USEFOR is USENET 
Format -- which is clearly not just the headers but the contents of 
articles that are valid USENET articles. 

USEFOR does not just tell us the correct syntax of the From header, it
tells us that the From header is MANDATORY. It does the same for several
other headers. Thus, it is clear that USEFOR is not just "syntax of
headers", it includes "syntax of articles".

In that job of "syntax of articles", USEFOR tells us that References is
optional. References is optional.

USEFOR and USEPRO are supposed to complement each other, not contradict.
Our Chair has said USEFOR is right not to talk about a mandate, although
he's said that because he thinks it can be slipped in after the fact in
USEPRO. I agree USEFOR is right not to talk about a mandate -- for a
different reason. If you want a mandate, do it honestly and in conformance
with RFC2119. Justify it and you can have it. So far, you haven't.
"Easier" and "play nice" don't even come close.

Date: Tue, 26 Apr 2005 12:11:54 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Suggested References texts
Sender: [email-masked]

>   pointing to that earlier article and any other "precursors".

USEFOR and RFC2822 contradict this. Since we've already decided to follow
RFC2822 in this matter, your text is wrong. RFC2822 is explicit in
providing guidance for processing of replies to replies that have no
References header; clearly a reply sans References header is allowed by
RFC2822.

>      Message-ID header content of the precursor.

RFC2822 contradicts this, and we have already decided that RFC2822 is the 
definition we are using. Further, these mandates have not been justified 
wrt RFC2119, which prohibits their use except for good cause.

>       NOTE: The two "MUST"s above ensure compliance with the
>       definition of the term "followup". 

This is absolutely ridiculous. RFC2119 does not allow mandates to be used  
just to "ensure compliance with a definition". There has to be an 
interoperatbility issue, and no such issue has been shown. This "MUST" is 
specious and violates existing standards, and contradicts our own draft
regarding article format.

>       with the weaker recomendation using "SHOULD" applied, in [RFC
>       2822], to the generation of "replies" in email. 

RFC2822 is not limited to "email", and it is dishonest to try to pretend 
that it is, especially when we defer to RFC2822 for news headers.

>perhaps for John) to write some text.

It is dishonest to ignore people who disagree with you, Charles. 

To: [email-masked]
Subject: Re: Suggested References texts
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 26 Apr 2005 12:08:24 -0700
Sender: [email-masked]

Date: Tue, 26 Apr 2005 13:41:58 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

John Stanley wrote:

> 
> I said:
> 
> 
>>The fact that I am the only one who even commented on it being gone is a
>>pretty good sign that nobody cares that it is gone.
> 
> 
> Ken Murchison <ken@xxxxxxxxxx> responds:
> 
> 
>>No, I knew that it wasn't there,

You have misquoted me here.  The above was written by Charles.

> 
> What are you saying "no" to? You did not comment on it being gone, at
> least not in any public forum I know of designed for this discussion.  
> Don't tell me I'm wrong when clearly I am not. The fact that you knew it
> wasn't there and did not comment is even stronger evidence that it not
> being there is ok by you. If you cared, and knew, you could have
> commented.
> 
> In fact, since your name is on the draft as an editor, you could have put 
> it back had you thought it necessary; the fact you had access to the 
> document and did not act to change it is an even stronger indication that 
> it is ok by you. And that it came OUT under your name is proof that it is 
> ok with you.
> 
> 
>>and there are a lot of other things that
>>should be in the Usefor draft and that have not made it there yet.

Again a misquote.  If you're going to quote text, please attribute it to 
the correct party.

> Since you conveniently reworded the problem into one of "should and should 
> not" instead of the actual "MUST and MUST NOT", you can easily say you see 
> no contradiction and then pretend that this lack of contradiction applies 
> to the real issue at hand. Nice trick. Didn't work this time.
> 
> You cannot say "this is optional" and "this is mandatory" at the same
> time. Pick one, stick with it, and accept the implications from your
> choice. I've covered the implications, and "mandatory option" is not one
> of them.

Thank you for your pleasant and constructive tone.

Is your argument that References shouldn't be required for followups 
(regardless of what USEFOR/USEPRO currently says) or that the current 
text is incorrect/ambiguous/contradictory/shitty?

If the former, please state so.  If the latter, please suggest some text 
for one or more of the documents that you feel is technically sound.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 09:49:15 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

I said:

>The fact that I am the only one who even commented on it being gone is a
>pretty good sign that nobody cares that it is gone.

Ken Murchison <ken@xxxxxxxxxx> responds:

>No, I knew that it wasn't there,

What are you saying "no" to? You did not comment on it being gone, at
least not in any public forum I know of designed for this discussion.  
Don't tell me I'm wrong when clearly I am not. The fact that you knew it
wasn't there and did not comment is even stronger evidence that it not
being there is ok by you. If you cared, and knew, you could have
commented.

In fact, since your name is on the draft as an editor, you could have put 
it back had you thought it necessary; the fact you had access to the 
document and did not act to change it is an even stronger indication that 
it is ok by you. And that it came OUT under your name is proof that it is 
ok with you.

>and there are a lot of other things that
>should be in the Usefor draft and that have not made it there yet.

This group has been working on this stuff for YEARS. "Not made it there, 
yet"? If it hasn't "made it there, yet", it's way too late. How many "last 
calls" has this group been through, already?

>I disagree. It is my opinion (and I believe our Chair's) that USEFOR only 
>document the syntax of an article. The use or non-use of any particular 
>"optional" header such as References belongs in either USEPRO or USEAGE.

If a header is MANDATORY in an article, then it is part of the syntax of 
that article. 

>I think you're taking the fact that neither RFC2822 nor USEFOR mandate 
>References too literally.

Excuse me? RFC2822 uses specific terms that say that the References header
is NOT mandatory, and even gives an example of how to deal with a reply
that does NOT contain a References header, and I'm being "too literal"?

This is a technical standard. Either it says what it means or it is 
broken. 

>USEFOR only documents what is necessary to make an article syntactically 
>correct, not what makes it useful in the Netnews world.

If a followup REQUIRES a References header, then a References header is 
REQUIRED to make that article "syntactically correct". Screw "useful", 
"useful" doesn't get you RFC2119 mandates. 

>In this regard, References is optional.

Yep. According to RFC2822 and draft-usefor-03, References is optional. In 
any article, even replies, and since replies are followups in our 
terminology, optional even in followups. 

>A "followup" article which doesn't have a References header is still 
>syntactically correct. 

Not if you think it MUST contain a References header. If you think it MUST 
contain a References header, a "followup that doesn't have a References 
header" is broken. Unfortunately, RFC2822 tells us how to process replies 
(one kind of followup) that don't contain a References header, so trying 
to pretend there is a mandate is just silly.

>USEPRO documents how to construct an article 
>("followup" or otherwise) which will play nice in the Netnews world.

Can we drop this stupid anthropomorphism? Articles don't "play", they are 
either correct or they are not. Transport agents either transport them or 
drop them. Reading agents either display them or do not. Articles without 
References headers exist and they are transported and displayed by the 
news system every day. They are not mandatory, and our draft now makes 
that clear. I'm fine with that. I'm not fine with this pretense that we 
can somehow sneak a mandate back into the standard after explicitely 
removing it. 

>In this regard, USEPRO can and should mandate that References be present 
>in "followup" articles.

Show me in RFC2119 where it uses the words "play nice" in regards to 
MUST/MUST NOT language. In fact, from what you've said about the 
References header, I see that RFC2119 says that we must not use RFC2119 
imperatives, since there is no requirement for interoperability in 
anything you've said. "Play nice" and "useful" are not reasons for 
RFC2119 language.

>USEPRO (and possibly USEAGE) will dictate when the optional headers 
>documented in USEFOR should and should not be used.

>I don't see a contradiction here, but maybe I'm wrong.

Since you conveniently reworded the problem into one of "should and should 
not" instead of the actual "MUST and MUST NOT", you can easily say you see 
no contradiction and then pretend that this lack of contradiction applies 
to the real issue at hand. Nice trick. Didn't work this time.

You cannot say "this is optional" and "this is mandatory" at the same
time. Pick one, stick with it, and accept the implications from your
choice. I've covered the implications, and "mandatory option" is not one
of them.

Date: Tue, 26 Apr 2005 12:37:58 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: Suggested References texts
Sender: [email-masked]

To: [email-masked]
Xref: clerew local.usefor:20722
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Suggested References texts
Date: Tue, 26 Apr 2005 15:12:10 GMT
Lines: 116
Sender: [email-masked]

Share and Enjoy!

Within USEFOR:

3.2.1  References 

Within USEPRO:

.......

Date: Tue, 26 Apr 2005 11:14:55 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

John Stanley wrote:

> 
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> 
>>Our Chair has ruled that the correct place to say such things is in USEPRO
>>rather than USEFOR.
> 
> 
> Then the Chair has effectively ruled that References is optional as
> defined in RFC2822 and that "followup" is a meaningless term, since nobody
> past the poster can identify what is and is not a followup using the
> optional definition of References. If the Chair thinks that we can say in
> one news standard that References is an optional header (by referring to
> the SHOULD appear and "may appear" definition in RFC2822) and then change
> our minds in mid-stream by saying that it is MANDATORY in another news
> standard, then he is demonstrating schizophrenia. Either it IS mandatory
> in followups or it is not.

It is mandatory as stated in USEPRO.

I'll reiterate what I stated elsewhere in the thread.

USEFOR documents the proper syntax of the set of headers typically used 
  in news articles.  USEPRO documents how to properly construct an 
article from this set per the Netnews "protocol" or "etiquette"

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 11:00:47 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

John Stanley wrote:

> 
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> 
>>Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx> writes:
> 
> 
>>>No References => no follow-up.
> 
> 
>>Which indeed follows from the fact that USEPRO says that a followup agent
>>MUST incorporate a References header (and do it correctly, of course).
> 
> 
> Unfortunately, RFC2822 says only SHOULD regarding the References header 
> and followups, and USEFOR says that the References header is defined as 
> found in RFC2822 (and for the pedants who cannot identify the context of 
> this discussion, I'll add "other than a few syntax changes that don't 
> change the requirements for usage, which is the context of this discussion 
> here.) So References in a followup is only a SHOULD, and if USEPRO says 
> otherwise, it contradicts both RFC2822 and USEFOR. While one of those we 
> can modify for good reason, the other is the result of our modifications, 
> and we did NOT modify the References header to make it mandatory in any 
> use.

I think you're taking the fact that neither RFC2822 nor USEFOR mandate 
References too literally.

USEFOR only documents what is necessary to make an article syntactically 
correct, not what makes it useful in the Netnews world.  In this regard, 
References is optional.  A "followup" article which doesn't have a 
References header is still syntactically correct.  This doesn't mean 
that its recommended or useful.

USEPRO documents how to construct an article ("followup" or otherwise) 
which will play nice in the Netnews world.  In this regard, USEPRO can 
and should mandate that References be present in "followup" articles. 
USEPRO (and possibly USEAGE) will dictate when the optional headers 
documented in USEFOR should and should not be used.

I don't see a contradiction here, but maybe I'm wrong.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:27:21 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Frank Ellermann <[email-masked]> writes:
> 
>> contents of an earlier article (the followup's "precursor"
>> indicated in the "References" header field).
> 
> 
> Yes, I had already concluded that something like that would be helpful in
> the definition. Russ seems to agree.

Done.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:24:39 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Russ Allbery <[email-masked]> writes:
> 
> 
>>Sounds right to me.
> 
> 
>>If it doesn't have References, it's not a follow-up, but in practice
>>people use References for things other than follow-ups, and I don't see
>>any reason to ban those uses.  So the question is whether we want to
>>artificially define all those things as follow-ups as well, or not worry
>>about it and just say that References is required for follow-ups but may
>>also be used for other things.
> 
> 
> Exactly. Those other uses for the References header deserve a brief
> mention somewhere, probably in USEFOR, but it would indeed be somewhat
> artificial to define those cases as "followups", and I personally do not
> want to do that.

I disagree.  It is my opinion (and I believe our Chair's) that USEFOR 
only document the syntax of an article.  The use or non-use of any 
particular "optional" header such as References belongs in either USEPRO 
  or USEAGE.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:16:21 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> John Stanley <[email-masked]> writes:
> 
> 
>>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> 
>>>The "MUST" in question arises from the long held consensus in this Working
>>>Group that the References header in News is NOT an "optional extra".
> 
> 
>>Not according to draft-usefor-03. According to draft-usefor-03, 
>>References is defined to be the same as in RFC2822, with the addition of a 
>>few bits that are not mandates for using it. This consensus you claim 
>>exists has a very odd way of being expressed in our products. Almost as if 
>>it doesn't really exist. 
> 
> 
>>The fact that I am the only one who even commented on it being gone is a
>>pretty good sign that nobody cares that it is gone.
> 
> 
> No, I knew that it wasn't there, and there are a lot of other things that
> should be in the Usefor draft and that have not made it there yet. But I
> do not want to be seen arguing with my co-editor in public if it can be
> avoided. Especially on issues that do not appear to be the subject of
> controversy.
> 
> 
> 
>>>That clear requirement has been in all our drafts up to article-13.
> 
> 
>>It is no longer a requirement at all in our draft.
> 
> 
> Indeed, but a draft is only a draft. There has been no WG decision to
> change the requirement for a References header in followups. Our Chair has
> taken the view that Usefor is not the place to say it, but that Usepro IS
> the proper place (and indeed there is already a corresponding MUST in
> Usepro, though it may need a little more amplification). I do not agree
> with the Chair on this one, but if nobody else speaks up and if his
> ruling stands, then Usepro is where it will be said (which also,
> incidentally, removes most of the cause of the disagreement there has been
> about how to word it all).

I am in agreement with the Chair that this discussion belongs in USEPRO 
(I thought we had discussed this offline, but perhaps not).

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:11:46 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor
Sender: [email-masked]

John Stanley wrote:

> 
> "Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:
> 
> 
>>And why does the Usefor draft contain the words
> 
> 
>>  None of the headers appearing in this section is required to appear
>>  in every article but some of them are required in certain types of
>>  article, such as followups.
> 
> 
> Because someone (an editor, perhaps?) overlooked this statement when
> removing the mandate for References? Remove the "such as" clause and the
> problem is fixed. Do it and lets move on. Problem solved.

Done.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:08:05 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Alexey Melnikov <[email-masked]>
CC: [email-masked]
Subject: Re: Obsolete vs. obsolescent
Sender: [email-masked]

Alexey Melnikov wrote:
>>
> In order to close this issue I suggest that the USEFOR document can 
> include few sentences describing what do we mean when calling a 
> particular header "obsolete".

Care to craft some text and tell me where you'd like to see it?

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Tue, 26 Apr 2005 10:02:19 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Sender: [email-masked]

Charles Lindsey wrote:

> In local.usefor you write:
> 
> 
>>------- Forwarded message -------
>>From: "Dr John Stockton" <[email-masked]>
>>To: "Charles. H. Lindsey" <[email-masked]>
>>Subject:  
>>http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
>>Date: Sun, 10 Apr 2005 19:44:55 +0100
> 
> 
> 
>>"1.1  Basic Concepts
>>   "Netnews" is a set of protocols for generating, storing and
>>   retrieving news "articles" (which are a subset of Email messages)"
> 
> 
>>They are surely not a subset of E-mail messages.  They might be
>>described as similar.  But E-mail is now commonly HTML & News should not
>>be; I'd omit the parenthetic analogy.
> 
> 
> Yes, it might be better to say "(whose format is a subset of that for
> Email messages)". Ken?

Yes, I'll make this change.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Mon, 25 Apr 2005 12:15:00 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Please show me where the word "followup" is to be found in RFC 2822.

I won't bother doing so, because we've already covered RFC2822 using the
term "reply" in the relevant section, and that all replies are followups,
using any common-sense or even news-based definition of the word followup.
Not all followups are necessarily replies, but every "reply" is "an
article posted in response to another".

Since RFC2822 clearly does NOT mandate References headers in all replies,
then it just as clearly exempts some followups -- the same replies that
have no mandate for its use are the same followups that have no mandate.
You cannot say that all followups MUST have a References header without
also saying all replies MUST, since the latter is a proper subset of the
former.

> USEFOR incorparates, by reference, many concepts from RFC 2822,

Including, explicitely, the References header. Thus RFC2822 not only tells 
us how to construct the "proper Email [sic]" that you grant in your 
previous paragraph, but a "proper News" article, because USEFOR says that 
RFC2822 applies to news, too.

>But nowhere does USEFOR
>equate the concepts of "Email Reply" with "Netnews Followup".

It doesn't need to. It is quite disengenuous of you to pretend that
RFC2822 says "Email Reply" in the section about References headers, and to
pretend that there has to be some equating of the two phrases for RFC2822
to apply. I've quoted the specific parts of RFC2822 that talk about
"replies", and they do not limit themselves to email, and USEFOR-03
explicitely expands any limit of "Email Reply" that you may fictionalize
for the References header and RFC2822 to "News reply", as well.

And you know very well that the two concepts do not have to be equivalent
for one to apply to the other. All replies are followups using standard
common-sense and common-practice news definitions for the same.  If you
say, as RFC2822 does, and draft-usefor-03 brings into the news context,
that not all replies require a References header, then you just as
clearly say not all followups require a References header. This is now
the third time in two days I've covered this ground for you, Charles.

>I grant you that the concepts are sufficiently similar that, wherever we
>say that News Followups are different in some way from Email Replies (e.g.
>MUST rather than SHOULD),

Except we DO NOT say this. We EXPLICITELY say that the RFC2822 definition
of References applies (with a few small syntax differences) but we do NOT
say that one of the differences is that References is a MUST. You can say
that you grant that the concepts are similar, even though you have just
argued that they are completely different and thus statements about one
thing do not apply to the other, but you cannot grant that we say that
there is a mandate, because draft-usefor-03 and RFC2822 clearly says
otherwise.

>Our Chair has ruled that the correct place to say such things is in USEPRO
>rather than USEFOR.

Then the Chair has effectively ruled that References is optional as
defined in RFC2822 and that "followup" is a meaningless term, since nobody
past the poster can identify what is and is not a followup using the
optional definition of References. If the Chair thinks that we can say in
one news standard that References is an optional header (by referring to
the SHOULD appear and "may appear" definition in RFC2822) and then change
our minds in mid-stream by saying that it is MANDATORY in another news
standard, then he is demonstrating schizophrenia. Either it IS mandatory
in followups or it is not.

I'll be happy with whichever position we take: either RFC1036 had it right 
and References is MUST/MUST NOT and followups are thus identifiable; or 
RFC1036 has it wrong, References is SHOULD/MAY and followups are not 
identifiable and thus a meaningless term. But since it has already been 
changed, it now requires justification for bringing back the MUST/anything 
language, since RFC2119 clearly limits such language to serious matters, 
and we can no longer use the 'it was pre-existing' excuse. It's not 
existing anymore; can you justify bringing it back?

I'll even suggest that if you cannot justify the return of a mandate (and
do not just unilaterally reinsert it as is your wont to do with our work
products), we can create a specific header to replace References, call it
"Thread-Me-Next-To:" or "Display-Near:" or something similar that makes it
clear this is a display issue and not a protocol one, and make that
optional. That's fine. It's being honest then about the value of the
information and not pretending it is some critical protocol
interoperability thing we're trying to solve.

And before you start playing the "show me where it says X" game with 
RFC1036 and MUST/MUST NOT, I'll point out that RFC1036 predates RFC2119 
but it uses clear terms like "required" and "prohibited", which map into 
RFC2119 MUST/MUST NOT. But we've been through that before, too, so I don't 
expect this preemptive note to have any effect on your argument.

>If a poster intends a followup, then a References header is REQUIRED.

No. Refer to RFC2822, which says that References is only SHOULD and 
"may", and even talks about messages that are clearly replies (contains 
In-Reply-To) but has no References.

>> That is a change, despite your repeated denials.

>Exactly. That has never been denied. 

You've repeatedly denied it. You keep saying we can use your proposed
wording because it makes no difference from what was there originally,
regarding MUST/MUST NOT and followups.  There's a difference from how
things are now. There's a change.

>You cannot tell from looking at the
>References header whether the poster intended to write a followup, or
>merely intended that it should be threaded like a followup (with all the
>other benefits accruing from a References header).

Let's put back some context to this, shall we? If you put back the
MUST/MUST NOT dichotomy for the References header, which has been removed
from draft-usefor-03 now (but still exists in news today due to RFC1036),
then you can tell precisely if the poster intended a followup or not
because a followup MUST have the header and a non-followup MUST NOT.  
(The third option is the poster is stupid/incompetent/ignorant and cannot
properly form a news article, but let's stick with properly formattted
news articles and not try to guess at the meaning of broken ones, ok?) It
is only when you water down the References header as it has now been that
you cannot tell what was intended. An article with a References header
(under draft-usefor-03) might be a followup or it might not. An article
without a References header might not be a followup, or it might be. You
have no way of knowing. You can only guess. Technical standards are broken
if they involve guessing about things that are supposed to be important, 
especially when they could easily be written to remove the guessing.

>So f*****g what? Why should the reader care?

If it is not important to the reader, then the References header
absolutely does not merit any RFC2119 language regarding its use, much
less MUST. It is not an interoperability issue in any case, and if the
information that it conveys is "so fucking what?" for the reader, then
lets make it obsolete and be done with it. I'm fine with that. References
is obsolete. The term "followup" is removed from the drafts. Let's
git-r-done. When can we expect you to produce the next version with these 
changes we've agreed to?

To: [email-masked]
Xref: clerew local.usefor:20720
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fixed (was: Broken Message-ID syntax)
Date: Mon, 25 Apr 2005 16:17:30 GMT
Lines: 131
Sender: [email-masked]

>>My preference is to retain the RFC 2822 naming, which accords with the
>>general principle that we do not change anything in RFC 2822 that we do
>>not need to change.
>>
>I concur with your preference.

In that case, we are ready for some syntax. I have given below what I now
think should be in the Message-ID section of USEFOR, including the
necessary syntax changes and rewordings of the descriptive text.

I have also taken the liberty of removing <msg-id-core> in favour of
<msg-id>, which I think we agreed in some earlier discussions on this list
(there will be a few consequential changes elsewhere in USEFOR).

Also, I have suggested a final paragraph regarding use of domains in the
<id-right>, which closes a possible shortcoming in the wording of RFC
2822, as spotted by Frank.

>>
>If WG members believe that better names should be used, I suggest they 
>talk to Pete Resnick first. As far as I remember he has already agreed 
>to publish 2822bis.

I don't think Pete has gone beyond agreeing in principle to 2822bis. Has
he actually published any Internet Drafts? If so, I will be needing to
comment upon them.

Here follows my suggested Message-ID text:

3.1.3  Message-ID 

   The Message-ID header contains a single unique message identifier.
   This document updates the <msg-id> construct from Section 3.6.4 of
   [RFC2822] so as to ensure that Internet Message Format Message-IDs
   are usable in widely deployed news software.  The global uniqueness
   requirement for <msg-id> in [RFC2822] is to be understood as applying
   across all protocols using such message identifiers, and across both
   Email and Netnews in particular.  A revised syntax for <msg-id> is
   given below, but the requirements and descriptive text from Section
   3.6.4 of [RFC2822] still apply.

|  message-id      =  "Message-ID:" SP [FWS] msg-id [FWS] CRLF
|
|  msg-id          =  "<" id-left "@" id-right ">"
|                     ; maximum length is 250 octets

   id-left         =  dot-atom-text / no-fold-quote

   id-right        =  dot-atom-text / no-fold-literal

|  no-fold-quote   =  DQUOTE
|                        (  "." *mqtext /
|                           *mqtext "." /
|                           *mqtext mqspecial *mqtext )
|                        DQUOTE
|
|  mqtext          =  atext / "." / mqspecial
|
   mqspecial       =  "(" / ")" /      ; same as specials except
                      "<" /            ; "\" and DQUOTE quoted
|                     "[" / "]" /      ; "." doubled and ">" omitted
                      ":" / ";" /
                      "@" / "," /
|                     ".." / "\\" / "\" DQUOTE

   no-fold-literal =  "[" *( mdtext / "\[" / "\]" / "\\" ) "]"
|
   mdtext          =  %d33-61 /        ; The rest of the US-ASCII
                      %d63-90 /        ; characters not including
                      %d94-126         ; ">", "[", "]", or "\"

   The msg-id-core MUST NOT be more than 250 octets in length.

      NOTE: The length restriction ensures that systems which accept
      message identifiers as a parameter when retrieving an article
      (e.g.  [NNTP]) can rely on a bounded length.

|  Observe that <msg-id >includes the < and >.

   Observe also that in contrast to the corresponding header in
|  [RFC2822]
| o the syntax does not allow comments within the Message-ID
|   header,
| o it ensures that no string of characters is quoted if it was already
|   a <dot-atom-text> (it must start or end with a ".", or contain at
|   least one mqspecial),
| o it ensures that no single character is prefixed by a "\" in the
|   form of a <uoted-pair> unless strictly necessary,
| o it excludes all control characters, and
| o there is no possibility for ">" or WSP to occur inside a
|   <msg-id>, whether quoted or not.
|
| This is to simplify processing by relaying and serving agents, and to
| ensure interoperability with existing implementations and compliance
| with [NNTP]. Thus, whereas under [RFC2822] the following <msg-id>s
| would be considered semantically equivalent,

|  <[email-masked]>
|  <"ab.cd"@example.com>
|  <"ab.\cd"@example.com>

   only the first of them is syntactically permitted by this standard,
   and hence a simple comparison of octets will always suffice to
   determine the identity of two <msg-id-core>s.

|  Also note that this updated ABNF applies wherever <msg-id> is used,
|  including the References header discussed in Section 3.2.1 and the
|  Supersedes header discussed in Section 3.2.5.

|    NOTE: It in RECOMMENDED in [RFC 2822] that, for ensuring global
|    uniqueness, the <id-right> be some domain identifier within whose
|    scope the uniqueness of the <id-left> can be guaranteed. When
|    following this recommendation, any <dot-atom-text> or
|    <no-fold-literal> used for the <id-right> are to be interpreted as
|    <domain>s as described in section 3.4.1 of [RFC 2822].

To: [email-masked]
Xref: clerew local.usefor:20719
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Mon, 25 Apr 2005 14:16:34 GMT
Lines: 69
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Unfortunately, RFC2822 says only SHOULD regarding the References header 
>and followups,

Please show me where the word "followup" is to be found in RFC 2822.

If you use your newsreader to send an Email reply to the author of a News
article, then indeed you (only) SHOULD include a References header (and
In-Reply-To as well to do the job properly).

If you use your newsreader to post a News followup to a News article
(usually in the same newsgroups as the precursor) then RFC 2822 is silent
on the issue (because the whole purpose of RFC 2822 is to tell you how to
construct a proper Email.

USEFOR incorparates, by reference, many concepts from RFC 2822, including
many headers and their meanings (semantics). But nowhere does USEFOR
equate the concepts of "Email Reply" with "Netnews Followup". Therefore
they remain different concepts with different rules.

I grant you that the concepts are sufficiently similar that, wherever we
say that News Followups are different in some way from Email Replies (e.g.
MUST rather than SHOULD), then we need to draw attention to this.

Our Chair has ruled that the correct place to say such things is in USEPRO
rather than USEFOR.

>You are apparently incapable of differentiating between "poster intends a 
>followup in what he sends" and "reader detects a followup in what he 
>gets." The previous standards and drafts have said that the intent for 
>something to be a followup by a poster REQUIRES a References header to be 
>inserted (MUST), and otherwise that header is prohibited. That means that 
>the READER can then determine what the poster intends to be a followup by 
>looking for a References header.

If a poster intends a followup, then a References header is REQUIRED.

If the poster does not intend a follouwp, but nevertheless includes a
References header (e.g. in a multi-part FAQ, or to stitch together a
message/partial), then what is the effect?

>And yes, Charles, when you say "non-followups MAY contain a References 
>header", you have removed that ability to DETECT at the reader's end of 
>the pipe what is and is not intended to be a followup. That is a change, 
>despite your repeated denials.

Exactly. That has never been denied. You cannot tell from looking at the
References header whether the poster intended to write a followup, or
merely intended that it should be threaded like a followup (with all the
other benefits accruing from a References header).

So f*****g what? Why should the reader care? So long as the reader (or his
agent) does the Right Thing (as intended by the poster in both cases), why
should anybody get upset?

To: [email-masked]
Xref: clerew local.usefor:20718
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Mon, 25 Apr 2005 13:47:06 GMT
Lines: 48
Sender: [email-masked]

>> How do we define the term "followup".

>| A "followup" is an article containing a response to the
>| contents of an earlier article (the followup's "precursor").

>Found in usefor-03.

Or as Alternative-1 in USEPRO.

>> or is it anything that has a References header.

Which is Alternative-2 in USEPRO.

I prefer Alternative-1, and I think most people are now of that view
(modulo minor wording tweaks).

>Not sure.  If it's the RfC 2822 idea of a reply coming from a
>mail2news gateway it probably is some "followup", but behind a
>gateway you never know.  Maybe it only had an In-Reply-To, the
>gateway tried to fix it, but it was a broken pipermail archive
>In-ReplyTo (the thread is then correct, but the "followup" is
>shown at the wrong place in the thread).

And lots of other ways it might have arisen. As Russ has said, referring
to some of those cases as followups would be "artificial".

>  contents of an earlier article (the followup's "precursor"
>  indicated in the "References" header field).

Yes, I had already concluded that something like that would be helpful in
the definition. Russ seems to agree.

Date: Sun, 24 Apr 2005 17:07:49 +0100
From: Alexey Melnikov <[email-masked]>
To: Frank Ellermann <[email-masked]>
CC: [email-masked]
Subject: Re: Obsolete vs. obsolescent
Sender: [email-masked]

Frank Ellermann wrote:

>Charles Lindsey wrote:
>  
>
>>"Obsolete" means this is no longer an official Netnews
>>header. It would be labelled as "obsolete" in the IANA
>>Registy of headers.
>>    
>>
>
>Bruce has created a draft with mail headers found in obsolete
>RfCs.  Among others Fcc: found in RfC 724 obsoleted by RfC 733.
>
>For news we'd declare See-Also: and Also-Control: as obsolete,
>as soon as Henry publishes s-o-1036 for information (obsoleted
>by - I'm an optimist here - draft-ietf-usefor-usefor-04.tx)
>  
>
In order to close this issue I suggest that the USEFOR document can 
include few sentences describing what do we mean when calling a 
particular header "obsolete".

>>"Obsolescent" means that is is still an official Netnews
>>header (just), but that its continued use is discouraged
>>(for reasons that are given), and that it is likely to be
>>declared fully obsolete in a future standard.
>>    
>>
>
>Now as pessimist:  There won't be any future standard, UseFor
>is the last news standard before its death.  So Lines: will be
>obsolescent "forever"
>
I tend to share your pessimism, but maybe for a slightly different reason.

> until the last news server goes jabber,
>or atom over soap based on beep, or whatever comes next.  Bye.
>  
>

Date: Sun, 24 Apr 2005 16:07:41 +0100
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked], Pete Resnick <[email-masked]>
Subject: Re: Fixed (was: Broken Message-ID syntax)
Sender: [email-masked]

Charles Lindsey wrote:

>In <[email-masked]> Frank Ellermann <[email-masked]> writes:
>  
>
>>Charles Lindsey wrote:
>>    
>>
>>>Yes, you have convinced me now.
>>>      
>>>
>>Great, we have a msg-id syntax.
>>    
>>
>>>there remains the issue of the naming of the syntax rules,
>>>      
>>>
>>We're talking about seven names.  So far we agree on msg-id,
>>and we disagree on names for LHS, RHS, and address-literal.
>>    
>>
>>Maybe we can solve it for the remaining three names without
>>bothering Alexey, Henry, or Ken:
>>    
>>
>
>Essentially, we either use the same names as RFC 2822 uses, saying that
>"the following syntax rules replace the corresponding rules in RFC 2822"
>(giving us id-left, id-right, no-fold-quote, no-fold literal, etc), or
>else we depart from the RFC 2822 names entirely, except for the msg-id at
>the top, in which case your names are as good as any. I don't see that
>there is anything in-between.
>
>My preference is to retain the RFC 2822 naming, which accords with the
>general principle that we do not change anything in RFC 2822 that we do
>not need to change.
>
I concur with your preference.

>If our Chair says that we can change these names, then
>let it be so. Alexey?
>  
>
If WG members believe that better names should be used, I suggest they 
talk to Pete Resnick first. As far as I remember he has already agreed 
to publish 2822bis.

Alexey

Date: Sun, 24 Apr 2005 16:16:28 +0100
From: Alexey Melnikov <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Russ Allbery wrote:

>John Stanley <[email-masked]> writes:
>  
>
>>USEFOR-03 says it is defined as found in RFC2822, and is
>>optional. Optional means "not mandatory". There is no MUST.
>>    
>>
>
>I think it's obvious that the References header has to be optional in that
>sense, since you can construct a well-formed Usenet message that doesn't
>have a References header.
>
>I'll repeat my earlier suggestion here, which is to simply define
>followups in such a way as to say that if something doesn't have a
>References header, it's not a followup.  That neatly avoids the whole
>issue without needing to worry about MUSTs and SHOULDs.  It becomes a
>definitional issue.  If it doesn't have References, it's not a followup.
>  
>
I fully agree.

>>Like I already said, once you remove the ability to detect followups by
>>making References optional, it doesn't matter when it may or may not be
>>used. In fact, it may be used anytime (that's what "optional" means),
>>even apparently when there is nothing at all for it to convey.
>>    
>>
>>By changing the requirement for References, we've solve a whole passle
>>of things that should be be CHANGED BACK to the way they were before you
>>decided to take it upon yourself to change them. Like "non-followups MAY
>>contain" References headers, which, despite your repeated claims to the
>>contrary, is a major change that nobody asked for.
>>    
>>
>
>Well, to be fair, I think it's a reasonable way of dealing with the issue.
>I'm not sure if that means I was asking for it, but I can see the logic
>behind it.
>
>The way I look at this, there are two separate issues that we keep mixing
>together:
>
> * What a news reader should do when constructing a posted reply to a
>   particular post.
>
> * How a news reader should display messages with References headers.
>
>The former is what we mean by a followup, and by definition it has to
>build a References header.  If it doesn't, it's doing something other than
>posting a followup.
>
>The latter is a separate issue, a UI issue.  It is expected that messages
>connected by References will be shown in a thread, subject to various user
>configuration and choices like whether changing the Subject should result
>in a separate thread in the user's display (something that should be a
>configuration option, not something the standard should be mandating
>either way, as there are people who prefer it both ways and there is no
>interoperability issue).
>
>There are various reasons why sometimes people like to connect messages
>that are not posted replies to each other, and that do not, or only barely
>do, fit into the notion of a followup.
>
>I don't think there's really anything else we can say that's more specific
>than that.
>  
>
I agree. And my guess is that RFC 2822 says so little on the subject 
exactly for the same reasons.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Obsolete vs. obsolescent (was: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt)
Date:  Sat, 23 Apr 2005 00:33:26 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Charles Lindsey wrote:

> "Obsolete" means this is no longer an official Netnews
> header. It would be labelled as "obsolete" in the IANA
> Registy of headers.

Bruce has created a draft with mail headers found in obsolete
RfCs.  Among others Fcc: found in RfC 724 obsoleted by RfC 733.

For news we'd declare See-Also: and Also-Control: as obsolete,
as soon as Henry publishes s-o-1036 for information (obsoleted
by - I'm an optimist here - draft-ietf-usefor-usefor-04.tx)

> "Obsolescent" means that is is still an official Netnews
> header (just), but that its continued use is discouraged
> (for reasons that are given), and that it is likely to be
> declared fully obsolete in a future standard.

Now as pessimist:  There won't be any future standard, UseFor
is the last news standard before its death.  So Lines: will be
obsolescent "forever" until the last news server goes jabber,
or atom over soap based on beep, or whatever comes next.  Bye.

Date: Fri, 22 Apr 2005 13:40:13 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>Since References is now optional, yes,

>No, they are NOT optional. Read Usepro. 

Read USEEFOR, where the header is defined, and then read RFC2822, which is
what USEFOR refers to. That "SHOULD" in RFC2822 means it is not mandatory,
it is just a "strong" recommendation. A recommendation is still just a
recommendation.

>Everything that comes out of a followup agent MUST have one.

Not according to RFC2822, which is where the header is actually defined.

>Please stop deliberately distorting what everyone else is saying.

Coming from someone who deliberately and repeatedly attempts to put words
in my mouth, your command is simply ludicrous. I'm not distorting
anything;  I'm reading the draft USEFOR standard and RFC2822. I suggest
you try doing so before you toss about claims of distortion.

>> something without References can be a followup.

>No.

Of course it can. RFC2822 says, and to avoid more condemnation from you 
that I am "distorting" something, I'll quote:

references      0*              1               SHOULD occur in some
                                                replies - see 3.6.4

"Some replies". Doesn't even go as far as to say "SHOULD occur in
all replies", just "some". And:

   Furthermore, reply messages SHOULD have "In-Reply-To:" and
   "References:" fields as appropriate, as described below.

Just SHOULD. And:

	 The
   "In-Reply-To:" field may be used to identify the message (or
   messages) to which the new message is a reply, while the
   "References:" field may be used to identify a "thread" of
   conversation.

"May". Not a requirement. Furthermore, RFC2822 continues:

   If the parent message does not contain
   a "References:" field but does have an "In-Reply-To:" field
   containing a single message identifier, 

So, RFC2822 explicitely covers an example case of a "reply" (followup, in
USENET terminology) that does NOT contain a References header.

>No, its primary purpose is to facilitate threading,

Threading is a user interface issue, and we do not deal with user
interface issues. Furthermore, the "threading" you speak of is due to the
followup nature of the articles it identifies as such. That makes its
PRIMARY purpose to IDENTIFY followups (and the articles to which this is a
followup); after that, the user agent can do what it wants with them when
it displays them.

>No. If you read USEPRO carefully, 

If you read RFC2822 carefully, which you must do if you want to understand
USEFOR, you will see that References is OPTIONAL, and SHOULD appear in
SOME replies. Not mandatory in anything.

>you will see that threading (or other
>means of presenting lists of articles for display) is an (optional)
>feature of the protocol;

It is ridiculous to argue that a header that implements an OPTIONAL
"feature of the protocol" is in any way MANDATORY. But then, it's not a
"feature of the protocol", it is just a user-interface issue, and that
makes it even more ridiculous to argue that it is mandatory. Nothing at
all cares about that header in the protocol; only the user agent that
wants to do real threading cares, and it has to be able to deal with
articles sans References headers already. It can deal with them; why can't 
you?

>...they are entitled to assume that the headers they rely on for
>the purpose (References, Subject, Date) have been properly constructed,

Nobody but YOU is talking about the syntax of the headers. The issue at 
hand is a mandate for CONTAINING a header. Said mandate does not exist. It 
is an etirely different issue to say "IF you contain header X, it MUST be 
constructed this way" than "you MUST contain header X". The former is 
without question. We do it all the time. The latter requires a 
justification for interoperability on the EXISTANCE of the header, not 
just ability to decode it if it there. 

I'm quite happy if you want to say "the References header MUST contain 
...". RFC2822 already deals with the question of when References headers 
are mandatory, and the answer is "they are not."

>No it doesn't. It says they should be used for "Replies", which is not
>quite the same thing.

Close enough. All replies are followups. When RFC2822 says that the header
SHOULD appear in replies, then whatever REPLIES it excludes from a mandate 
are also FOLLOWUPs that are excluded.

>Well you are giving a good impression of not caring :-( . And even if you
>don't care, then I still do.

No, Charles, you do not. When you promote the concept "non-followups MAY 
contain References header", you exhibit a complete lack of caring about 
the ability to detect what is and is not a followup. It is only when the 
References header is in the current (RFC1036) MUST/MUST NOT version can 
you use it to identify a followup. MUST/MAY throws that ability away.

>> and 2) the existing standard does,

>Which is a good reason for keeping it so,

One cannot "keep" what one has already let go. draft-usefor-03 removed the
mandate. It's gone. If you want it back, justify it. "existing standard
says"  wasn't sufficient to keep it in usefor-03, so it's not sufficient
to put it back. "Charles says" is even less convincing.

>... and we should not change it
>without a clear agreement in this WG to do so.

Given your history of making changes to the drafts without any input from
the WG at all, much less a "clear agreement", this sudden religion you
have gotten for sticking to the process is, shall I say, questionable. And 
considering that we are discussing a change to the draft that was made 
with NO input from the WG in the first place ...

>>Currently, you cannot construct a well-formed USENET message that is a
>>followup without a References header. That is a direct effect of the
>>language in RFC1036 that says "It is required for all follow-up messages".

>And it's a direct effect of what USEPRO says.

Baloney. USEPRO is a DRAFT standard. It has no effect on existing 
requirements, only on what might be required some day maybe. When I say 
"currently ...", USEPRO is irrelevant. IF USEFOR-03 ever makes it to 
standard, then USEFOR will tell me that I CAN construct a well-formed
USENET message that is a followup without a References header, because 
RFC2822 says I can. POOF goes your caring about what is and is not a 
followup.

>>We are not debating what is and is not a followup here.

>Yes we are.

Maybe you are, but the topic of THIS discussion is the lack of mandate for 
a References header, and that has NOTHING to do with what is or is not a 
followup, only on whether there is a mandate for using that header in any 
circumstance. RFC2822 says SHOULD. That is not a mandate. End of story.

>And the opinion I am hearing (even from you now) is that it is
>whatever a followup agent produces. 

Stop putting words in my mouth, Charles. I'm getting fucking tired of it.
This is no longer just a difference of opinion, it's you pretending to 
speak for me, and you do NOT have the right to do that. Period. 

Date: Fri, 22 Apr 2005 16:18:24 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

> Either we say "RFC1036 was right" regarding the References header and
> followups and propogate that, with all the implications, into the new
> standard, or we say "RFC1036 was wrong" and propogate that, with all the
> implications, into the new standard. This half of one, half of the other
> attitude is ridiculous. If RFC1036 is right, then Rererences is a
> MUST/MUST NOT header, and followup is a valuable concept because we can
> identify them at the recipient's end. If RFC1036 is wrong, then References
> is a MAY/SHOULD/SOMETIMES header and followup is meaningless because we
> cannot identify them.
> 
> 

Me too.

John has been pointing out problems with this all along,
much past the point of getting ignored.  So I want to be on record that
I tend to think this paragraph sums it up.

There is no reason USEFOR should get away with the result of something like a
bad Abbot and Costello routine.

Date: Fri, 22 Apr 2005 12:52:39 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

>I don't want to use any sort of SHOULD or MUST in this area (beyond the
>one we inherit from RFC 2822). 

Ok. I'm happy with that. But we cannot then change our minds in mid-stream 
and put a SHOULD NOT or MUST into other pieces of the news standards, 
since that would contradict USEFOR and RFC2822.

>I want to say that, in a Usenet context, a
>followup is *defined* to be a message posted in reply to another message
>with a References header pointing to the message being replied to.

So what? Not "so what, you want to say", but "so what if a followup is 
defined that way?" You can't identify one as such using the RFC2822 
headers, so why bother putting a special name on it? It's like defining 
a "narquark" as "an article posted by a person with a mole on his left 
shoulder." If you cannot identify "narquarks", you aren't able to treat 
them specially, so they are still just "articles" and get treated like any 
other.

In other words, if it is important enough to DEFINE what a followup is
(presumably so it can be treated specially), then it is important enough
to provide a mechanism to identify followups at the point you want to
apply that special treatment. The only person who really knows "this is a
response to somethign else" is the poster. Without a References MUST/MUST
NOT dichotomy, you cannot identify them once they leave his hands. You get
"well, this sorta looks like a followup" and MAYBE it is, maybe it isn't.

(And in case the argument that "it guesses right often enough", I'll point 
out that this is a TECHNICAL standard and TECHNICAL standards don't 
promote guessing, especially when they could promote fact just as easily.)

>In other words, if your news posting software doesn't insert a References
>header, it's still posting well-formed messages.  It's just not posting
>*followups*, by definition.  

Well, I guess we have to disagree here. I don't think you can isolate the 
concept of a "followup" from the intent of the poster. An article that 
someone posts using F in trn, where he puts in a new subject and entirely 
new content, but forgets to remove the References header, is not really a 
followup of any kind. The recipient would treat it as such based on the 
References header, but it really isn't. It's really an incorrectly 
formatted article (today) because it contains a header it is prohibited 
from containing based on its purpose.

>In practice, I don't think there's any need to distinguish, for UI
>purposes, between different reasons why an article cites a previous
>article with References.  

Then "followup" is a meaningless concept, since it cannot be identified 
for special treatment. 

>The function the References header provides is enabling construction of
>threads.  This is closely related to the notion of followups but, to me,
>is not identical to it.

What is a "thread" if there is no "followup"? What is a "thread" when a 
"thread" may contain any collection of articles at all, not just ones that 
are somehow related in the real world?

So, this looks like a complete turnaround in what I'm saying, huh? No 
followups, no threads, References meaningless? Nope. Here's the crux:

Either we say "RFC1036 was right" regarding the References header and
followups and propogate that, with all the implications, into the new
standard, or we say "RFC1036 was wrong" and propogate that, with all the
implications, into the new standard. This half of one, half of the other
attitude is ridiculous. If RFC1036 is right, then Rererences is a
MUST/MUST NOT header, and followup is a valuable concept because we can
identify them at the recipient's end. If RFC1036 is wrong, then References
is a MAY/SHOULD/SOMETIMES header and followup is meaningless because we
cannot identify them.

Date: Fri, 22 Apr 2005 11:31:48 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: References
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx> writes:

>>No References => no follow-up.

>Which indeed follows from the fact that USEPRO says that a followup agent
>MUST incorporate a References header (and do it correctly, of course).

Unfortunately, RFC2822 says only SHOULD regarding the References header 
and followups, and USEFOR says that the References header is defined as 
found in RFC2822 (and for the pedants who cannot identify the context of 
this discussion, I'll add "other than a few syntax changes that don't 
change the requirements for usage, which is the context of this discussion 
here.) So References in a followup is only a SHOULD, and if USEPRO says 
otherwise, it contradicts both RFC2822 and USEFOR. While one of those we 
can modify for good reason, the other is the result of our modifications, 
and we did NOT modify the References header to make it mandatory in any 
use.

>A pure logical deduction indeed. But for the removal of all doubt, and to
>emphasise the difference from the treatment of "replies" in RFC 2822, we
>used to say that as well. Now we don't (because our Chair so decrees), but
>because it was redundant nothing has actually changed (John's claims to
>the contrary notwithstanding).

Stop putting words in my mouth, Charles. 

>Is is the the thing which is produced (as carefully defined in USEPRO) by
>a "followup agent", or is it anything that has a References header. I
>prefer the former (and you seem to have been using it in that sense), and
>John has been arguing for the latter. Technically, it makes no difference
>so long as we use the term consistently.

Stop putting words in my mouth, Charles. You don't understand what I've
said, so you are the last person who ought to be pretending you do. You
being the editor and able to make willy-nilly changes to the drafts does
NOT give you the right to put words specifically in MY mouth, even though
you are putting words regularly into the mouths of the group as a whole.

You are apparently incapable of differentiating between "poster intends a 
followup in what he sends" and "reader detects a followup in what he 
gets." The previous standards and drafts have said that the intent for 
something to be a followup by a poster REQUIRES a References header to be 
inserted (MUST), and otherwise that header is prohibited. That means that 
the READER can then determine what the poster intends to be a followup by 
looking for a References header.

And yes, Charles, when you say "non-followups MAY contain a References 
header", you have removed that ability to DETECT at the reader's end of 
the pipe what is and is not intended to be a followup. That is a change, 
despite your repeated denials.

Date: Fri, 22 Apr 2005 11:16:59 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

> No, USEPRO-03 does not say it is optional.

Read what I wrote, Charles, and respond to that. I said:

>USEFOR-03 says it is defined as found in RFC2822, and is optional. 

USEFOR is where the definition appears, and that definition defers to 
RFC2822 for almost everything. One of the areas it does NOT change RFC2822 
is in the requirement for use of the header, which says "SHOULD". There is 
no MUST, either in RFC2822 or USEFOR. The header is optional, even in 
followups.

Now, I realize that YOU say that RFC2119 "SHOULD" is a requirement of some
kind, but RFC2119 actually says that it is a recommendation, and
"required" and "recommended" are two very different words with two very 
different meanings, even when one of them carries an additional adjective 
"STRONG".

To: [email-masked]
Xref: clerew local.usefor:20706
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Fri, 22 Apr 2005 13:29:11 GMT
Lines: 106
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>Since References is now optional, yes,

No, they are NOT optional. Read Usepro. Everything that comes out of a
followup agent MUST have one. Please stop deliberately distorting what
everyone else is saying.

> something without References can be 
>a followup.

No.

 And something WITH a References can be something other than a 
>followup.

Yes.

>The only purpose for the References header is to identify followups and to
>what article the present one is a followup.

No, its primary purpose is to facilitate threading, but it is also useful
to identify followups, as you say, and also to enable quick retrieval of
precursors by clicking on them. And to stitch message/partials together.
And maybe other things.

> Previous thought on the matter
>was that this function ('identify followups') was important enough to
>justify an RFC2119 mandate. It is, after all, how true threading works and
>the only way it can be accomplished. But threading is a display issue, 
>not an interoperability one, so it was an abuse of the language to 
>pretend it merited an RFC2119 "MUST". 

No. If you read USEPRO carefully, you will see that threading (or other
means of presenting lists of articles for display) is an (optional)
feature of the protocol; i.e. implememtors MAY provide it and, if they so
choose, then they are entitled to assume that the headers they rely on for
the purpose (References, Subject, Date) have been properly constructed,
and this merits MUST or SHOULD wording in the appropriate places.

>Russ Allbery <rra@xxxxxxxxxxxx>:

>>So the question is whether we want to
>>artificially define all those things as follow-ups as well, or not worry
>>about it and just say that References is required for follow-ups but may
>>also be used for other things.

>RFC2822 says that References SHOULD be used in followups.

No it doesn't. It says they should be used for "Replies", which is not
quite the same thing. However, followups and replies are sufficiently
similar that if we put a MUST somewhere regarding followups, we need to
draw attention to the difference from the treatment of replies by RFC
2822.

>And if you remove the one function that the header performs today 
>("identify a followup"), then why bother keeping that header? How do you 
>differentiate between "this article is a followup", and "this article 
>contains a References header for some other reason"? And if you say "who 
>cares?", I'll say 1) I do,

Well you are giving a good impression of not caring :-( . And even if you
don't care, then I still do.

> and 2) the existing standard does,

Which is a good reason for keeping it so, and we should not change it
without a clear agreement in this WG to do so.

>>I think it's obvious that the References header has to be optional in that
>>sense, since you can construct a well-formed Usenet message that doesn't
>>have a References header.

>Currently, you cannot construct a well-formed USENET message that is a
>followup without a References header. That is a direct effect of the
>language in RFC1036 that says "It is required for all follow-up messages".

And it's a direct effect of what USEPRO says.

>>I'll repeat my earlier suggestion here, which is to simply define
>>followups in such a way as to say that if something doesn't have a
>>References header, it's not a followup.

>We are not debating what is and is not a followup here.

Yes we are. And the opinion I am hearing (even from you now) is that it is
whatever a followup agent produces. I think, though, I would also want to
include the case where a user constructs a followup manually (possibly to
more than one precursor, as you seem to do) and generates a suitable
References header manually.

To: [email-masked]
Xref: clerew local.usefor:20705
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Fri, 22 Apr 2005 12:09:06 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I got this part, but it doesn't answer my confusion above.  What's the
>difference between obsolete and obsolescent?

"Obsolete" means this is no longer an official Netnews header. It would be
labelled as "obsolete" in the IANA Registy of headers.

"Obsolescent" means that is is still an official Netnews header (just),
but that its continued use is discouraged (for reasons that are given),
and that it is likely to be declared fully obsolete in a future standard.

Our earlier drafts uses the words "is to be regarded as obsolete", which
falls slightly short of "is obsolete" and would IMO be better expressed as
"obsolescent", if indeed that is the meaning we wish to convey (it was
certainly the meaning I intended to convey in those drafts).

To: [email-masked]
Subject: Re: References
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 21 Apr 2005 21:25:34 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> ACK.   You could add "it has a References header field" to be
> absolutely sure:

>   contents of an earlier article (the followup's "precursor"
>   indicated in the "References" header field).

This is the definition that I'd go for.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: References
Date:  Fri, 22 Apr 2005 05:25:05 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 37
Sender: [email-masked]

Charles Lindsey wrote:

> How do we define the term "followup".

| A "followup" is an article containing a response to the
| contents of an earlier article (the followup's "precursor").

Found in usefor-03.  It's used for things like "Followup-To: ".
Just above the definition of "Followup agent" in chapter 1.5
"Definitions".

> Is is the the thing which is produced (as carefully defined
> in USEPRO) by a "followup agent"

Sure.

> or is it anything that has a References header.

Not sure.  If it's the RfC 2822 idea of a reply coming from a
mail2news gateway it probably is some "followup", but behind a
gateway you never know.  Maybe it only had an In-Reply-To, the
gateway tried to fix it, but it was a broken pipermail archive
In-ReplyTo (the thread is then correct, but the "followup" is
shown at the wrong place in the thread).

> I prefer the former (and you seem to have been using it in
> that sense)

ACK.   You could add "it has a References header field" to be
absolutely sure:

  A "followup" is an article containing a response to the
  contents of an earlier article (the followup's "precursor"
  indicated in the "References" header field).

                                Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20700
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References
Date: Thu, 21 Apr 2005 16:29:29 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Sounds right to me.

>If it doesn't have References, it's not a follow-up, but in practice
>people use References for things other than follow-ups, and I don't see
>any reason to ban those uses.  So the question is whether we want to
>artificially define all those things as follow-ups as well, or not worry
>about it and just say that References is required for follow-ups but may
>also be used for other things.

Exactly. Those other uses for the References header deserve a brief
mention somewhere, probably in USEFOR, but it would indeed be somewhat
artificial to define those cases as "followups", and I personally do not
want to do that.

>The second seems clearer and simpler to me.

Yes, that now seems to be where we are headed.

To: [email-masked]
Xref: clerew local.usefor:20699
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: References (was: Differences between RFC 2822 and Usefor (fwd))
Date: Thu, 21 Apr 2005 16:22:39 GMT
Lines: 62
Sender: [email-masked]

>> There has been no WG decision to change the requirement for
>> a References header in followups.

>I'm completely lost with this stuff.  What's the precise
>problem ?  Something without References is never a follow-up.

>No References => no follow-up.

Which indeed follows from the fact that USEPRO says that a followup agent
MUST incorporate a References header (and do it correctly, of course).

>If that's the case we obviously also have:

>Follow-up => references.

A pure logical deduction indeed. But for the removal of all doubt, and to
emphasise the difference from the treatment of "replies" in RFC 2822, we
used to say that as well. Now we don't (because our Chair so decrees), but
because it was redundant nothing has actually changed (John's claims to
the contrary notwithstanding).

>Based on this premise we don't have (and IMHO don't want):

>References => follow-up

But there is just one niggle here (yes, I agree we want sometimes to use
References for multi-part FAQs and other special cases).

How do we define the term "followup".

Is is the the thing which is produced (as carefully defined in USEPRO) by
a "followup agent", or is it anything that has a References header. I
prefer the former (and you seem to have been using it in that sense), and
John has been arguing for the latter. Technically, it makes no difference
so long as we use the term consistently.

>> I do not agree with the Chair on this one

>I'm not even sure where you disagree.  Is it related to
>anything I said above ?

It is because I preferred to mention it in both places, for the removal of
all doubt as stated above, and also to provide a convenient place to
introduce the special cases (FASQs etc) and to exaplain the semantics of
the header properly. That still has to be done somewhere, and probably in
USEFOR.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: References
Date:  Fri, 22 Apr 2005 03:36:29 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

Russ Allbery wrote:

> people normally associate the idea of followup with a reply

People: yes.  Anybody who knows RfC 2822 and (s-o-) 1036: no.

A followup is defined in usepro, a reply is defined in 2822,
they are to great parts unrelated and / or different.  There
is not one string "followup" in the complete 2822 text.  Bye.

To: [email-masked]
Subject: Re: References
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 21 Apr 2005 15:00:54 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> RFC2822 says that References SHOULD be used in followups. This is not a
> requirement, it is only a recommendation. What is the justification for
> making References a requirement? What interoperability issue exists? 
> What breaks if a References header is not in an article, and don't say
> "news readers", because news readers deal all the time with articles
> that don't have References headers. They have to, since currently the
> standard says that non-followups must not have them. I've yet to see a
> news reader that doesn't deal with non-followups; I suggest that if you
> have one, you get a better newsreader.

I don't want to use any sort of SHOULD or MUST in this area (beyond the
one we inherit from RFC 2822).  I want to say that, in a Usenet context, a
followup is *defined* to be a message posted in reply to another message
with a References header pointing to the message being replied to.

In other words, if your news posting software doesn't insert a References
header, it's still posting well-formed messages.  It's just not posting
*followups*, by definition.  (So if your software calls such messages
followups, it has a serious definitional problem.)

I think we're mostly agreeing.  I want to take this approach since it
avoids the whole question of whether things really break without
References headers.

> And if you remove the one function that the header performs today
> ("identify a followup"), then why bother keeping that header? How do you
> differentiate between "this article is a followup", and "this article
> contains a References header for some other reason"?

In practice, I don't think there's any need to distinguish, for UI
purposes, between different reasons why an article cites a previous
article with References.  I just think it's a definitional stretch to call
all such messages followups, since people normally associate the idea of
followup with a reply and clearly not all of those articles are replies.

> And if you say "who cares?", I'll say 1) I do, and 2) the existing
> standard does, and 3) if nobody does, then get rid of the header, since
> it performs no useful function.

The function the References header provides is enabling construction of
threads.  This is closely related to the notion of followups but, to me,
is not identical to it.

> True wrt RFC1036. Untrue wrt draft-usefor-03. SHOULD is a
> recommendation.  However strong you think it is, it is still only a
> recommendation.

Right, I agree that what I'm pushing for is not what's in the current
language.

To: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 21 Apr 2005 12:04:23 -0700
Sender: [email-masked]

>> Then I don't understand the distinction you're drawing between obsolete
>> and obsolescent.  As near as I could tell, the only difference in
>> practice was that servers were rejecting the former headers, but now
>> you're saying that they shouldn't even do that.

> Generally speaking, our draft has always said (and I think Usefor still
> says) that if you see a header that you do not recognize as a news
> header, you just leave it be and ignore it. Essentially the same applies
> in Mail.

I got this part, but it doesn't answer my confusion above.  What's the
difference between obsolete and obsolescent?

Date: Thu, 21 Apr 2005 10:49:17 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: References
Sender: [email-masked]

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>I'm completely lost with this stuff.  What's the precise
>problem ?  Something without References is never a follow-up.

Since References is now optional, yes, something without References can be 
a followup. And something WITH a References can be something other than a 
followup. The current language says that followups SHOULD have that 
header, it does not prohibit it in anything else.

>If that's the case we obviously also have:
>Follow-up => references.

Not obvious. The header is optional. See RFC2822.

>Based on this premise we don't have (and IMHO don't want):
>References => follow-up

The only purpose for the References header is to identify followups and to
what article the present one is a followup. Previous thought on the matter
was that this function ('identify followups') was important enough to
justify an RFC2119 mandate. It is, after all, how true threading works and
the only way it can be accomplished. But threading is a display issue, 
not an interoperability one, so it was an abuse of the language to 
pretend it merited an RFC2119 "MUST". 

If you have some other function in mind, please enlighten us, and then
justify any mandates you want to apply. What interoperability problem does 
making References mandatory solve?

>Now to the claim that it's exactly the same as in RfC 2822:
>That's not exactly true. 

Of course it isn't true. Nobody has said it is. Read draft-usefor-03 to
see the differences.  None of those differences deals with a mandate for
use. Notice that "we" were able to insert the differences that "we" 
thought were important from RFC2822. "We" apparently didn't think a MUST 
was important; I agree. Let's move on.

Russ Allbery <rra@xxxxxxxxxxxx>:

>So the question is whether we want to
>artificially define all those things as follow-ups as well, or not worry
>about it and just say that References is required for follow-ups but may
>also be used for other things.

RFC2822 says that References SHOULD be used in followups. This is not a 
requirement, it is only a recommendation. What is the justification for 
making References a requirement? What interoperability issue exists? What 
breaks if a References header is not in an article, and don't say "news 
readers", because news readers deal all the time with articles that don't 
have References headers. They have to, since currently the standard says 
that non-followups must not have them. I've yet to see a news reader that 
doesn't deal with non-followups; I suggest that if you have one, you get a 
better newsreader.

And if you remove the one function that the header performs today 
("identify a followup"), then why bother keeping that header? How do you 
differentiate between "this article is a followup", and "this article 
contains a References header for some other reason"? And if you say "who 
cares?", I'll say 1) I do, and 2) the existing standard does, and 3) if 
nobody does, then get rid of the header, since it performs no useful 
function.

>> So what's the problem ?  It's an optional header field as
>> in RfC 2822, and its presence does not automatically mean
>> "follow-up", it could be also something else.

>Exactly.

So the purpose of References headers is now deprecated. That's an argument 
for making them optional (as we have already done), not for making them a 
requirement.

>I think it's obvious that the References header has to be optional in that
>sense, since you can construct a well-formed Usenet message that doesn't
>have a References header.

Currently, you cannot construct a well-formed USENET message that is a
followup without a References header. That is a direct effect of the
language in RFC1036 that says "It is required for all follow-up messages".
While the article may appear to be well-formed, it is technically in
violation of the standards and thus is not valid. This is exactly the
reason why I argued that a specific proposed moderated newsgroup that was
intended to contain only followups should enforce that requirement, and, 
as I recall, you agreed with the technical argument but disagreed with 
doing that for social/political reasons. Correct me if I am wrong in my 
recollection.

Once you say it is optional, you cannot also say it is mandatory. If 
USEFOR defines valid USENET format messages, and USEFOR says that a header 
is optional, then any message that does not contain that header can be 
valid. Another standard for the same medium that says "this article is not 
valid", when USEFOR says it is, is contradictory and broken.

>I'll repeat my earlier suggestion here, which is to simply define
>followups in such a way as to say that if something doesn't have a
>References header, it's not a followup.

We are not debating what is and is not a followup here. We are discussing 
the fact that our draft now defers the definition of References to RFC2822 
with certain limited differences, none of which deal with changing the 
RFC2119 status of that header. 

References is now a SHOULD, not a MUST. If it is to be changed to a MUST,
then justify the change. If you cannot justify the change, it ought not be
made. This group has suffered too many times from unilateral changes made
at the whim of the person who controls the text to allow it to continue.
Charles says it needs to be changed back;  Charles ought to justify that
"need" or drop it.

> * What a news reader should do when constructing a posted reply to a
>   particular post.

>[That] is what we mean by a followup, and by definition it has to
>build a References header.  If it doesn't, it's doing something other than
>posting a followup.

True wrt RFC1036. Untrue wrt draft-usefor-03. SHOULD is a recommendation.
However strong you think it is, it is still only a recommendation.

To: [email-masked]
Xref: clerew local.usefor:20697
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Thu, 21 Apr 2005 15:44:40 GMT
Lines: 34
Sender: [email-masked]

>> Ah! But in that case I don't think servers should be rejecting such
>> headers. If the article has been wandering in any out of both email and
>> news systems, then it is sometimes useful to have the full history
>> recorded (though I might agree that if there was a mailing list involved
>> I might not want to know what happened before it reached the mailing
>> list).

>Then I don't understand the distinction you're drawing between obsolete
>and obsolescent.  As near as I could tell, the only difference in practice
>was that servers were rejecting the former headers, but now you're saying
>that they shouldn't even do that.

Generally speaking, our draft has always said (and I think Usefor still
says) that if you see a header that you do not recognize as a news header,
you just leave it be and ignore it. Essentially the same applies in Mail.

The only time you should be worrying is if you DO recognize it as a news
header, and can see that it is malformed and likely to cause trouble
elsewhere (and especially so if you are an injecting agent).

To: [email-masked]
Xref: clerew local.usefor:20698
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Date: Thu, 21 Apr 2005 15:55:38 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>Our Chair has
>>taken the view that Usefor is not the place to say it, but that Usepro IS
>>the proper place (and indeed there is already a corresponding MUST in
>>Usepro, though it may need a little more amplification).

>USEFOR-03 says it is defined as found in RFC2822, and is optional. 
>Optional means "not mandatory". There is no MUST.

No, USEPRO-03 does not say it is optional. It is silent on the issue (and
our Chair confirms this by saying that RFC 2822 contains both format and
protocol material, and so if you want to see whether some protocol
material in RFC 2822 still applies, then you have to look at USEPRO). I am
not sure I like that way of looking at it, but it is what our Chair says
he wants.

Date: Thu, 21 Apr 2005 11:45:17 +0200
From: Sebastian Brocks <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Russ Allbery schrieb:

> I don't think there's really anything else we can say that's more specific
> than that.

I agree.

greetings, Sebastian

To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 20 Apr 2005 17:56:41 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> USEFOR-03 says it is defined as found in RFC2822, and is
> optional. Optional means "not mandatory". There is no MUST.

I think it's obvious that the References header has to be optional in that
sense, since you can construct a well-formed Usenet message that doesn't
have a References header.

I'll repeat my earlier suggestion here, which is to simply define
followups in such a way as to say that if something doesn't have a
References header, it's not a followup.  That neatly avoids the whole
issue without needing to worry about MUSTs and SHOULDs.  It becomes a
definitional issue.  If it doesn't have References, it's not a followup.

> Like I already said, once you remove the ability to detect followups by
> making References optional, it doesn't matter when it may or may not be
> used. In fact, it may be used anytime (that's what "optional" means),
> even apparently when there is nothing at all for it to convey.

> By changing the requirement for References, we've solve a whole passle
> of things that should be be CHANGED BACK to the way they were before you
> decided to take it upon yourself to change them. Like "non-followups MAY
> contain" References headers, which, despite your repeated claims to the
> contrary, is a major change that nobody asked for.

Well, to be fair, I think it's a reasonable way of dealing with the issue.
I'm not sure if that means I was asking for it, but I can see the logic
behind it.

The way I look at this, there are two separate issues that we keep mixing
together:

 * What a news reader should do when constructing a posted reply to a
   particular post.

 * How a news reader should display messages with References headers.

The former is what we mean by a followup, and by definition it has to
build a References header.  If it doesn't, it's doing something other than
posting a followup.

The latter is a separate issue, a UI issue.  It is expected that messages
connected by References will be shown in a thread, subject to various user
configuration and choices like whether changing the Subject should result
in a separate thread in the user's display (something that should be a
configuration option, not something the standard should be mandating
either way, as there are people who prefer it both ways and there is no
interoperability issue).

There are various reasons why sometimes people like to connect messages
that are not posted replies to each other, and that do not, or only barely
do, fit into the notion of a followup.

I don't think there's really anything else we can say that's more specific
than that.

To: [email-masked]
Subject: Re: References
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 20 Apr 2005 17:32:02 -0700
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Based on this premise we don't have (and IMHO don't want):

> References => follow-up

> That's elementary logic as far as I'm concerned.  We don't
> need your co-editor or Alexey for this issue, I'm 100% sure.

Sounds right to me.

If it doesn't have References, it's not a follow-up, but in practice
people use References for things other than follow-ups, and I don't see
any reason to ban those uses.  So the question is whether we want to
artificially define all those things as follow-ups as well, or not worry
about it and just say that References is required for follow-ups but may
also be used for other things.

The second seems clearer and simpler to me.

> Now to the claim that it's exactly the same as in RfC 2822:
> That's not exactly true.  We truncate References, RfC 2822
> does not, that's a Usepro issue.  We separate References by
> at least one WSP, RfC 2822 does not, that's a Usefor issue.

Yup.

> RfC 2822 fixes In-Reply-To without References, we'd expect
> the same from a mail2news gateway, but don't need it within
> news.  It doesn't hurt, but it's not required.

Yup.

> So what's the problem ?  It's an optional header field as
> in RfC 2822, and its presence does not automatically mean
> "follow-up", it could be also something else.

Exactly.

Date: Wed, 20 Apr 2005 17:24:47 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>There has been no WG decision to
>change the requirement for a References header in followups.

Not according to USEFOR-03.

>Our Chair has
>taken the view that Usefor is not the place to say it, but that Usepro IS
>the proper place (and indeed there is already a corresponding MUST in
>Usepro, though it may need a little more amplification).

USEFOR-03 says it is defined as found in RFC2822, and is optional. 
Optional means "not mandatory". There is no MUST.

>But in that case, I will have to find somewhere in Usepro to say that the
>References header MAY also be used in other cases such as multipart FAQs,

If it is optional, then it is optional. You don't need to say it MAY be 
used when USEFOR-03 already says it may be used.

Like I already said, once you remove the ability to detect followups by 
making References optional, it doesn't matter when it may or may not be 
used. In fact, it may be used anytime (that's what "optional" means), even 
apparently when there is nothing at all for it to convey.

By changing the requirement for References, we've solve a whole passle of 
things that should be be CHANGED BACK to the way they were before you 
decided to take it upon yourself to change them. Like "non-followups MAY 
contain" References headers, which, despite your repeated claims to the 
contrary, is a major change that nobody asked for.

>News readers that try to do threading care very much if it is missing.

That is not an interoperability issue, and you know it. It doesn't cause
the news reader to break, unless it is a VERY poorly designed news reader.
How does it deal with non-followups today, which do not contain References
headers, or broken followups, which also do not contain them, if they
cannot deal with any article tomorrow that does not contain one? If they
break today, they are patently broken and need to be fixed. I know of none 
that are that broken. Do you?

And, by the way, it's software. Until it gains sentience, it doesn't care 
about anything.

>No, the requirement has not gone.

Read the draft-03. It is gone. Explicit language that makes References 
optional. "As defined in RFC2822". No MUST. A MUST that doesn't exist is 
not a requirement.

>The mandate in question is already in USEPRO. And always has been.

But is missing completely in USEFOR, where the header is defined. 

It's gone. It's time to stop pretending that it's there, or that it wasn't 
a deliberate change to remove it. I'm ok with it gone; why are you 
suddenly trying to say that it needs to be put back? You didn't say that 
prior to my commenting on it, it must have been ok -- you had ample time 
to comment on the draft prior to my speaking up, and did not.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  References (was: Differences between RFC 2822 and Usefor (fwd))
Date:  Thu, 21 Apr 2005 02:09:25 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 46
Sender: [email-masked]

Charles Lindsey wrote:

> I do not want to be seen arguing with my co-editor in public
> if it can be avoided.

It would be rather nice if your co-editor says something in
public, like "hi there" for starters.

> There has been no WG decision to change the requirement for
> a References header in followups.

I'm completely lost with this stuff.  What's the precise
problem ?  Something without References is never a follow-up.

No References => no follow-up.

If that's the case we obviously also have:

Follow-up => references.

Based on this premise we don't have (and IMHO don't want):

References => follow-up

That's elementary logic as far as I'm concerned.  We don't
need your co-editor or Alexey for this issue, I'm 100% sure.

Now to the claim that it's exactly the same as in RfC 2822:
That's not exactly true.  We truncate References, RfC 2822
does not, that's a Usepro issue.  We separate References by
at least one WSP, RfC 2822 does not, that's a Usefor issue.

RfC 2822 fixes In-Reply-To without References, we'd expect
the same from a mail2news gateway, but don't need it within
news.  It doesn't hurt, but it's not required.

So what's the problem ?  It's an optional header field as
in RfC 2822, and its presence does not automatically mean
"follow-up", it could be also something else.

> I do not agree with the Chair on this one

I'm not even sure where you disagree.  Is it related to
anything I said above ?
                        Bye, Frank

To: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 20 Apr 2005 16:44:43 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Ah! But in that case I don't think servers should be rejecting such
> headers. If the article has been wandering in any out of both email and
> news systems, then it is sometimes useful to have the full history
> recorded (though I might agree that if there was a mailing list involved
> I might not want to know what happened before it reached the mailing
> list).

Then I don't understand the distinction you're drawing between obsolete
and obsolescent.  As near as I could tell, the only difference in practice
was that servers were rejecting the former headers, but now you're saying
that they shouldn't even do that.

To: [email-masked]
Xref: clerew local.usefor:20688
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Date: Wed, 20 Apr 2005 20:03:27 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>The "MUST" in question arises from the long held consensus in this Working
>>Group that the References header in News is NOT an "optional extra".

>Not according to draft-usefor-03. According to draft-usefor-03, 
>References is defined to be the same as in RFC2822, with the addition of a 
>few bits that are not mandates for using it. This consensus you claim 
>exists has a very odd way of being expressed in our products. Almost as if 
>it doesn't really exist. 

>The fact that I am the only one who even commented on it being gone is a
>pretty good sign that nobody cares that it is gone.

No, I knew that it wasn't there, and there are a lot of other things that
should be in the Usefor draft and that have not made it there yet. But I
do not want to be seen arguing with my co-editor in public if it can be
avoided. Especially on issues that do not appear to be the subject of
controversy.

>>That clear requirement has been in all our drafts up to article-13.

>It is no longer a requirement at all in our draft.

Indeed, but a draft is only a draft. There has been no WG decision to
change the requirement for a References header in followups. Our Chair has
taken the view that Usefor is not the place to say it, but that Usepro IS
the proper place (and indeed there is already a corresponding MUST in
Usepro, though it may need a little more amplification). I do not agree
with the Chair on this one, but if nobody else speaks up and if his
ruling stands, then Usepro is where it will be said (which also,
incidentally, removes most of the cause of the disagreement there has been
about how to word it all).

But in that case, I will have to find somewhere in Usepro to say that the
References header MAY also be used in other cases such as multipart FAQs,
as well as pointing out the divergence from the SHOULD in RFC 2822.

>At this point, the mandate is not part of the draft, and putting one in
>would (or at least, should, in an honest system) require a strong
>demonstration of interoperability issues. Since news systems simply do not
>care if the header is in the article or not, there is no way to
>demonstrate an interoperability issue, and thus RFC2119 language is not
>appropriate. 

News readers that try to do threading care very much if it is missing.

To: [email-masked]
Xref: clerew local.usefor:20686
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fixed (was: Broken Message-ID syntax)
Date: Wed, 20 Apr 2005 19:42:16 GMT
Lines: 38
Sender: [email-masked]

>> Yes, you have convinced me now.

>Great, we have a msg-id syntax.

>> there remains the issue of the naming of the syntax rules,

>We're talking about seven names.  So far we agree on msg-id,
>and we disagree on names for LHS, RHS, and address-literal.

>Maybe we can solve it for the remaining three names without
>bothering Alexey, Henry, or Ken:

Essentially, we either use the same names as RFC 2822 uses, saying that
"the following syntax rules replace the corresponding rules in RFC 2822"
(giving us id-left, id-right, no-fold-quote, no-fold literal, etc), or
else we depart from the RFC 2822 names entirely, except for the msg-id at
the top, in which case your names are as good as any. I don't see that
there is anything in-between.

My preference is to retain the RFC 2822 naming, which accords with the
general principle that we do not change anything in RFC 2822 that we do
not need to change. If our Chair says that we can change these names, then
let it be so. Alexey?

To: [email-masked]
Xref: clerew local.usefor:20687
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Wed, 20 Apr 2005 19:45:12 GMT
Lines: 32
Sender: [email-masked]

>>> I think I see what you're saying -- it's not obsolete in the sense that
>>> Path is obsolete (where news servers actually reject messages containing
>>> the header).  I'm fine with the wording; I guess I don't really care
>>> whether that means "obsolete" or "obsolescent" as long as the key point is
>>> expressed: don't use the header and don't bother creating it.

>> Eh? Since when has Path been obsolete. Are you confusing it with some
>> other ancient header?

>Er, sorry, not Path, Received.  Bleh.

Ah! But in that case I don't think servers should be rejecting such
headers. If the article has been wandering in any out of both email and
news systems, then it is sometimes useful to have the full history recorded
(though I might agree that if there was a mailing list involved I might
not want to know what happened before it reached the mailing list).

To: [email-masked]
Xref: clerew local.usefor:20690
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Date: Wed, 20 Apr 2005 20:12:34 GMT
Lines: 41
Sender: [email-masked]

>>And why does the Usefor draft contain the words
>>
>>   None of the headers appearing in this section is required to appear
>>   in every article but some of them are required in certain types of
>>   article, such as followups.
>>
>Which is also followed by the following sentence:
>          Further discussion of these requirements appears in [USEPRO] 
>and [USEAGE].

Which sentence should not be there IMHO.

>This is a clear indication that USEPRO is a proper place to define any 
>additional requirements.

>You can add it to USEPRO to "Duties of a Followup Agent" as per above.

Yes, that is possible. Indeed it is already there, but might need a little
more saying about it there, including pointing out the divergence from RFC
2822 and something about using it in non-followup situations such as
mulipart FAQs and message/partial.

Incidentally, Usefor also needs the semantics of this header explaining
(such as the significance attaching to the order in which the precursors
are listed), so maybe the mention of multipart FAQs and message/partial
could be slipped in there.

To: [email-masked]
Xref: clerew local.usefor:20689
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Date: Wed, 20 Apr 2005 20:06:14 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>The side effect is now that THIS requirement is gone, there is no way of
>identifying what is and is not a followup no matter WHAT definition of
>followup may be used, so there is no reason to argue about what is and is
>not a followup anymore. If you think that not being able to identify what
>is a followup isn't important (i.e., it's ok for non-followups to contain
>References headers) then you cannot possibly think it is important enough
>to identify followups that the header designed for that purpose has to be
>mandatory. Be consistent -- either it's important or it isn't.

No, the requirement has not gone.

>Don't think that the mandate can be slipped back into USEPRO, since any
>mandate in USEPRO would contradict USEFOR, and that would look stupid. If
>USEFOR says it's optional, then USEPRO cannot claim that it is an
>interoperability issue and is thus not optional.

The mandate in question is already in USEPRO. And always has been.

Date: Sun, 17 Apr 2005 22:05:31 +0100
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor
Sender: [email-masked]

Charles Lindsey wrote:

>In <[email-masked]> Alexey Melnikov <[email-masked]> writes:  
>
>>Charles Lindsey wrote:
>>    
>>
>>>The text regarding the References header in the new usefor-03 draft is
>>>just plain WRONG.
>>>      
>>>
>>>But the requirement for
>>>the header to be present for followups has been omitted, and it needs to
>>>be put back (modulo the relatively minor matter of the precise wording).
>>>      
>>>
>>No it is not, because this is not a USEFOR business to say anything 
>>about followups. Any given article doesn't become valid/invalid because 
>>of presence or lack of References. USEFOR doesn't deal with a thread of 
>>messages, this is a protocol issue.
>>    
>>
>
>Then why is RFC 2822 allowed to say that the References header SHOULD be
>present in the corresponding email situation?
>
Because RFC 2822 is very much like a combined USEFOR+USEPRO. All I am 
trying to say is that USEFOR is not the proper place to state any 
requirement that doesn't apply to all articles.

>And why does the word "followup" occur 10 times in the Usefor draft?
>
6 of them are used when defining different terms, 3 are used when 
describing Followup-to (naturally), 1 is used in the sentence you've 
quoted below.
IMHO, this is quite reasonable. I would like to avoid any use of the 
term in USEFOR, but this is just not practical.

>And why does the Usefor draft contain the words
>
>   None of the headers appearing in this section is required to appear
>   in every article but some of them are required in certain types of
>   article, such as followups.
>
Which is also followed by the following sentence:
          Further discussion of these requirements appears in [USEPRO] 
and [USEAGE].

This is a clear indication that USEPRO is a proper place to define any 
additional requirements.

>And it says
>
>   The References header is the same as that specified in Section 3.6.4
>   of [RFC2822] with the added restrictions detailed in Section 2.2 and
>   those listed below:
>
>Which clearly implies that we have signed up to everything in RFC 2822
>Section 3.6.4 except as detailed otherwise, which means that we have
>signed up to that "SHOULD", which we haven't.
>  
>
You can add it to USEPRO to "Duties of a Followup Agent" as per above.

>Netnews departs from RFC 2822 in various ways, which we document. Changing
>that "SHOULD" to a "MUST" is one of them (the identical SHOULD->MUST for
>Message-ID is another, but I do not see any objection to saying that).
>I understand that we intend to document these differences in an Appendix
>to Usefor, but how can we document differences that we are not allowed to
>describe?
>  
>

Date: Mon, 18 Apr 2005 00:03:51 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Path header delimiters
Sender: [email-masked]

On Mon, 11 Apr 2005, Frank Ellermann wrote:
> More fun:  Any news about publishing s-o-1036 as RfC 4036 ?

Despite good intentions, I've been distracted and haven't gotten to the
point of making a submission.  Early next month, I hope.  (No chance
between now and then, too much to do.)

> Less fun:  Charles and I need a volunteer for some kind of
> "rough consensus" about the names (sic!) of some msg-id ABNF
> productions, could you please toss a coin or add a comment ?

I've been hoping that stuff would sort itself out, because I haven't
been paying enough attention to have an opinion on it.

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 13 Apr 2005 14:39:40 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>And why does the Usefor draft contain the words

>   None of the headers appearing in this section is required to appear
>   in every article but some of them are required in certain types of
>   article, such as followups.

Because someone (an editor, perhaps?) overlooked this statement when
removing the mandate for References? Remove the "such as" clause and the
problem is fixed. Do it and lets move on. Problem solved.

>Which clearly implies that we have signed up to everything in RFC 2822
>Section 3.6.4 except as detailed otherwise, which means that we have
>signed up to that "SHOULD", which we haven't.

Clearly the text in usefor-03 says we HAVE signed up for RFC2822, since it
says that. "as defined in ... with these limited exceptions" means "as 
defined in" with a few exceptions, none of which is a mandate for use in 
any kind of article, only mandates about what it contains when it is used.

>Netnews departs from RFC 2822 in various ways, which we document.

Yep. And one of those ways is no longer a mandatory References header in
this nebulous "followup" thing. Undocumented "MUSTS" are not MUSTS. 
"Charles says..." is not RFC2119 justification.

The side effect is now that THIS requirement is gone, there is no way of
identifying what is and is not a followup no matter WHAT definition of
followup may be used, so there is no reason to argue about what is and is
not a followup anymore. If you think that not being able to identify what
is a followup isn't important (i.e., it's ok for non-followups to contain
References headers) then you cannot possibly think it is important enough
to identify followups that the header designed for that purpose has to be
mandatory. Be consistent -- either it's important or it isn't.

Don't think that the mandate can be slipped back into USEPRO, since any
mandate in USEPRO would contradict USEFOR, and that would look stupid. If
USEFOR says it's optional, then USEPRO cannot claim that it is an
interoperability issue and is thus not optional.

Date: Wed, 13 Apr 2005 14:22:02 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor (fwd)
Sender: [email-masked]

Third attempt at one message. This "open list" does not seem to be 
accepting messages ... 

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>The "SHOULD" in question is in the first paragraph of section 3.6.4 of RFC
>2822.

I know where the SHOULD comes from. RFC2822 isn't "in question", it is an
existing standard. The question is where you see a MUST in USEFOR. I don't 
see one. It simply isn't there. 

>The "MUST" in question arises from the long held consensus in this Working
>Group that the References header in News is NOT an "optional extra".

Not according to draft-usefor-03. According to draft-usefor-03, 
References is defined to be the same as in RFC2822, with the addition of a 
few bits that are not mandates for using it. This consensus you claim 
exists has a very odd way of being expressed in our products. Almost as if 
it doesn't really exist. 

The fact that I am the only one who even commented on it being gone is a
pretty good sign that nobody cares that it is gone.

>And you yourself have been at the forefront in upholding that consensus.

Hardly. I've repeatedly said "IF we are going to say that References
qualifies for RFC2119 mandates, THEN we should not water it down by saying
that non-followups MAY contain them." That first clause is called a
"conditional". If we DON'T say that References qualifies for such a
mandate (and I've argued that it is hard to justify one, if you recall),
then the situation is different. Like it is now. We currently do NOT say
it is mandatory. Once we say that, the rest of the argument about WHEN it
is mandatory goes away.

>That clear requirement has been in all our drafts up to article-13.

It is no longer a requirement at all in our draft. As you have been
quick to tell me when I object to changes YOU make, things change. Our 
current draft does NOT make it a requirement; it is lunacy to claim that 
this lack of mandate reflects a "consensus" that there is a mandate, or 
that the complete lack of comment about the change (other than your 
expressed belief that a missing mandate is still a mandate) is a better 
sign of consensus than your word for it.

At this point, the mandate is not part of the draft, and putting one in
would (or at least, should, in an honest system) require a strong
demonstration of interoperability issues. Since news systems simply do not
care if the header is in the article or not, there is no way to
demonstrate an interoperability issue, and thus RFC2119 language is not
appropriate. 

I've accepted the change, why is the editor whose name is on it as author
trying to claim it doesn't exist?

>The text regarding the References header in the new usefor-03 draft is
>just plain WRONG.

Says you. If it was wrong, it should not have been sent off as our work 
product without a single person here seeing it. Well, ok, maybe someone 
here did see it, but it sure wasn't announced for review prior to it 
becoming official. 

>But the requirement for
>the header to be present for followups has been omitted, and it needs to
>be put back (modulo the relatively minor matter of the precise wording).

You need to PROVE that NEED before you make a change to the draft. You
saying "we need this" doesn't meet the requirements of RFC2119 and you
know it. Were that all it takes, then we would be documenting PAM and MCT 
and making THEM mandatory. But, no. It doesn't work that way. You're 
taking all mention of those headers OUT of the drafts that still refer to 
them.

>And there is also an error in the syntax of the References header, which
>Frank Ellermann pointed out some while back, and which needs to be
>corrected.

Please stop trying to equate a syntax error with a change you want that 
has nothing to do with syntax. Making References optional (in accordance 
with RFC2822) is a different kind of change, and it's been made. Making it
MANDATORY needs strong justification, and I'll await your attempts at 
same. "Because it was that way before" doesn't work when I object to a 
change you decide to make, don't pretend that you can use it as an excuse.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Fixed (was: Broken Message-ID syntax)
Date:  Wed, 13 Apr 2005 22:25:21 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 96
Sender: [email-masked]

Charles Lindsey wrote:

> Yes, you have convinced me now.

Great, we have a msg-id syntax.

> there remains the issue of the naming of the syntax rules,

We're talking about seven names.  So far we agree on msg-id,
and we disagree on names for LHS, RHS, and address-literal.

Maybe we can solve it for the remaining three names without
bothering Alexey, Henry, or Ken:

1 - "unique-quote", the LHS if it must be quoted.  If you hate
    "unique" a very traditional name would be "local".  For
    a post-modern touch we could take "id".  Maybe "quote"
    was too sloppy and "quoted" is better.  Six possibilities:

    unique-quote        local-quote        id-quote
    unique-quoted       local-quoted       id-quoted

2 - "unique-part", the syntactically irrelevant stuff for the
    question "to quote or not to quote".  Obviously we'd want
    the same prefix as in (1).  Maybe "text" is better if we
    don't need it in (3):

    unique-part         local-part         id-part
    unique-text         local-text         id-text

3 - "unique-literal", something that requires quoting.  The
    "literal" is somewhat dubious, 2822 and usefor-03 use it
    for the RHS, but here we are in the LHS   Let's say that
    "literal" was a bad idea.   Depending on (2) "text" could
    be fine.  I also like "special" (as in your "mqspecial"):

    unique-text         local-text         id-text
    unique-special      local-special      id-special

JFTR the three names where we are unable to find a compromise:

4 - "address-literal" vs. "no-fold-literal".  You want the same
    name as in 2822 with a different syntax.  For this reason I
    want a different name reflecting the semantics.

    We've established that the semantics is a "domain-literal"
    in RfC 2822, but that's already used for an 2822-construct
    with CFWS and FWS.   A related source is RfC 2821, there
    the name is "address-literal".  IMHO a typical case of "if
    it quacks like a duck...".

5 - "mdomain" vs. "id-right".  As in (4), you want the name as
    in RfC 2822 with a different syntax.  I want a name that
    reflects the semantics "domain".

    We've spent hours, days, and weeks with the prolem of the
    RHS in a Message-ID, because you said that it's no domain.

    I've shown that that's _not_ what 2822 means.  AFAIK Bruce
    also disagrees with you.  STD 11, RfC 1036, and s-o-1036
    disagree with you.  Abusing foreign namespaces is a serious
    problem.

    If you don't like "mdomain" you could pick "msg-domain" or
    "id-domain".  But "id-right" is OUT as far as I'm concerned.

6 - "unique" vs. "id-left".  A similar case as in (5).  Maybe
    you feel that "unique" is too traditional, and "local" is
    even more traditional.  OTOH it's true, so we could only
    beautify it a bit:

    msg-unique          id-unique
    msg-local           id-local

We can use the prefix "id-" in (6), (5), (3), (2), and (1).
Or the prefix "msg-" in (6), (5), we already have it in msg-id.

> which I think needs a ruling from the Chair.

Alexey as King Canute decreeing that the RHS is NOT a domain ?
I don't think so.

> the wording needs to explain this additional problem

Not necessarily, leading / trailing / adjacent dots aren't new,
it's exactly the same problem as in RfC 2822.

> also the reason for exlcusion of NO-WS-CONTROL

Maybe, but please state them all:  It was never used, it never
worked, it would be a PITA in news-URLs, it wasn't allowed in
any news standard.  More than one line of text, so that we have
a line between these lines saying "2822 GOT IT WRONG - PERIOD."

                           Bye, Frank

To: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 13 Apr 2005 13:02:53 -0700
Sender: [email-masked]

>> I think I see what you're saying -- it's not obsolete in the sense that
>> Path is obsolete (where news servers actually reject messages containing
>> the header).  I'm fine with the wording; I guess I don't really care
>> whether that means "obsolete" or "obsolescent" as long as the key point is
>> expressed: don't use the header and don't bother creating it.

> Eh? Since when has Path been obsolete. Are you confusing it with some
> other ancient header?

Er, sorry, not Path, Received.  Bleh.

To: [email-masked]
Xref: clerew local.usefor:20676
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Date: Wed, 13 Apr 2005 15:59:00 GMT
Lines: 55
Sender: [email-masked]

>>The text regarding the References header in the new usefor-03 draft is
>>just plain WRONG.

>>But the requirement for
>>the header to be present for followups has been omitted, and it needs to
>>be put back (modulo the relatively minor matter of the precise wording).
>>  
>>
>No it is not, because this is not a USEFOR business to say anything 
>about followups. Any given article doesn't become valid/invalid because 
>of presence or lack of References. USEFOR doesn't deal with a thread of 
>messages, this is a protocol issue.

Then why is RFC 2822 allowed to say that the References header SHOULD be
present in the corresponding email situation?

And why does the word "followup" occur 10 times in the Usefor draft?

And why does the Usefor draft contain the words

   None of the headers appearing in this section is required to appear
   in every article but some of them are required in certain types of
   article, such as followups.

And it says

   The References header is the same as that specified in Section 3.6.4
   of [RFC2822] with the added restrictions detailed in Section 2.2 and
   those listed below:

Which clearly implies that we have signed up to everything in RFC 2822
Section 3.6.4 except as detailed otherwise, which means that we have
signed up to that "SHOULD", which we haven't.

Netnews departs from RFC 2822 in various ways, which we document. Changing
that "SHOULD" to a "MUST" is one of them (the identical SHOULD->MUST for
Message-ID is another, but I do not see any objection to saying that).
I understand that we intend to document these differences in an Appendix
to Usefor, but how can we document differences that we are not allowed to
describe?

To: [email-masked]
Xref: clerew local.usefor:20674
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header delimiters
Date: Wed, 13 Apr 2005 15:24:17 GMT
Lines: 31
Sender: [email-masked]

>> Anyway, it a minor detail as regards scheme E, which seems to be gaining
>> added consenus. Does anyone else want to play?

>Giving semantic meaning to a doubled delimiter makes me nervous, just
>because that's something one generally doesn't do in protocols of this
>type.  But that's just a vague, untargetted concern and I can't think of
>concrete problems that it would solve.

>It's the best solution I've seen so far, so I'm happy to go with it unless
>someone comes up with something better.  I do think that this is an
>important problem for us to solve.

OK, that makes it quite a smooth consensus compared to some we have had :-) .

I shall write some text to incorporate it next week (no time at the moment
- visiting grandchildren and a daughter getting married at the weekend).

To: [email-masked]
Xref: clerew local.usefor:20677
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Broken Message-ID syntax
Date: Wed, 13 Apr 2005 16:04:57 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Module @ ~ unique-literal it can match in two different ways: 

>@  ~ unique-literal           @. ~ unique-part
>.@ ~ unique-part              @  ~ unique-literal

>| unique-part  = 1*( atext / "." / unique-literal )

>Note the dot "." in unique-part, this should work as desiged.

Yes, you have convinced me now.

So there remains the issue of the naming of the syntax rules, which I
think needs a ruling from the Chair.

Also, the wording needs to explain this additional problem (and also the
reason for exlcusion of NO-WS-CONTROL). At the very least, the three
examples given should be changed to:

   <[email-masked]>
   <"ab.cd"@example.com>
   <"ab.\cd"@example.com>

To: [email-masked]
Xref: clerew local.usefor:20675
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Wed, 13 Apr 2005 15:27:14 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I think I see what you're saying -- it's not obsolete in the sense that
>Path is obsolete (where news servers actually reject messages containing
>the header).  I'm fine with the wording; I guess I don't really care
>whether that means "obsolete" or "obsolescent" as long as the key point is
>expressed: don't use the header and don't bother creating it.

Eh? Since when has Path been obsolete. Are you confusing it with some
other ancient header?

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Broken Message-ID syntax
Date:  Wed, 13 Apr 2005 01:57:50 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 47
Sender: [email-masked]

Charles Lindsey wrote:
 
> anything with an "@" in it (or various other weird
> characters) clearly MUST be quoted,

ACK

> your syntax does not allow me to say <"@.@"@example.com>

NAK

| msg-id       =  "<" unique "@" mdomain ">"
| unique       = dot-atom-text / ( DQUOTE unique-quote DQUOTE )

Skipping some details for your msg-id <"@.@"@example.com> I get
example.com    ~ mdomain
"@.@"          ~ ( DQUOTE unique-quote DQUOTE )

Modulo DQUOTE that's @.@ ~ unique-quote

| unique-quote = ( "." [unique-part] ) /
|                ( [unique-part] "." ) /
|                ( [unique-part] unique-literal [unique-part] )

@.@ doesn't start with a dot, it doesn't end with a dot, so
let's try:  @.@ ~ [unique-part] unique-literal [unique-part]

Module @ ~ unique-literal it can match in two different ways: 

@  ~ unique-literal           @. ~ unique-part
.@ ~ unique-part              @  ~ unique-literal

| unique-part  = 1*( atext / "." / unique-literal )

Note the dot "." in unique-part, this should work as desiged.

> though it would allow <"@..@"@example.com>

Any number of dots, zero, one, two, or more.

> it will be even uglier after the "@.@" bug is fixed :-( .

First I want to see a bug before we make it uglier than it is.
It's already ambiguous, it can't get worse, unless it's plain
wrong.
                           Bye, Frank

Date: Tue, 12 Apr 2005 20:20:10 +0100
To: Russ Allbery <[email-masked]>
Cc: Dr John Stockton <[email-masked]>, [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Sender: [email-masked]

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes
>
>John Stockton <[email-masked]> writes:
>
>> I use Turnpike.  It has kill-rules acting on article headers.  With a
>> general rule using a RegExp one can kill or accept by a wide variety of
>> criteria.  But in the controlling dialogue of the version that I use
>> there is specific provision for "Number of lines ... (as recorded in the
>> header)" - enter a number, rather than compose a RegExp - evidently, an
>> Internet Expert must have considered Lines to be useful information..
>
>> I find this useful in newsgroups where a few users tend to post whole
>> web pages, computer programs, or essays.
>
>Yes.  I'm not disagreeing about the usefulness of that sort of killfile
>rule.  I'm saying that Turnpike should apply that rule against the
>overview information, which is guaranteed to be correct.  

which it can already do :)

.... assuming of course that its heuristics (and configuration) mean
that it is indeed fetching the overview info and it isn't just using
HEAD commands (that will always "work") ...  in the real world, clients
currently have to guess (or be told) what facilities to use; and tend to
err in favour of features that are bound to be available...

... hence the value of NNTP v2 :)

>If it's using the Lines header in the article, it's getting occasionally
>bogus results right now.  The standard is intended to tell Turnpike and
>other news readers that do this to stop and use the overview information
>instead.  Declaring this to be a bug in Turnpike is fully intentional.

John should add  USEXOVER=YES to the appropriate [NEWS xxx] part of his
configuration file  :)  and the "bug" will not bite him

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

To: [email-masked]
Subject: Re: Path header delimiters
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 12 Apr 2005 12:04:04 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Anyway, it a minor detail as regards scheme E, which seems to be gaining
> added consenus. Does anyone else want to play?

Giving semantic meaning to a doubled delimiter makes me nervous, just
because that's something one generally doesn't do in protocols of this
type.  But that's just a vague, untargetted concern and I can't think of
concrete problems that it would solve.

It's the best solution I've seen so far, so I'm happy to go with it unless
someone comes up with something better.  I do think that this is an
important problem for us to solve.

To: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 12 Apr 2005 11:59:29 -0700
Sender: [email-masked]

>> Kill Rules are based on overview information, not on article headers
>> except in unusual situations (since killing on full headers is very
>> slow).  Overview contains its own line count that has nothing to do
>> with an article header; this is specified by the NNTP protocol.

> I think it is up to individual implementors of reading agents whether to
> provide kill file facilities on all headers, or only on those in the
> overview; some do, some don't.

Except for Lines.  Lines should always be applied against overview.

> Indeed, but any killfile rule is likely to be expressed as "kill all
> articles where Lines > 1000", in which case absolute accuracy of the
> line count hardly matters. But I agree that there are better ways of
> killing overly long articles.

Like EXACTLY THAT KILLFILE RULE, but applied against the
guaranteed-accurate overview information.

> The wording actually used, which has not changed since our very early
> drafts, was

>    This header is to be regarded as obsolete, and it will likely be
>    removed entirely in a future version of this standard. In the
>    meantime, its use is deprecated.

> Which to me indicates "It is not obsolete yet, but it soon will be, so
> it is not really a good idea to use it". I think we did it that way
> because we wanted to document the correct way to do the count (because
> implementations regularly did it wrong), but there was still a
> widespread usage of it, which was likely to continue for some while.

> To my mind, the proper word to describe that situation is "obsolescent",
> rather than "obsolete".

I think I see what you're saying -- it's not obsolete in the sense that
Path is obsolete (where news servers actually reject messages containing
the header).  I'm fine with the wording; I guess I don't really care
whether that means "obsolete" or "obsolescent" as long as the key point is
expressed: don't use the header and don't bother creating it.

To: Dr John Stockton <[email-masked]>
Cc: [email-masked]
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 12 Apr 2005 11:52:37 -0700
Sender: [email-masked]

John Stockton <[email-masked]> writes:

> I use Turnpike.  It has kill-rules acting on article headers.  With a
> general rule using a RegExp one can kill or accept by a wide variety of
> criteria.  But in the controlling dialogue of the version that I use
> there is specific provision for "Number of lines ... (as recorded in the
> header)" - enter a number, rather than compose a RegExp - evidently, an
> Internet Expert must have considered Lines to be useful information..

> I find this useful in newsgroups where a few users tend to post whole
> web pages, computer programs, or essays.

Yes.  I'm not disagreeing about the usefulness of that sort of killfile
rule.  I'm saying that Turnpike should apply that rule against the
overview information, which is guaranteed to be correct.  You'll get
exactly the same results, faster, and more reliably.

If it's using the Lines header in the article, it's getting occasionally
bogus results right now.  The standard is intended to tell Turnpike and
other news readers that do this to stop and use the overview information
instead.  Declaring this to be a bug in Turnpike is fully intentional.

>> I believe that we did.  We intended it to be declared obsolete in the
>> sense that nothing should generate it and nothing should use it.

> That will not meet with universal agreement.

It generally does as soon as people understand what I mean when I say that
a better version of that information is already in the overview data.

Date: Tue, 12 Apr 2005 12:59:52 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Path header delimiters
Sender: [email-masked]

> Anyway, it a minor detail as regards scheme E, which seems to be
> gaining added consenus. Does anyone else want to play?

Sure.  consensus++

Seth

Date: Tue, 12 Apr 2005 17:00:35 +0100
From: Alexey Melnikov <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor
Sender: [email-masked]

Charles Lindsey wrote:

>>>3.2.1  References
>>>      
>>>
>>>  The References header is the same as that specified in Section 3.6.4
>>>  of [RFC2822] with the added restrictions detailed in Section 2.2 and
>>>  those listed below:
>>>      
>>>
>>Section 2.2 does not say anything about specific headers (and nothing at
>>all about References), and the remaining part of 3.2.1 says nothing about
>>followups and MUST. RFC2822 says "SHOULD". 
>>    
>>
>
>The text regarding the References header in the new usefor-03 draft is
>just plain WRONG.
>
>Yes. the bit about section 2.2 covers the obligatory SP after the ':', as
>required in all headers, and the "listed below" refers to the avoidance of
>comments (MUST accept but do not generate yet).
>

>But the requirement for
>the header to be present for followups has been omitted, and it needs to
>be put back (modulo the relatively minor matter of the precise wording).
>  
>
No it is not, because this is not a USEFOR business to say anything 
about followups. Any given article doesn't become valid/invalid because 
of presence or lack of References. USEFOR doesn't deal with a thread of 
messages, this is a protocol issue.

>And there is also an error in the syntax of the References header, which
>Frank Ellermann pointed out some while back, and which needs to be
>corrected.
>
This should be fixed or at least discussed.

To: [email-masked]
Xref: clerew local.usefor:20664
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Broken Message-ID syntax
Date: Tue, 12 Apr 2005 11:31:32 GMT
Lines: 41
Sender: [email-masked]

>> It prevents the msg-id
>> <"foo.bar.baz"@example.com>
>> as required

>Yes, that's the idea:
>Leading dot or trailing dot or unique-literal <=> quote
>No leading/trailing dot and no unique-literal <=> don't quote

>> it also prevents
>> <"@.@"@example.com>

>"@" is a "unique literal" => quote.  Working as designed.
>"@" is no atext => no dot-atom-text => unquoted is illegal.
>Where's the problem ?

The problem is that anything with an "@" in it (or various other weird
characters) clearly MUST be quoted, but your syntax does not allow me to
say <"@.@"@example.com> (though it would allow <"@..@"@example.com>).

>In the proposed "unique" ABNF adjacent dots are handled by
>".." in "unique-literal".  Ugly like hell, but who cares if
>it's correct.

And I suspect it will be even uglier after the "@.@" bug is fixed :-( .
But let us get a working correct syntax first, and then we can see whether
there is a neater way to express it.

To: [email-masked]
Xref: clerew local.usefor:20663
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Tue, 12 Apr 2005 11:23:40 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>> "Dr John Stockton" <[email-masked]> writes:

>>> "3.3.1  Lines"

>>> This may not be needed for the transmission of News; but it is very
>>> useful for Kill Rules.  Therefore, IMHO it should be compulsory, not
>>> obsolete (unless replaced by, say, 'Bytes').

>Kill Rules are based on overview information, not on article headers
>except in unusual situations (since killing on full headers is very slow).
>Overview contains its own line count that has nothing to do with an
>article header; this is specified by the NNTP protocol.

I think it is up to individual implementors of reading agents whether to
provide kill file facilities on all headers, or only on those in the
overview; some do, some don't.

>You will find, if you analyze articles in the wild, that a significant
>percentage of Lines headers are wrong.

Indeed, but any killfile rule is likely to be expressed as "kill all
articles where Lines > 1000", in which case absolute accuracy of the line
count hardly matters. But I agree that there are better ways of killing
overly long articles.

>> I think the WG agreed early on that this header had passed its
>> sell-by-date, though I do not think we intended it to be declared
>> "obsolete" as the current Usefor draft implies.

>I believe that we did.  We intended it to be declared obsolete in the
>sense that nothing should generate it and nothing should use it.

The wording actually used, which has not changed since our very early
drafts, was

   This header is to be regarded as obsolete, and it will likely be
   removed entirely in a future version of this standard. In the
   meantime, its use is deprecated.

Which to me indicates "It is not obsolete yet, but it soon will be, so it
is not really a good idea to use it". I think we did it that way because
we wanted to document the correct way to do the count (because
implementations regularly did it wrong), but there was still a widespread
usage of it, which was likely to continue for some while.

To my mind, the proper word to describe that situation is "obsolescent",
rather than "obsolete".

To: [email-masked]
Xref: clerew local.usefor:20665
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header delimiters
Date: Tue, 12 Apr 2005 11:36:53 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Henry Spencer <[email-masked]> writes:

>On Mon, 11 Apr 2005, Frank Ellermann wrote:
>> > I would suggest would be to use "AKA" -- Also Known As --
>> > instead of MISMATCH
>> 
>> Two problems:  MISMATCH is already "common practice"...

>As a standalone item in the path content?  I think not.  There is some
>vaguely-related existing practice of appending it to server names, which
>might be considered a minor point in its favor.

You are damning it with faint praise :-) .

Actually, I think I prefer MISMATCH, not only because it is close to some
existing practice, but also because it is a conspicuous word that will
shout out at you "there is something fishy about this Path" whenever you see
it.

Anyway, it a minor detail as regards scheme E, which seems to be gaining
added consenus. Does anyone else want to play?

To: [email-masked]
Xref: clerew local.usefor:20662
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Differences between RFC 2822 and Usefor
Date: Tue, 12 Apr 2005 11:08:04 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>>                Differences between RFC 2822 and Usefor.
>>		----------------------------------------

>>2.11 A References header MUST be provided for followups (as opposed to 
>>SHOULD be provided for replies in RFC 2822).

>Would you care to explain where you are getting this difference? In
>draft-ietf-usefor-usefor-03.txt, the References header is defined thusly:

The "SHOULD" in question is in the first paragraph of section 3.6.4 of RFC
2822. In consequence of that SHOULD, many MUAs still regard providing a
References header as an "optional extra", with the consequence that
attempts to display threading in mailing lists (including this one)
usually succeed only in showing broken threads.

The "MUST" in question arises from the long held consensus in this Working
Group that the References header in News is NOT an "optional extra". And
you yourself have been at the forefront in upholding that consensus. There
is a disagreement as to how exactly that requirement should be expressed,
and as to how the wording should reflect the use of the References header
in the case of multipart FAQs and the like, but there is no disagreement
at all that it MUST be provided in cases where a poster follows up
(responds/replies/whatever) to an article from an earlier poster. That
clear requirement has been in all our drafts up to article-13.

>>   The References header is the same as that specified in Section 3.6.4
>>   of [RFC2822] with the added restrictions detailed in Section 2.2 and
>>   those listed below:

>Section 2.2 does not say anything about specific headers (and nothing at
>all about References), and the remaining part of 3.2.1 says nothing about
>followups and MUST. RFC2822 says "SHOULD". 

The text regarding the References header in the new usefor-03 draft is
just plain WRONG.

Yes. the bit about section 2.2 covers the obligatory SP after the ':', as
required in all headers, and the "listed below" refers to the avoidance of
comments (MUST accept but do not generate yet). But the requirement for
the header to be present for followups has been omitted, and it needs to
be put back (modulo the relatively minor matter of the precise wording).

And there is also an error in the syntax of the References header, which
Frank Ellermann pointed out some while back, and which needs to be
corrected.

Date: Tue, 12 Apr 2005 10:31:54 +0100
From: Alexey Melnikov <[email-masked]>
To: Russ Allbery <[email-masked]>
CC: [email-masked], Dr John Stockton <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Sender: [email-masked]

Russ Allbery wrote:

>>"Dr John Stockton" <[email-masked]> writes:
>>    
>>
>>>"3.3.1  Lines"
>>>      
>>>
>>>This may not be needed for the transmission of News; but it is very
>>>useful for Kill Rules.  Therefore, IMHO it should be compulsory, not
>>>obsolete (unless replaced by, say, 'Bytes').
>>>      
>>>
>
>Kill Rules are based on overview information, not on article headers
>except in unusual situations (since killing on full headers is very slow).
>Overview contains its own line count that has nothing to do with an
>article header; this is specified by the NNTP protocol.
>
>You will find, if you analyze articles in the wild, that a significant
>percentage of Lines headers are wrong.
>  
>
>>I think the WG agreed early on that this header had passed its
>>sell-by-date, though I do not think we intended it to be declared
>>"obsolete" as the current Usefor draft implies.
>>    
>>
>I believe that we did.  We intended it to be declared obsolete in the
>sense that nothing should generate it and nothing should use it.
>  
>
Exactly. I don't think any clarification in the document is needed.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header delimiters
Date:  Mon, 11 Apr 2005 21:58:39 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Henry Spencer wrote:

 [MISMATCH] 
> As a standalone item in the path content?  I think not.

Probably I was wrong, a quick check confirmed your version
".MISMATCH" added to the host name, not "!MISMATCH".

> This would be a nuisance for its proprietors, yes, but not
> a problem for the net as a whole.

Also true, the former host TV apparently found a better name.
A potential host AKA after the introduction of TLD AKA could
also do this as news server.  Besides ...!news.aka!AKA!...
would be fun.

More fun:  Any news about publishing s-o-1036 as RfC 4036 ?
It's apparently not yet in the RfC editor queue.

Less fun:  Charles and I need a volunteer for some kind of
"rough consensus" about the names (sic!) of some msg-id ABNF
productions, could you please toss a coin or add a comment ?

                       Bye, Frank

Date: Mon, 11 Apr 2005 15:09:32 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Path header delimiters
Sender: [email-masked]

On Mon, 11 Apr 2005, Frank Ellermann wrote:
> > I would suggest would be to use "AKA" -- Also Known As --
> > instead of MISMATCH
> 
> Two problems:  MISMATCH is already "common practice"...

As a standalone item in the path content?  I think not.  There is some
vaguely-related existing practice of appending it to server names, which
might be considered a minor point in its favor.

> and we
> hope that no UUCP host MISMATCH existed, and that no TLD with
> this name will be introduced.  For "AKA" I'm less sure...

We can quite reasonably have the same hopes for AKA.  Remember also that
it doesn't greatly matter whether AKA is potentially a valid server name,
because such a server can be known by a nickname (e.g. "news.aka") for
news purposes.  This would be a nuisance for its proprietors, yes, but not
a problem for the net as a whole. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header delimiters
Date:  Mon, 11 Apr 2005 20:56:47 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Henry Spencer wrote:
 
> I would suggest would be to use "AKA" -- Also Known As --
> instead of MISMATCH

Two problems:  MISMATCH is already "common practice", and we
hope that no UUCP host MISMATCH existed, and that no TLD with
this name will be introduced.  For "AKA" I'm less sure, funny
host names are AI, IO, PH, PN, TM, and TW - in theory these
hosts could be news servers. 
                               Bye, Frank

To: [email-masked], "Dr John Stockton" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 11 Apr 2005 11:45:43 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> "Dr John Stockton" <[email-masked]> writes:

>> "3.3.1  Lines"

>> This may not be needed for the transmission of News; but it is very
>> useful for Kill Rules.  Therefore, IMHO it should be compulsory, not
>> obsolete (unless replaced by, say, 'Bytes').

Kill Rules are based on overview information, not on article headers
except in unusual situations (since killing on full headers is very slow).
Overview contains its own line count that has nothing to do with an
article header; this is specified by the NNTP protocol.

You will find, if you analyze articles in the wild, that a significant
percentage of Lines headers are wrong.

> I think the WG agreed early on that this header had passed its
> sell-by-date, though I do not think we intended it to be declared
> "obsolete" as the current Usefor draft implies.

I believe that we did.  We intended it to be declared obsolete in the
sense that nothing should generate it and nothing should use it.

Date: Mon, 11 Apr 2005 10:43:30 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>(Oddly, I noticed that John Stanley was in favour of Posted-And-Mailed -

Why is this so odd, Charles? I was one of the people who worked on the 
draft that tried to define PAM and MCT. Do you think I would have spent my
time on that were I opposed to them?

>he is quite right, but if Mail- Copies-To has to wait, then
>Posted-And-Mailed must wait too.)

If our task is to document existing practice, and if you were correct in
writing:

>It is still not clear (to me) what the objection to keeping them is, and I
>see no merit at all in #2b (since these headers are in moderately common
>use, and the "experiment" has, in effect, been done).

as Alexy quoted you as saying, then it would be incorrect NOT to include 
them. Are they not "current practice", are they NOT in moderately common 
use, or are we sticking our heads in the sand and ignoring the task we
are supposed to accomplish here?

I notice that Alexy did not debate the validitiy of your statement, he 
just decided the issue is closed and we will not document existing 
practice here.

Date: Mon, 11 Apr 2005 10:33:50 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Differences between RFC 2822 and Usefor
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>                Differences between RFC 2822 and Usefor.
>		----------------------------------------

>2.11 A References header MUST be provided for followups (as opposed to 
>SHOULD be provided for replies in RFC 2822).

Would you care to explain where you are getting this difference? In

>   The References header is the same as that specified in Section 3.6.4
>   of [RFC2822] with the added restrictions detailed in Section 2.2 and
>   those listed below:

Section 2.2 does not say anything about specific headers (and nothing at
all about References), and the remaining part of 3.2.1 says nothing about
followups and MUST. RFC2822 says "SHOULD". 

There is no section 2.11 at all, much less one that creates a difference
between RFC2822 and USEFOR. So, it appears to me that the artificial
requirement for References has been removed from the draft, as would be 
proper based upon a strict reading of RFC2119 and when a MUST is 
appropriate. So be it. That's one way of solving the debate between
"followups MUST/non-followups MUST NOT" and "followups MUST/non-followups
MAY". Not the way I would have picked, but one that is certainly RFC2119
conforming. 

Of course, now that this change has been made, it will require a full
justification for putting back an extraneous (w.r.t. RFC2119) MUST
condition. There is no interoperability issue, nothing breaks when it
isn't there, so it will take some special arguments to justify replacing
it.

Date: Mon, 11 Apr 2005 13:12:43 +0100 (BST)
From: Charles Lindsey <[email-masked]>
To: [email-masked]
Cc: "Dr John Stockton" <[email-masked]>
Subject: Re: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Sender: [email-masked]

In local.usefor you write:

>------- Forwarded message -------
>From: "Dr John Stockton" <[email-masked]>
>To: "Charles. H. Lindsey" <[email-masked]>
>Subject:  
>http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
>Date: Sun, 10 Apr 2005 19:44:55 +0100

>I've also been reading
>http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt,
>April 2005.

I think most of these points are for Ken Murchison to answer. So I shall
just make a few comments.

>"Changes since draft-ietf-usefor-usefor-01
>...
>    o  Only allow "UT and "GMT" in Date header; disallow all other <obs-
>       zone>."

>AIUI, it is UTC and not UT that is distributed by time servers, in which
>case a label UT would in practice be incorrect in principle.  UT != UTC.
>Anyway, IIRC, UT is GMT, except when it is the University of Texas.

The only reason these two <zone>s remain in the draft is that they are in
current widespread use (well, GMT is anyway). So they must continue to be
accepted, even though they MUST NOT be generated in future. Generally
speaking, neither news nor email software cares in the least about the
subtleties of UTC vs GMT. Times, as they appear in Date headers, are just
approximations to the time the article was written, and cannot be assumed
to be accurate to better than a few minutes.

>Note that while input of UTC should allow 23:59:60,...

That format for time is already permitted by RFC 2822, though I doubt any
operating system ever produces it in real practice.

>"1.1  Basic Concepts
>    "Netnews" is a set of protocols for generating, storing and
>    retrieving news "articles" (which are a subset of Email messages)"

>They are surely not a subset of E-mail messages.  They might be
>described as similar.  But E-mail is now commonly HTML & News should not
>be; I'd omit the parenthetic analogy.

Yes, it might be better to say "(whose format is a subset of that for
Email messages)". Ken?

>"1.5  Definitions
>...
>      When an article is posted to more than one newsgroup, it
>    is said to be "crossposted"; ..."

>'When a single [copy of an ] article ...' ???

I am not convinced that needs changing.

>"2.2  Headers"    and elsewhere.

>The common understanding of "Header[s]" is "That cabalistic stuff before
>the message starts", whereas Header is evidently used here to refer to a
>logical line starting with such as "Date: ".

We took a conscious decision to use the terminology in widespread use,
rather than that defined in RFC 2822. The new NNTP draft is following that
same approach. So the "cabalistic stuff" is the "Headers" (plural) and the
Date header, for example, is a "Header" (singular).

>ISTM that, clear definitions notwithstanding, there is an undue risk of
>reader error unless there is a very clear written distinction in all
>references to the entire heading material, the logical (de-folded)
>lines, and the physical (to CRLF) lines.  That part got me confused.

I think (hope) that the draft always makes it clear from the context when
a single "header line" is being discussed.

>"3.1.2  Date"

>ISTM that "GMT" is useful to the human reader, as indicating a choice of
>the world standard time rather than the local one.  My Turnpike here
>uses +0000 in Winter Time, which carries an expectation that it will use
>+0100 in Summer Time, and that the relevant agent is in the London time
>zone.  GMT would carry no such implication.  Then agents such as Google
>should be encouraged to use GMT rather than -0800, if they cannot use
>the author's zone.  In other words, I suggest un-deprecation of GMT
>(and/or UT/UTC).

I don't think there has ever been any expectation that you can guess, from
somebody's Date header, whether that somebody lives in a location where
daylight saving applies. And I believe Madrid also uses +0000 in Winter
(and if Madrid doesn't, then Lisbon surely will - and what about
Casablanca)?

Generally speaking, the poster's timezone, as written, should be preserved
as the article propagates or gets stored (indeed, gratuitous changes of
headers are forbidden by Usepro). I find it useful to know whether someone
really was taking the trouble to stay up late to respond to my messages.

>Presumably absence of a numeric offset indication, while not allowed,
>would be treated as +0000.

Possibly.

>"3.3.1  Lines"

>This may not be needed for the transmission of News; but it is very
>useful for Kill Rules.  Therefore, IMHO it should be compulsory, not
>obsolete (unless replaced by, say, 'Bytes').

I think the WG agreed early on that this header had passed its
sell-by-date, though I do not think we intended it to be declared
"obsolete" as the current Usefor draft implies. I think "obsolescent" is
the right term. At least the draft does define exactly which lines are
supposed to get counted, which had always been a bit vague hitherto.

>"5.  Security Considerations
>...
>    Agents that generate message-ids for news articles
>    SHOULD ensure that they are unpredictable."

>It would be undesirable to lose the ability to kill-rule on partial
>message-ID, or to kill spam addressed to message-IDs using partial ID,
>or to search news databases on partial ID.  Therefore, I suggest
>something like "incompletely predictable".  The right hand side in fact
>should be predictable.

RFC 2822 RECOMMENDS (which is a pretty strong term) that the RHS
should be the domain of the sender (or of his ISP), and Usefor inherits
that RECCOMENDATION.

To: [email-masked]
Xref: clerew local.usefor:20651
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: http://www.ietf.org/internet-drafts/draft-ietf-usefor-useage-01.txt
Date: Mon, 11 Apr 2005 12:14:32 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>"Charles Lindsey" <[email-masked]> wrote:
>> In <[email-masked]> "Dr John Stockton"
>> <[email-masked]> writes:

>>>      Moreover, articles crossposted to many newsgroups SHOULD be
>>>    considered to have been read once they have been seen in any of
>>>    those groups."
>>
>>>But "many" is not the right word, surely.  Cross-posting can be to 2
>>>groups, and 2 < a few < many.
>>
>> s/many/several/

>How about "multiple"?

I think "several" is good enough. Anyone else preferring "multiple"?

Date: Mon, 11 Apr 2005 11:51:13 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Path header delimiters
Sender: [email-masked]

On Sat, 9 Apr 2005, Charles Lindsey wrote:
> E. A variant on scheme D
> Path: good-site.com!!mallet.com!MISMATCH!dodgy.com!old-site.com!
>       new-site.com!!injector.com!POSTED!not-for-mail
> This avoids any delimiter other than '!', but it assumes that two
> delimiters in succession will not cause any trouble. But it is clear that
> RFC 1036 permits such usage...

This sounds good to me.

The one slight amendment I would suggest would be to use "AKA" -- Also
Known As -- instead of MISMATCH, making it a bit clearer just what's going
on.  (I struggled to find a concise way of writing "claimed to be", which
is the underlying idea, and this was the best I could think of.)  I support 
scheme E even if this suggestion is not accepted. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Broken Message-ID syntax 
Date:  Mon, 11 Apr 2005 16:32:03 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 44
Sender: [email-masked]

Charles Lindsey wrote:
 
> you can count me out for any name change for <id-left> and
> <id-right>.

That's why we need a third opinion like Alexejs's about this
point, or a better question.  A recent joke on ietf.general:

   /\          /\
__/  \__   ___/  \___/\___   _/\__/\__/\_

consensus  rough consensus   bad question

 [ABNF]
> Well not with those names :-( .

As long as the RHS contains "domain" and the "address-literal"
is an "address-literal" you could beutify it, all productions
starting with "id-" could be a good idea (id-local, id-domain,
etc.)

> It prevents the msg-id
> <"foo.bar.baz"@example.com>
> as required

Yes, that's the idea:
Leading dot or trailing dot or unique-literal <=> quote
No leading/trailing dot and no unique-literal <=> don't quote

> it also prevents
> <"@.@"@example.com>

"@" is a "unique literal" => quote.  Working as designed.
"@" is no atext => no dot-atom-text => unquoted is illegal.
Where's the problem ?

> the "[[Adjacent dots should not be allowed]]" remark in
> the draft is not a correct description of the problem.

In the proposed "unique" ABNF adjacent dots are handled by
".." in "unique-literal".  Ugly like hell, but who cares if
it's correct.
                           Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20650
Newsgroups: local.usefor
Path: clerew!clerew!x
Subject: Fwd: http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
To: [email-masked]
From: "Charles Lindsey" <[email-masked]>
Date: Mon, 11 Apr 2005 10:56:25 +0100
Sender: [email-masked]

------- Forwarded message -------
From: "Dr John Stockton" <[email-masked]>
To: "Charles. H. Lindsey" <[email-masked]>
Subject:  
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt
Date: Sun, 10 Apr 2005 19:44:55 +0100

I've also been reading
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt,
April 2005.

"Changes since draft-ietf-usefor-usefor-01
...
    o  Only allow "UT and "GMT" in Date header; disallow all other <obs-
       zone>."

AIUI, it is UTC and not UT that is distributed by time servers, in which
case a label UT would in practice be incorrect in principle.  UT != UTC.
Anyway, IIRC, UT is GMT, except when it is the University of Texas.

Note that while input of UTC should allow 23:59:60, in this context
output should not generate it, since an error of <= one second is better
than a risk of a receiving agent getting upset about it.  A posting
agent can just wait for a leap second to have finished.

However, I see that the tenor of 3.1.2 is incompatible with the
introduction of UTC.

"1.1  Basic Concepts
    "Netnews" is a set of protocols for generating, storing and
    retrieving news "articles" (which are a subset of Email messages)"

They are surely not a subset of E-mail messages.  They might be
described as similar.  But E-mail is now commonly HTML & News should not
be; I'd omit the parenthetic analogy.

"1.2  Scope
...
    An best ..."   Ouch.

"1.5  Definitions
...
      When an article is posted to more than one newsgroup, it
    is said to be "crossposted"; ..."

'When a single [copy of an ] article ...' ???

"2.2  Headers"    and elsewhere.

The common understanding of "Header[s]" is "That cabalistic stuff before
the message starts", whereas Header is evidently used here to refer to a
logical line starting with such as "Date: ".

ISTM that, clear definitions notwithstanding, there is an undue risk of
reader error unless there is a very clear written distinction in all
references to the entire heading material, the logical (de-folded)
lines, and the physical (to CRLF) lines.  That part got me confused.

"3.1.2  Date"

ISTM that "GMT" is useful to the human reader, as indicating a choice of
the world standard time rather than the local one.  My Turnpike here
uses +0000 in Winter Time, which carries an expectation that it will use
+0100 in Summer Time, and that the relevant agent is in the London time
zone.  GMT would carry no such implication.  Then agents such as Google
should be encouraged to use GMT rather than -0800, if they cannot use
the author's zone.  In other words, I suggest un-deprecation of GMT
(and/or UT/UTC).

Presumably absence of a numeric offset indication, while not allowed,
would be treated as +0000.

"3.3.1  Lines"

This may not be needed for the transmission of News; but it is very
useful for Kill Rules.  Therefore, IMHO it should be compulsory, not
obsolete (unless replaced by, say, 'Bytes').

"5.  Security Considerations
...
    Agents that generate message-ids for news articles
    SHOULD ensure that they are unpredictable."

It would be undesirable to lose the ability to kill-rule on partial
message-ID, or to kill spam addressed to message-IDs using partial ID,
or to search news databases on partial ID.  Therefore, I suggest
something like "incompletely predictable".  The right hand side in fact
should be predictable.

Example : the first two of the last four characters of a Turnpike
message-ID are, for a given setup, constant over long periods of time;
mine are currently Fw.  Therefore, it could be useful for me to kill all
mail to .*fw..@<here>, provided that I have no user called
halfwit@<here> or similar.  AIUI, newer Turnpike allows the user to
choose the last few characters of the left part, which is even better as
a kill target.

Regards,

From: "Ruud H.G. van Tol" <[email-masked]>
To: <[email-masked]>
Subject: Re: http://www.ietf.org/internet-drafts/draft-ietf-usefor-useage-01.txt
Date: Mon, 11 Apr 2005 12:15:50 +0200
Organization: Chaos rules.
Sender: [email-masked]

Seth Breidbart:
> Charles Lindsey:
>> Dr John Stockton:

>>> In particular, that the form used, after removal of any
>>> ".invalid", MUST NOT be a possible address for someone else, 
>>> present or reasonable future.

>> I am not sure. Sometimes people just add ".invalid" to their own
>> genuine address.

> That's why it says "someone _else_".

Same observation here. The "for" didn't emphasize the issue of 
ownership (or permission) though, so it was easy to read it 
otherwise, like the "someone else" being the spammert.

-- 
Grtz, Ruud

Date: Sun, 10 Apr 2005 22:50:24 -0400 (EDT)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: http://www.ietf.org/internet-drafts/draft-ietf-usefor-useage-01.txt
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> wrote:
> In <[email-masked]> "Dr John Stockton"
> <[email-masked]> writes:

>>  In particular, that the form used, after removal of any
>>".invalid", MUST NOT be a possible address for someone else, present
>>or reasonable future.
>
> I am not sure. Sometimes people just add ".invalid" to their own genuine
> address.

That's why it says "someone _else_".

>>      Moreover, articles crossposted to many newsgroups SHOULD be
>>    considered to have been read once they have been seen in any of
>>    those groups."
>
>>But "many" is not the right word, surely.  Cross-posting can be to 2
>>groups, and 2 < a few < many.
>
> s/many/several/

How about "multiple"?

Seth

To: [email-masked]
Xref: clerew local.usefor:20647
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: http://www.ietf.org/internet-drafts/draft-ietf-usefor-useage-01.txt
Date: Sun, 10 Apr 2005 22:21:14 GMT
Lines: 370
Sender: [email-masked]

In <[email-masked]> "Dr John Stockton" <[email-masked]> writes:

>I've been reading
>http://www.ietf.org/internet-drafts/draft-ietf-usefor-useage-01.txt,
>March 2005.

>I don't think I'm qualified to address the mailing list of the Usenet
>Format Working Group;

If this group were confined to "qualified" people, there would be no group
:-).

I have copied your message to the group, and am CCing this reply to you,
but you will need to subscribe if you want to see any further responses.

>"1.2.  Objectives
>...
>         NOTE: The extreme irritation caused to other readers by such
>         violations is not to be underestimated; ..."

>And by false accusations of such violations!

Yes, but don`t let's over-egg the pudding :-) .

>"2.2.  Textual Notations
>...
>         NOTE: While such explanatory notes may seem superfluous in
>         principle, they often help the less-than-omniscient reader
>         understand the true intent of the specification in cases where
>         the wording is not entirely clear."

>The wording should always be entirely clear, in the opinion of the
>author!  But the meaning may well not be entirely obvious to such a
>reader.

Yes. It's a nice quote by Henry Spencer from Son-of-1036 - too good not to
use it.

>"3.1.  The Well-Behaved Posting Agent
>    The implementor of a posting agent SHOULD make it possible for a
>    suitably perseverent poster to generate any article, ..."

>I don't believe that "perseverent" is a real word, except maybe in the
>full OED; "perseverant" is in Chambers' Dictionary;

OK, I will follow Chambers.

>Last sentence of 3.1 contains "imppose".

Fixed.

>"3.1.1.  Construction of Headers
>...
>    Posting agents SHOULD permit the poster to include headers of
>    arbitrary length (and MUST permit at least 79 characters)."

>You must mean "header lines" or "characters per line" ?

No, I meant headers. The next sentence tells them to fold, preferably
before 79, but certainly before 998.

>...
>    "NOTE: ... standard 80-column screen"
>Since use of text-only screens is now rare, how about something more
>general, such as 80-column display, or viewport ?

OK, I now say "display".

>...
>    "... WSP ..."
>Undefined in useage-01;...

Defined in Usefor, by way of RFC 2822.

>"3.1.1.1.  Date
>...
>       Date: Sat, 26 May 2001 11:13:00 -0500 (EST)"

>To rub it in, you could observe (after checking) that EST is used in
>Australia, probably (+1000).

If someone can quote the Autralian rule authoritatively, and what "EST"
stands for there, I might even be tempted to do just that.

>I'd like to see full rigorous ISO 8601 dates being allowed, with day-of-
>week as an optional but recommended comment -
>       Date : 2001-05-26 11:13:00 -0500 (EST) (Sat)

It would be nice, but the email/news Date format is too entrenched to
change it now.

>In a time example, I'd choose an hour greater than 12, to expand the
>experience of US readers.

I think US readers are sufficiently familiar with the 24 hour clock as
seen in Date headers. But I have changed it anyway.

>"3.1.1.2.  From"

>Contains the word "ro".

Oops!

>    "NOTE: ..."

>ISTM that it might be better to bow to /force majeure/, and indicate,
>maybe by reference to another document, what is and what is not
>reasonable common practice.  In particular, that the form used, after
>removal of any ".invalid", MUST NOT be a possible address for someone
>else, present or reasonable future.

I am not sure. Sometimes people just add ".invalid" to their own genuine
address. Not always spammer proof, but most spammers are stupid (Rule #3,
or something) - I get massive amounts of spam directed at the Message-ID of
my articles from 3 years back. The main purpose of that sentence was to
encourage people NOT to munge their addresses WITHOUT adding .invalid as
well.

>"3.1.1.4.  Subject
>...
>    regognized" !

Oops!

>"3.1.1.7.  Organization
>...
>    ... unless ... and unless ... "

>ISTM that in English that's syntactically unreasonable!  "unless ... or
>... "?

No, I meant 'and'. Applying De Morgan's rule, I said "DIScouage UNLESS
acceptable AND UNLESS useful", which is the same as "ENcourage IF
acceptable OR useful" (or would be if there wasn't a possible middle to
exclude).

>"3.1.2.1.  Signatures
>    A "personal signature" is a short closing text automatically added"

>ISTM that it is not necessarily automatic.

OK. How about "added (usually automatically)"?

>Since sigs are not normally quoted, you *might* agree with the
>Implementors of Turnpike that sig lines can have as many as 79
>characters each.

See my own signature below :-) .

>"3.1.2.3.  Content-Transfer-Encoding
>...
>       moreover, Usenet articles are very likely to include trailing
>       whitespace in the form of a personal signature (3.1.2.1)."

>3.1.2.1 is clear that the signature follows, and does not include, the
>separator, which has a whitespace.

OK. 'in the form of the "-- " which introduces a personal signature'.

>"3.2.1.1.  Subject
>...
>    1. Although the "Re" (which is an abbreviation for the Latin "In re",
>       meaning "in the matter of", and not an abbreviation of "Reference"
>       as is sometimes erroneously supposed) may be understood by English
>       speakers, and indeed by speakers of most European Languages, its
>       use in a newsgroup where articles were customarily written in
>       Arabic, or Hindi, or Chinese would be less than helpful."

>That seems disrespectful to Eastern intelligence.  If they can manage to
>use News, they should be able to recognise the odd alien term (however
>displayed) occurring always in a specific context.

Apparently a lot or orientals are very unfamiliar with the Latin alphabet,
but nevertheless are beginning to use mail and even news in their own
languages. Granted they cannot avoid it 100% yet. There are domain-names
(even if puny-coded), local-parts and newsgroup-names and much else that
is still in ASCII, but those issues are being worked on (though slowly).

>"3.2.1.1.1.  Examples
>...
>    Software can always recognize
>    that such changes have occurred from the References header."

>I don't see that software can do that.  ISTM that what you mean is
>something different.

No. Every followup MUST have a References header, so if software sees a
References header it knows it is a followup (or something to be treated
like a followup). So it can adjust its display accordingly (e.g. by
threading); it does not need to see the "Re: ". From the software writer's
POV, "Re: " is just some awkward characters that he has to detect and
ignore in order to get his threading (well, some forms of threading) to
work properly.

>"3.2.1.3.  Mail-Copies-To

Has now been removed from all our drafts.

>If it is permissible for a mail article to contain a newsgroups line
>(I've seen it), then ISTM that a mail reading agent should give clear
>warning.

The intention is that the meaning of a Newsgroups header, if it is seen in
an email message, is to indicate that it was also posted to that newsgroup
(but replies to that email message should not include such a header unless
they are posted to the newsgroup as well).

>"3.2.1.4.  References
>...
>     carefully put their
>     by precursors.]"

>"there" !

Oops! But it was not in text that was meant to go into the final version.

>"3.2.2.1.  Quoting and Attributions
>...
>    SHOULD be so dintinguished ..." - distinguished.

Genuine Oops! that one.
>...
>    "   The followup agent SHOULD also precede the quoted content by an
>    "attribution line" (however, ..."

>I'd like to see a terminological amendment, to remove any taint of a
>suggestion that the attribution is a single physical line.

Point taken. s/"attribution line"/"attribution"/ and consequential changes
elsewhere.

>"  The attribution MAY contain also a single <newsgroup-name> (the one
>    from which the followup is being made), the precursor's message
>    identifier and/or the precursor's Date and Time."

>'single' .. usually; but it can be convenient to change that manually to
>indicate the degree of cross-posting.

Sure. You edit anything manually as you want. But nobody wants to see more
than one <newsgroup-name> in the attribution, and I have never seen a user
agent that put more than one (in fact they often don't put any).

>'Date and Time' - IMHO, the Zone of the date & time is needed and should
>be mentioned; and there should be consideration of whether this should
>be a direct copy of the original Date:, whether it may be translated
>into the responding poster's (or ISO) notation, whether it may be
>adjusted to GMT or responder's zone.  I do dislike being attributed as
>having written on such as 3/21/05, or during AM/PM.

I think normal practice is to use the same date-time format as in the Date
header. Does anyone know of a system that changes the zone from whatever
was actually in the Date header of the precursor? Basically, we are trying
to document current best practice here rather than invent anything new.

>"   o The various fields may be separated by arbitrary text "
>                                             ^ brief

OK. "arbitrary (but brief) text". Yes, we have all seen some horrors that
people have inserted in there, and there is not much we can do to stop
them. But no harm in giving a hint and hoping they take it :-( .

>"3.3.2.  Presentation of Articles
>3.3.2.1.  Threading

>    4. Construct a tree in " ...

>ISTM that it *might* be useful for a reading agent to display an
>indication, for example, of the difference in tree-depth between that
>article and the most previously read article in that thread, presumably
>as a signed number, possibly background-coloured for sign.  Of course,
>all reading-agents should make thread-structure capable of being fully
>seen; but it's not necessarily obvious when going from article to
>article.  Just a thought.

I think this is an area where newsreader implementors should be encouraged
to experiment. If you don't like what your newsreqder does, then you
should go out and buy a better one. I think the purpose of that whole
section was to point out various possibilities, each of which has
disadvantages if used on its own, and to encourage experimentation with
hybrids. There are some agents out there that seem to do a pretty good
job, and others that don't.

>"3.3.2.2.  Killfiles
>      Moreover, articles
>    crossposted to many newsgroups SHOULD be considered to have been read
>    once they have been seen in any of those groups."

>But "many" is not the right word, surely.  Cross-posting can be to 2
>groups, and 2 < a few < many.

s/many/several/

>"3.3.3.  Interpretation of Bodies
>...
>    Tab (US-ASCII 9) SHOULD be interpreted as sufficient horizontal white
>    space to reach the next of a set of fixed positions (customarily set
>    at every 8th character)."

>"... after every 8th character)."

OK.

>"4.1.  Construction of Headers
>    According to [USEPRO], an injecting agent MAY add other headers not
>    already provided by the poster, but SHOULD NOT alter, delete, or
>    reorder any existing header."

>It might be useful to attempt to post a copy of that to certain
>moderated newsgroups, sic, as comp.lang.asm.x86 :-( .  And, perhaps, to
>strengthen SHOULD NOT, if possible.

No. Moderation comes before injection. Usepro makes it clear that
moderators have some discretion to establish a "house style". In a
well-managed hierarchy, there will be charters and moderation rules and
means to enforce them. OTOH, some charters overdo it with pages and pages
of rules governing the moderation process (and you and I know which group
I am talking about :-( ). Our drafts need to stand well back from such
issues.

>"6.1.1.  The 'newgroup' and 'mvgroup' Control Messages"
>...

>And 'rmgroup' ????

Actually no. That section was concerned with the detailed contents of
newgroup etc messages, and rmgroup messages don't have any details - it's
all or nothing :-) .

>"9.2.  Construction of Bodies
>    Posters SHOULD avoid using control characters and escape sequences
>    except for tab (US-ASCII 9), formfeed (US-ASCII 12) and, possibly,
>    backspace (US-ASCII 8), for reasons already explained in section
>    3.3.3."

>I thought that CR & LF were control characters too.

Sure, but the requirements for CRLF are well documented in Usefor (via RFC
2822).

>"12.  Contact Address
>Editor
>         Charles. H. Lindsey"
>                ^ One wonders what the dot is for.

One does indeed!

>P.S. I have discovered that the document bears a non-functional E-mail
>address; sending the above to :-

>"        Email: [email-masked]"

>gave

>"A message that you sent could not be delivered to one or more of its
>recipients.

Ah! Manchester have finally pulled the plug on my old email address.

To: [email-masked]
Xref: clerew local.usefor:20648
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Broken Mesage-ID syntax (was: Issues outstanding)
Date: Sun, 10 Apr 2005 23:19:09 GMT
Lines: 74
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Alexey Melnikov wrote:

>> NO-WS-CTL have been removed. Can people check that the new
>> syntax is Ok?

>It is not.  The "pseudo-consensus" between Charles and me was
>to replace msg-id-core by msg-id everywhere, and to fix the
>dot-atom-text issue with leading / trailing dots plus dot-dot.

Yes, I asked Ken to make that msg-id change, but he hasn't done it yet.
The dot-atom-text issue is noted, but not fixed yet.

>Two (Charles + me) is a bad number, could somebody like you
>please volunteer as third for a proper "rough consensus" about
>the last open msg-id point, a better name for the "LHS @ RHS"
>productions ?  I've interpreted an unrelated article from Bruce
>elsewhere as support for "unique @ domain" productions, but
>that's obviously dubious,

You can count me in for getting rid of <msg-id-core>, but you can count me
out for any name change for <id-left> and <id-right>.

>> I suspect that some minor issues raised by Frank are yet to
>> be addressed.

>The msg-id and dot stuff isn't minor, the former is important,
>the latter is erroneous.  Here's the latest ABNF for a msg-id:

>| msg-id          =  "<" unique "@" mdomain ">"

>| unique          = dot-atom-text / ( DQUOTE unique-quote DQUOTE )
>| unique-quote    = ( "." [unique-part] ) /
>|                   ( [unique-part] "." ) /
>|                   ( [unique-part] unique-literal [unique-part] )
>| unique-part     = 1*( atext / "." / unique-literal )
>| unique-literal  = "(" / ")" / "," / ; all specials, minus ">",
>|                   "[" / "]" / "@" / ; minus DQUOTE, minus "\",
>|                   ":" / ";" / "<" / ; minus single ".", plus:
>|                   ".." / "\\" / ( "\" DQUOTE )

>| mdomain         = dot-atom-text / ("[" address-literal "]")
>| address-literal = 1*( %d33-61 /     ; printable ASCII minus
>|                       %d63-90 /     ; ">", "[", "\", "]"
>|                       %d94-126 /    ; plus "\[", "\\, "\]"
>|                       "\[" / "\\" / "\]" )

Well not with those names :-( .

But it is still not technically correct. It prevents the msg-id

<"foo.bar.baz"@example.com>

as required (because under RFC 2822 that is indistinguishable from
<[email-masked]>), but it also prevents

<"@.@"@example.com>

and there is no reason to do that (ugly though it may be).

BTW, the "[[Adjacent dots should not be allowed]]" remark in the draft is
not a correct description of the problem.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Broken Mesage-ID syntax (was: Issues outstanding)
Date:  Sun, 10 Apr 2005 17:38:33 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 53
Sender: [email-masked]

Alexey Melnikov wrote:

> NO-WS-CTL have been removed. Can people check that the new
> syntax is Ok?

It is not.  The "pseudo-consensus" between Charles and me was
to replace msg-id-core by msg-id everywhere, and to fix the
dot-atom-text issue with leading / trailing dots plus dot-dot.

Two (Charles + me) is a bad number, could somebody like you
please volunteer as third for a proper "rough consensus" about
the last open msg-id point, a better name for the "LHS @ RHS"
productions ?  I've interpreted an unrelated article from Bruce
elsewhere as support for "unique @ domain" productions, but
that's obviously dubious,

> I suspect that some minor issues raised by Frank are yet to
> be addressed.

The msg-id and dot stuff isn't minor, the former is important,
the latter is erroneous.  Here's the latest ABNF for a msg-id:

| msg-id          =  "<" unique "@" mdomain ">"

| unique          = dot-atom-text / ( DQUOTE unique-quote DQUOTE )
| unique-quote    = ( "." [unique-part] ) /
|                   ( [unique-part] "." ) /
|                   ( [unique-part] unique-literal [unique-part] )
| unique-part     = 1*( atext / "." / unique-literal )
| unique-literal  = "(" / ")" / "," / ; all specials, minus ">",
|                   "[" / "]" / "@" / ; minus DQUOTE, minus "\",
|                   ":" / ";" / "<" / ; minus single ".", plus:
|                   ".." / "\\" / ( "\" DQUOTE )

| mdomain         = dot-atom-text / ("[" address-literal "]")
| address-literal = 1*( %d33-61 /     ; printable ASCII minus
|                       %d63-90 /     ; ">", "[", "\", "]"
|                       %d94-126 /    ; plus "\[", "\\, "\]"
|                       "\[" / "\\" / "\]" )

Some weeks later the "no-pseudo-consensus" arrived at this:

  msg-id = "<" id-local "@" id-domain ">"
  msg-id = "<" msg-local "@" msg-domain ">"
  msg-id = "<" unique "@" mdomain ">"

There it ended with another "pseudo-consensus" to wait for the
next draft and a ruling from the chair (= you).  The latter was
of course a joke, but maybe you can toss a coin - in that case
don't forget Charles' msg-id = "<" id-left "@" id-right ">"

                      Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20643
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Path header delimiters
Date: Sat, 9 Apr 2005 21:06:31 GMT
Lines: 138
Sender: [email-masked]

Our Chair has ruled that this matter it still open, so here is a summary
of where I think we are at.

Many years ago, when this WG was first formed, we were concerned that
malefactors regularly preloaded the Path header with all sorts of sites
(usually culled for some other article), in an attempt to disguise the
point at which it had been injected. We therefore proposed to modify the
Path header so that relaying agents would be able to make the following
assertions when they add a new path-identity to the Path-header:

#1  I am the injecting site.
#2  I have checked the identity of the previous site, and I believe the
    path-identity inserted by that site to be correct.
#3 I have checked the identity of the previous site, and I do not believe
    the path-identity claimed by that site; here is what I believe to be
    the true identity of that site.
#4  I have made no checks on the identity of the previous site.

The Path header is to News as the sequence of Received headers is to
Email, and Received headers currently convey much the same information,
and very useful they are when trying to trace the origin of some spam or
other malefaction. I think we are still agreed that such a feature in the
Path header is desirable - the point at issue is how to convey this
information in a backwards compatible manner.

The original suggestion was to introduce the delimiters '%', '/', and '?'
in addition to the usual '!', it being apparently the case that RFC 1036
already allowed all these characters, and many more, to be used as
delimiters, and it appeared that existing implementations did indeed
accept them. However, RFC 1036 is somewhat vague on exactly what the
allowed set of characters was.

A few months ago, Bruce Lilly pointed out that maybe some implementations
were not so accomodating, and pointed to the implementation of BNews
(written by the author of RFC 1036) which actually accepted a rather small
set. Following from that, various alternative schemes were proposed, as
follows.

In the following running example
    injector.com always uses #1
    new-site.com always uses #2 or #3
    good-site.com always uses #2 or #3
    old-site always uses #4
    dodgy.com was a bogus identity actually inserted by mallet.com

A. Current draft:
-----------------
 
Uses '%' for #1, '/' for #2, '?' for #3 amd '!' for #4
 
Path: good-site.com/mallet.com?dodgy.com!old-site.com!
      new-site.com/injector.com%not-for-mail
 
B. Henry's proposal <http://www.landfield.com/2004/Jul/0236.html>:
------------------------------------------------------------------
 
Uses '@' for #1, ',' for #2, ' ' for #3 amd '!' for #4, since it is clear
from RFC 1036 that all of those are intended to be usable as delimiters.
 
Path: good-site.com,mallet.com dodgy.com!old-site.com!
      new-site.com,injector.com@not-for-mail
 
Observe that the ' ' delimiter turns up rather conveniently as a separator
between the correct and bogus identities of mallet.com. One would need to
discuss whether FWS as well as SP should delimit this case.
 
C. The Diablo scheme
--------------------
 
I still have not been able to find documentation on this, but from
observed instances it appears to work as follows:
 
Path: good-site.com!mallet.com.MISMATCH!dodgy.com!old-site.com!
      new-site.com!injector.com.POSTED!not-for-mail
 
I see 2 problems with this one:
 
1: Any site which peers with injector.com (e.g. new-site.com) would
normally scan the received Path for occurrences of "injector.com", and
would send the article back to injector.com if it was not found (which, of
course, it isn't here because it recorded itself as "injector.com.POSTED").
 
2: It provides no distinction betwen cases #2 and #4, which rather defeats
the object of the whole exercise.

D. Another possible scheme
--------------------------

If you want to avoid all delimiters other than '!', and to overcome the
problems with the Diablo scheme, then here is one which relies on special
keywords "M", "MISMATCH" and "POSTED" in places where the current syntax
would expect a path-identity.

Path: good-site.com!M!mallet.com!MISMATCH!dodgy.com!old-site.com!
      new-site.com!M!injector.com!POSTED!not-for-mail

It makes the Path a little longer, but not unacceptably so, and assumes
that those keywords will never represent real sites.

E. A variant on scheme D
------------------------

Instead of the keyword 'M' to indicate case #2, just place two '#'
delimiters in succession, giving:

Path: good-site.com!!mallet.com!MISMATCH!dodgy.com!old-site.com!
      new-site.com!!injector.com!POSTED!not-for-mail

This avoids any delimiter other than '!', but it assumes that two
delimiters in succession will not cause any trouble. But it is clear that
RFC 1036 permits such usage, because it uses two delimiters in succession
in some of its own examples.

My own view is that we should adopt scheme E. It is much easier to
recognize what is meant, rather than having to remember exactly what '%',
'/', and '?' mean. In the discussion since I posted that last list of
outstanding issues, Frank Ellerman agreed that we should go with this
scheme (having carefully examined those examples in RFC 1036), and nobody
suggested anything different. So that scheme is the front runner at the
moment. I could easily propose modifications to the existing texts to
incorporate that scheme, but I don't want to embark on that unless we are
agreed it is the way to go.

So does anyone else want to comment, and can I assume that, if there are
no objections, we go with scheme E? 

To: [email-masked]
Xref: clerew local.usefor:20642
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Differences between RFC 2822 and Usefor
Date: Sat, 9 Apr 2005 20:00:42 GMT
Lines: 318
Sender: [email-masked]

I published a list of differences from RFC 2822 as they stood in the old
draft-13 last July (and it has been on our web site ever since). Most of
those differences were agreed years ago when this WG was first formed and
have been in the drafts ever since. I don't recall any disagreement with
the list when I published it.

There have been a few things changed since then (like the MIME-style
parameters have gone), and also things are described somewhat differently
in USEFOR, so I have updated it. The new texts follows (and it will
shortly be put on the website), and after that there are the diffs from the
previous one.

Note that many of these agreed changes have not made it into the new
USEFOR yet, and so I have marked the missing ones with "**".

                Differences between RFC 2822 and Usefor.
		----------------------------------------

08 April 2005

NOTE: Some items have been removed from the previous version, hence some
discontinuities in the listing below.

A '**' before an item indicates that the restriction in question has not yet
been incorporated into the USEFOR document.

1. Differences enforced syntactically.
--------------------------------------

1.1  There is a REQUIRED SP (not even WSP) after the ':' in each header
     (even if the header has no content).

1.2  An <unstructured> MUST have at least one character. Thus the Subject
     (for example) of an article cannot be empty.

1.3**In the References-header, there is an obligatory CFWS (currently
     restricted to FWS by 2.1.1 below) in between each msg-id.

NOTE: all the above were in RFC 1036, and the first is also required by
NNTP.

1.5**Body lines are restricted to 998 characters plus CRLF (as in RFC
     2822). However, all agents SHOULD, and relaying agents MUST, process
     lines of arbitrary length.

1.8  All the "obsolete" syntax in RFC 2822 is NOT REQUIRED (though it MAY
     be recognized). There are two small exceptions, listed below.

1.8.1  In the Date-header, the obsolete "UT" and "GMT" forms of zone MUST
       be recognized (because of their current widespread use), but MUST
       NOT be generated.

1.8.2**The (mis-named) "obsolete" syntax for phrase from RFC 2822, which
       allows for 'John D. Smith' to remain unquoted, is retained (but
       renamed as extended-phrase).

NOTE: NNTP imposes similar restrictions.

1.11 Message-ID

1.11.1 No CFWS allowed (only FWS, which in practice means only non-folded
       WSP).

1.11.2 No quoted-pairs in msg-id, except for
           '\\' and '\"' in id-left
           '\[', '\]' and '\\' in id-right

1.11.3 No quoted-strings in id-left unless they contain one of the
       "specials".

1.11.4 No '>' anywhere within a msg-id, even within a quoted-string (to
       conform with RFC 1036).

1.11.5 All these restrictions on msg-id apply also to the References- and
       Supersedes-headers.

1.11.6 No control characters in msg-id.

2. Differences enforced by verbiage.
------------------------------------

2.1  The following MUST be accepted, but SHOULD NOT be generated (yet):

2.1.1**Comments, except after a mailbox (where there is a now-deprecated
       convention for indicating the mailbox owner), or at the end of a
       date-time (which conventionally indicates the timezone). They are,
       however, freely allowed in headers that are newly defined in Usefor.

2.1.2**Extended-phrases (see 1.8.2 above).

2.2  The content of the first line of a header MUST NOT consist of WSP
     only (though such SHOULD be accepted). Observe that continuation
     lines of headers also MUST NOT consist of WSP only, as in RFC 2822.

2.3  Headers with empty content are deprecated (but if present that SP
     after the ':" is still required).

2.4  All agents MUST support header lines up to 998 octets, but there is
     no RECOMENDED limit of 78 characters as in RFC 2822. There is mention
     of a purely advisory limit of 79 (with a reference to USEAGE).

2.5  Relaying agents MUST NOT refold headers in transit.

2.6  There must not be more than one header with a given header-name,
     except where explicitly sanctioned by the appropriate standard. In
     particular, there MUST NOT be more than one Keywords-header.

2.7  The length of a msg-id MUST NOT exceed 250 octets.

2.8**The body of an article SHOULD NOT be empty.

2.9  (was 1.6) RFC 2047 and RFC 2231 are fully integrated into the Netnews.

2.10 (was 1.7) All the Content-* MIME headers are considered to be
     incorporated into Netnews and MUST be accepted in articles at, least
     to the extent required by RFC 2049.

2.11 A References header MUST be provided for followups (as opposed to SHOULD
     be provided for replies in RFC 2822).

3. Rules specific to Netnews headers.
-------------------------------------

3.2  Comments (but not FWS) are forbidden in the Newsgroups-,
     Distribution- Path- and Followup-To-headers (also see 1.11.1 above
     for Message-ID).

3.3**WSP and folding in Newsgroup- and Followup-To-headers MUST be
     accepted, but SHOULD NOT be generated (yet).

NOTE: The effect of all these differences still preserves the property
that the articles that Usefor permits to be generated form a proper subset
of the articles that are required to be acceptable to RFC 2822.

*** rfc2822-diffs.old	Thu Jul  1 23:24:57 2004
--- rfc2822-diffs.txt	Fri Apr  8 22:05:05 2005
***************
*** 3,6 ****
  
! 01 July 2004
  
  1. Differences enforced syntactically.
--- 3,12 ----
  
! 08 April 2005
  
+ NOTE: Some items have been removed from the previous version, hence some
+ discontinuities in the listing below.
+ 
+ A '**' before an item indicates that the restriction in question has not
yet
+ been incorporated into the USEFOR document.
+ 
  1. Differences enforced syntactically.
***************
*** 11,13 ****
  
! 1.2 An 'unstructured' MUST have at least one character. Thus the Subject
       (for example) of an article cannot be empty.
--- 17,19 ----
  
! 1.2 An <unstructured> MUST have at least one character. Thus the Subject
       (for example) of an article cannot be empty.
***************
*** 14,16 ****
  
! 1.3  In the References-header, there is an obligatory CFWS (currently
       restricted to FWS by 2.1.1 below) in between each msg-id.
--- 20,22 ----
  
! 1.3**In the References-header, there is an obligatory CFWS (currently
       restricted to FWS by 2.1.1 below) in between each msg-id.
***************
*** 20,26 ****
  
! 1.4  The allowed characters in a header-name are restricted to ALPHA,
! DIGIT and embedded '-'. However, agents SHOULD accept all printables
!      except SP and ':'.
! 
! 1.5  Body lines are restricted to 998 characters plus CRLF (as in RFC
       2822). However, all agents SHOULD, and relaying agents MUST,
process
--- 26,28 ----
  
! 1.5**Body lines are restricted to 998 characters plus CRLF (as in RFC
       2822). However, all agents SHOULD, and relaying agents MUST,
process
***************
*** 28,38 ****
  
- 1.6  RFC 2047 and RFC 2231 are fully integrated into the syntax. Thus
-      encoded-words are explicitly included within 'unstructured',
-      'ccontent' and 'phrase'. Moreover, it is the RFC 2231 version of
-      encoded-word that is used.
- 
- 1.7 All the Content-* MIME headers are considered to be incorporated
into
- the syntax (i.e. they are to be accepted in articles, though not all
- of them are required to have their semantic intentions implemented).
- 
  1.8 All the "obsolete" syntax in RFC 2822 is NOT REQUIRED (though it MAY
--- 30,31 ----
***************
*** 44,46 ****
  
! 1.8.2 The (mis-named) "obsolete" syntax for phrase from RFC 2822, which
         allows for 'John D. Smith' to remain unquoted, is retained (but
--- 37,39 ----
  
! 1.8.2**The (mis-named) "obsolete" syntax for phrase from RFC 2822, which
         allows for 'John D. Smith' to remain unquoted, is retained (but
***************
*** 68,70 ****
--- 61,65 ----
  
+ 1.11.6 No control characters in msg-id.
  
+ 
  2. Differences enforced by verbiage.
***************
*** 74,83 ****
  
! 2.1.1 Comments, except after a mailbox (where there is a now-deprecated
         convention for indicating the mailbox owner), or at the end of a
!        date-time (which conventionally indicates the timezone).
!        BUG: they ought to be freely allowed in headers that are newly
! defined in Usefor (and indeed the text positively encourages them
!        in some of those places).
  
! 2.1.2  Extended-phrases (see 1.8.2 above).
  
--- 69,76 ----
  
! 2.1.1**Comments, except after a mailbox (where there is a now-deprecated
         convention for indicating the mailbox owner), or at the end of a
! date-time (which conventionally indicates the timezone). They are,
! however, freely allowed in headers that are newly defined in Usefor.
  
! 2.1.2**Extended-phrases (see 1.8.2 above).
  
***************
*** 88,91 ****
  2.3  Headers with empty content are deprecated (but if present that SP
! after the ':" is still required). Injecting agents SHOULD delete such
!      headers, but other agents MUST propagate them.
  
--- 81,83 ----
  2.3  Headers with empty content are deprecated (but if present that SP
!      after the ':" is still required).
  
***************
*** 103,107 ****
  
! 2.8  The body of an article SHOULD NOT be empty.

  3. Rules specific to Netnews headers.
--- 95,107 ----
  
! 2.8**The body of an article SHOULD NOT be empty.
  
+ 2.9 (was 1.6) RFC 2047 and RFC 2231 are fully integrated into the
Netnews.
  
+ 2.10 (was 1.7) All the Content-* MIME headers are considered to be
+ incorporated into Netnews and MUST be accepted in articles at, least
+      to the extent required by RFC 2049.
+ 
+ 2.11 A References header MUST be provided for followups (as opposed to
SHOULD
+      be provided for replies in RFC 2822).
+ 
  3. Rules specific to Netnews headers.
***************
*** 109,118 ****
  
- 3.1  All structured headers have MIME-style extension-parameters, with
- x-attributes or to be defined in future standards. Some have explicit
- parameters defined in this standard. However, this does not apply to
- headers which are taken from RFC 2822 or other mail standards, nor to
-      the Mail-Copies-To, Complaints-to and Supersedes-header defined in
-      this standard. Nevertheless, such parameters SHOULD be recognized
-      (and ignored) in all headers.
- 
  3.2  Comments (but not FWS) are forbidden in the Newsgroups-,
--- 109,110 ----
***************
*** 121,128 ****
  
! 3.3  The following MUST be accepted, but SHOULD NOT be generated (yet):
  
- 3.3.1  MIME-style parameters in headers defined prior to this standard.
- 
- 3.3.2  WSP and folding in Newsgroup- and Followup-To-headers.
- 
  NOTE: The effect of all these differences still preserves the property
--- 113,117 ----
  
! 3.3**WSP and folding in Newsgroup- and Followup-To-headers MUST be
!      accepted, but SHOULD NOT be generated (yet).
  
  NOTE: The effect of all these differences still preserves the property

To: [email-masked]
Xref: clerew local.usefor:20641
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Babble from Frank Ellermann <[email-masked]>
Date: Sat, 9 Apr 2005 19:33:52 GMT
Lines: 48
Sender: [email-masked]

>>> it's a valid RfC 2822 mail header field.
>> It is indeed, but since it is in breach of RFC 2047

>It shouldn't, I've sent it in two lines, 2nd line 76 characters:
>Subject:
> =?UTF-8*de-DE-1996?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
>....5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.

Ah! It was all on one line when I received it, so I didn't understand what
your problem was.

>http://article.gmane.org/gmane.ietf.usenet.format:28154:raw
>says that "something" between me and GmaNe "fixed" this, but
>generally my UA is innocent if it comes to attempts of being
>smart.

>> Subject: =?UTF-8*de-DE-?B?RG9uYXVkYW1wZnNjaGlmZmZh?=
>>          =?UTF-8*de-DE-?B?aHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=

>Sure, you and me know this, but I'm talking about existing MUAs,
>mail2news gateways, injection agents, servers, and newsreaders.

Well if the sending agent cannot figure how to do it that way, then it
will just have to leave it on one long line. Then it becomes the reading
agent's problem :-) .

>> BTW, your message achieved a Spammassassin score of 3 :-( .

>1st UTF-8, 2nd B64, what was the 3rd ?

SpamAssassin hits were
FORGED_RCVD_HELO RCVD_BY_IP RCVD_NUMERIC_HELO SUBJECT_EXCESS_BASE64
SUBJECT_NOVOWEL

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: =?UTF-8*de-DE-1996?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
Date:  Sat, 09 Apr 2005 06:55:19 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 9
Sender: [email-masked]

I've just tested it with two news servers, and both rejected a
"Subject:" SP CRLF SP "stuff" CRLF

So "fixing" the FWS issue by removing the separate MUST about
non-empty header body lines Usefor-03 would be a very bad idea.

Maybe I should a post an I-D "Subject: Re: considered effective
at least" mentioning this case.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  =?UTF-8*de-DE-1996?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
Date:  Sat, 09 Apr 2005 06:14:38 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Charles Lindsey wrote:

>> it's a valid RfC 2822 mail header field.
> It is indeed, but since it is in breach of RFC 2047

It shouldn't, I've sent it in two lines, 2nd line 76 characters:
Subject:
 =?UTF-8*de-DE-1996?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
....5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.

http://article.gmane.org/gmane.ietf.usenet.format:28154:raw
says that "something" between me and GmaNe "fixed" this, but
generally my UA is innocent if it comes to attempts of being
smart.

> Subject: =?UTF-8*de-DE-?B?RG9uYXVkYW1wZnNjaGlmZmZh?=
>          =?UTF-8*de-DE-?B?aHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=

Sure, you and me know this, but I'm talking about existing MUAs,
mail2news gateways, injection agents, servers, and newsreaders.

> BTW, your message achieved a Spammassassin score of 3 :-( .

1st UTF-8, 2nd B64, what was the 3rd ?  Bye, Frank

To: [email-masked]
Xref: clerew local.usefor:20638
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Issues outstanding
Date: Fri, 8 Apr 2005 21:32:34 GMT
Lines: 188
Sender: [email-masked]

In <[email-masked]> Alexey Melnikov <[email-masked]> writes:

>Ken has just published a new revision of USEFOR (03). Some comments on 
>the changes and how they relate to the list of issues below:

There is still an awful lot of stuff that is meant to be covered in
USEFOR, but still isn't. But that is for another thread.

>>>We are coming to the point where there is little more that can be done on
>>>the documents we are supposed to be producing without deciding how various
>>>outstanding issues are to be resolved.
>>
>>1. Complaints-To
>>
>>I think the conclusion we reached on this was to have a
>>'mail-complaints-to' parameter in the Injection-Info header with an
>><address-list> for its parameter. And we decided not to have any provision
>>for URLs at this time, though a url-complaints-to parameter could be added
>>as a future extension if there was a demand for it.
>>
>>If that is agreed, then this issue is CLOSED,
>>
>This change is done as well as description of different parameters.

Yes, but not quite as agreed, because the token he has given for the
parameter is 'complaints-to', rather than 'mail-complaints-to' as we
agreed. That will not satisfy the people who wanted to leave room for a
'url-complaints-to' parameter as a future extension.

>>except for deciding whether
>>multiple <address>s meant you were supposed to reply to ALL of them, or to
>>ANY ONE of them. Input on this is still needed.
>>
>This minor issue is still open.

As currently written, the <value> of that parameter is an
<address-list>, which by default means "send to them all" (cf. the
Reply-To header).

>My personal opinion is that if there are multiple email addresses they 
>all should be treated as equal and mailing to ANY of them should suffice.

That could be achieved by adding some wording to say so. I am easy either
way, so we need to hear more opinions.

>>2. Path header delimiters
>>  
>>
>This is still open.
>[...]

Shame! I though we had more-or less agreed on that one. OK, I shall start
another thread to discuss that.

>>3. Mail-Copies-To and Posted-And Mailed
>>
>>>1.  Include them as in draft-13
>>>2a. Defer them to a future document (standards-track)
>>>2b. Defer them to a future document (experimental)
>>>3.  Drop them entirely
>>
>This issue is closed now: the headers will not appear in the USEFOR 
>document. The choice between 2a/2b/3 is up to the WG.

Actually, I have now come to the conclusion that this problem is far worse
with mailing lists than it is on Usenet, so it might be better to fix it
there, maybe based on Mail-Followups-To with some extra features to
incorporate News. Indeed, there was some discussion on the ietf-822 list
about Mail-Followups-To as a possible solution, with many in favour but
two diehards implacably opposed (for completely opposite reasons). Ours is
not the only list that suffers from long discussions with no decision at
the end :-( .

(Oddly, I noticed that John Stanley was in favour of Posted-And-Mailed -
he is quite right, but if Mail- Copies-To has to wait, then
Posted-And-Mailed must wait too.)

I shall now remove all mention of both these headers from USEPRO.

>>4.  Terminology for followups.
>>
>>This one is still OPEN. There are two alternative texts in USEPRO, but the
>>matching alternative texts for USEFOR are not in place yet (I hope Ken is
>>working on them). So I am happy to let this one be for now. There is no
>>technical issue involved - just a question of how to define things.
>>
>I don't believe that anything in the USEFOR should be changed, so this 
>issue concerns the USEPRO document at best.

Could you please explain your reasoning here?

ISTM that any header described in USEFOR needs the following information,
as appropriate:

1. A brief statement of what the header is supposed to achieve.
2. Its syntax.
3. Its semantics (i.e. what information is conveyed by its various
syntactic parts).
4. Any restrictions or requirements on when it is to be used (e.g. it is
"mandatory", or it MUST/SHOULD [NOT] be present if such and such other
circumstances pertain).

Following #4, there has always been a statement like the following
associated with this header:

   A followup MUST have a References-header, and an article that is not
   a followup MUST NOT have a References-header.

It is particularly important to say that here, because it is a change from
RFC 2822, where the word used is only SHOULD.

Now the precise wording of that varies according to exactly how the term
"followup" is defined (and that is what this issue is all about). But both
sides to this argument are agreed that that "MUST" needs to be said, and
this is (AFAICS) the only place where it could be said.

Yes, USEPRO describes in detail how to construct this header in the
particular case of responses/replies to earlier articles, but there are
other applications for it also, for example multipart FAQs and
message/partial (and again it is common ground that these are legitimate
applications).

>>5. Review Injection-Info syntax (this might be related to Complaints-To)
>>
>The updated USEFOR draft now includes description of different parameters.
>If people want an alternative syntax, please speak up now!

Well nobody has spoken on this for a long time. I think the important
thing is that Russ declared that he could live with this syntax, and I
think his opinion is important. But I think we also need some deprecatory
remarks to discourage the wilder (and unnecessary) extremes of RFC 2231
(and please can we use that lovely word "gibbous" in them :-) ).

It is nice to see all the other Injection-Info parameters fully described.
I have a few niggles about the precise details, and there are issues
regarding the way they have been introduced syntactically, but that too is
for another thread.

>>6. Remove filename parameter from the Archive header.
>>
>In the latest USEFOR draft the filename parameter was replaced by a 
>generic parameters.

Fine!

>>7. FWS issue in headers.
>>
>>Frank was very keen to introduce *FWS rather than *CFWS or *FWS in various
>>headers to cope with the rule that folding should not result in empty lines,
>>or even in lines with empty content. It was established, however, that the
>>present verbiage covering this issue would still be needed because it was not
>>possible to solve all such cases syntactically. I argued that there was no
>>point in changing only those cases where it would work, thereby introducing
>>differences from RFC 2822. Note that this issue involves no technical change -
>>just the method of description.
>>
>>Frank received no other support, and I propose to do nothing. If Ken wants to
>>make these changes to USEFOR, then so be it. I regard this one as CLOSED.
>>
>I tend to agree.

>>8. Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.
>>
>>We agreed to get rid of NO-WS-CTL (it would have been incompatible with
>>the new NNTP draft), but our Chair rules that further departures from RFC
>>2822 were not to be allowed. So I think this is CLOSED.

>NO-WS-CTL have been removed. Can people check that the new syntax is Ok?
>I suspect that some minor issues raised by Frank are yet to be addressed.

Yes, there is a placeholder "[[Adjacent dots should not be allowed]]" for
Frank's problem, but we still need some syntax to plug in there.

We also need some wording drawing attention to this extra departure from
RFC 2822, and referring to [NNTP] which necessitated it.

To: [email-masked]
Xref: clerew local.usefor:20637
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: =?UTF-8*de-DE-?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
Date: Fri, 8 Apr 2005 20:11:05 GMT
Lines: 55
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Alexey Melnikov wrote:

>>> Frank received no other support, and I propose to do
>>> nothing. If Ken wants to make these changes to USEFOR, then
>>> so be it. I regard this one as CLOSED.

>> I tend to agree.

>One last attempt, the subject of this article is a German fun
>word:  Donaudampfschifffahrtskapitänsmütze.  The 3 "fff" are a
>case of de-DE-1996, I use RfC 2231 and one B64 UTF-8 word.

>It is too long for RfC2047, therefore it's folded with a FWS.
>This violates a MUST in Usefor-03, but it's a valid RfC 2822
>mail header field.

It is indeed, but since it is in breach of RFC 2047, a conforming reading
agent SHOULD render it exactly as received ('=?...?=' and all). However,
some kind agents _might_ try to decode it for you.

The proper way to create that header is as:

Subject: =?UTF-8*de-DE-?B?RG9uYXVkYW1wZnNjaGlmZmZh?=
         =?UTF-8*de-DE-?B?aHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=

That works because, if you have two encoded words with nothing but FWS
between them, then the decoder is required to ignore the FWS. And you have
to be careful where you make the split, because each encoded-word must
encode an integral number of octets (so after a multiple of 4 for base64,
and watch out for '=xy' in Q-P). And on top of that you must not split any
multi-octet character between the encoded words (which could be tricky in
UTF-8, where the number of octets per character is variable).

But any decent MUA ought to be able to do it. I tried it in Opera, and it
produced:

Subject: =?utf-8?Q?Donaudampfschifffahrtskapit?= =?utf-8?Q?=C3=A4nsm=C3=BCtze?=

OK, it used Q-P rather than Base 64, but it was quite happy to display the
original word without any spaces in the middle when read back.

BTW, your message achieved a Spammassassin score of 3 :-( .

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  =?UTF-8*de-DE-1996?B?RG9uYXVkYW1wZnNjaGlmZmZhaHJ0c2thcGl0w6Ruc23DvHR6ZQ==?=
Date:  Fri, 08 Apr 2005 08:10:53 +0200
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Alexey Melnikov wrote:

>> Frank received no other support, and I propose to do
>> nothing. If Ken wants to make these changes to USEFOR, then
>> so be it. I regard this one as CLOSED.

> I tend to agree.

One last attempt, the subject of this article is a German fun
word:  Donaudampfschifffahrtskapit=E4nsm=FCtze.  The 3 "fff" are a
case of de-DE-1996, I use RfC 2231 and one B64 UTF-8 word.

It is too long for RfC2047, therefore it's folded with a FWS.
This violates a MUST in Usefor-03, but it's a valid RfC 2822
mail header field.

So now what, shoot my UA (in theory, in practice I edited the
subject manually) ?  Let some "injection agent" fix it ?  Or
reject it ?  What about mail2news gateways ?  =

                        Bye, Frank

Date: Thu, 07 Apr 2005 12:10:54 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: Issues outstanding
Sender: [email-masked]

Ken has just published a new revision of USEFOR (03). Some comments on 
the changes and how they relate to the list of issues below:

Charles Lindsey wrote:

>Some while back I posted a list of issues (and our Chair added some more).
>We have now reached the point where we cannot continue working on our
>drafts until these are resolved.
>
>So here is the list again, with my comments on where we are at on each
>one. SO PLEASE CAN WE HAVE INPUT ON THESE, especially on the ones which
>still appear to be OPEN?
>  
>
>>We are coming to the point where there is little more that can be done on
>>the documents we are supposed to be producing without deciding how various
>>outstanding issues are to be resolved.
>>    
>>
>
>1. Complaints-To
>
>  
>
>>I published a list of 4 options (and invited other options) in
>>http://www.imc.org/ietf-usefor/mail-archive/msg00151.html. Only three
>>people have expressed any preference amongst them. I think #4 is dead, and
>>#2 is the one most people could live with (but on such a small sample,
>>that is hardly meaningful).
>>    
>>
>>#2 was to do it in Injection-Info rather than in a Complaints-To header,
>>and to provide only a mail-complaints-to parameter (which would leave open
>>the option to provide a separate url-complaints parameter as a future
>>extension).    
>>
>
>I think the conclusion we reached on this was to have a
>'mail-complaints-to' parameter in the Injection-Info header with an
><address-list> for its parameter. And we decided not to have any provision
>for URLs at this time, though a url-complaints-to parameter could be added
>as a future extension if there was a demand for it.
>
>If that is agreed, then this issue is CLOSED,
>
This change is done as well as description of different parameters.

>except for deciding whether
>multiple <address>s meant you were supposed to reply to ALL of them, or to
>ANY ONE of them. Input on this is still needed.
>
This minor issue is still open.

My personal opinion is that if there are multiple email addresses they 
all should be treated as equal and mailing to ANY of them should suffice.

>
>2. Path header delimiters
>  
>
This is still open.
[...]

>  
>
>3. Mail-Copies-To and Posted-And Mailed
>
>  
>
>>Available options appear to be:
>>
>>1.  Include them as in draft-13
>>2a. Defer them to a future document (standards-track)
>>2b. Defer them to a future document (experimental)
>>3.  Drop them entirely
>>    
>>
>>Earlier discussions were inconclusive. I gather our Chair prefers #2 (a or
>>b), but he has made no definitive pronouncement.
>>    
>>
>
>I think all that the discussion established was that it was as much effort to
>remove them (from USEPRO) as to add them (to USEFOR). It is still not clear
>(to me) what the objection to keeping them is, and I see no merit at all in
>#2b (since these headers are in moderately common use, and the "experiment"
>has, in effect, been done).
>
>So this issue is still OPEN.
>
This issue is closed now: the headers will not appear in the USEFOR 
document. The choice between 2a/2b/3 is up to the WG.

>
>
>4.  Terminology for followups.
>
>  
>
>>1. A followup is a response, and MUST have a References header. A part of
>>  a multi-part FAQ (or anything similar) is not a followup, but it MAY
>>  nevertheless have a References header.
>>
>>2. A followup is a response, or a part of a multi-part FAQ (or anything
>>  similar). A followup MUST have a References header, and anything else
>>  MUST NOT have one.
>>
>>It has been established that there is no technical difference between
>>these formulations. It is just a matter of wording, so a simple majority
>>for one of the other should settle the matter.
>>
>>There are alternative definitions in USEPRO, but no corresponding wordings
>>for the References header in USEFOR yet, so maybe we should wait until
>>there are.
>>    
>>
>
>This one is still OPEN. There are two alternative texts in USEPRO, but the
>matching alternative texts for USEFOR are not in place yet (I hope Ken is
>working on them). So I am happy to let this one be for now. There is no
>technical issue involved - just a question of how to define things.
>
I don't believe that anything in the USEFOR should be changed, so this 
issue concerns the USEPRO document at best.

>
>
>5. Review Injection-Info syntax (this might be related to Complaints-To)
>
>I invited proposals from anyone who wanted to pursue this. I received none, so
>I think this one is CLOSED.
>
The updated USEFOR draft now includes description of different parameters.
If people want an alternative syntax, please speak up now!

>
>We all agree that RFC 2231 is ugly, but most of it is quite unnecessary in
>Netnews. I would be happy for this to be pointed out in USEFOR with suitably
>discouraging wording.
>
>6. Remove filename parameter from the Archive header.
>
>I think we concluded that the filename-parameter (and perhaps other
>parameters) might well be useful in the future, but there was no need to
>define them now. Therefore, we should just keep provision for MIME-style
>parameters in this header (so software would be required to ignore such
>parameters for now), but leave the definition of any actual parameters for
>future extensions.
>
In the latest USEFOR draft the filename parameter was replaced by a 
generic parameters.

>
>
>7. FWS issue in headers.
>
>Frank was very keen to introduce *FWS rather than *CFWS or *FWS in various
>headers to cope with the rule that folding should not result in empty lines,
>or even in lines with empty content. It was established, however, that the
>present verbiage covering this issue would still be needed because it was not
>possible to solve all such cases syntactically. I argued that there was no
>point in changing only those cases where it would work, thereby introducing
>differences from RFC 2822. Note that this issue involves no technical change -
>just the method of description.
>
>Frank received no other support, and I propose to do nothing. If Ken wants to
>make these changes to USEFOR, then so be it. I regard this one as CLOSED.
>
I tend to agree.

>
>
>8. Define a Message-ID compatible with NNTP, get rid of NO-WS-CTL.
>
>We agreed to get rid of NO-WS-CTL (it would have been incompatible with
>the new NNTP draft), but our Chair rules that further departures from RFC
>2822 were not to be allowed. So I think this is CLOSED.
>
>
>  
>

NO-WS-CTL have been removed. Can people check that the new syntax is Ok?
I suspect that some minor issues raised by Frank are yet to be addressed.

> So could people who disagree with the ones I have marked CLOSED please
>speak up, and otherwise will our Chair please confirm that they are
>CLOSED.
>
>And please may we have discussion of the ones still OPEN, especially the
>Path header one.
>
>  
>
Alexey

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usefor-03.txt
Date: Wed, 06 Apr 2005 15:35:27 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: News Article Format
	Author(s)	: C. Lindsey, et al. 
	Filename	: draft-ietf-usefor-usefor-03.txt
	Pages		: 31
	Date		: 2005-4-6
	
This document specifies the syntax of network news (Netnews) articles
   in the context of the 'Internet Message Format' (RFC 2822) and
   'Multipurpose Internet Mail Extensions (MIME)' (RFC 2045).  This
   document supersedes RFC 1036, updating it to reflect current practice
   and incorporating incremental changes specified in other documents.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-03.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-usefor-usefor-03.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usefor-03.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usefor-03.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usefor-03.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--