Subject: References

From: John Stanley <[email-masked]>
Date: Thu, 21 Apr 2005 10:49:17 -0700 (PDT)

Frank Ellermann <nobody@xxxxxxxxxxxxxxxxx>:

>I'm completely lost with this stuff.  What's the precise
>problem ?  Something without References is never a follow-up.

Since References is now optional, yes, something without References can be 
a followup. And something WITH a References can be something other than a 
followup. The current language says that followups SHOULD have that 
header, it does not prohibit it in anything else.

>If that's the case we obviously also have:
>Follow-up => references.

Not obvious. The header is optional. See RFC2822.

>Based on this premise we don't have (and IMHO don't want):
>References => follow-up

The only purpose for the References header is to identify followups and to
what article the present one is a followup. Previous thought on the matter
was that this function ('identify followups') was important enough to
justify an RFC2119 mandate. It is, after all, how true threading works and
the only way it can be accomplished. But threading is a display issue, 
not an interoperability one, so it was an abuse of the language to 
pretend it merited an RFC2119 "MUST". 

If you have some other function in mind, please enlighten us, and then
justify any mandates you want to apply. What interoperability problem does 
making References mandatory solve?

>Now to the claim that it's exactly the same as in RfC 2822:
>That's not exactly true. 

Of course it isn't true. Nobody has said it is. Read draft-usefor-03 to
see the differences.  None of those differences deals with a mandate for
use. Notice that "we" were able to insert the differences that "we" 
thought were important from RFC2822. "We" apparently didn't think a MUST 
was important; I agree. Let's move on.

Russ Allbery <rra@xxxxxxxxxxxx>:

>So the question is whether we want to
>artificially define all those things as follow-ups as well, or not worry
>about it and just say that References is required for follow-ups but may
>also be used for other things.

RFC2822 says that References SHOULD be used in followups. This is not a 
requirement, it is only a recommendation. What is the justification for 
making References a requirement? What interoperability issue exists? What 
breaks if a References header is not in an article, and don't say "news 
readers", because news readers deal all the time with articles that don't 
have References headers. They have to, since currently the standard says 
that non-followups must not have them. I've yet to see a news reader that 
doesn't deal with non-followups; I suggest that if you have one, you get a 
better newsreader.

And if you remove the one function that the header performs today 
("identify a followup"), then why bother keeping that header? How do you 
differentiate between "this article is a followup", and "this article 
contains a References header for some other reason"? And if you say "who 
cares?", I'll say 1) I do, and 2) the existing standard does, and 3) if 
nobody does, then get rid of the header, since it performs no useful 
function.

>> So what's the problem ?  It's an optional header field as
>> in RfC 2822, and its presence does not automatically mean
>> "follow-up", it could be also something else.

>Exactly.

So the purpose of References headers is now deprecated. That's an argument 
for making them optional (as we have already done), not for making them a 
requirement.

>I think it's obvious that the References header has to be optional in that
>sense, since you can construct a well-formed Usenet message that doesn't
>have a References header.

Currently, you cannot construct a well-formed USENET message that is a
followup without a References header. That is a direct effect of the
language in RFC1036 that says "It is required for all follow-up messages".
While the article may appear to be well-formed, it is technically in
violation of the standards and thus is not valid. This is exactly the
reason why I argued that a specific proposed moderated newsgroup that was
intended to contain only followups should enforce that requirement, and, 
as I recall, you agreed with the technical argument but disagreed with 
doing that for social/political reasons. Correct me if I am wrong in my 
recollection.

Once you say it is optional, you cannot also say it is mandatory. If 
USEFOR defines valid USENET format messages, and USEFOR says that a header 
is optional, then any message that does not contain that header can be 
valid. Another standard for the same medium that says "this article is not 
valid", when USEFOR says it is, is contradictory and broken.

>I'll repeat my earlier suggestion here, which is to simply define
>followups in such a way as to say that if something doesn't have a
>References header, it's not a followup.

We are not debating what is and is not a followup here. We are discussing 
the fact that our draft now defers the definition of References to RFC2822 
with certain limited differences, none of which deal with changing the 
RFC2119 status of that header. 

References is now a SHOULD, not a MUST. If it is to be changed to a MUST,
then justify the change. If you cannot justify the change, it ought not be
made. This group has suffered too many times from unilateral changes made
at the whim of the person who controls the text to allow it to continue.
Charles says it needs to be changed back;  Charles ought to justify that
"need" or drop it.

> * What a news reader should do when constructing a posted reply to a
>   particular post.

>[That] is what we mean by a followup, and by definition it has to
>build a References header.  If it doesn't, it's doing something other than
>posting a followup.

True wrt RFC1036. Untrue wrt draft-usefor-03. SHOULD is a recommendation.
However strong you think it is, it is still only a recommendation.

---

From: Russ Allbery <[email-masked]>
Date: Thu, 21 Apr 2005 15:00:54 -0700

John Stanley <[email-masked]> writes:

> making References a requirement? What interoperability issue exists? 
> What breaks if a References header is not in an article, and don't say
> "news readers", because news readers deal all the time with articles
> that don't have References headers. They have to, since currently the
> standard says that non-followups must not have them. I've yet to see a
> news reader that doesn't deal with non-followups; I suggest that if you
> have one, you get a better newsreader.

I don't want to use any sort of SHOULD or MUST in this area (beyond the
one we inherit from RFC 2822).  I want to say that, in a Usenet context, a
followup is *defined* to be a message posted in reply to another message
with a References header pointing to the message being replied to.

In other words, if your news posting software doesn't insert a References
header, it's still posting well-formed messages.  It's just not posting
*followups*, by definition.  (So if your software calls such messages
followups, it has a serious definitional problem.)

I think we're mostly agreeing.  I want to take this approach since it
avoids the whole question of whether things really break without
References headers.

> contains a References header for some other reason"?

In practice, I don't think there's any need to distinguish, for UI
purposes, between different reasons why an article cites a previous
article with References.  I just think it's a definitional stretch to call
all such messages followups, since people normally associate the idea of
followup with a reply and clearly not all of those articles are replies.

> And if you say "who cares?", I'll say 1) I do, and 2) the existing
> standard does, and 3) if nobody does, then get rid of the header, since
> it performs no useful function.

The function the References header provides is enabling construction of
threads.  This is closely related to the notion of followups but, to me,
is not identical to it.

> True wrt RFC1036. Untrue wrt draft-usefor-03. SHOULD is a
> recommendation.  However strong you think it is, it is still only a
> recommendation.

Right, I agree that what I'm pushing for is not what's in the current
language.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 22 Apr 2005 03:36:29 +0200

Russ Allbery wrote:

> people normally associate the idea of followup with a reply

People: yes.  Anybody who knows RfC 2822 and (s-o-) 1036: no.

A followup is defined in usepro, a reply is defined in 2822,
they are to great parts unrelated and / or different.  There
is not one string "followup" in the complete 2822 text.  Bye.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 22 Apr 2005 13:29:11 GMT

>Since References is now optional, yes,

No, they are NOT optional. Read Usepro. Everything that comes out of a
followup agent MUST have one. Please stop deliberately distorting what
everyone else is saying.

> something without References can be 
>a followup.

No.

 And something WITH a References can be something other than a 
>followup.

Yes.

>what article the present one is a followup.

No, its primary purpose is to facilitate threading, but it is also useful
to identify followups, as you say, and also to enable quick retrieval of
precursors by clicking on them. And to stitch message/partials together.
And maybe other things.

> Previous thought on the matter

No. If you read USEPRO carefully, you will see that threading (or other
means of presenting lists of articles for display) is an (optional)
feature of the protocol; i.e. implememtors MAY provide it and, if they so
choose, then they are entitled to assume that the headers they rely on for
the purpose (References, Subject, Date) have been properly constructed,
and this merits MUST or SHOULD wording in the appropriate places.

>RFC2822 says that References SHOULD be used in followups.

No it doesn't. It says they should be used for "Replies", which is not
quite the same thing. However, followups and replies are sufficiently
similar that if we put a MUST somewhere regarding followups, we need to
draw attention to the difference from the treatment of replies by RFC
2822.

>cares?", I'll say 1) I do,

Well you are giving a good impression of not caring :-( . And even if you
don't care, then I still do.

> and 2) the existing standard does,

Which is a good reason for keeping it so, and we should not change it
without a clear agreement in this WG to do so.

>language in RFC1036 that says "It is required for all follow-up messages".

And it's a direct effect of what USEPRO says.

>We are not debating what is and is not a followup here.

Yes we are. And the opinion I am hearing (even from you now) is that it is
whatever a followup agent produces. I think, though, I would also want to
include the case where a user constructs a followup manually (possibly to
more than one precursor, as you seem to do) and generates a suitable
References header manually.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5