Subject: Re: Differences between RFC 2822 and Usefor (fwd)

From: John Stanley <[email-masked]>
Date: Tue, 26 Apr 2005 13:28:08 -0700 (PDT)

Ken Murchison <ken@xxxxxxxxxx>:

>You have misquoted me here. The above was written by Charles.

You are correct. My mistake, I'm sorry. I'm used to the standard methods
of quoting material (using '>') and not this blue-line crap that doesn't
cut-and-paste. I'm also used to people eliding things they are NOT
responding to and not including large volumes of text just to add one line
at the bottom.

> Thank you for your pleasant and constructive tone.

You are welcome.  I'm sorry you are unhappy that you got caught trying to
reword the problem to make it go away. "Should/should not" is not the
issue and pretending it is is pretty insulting to the person you tried
pulling that on. Excuse me for pointing out that I caught you.

>Is your argument that References shouldn't be required for followups 
>(regardless of what USEFOR/USEPRO currently says) or that the current text 
>is incorrect/ambiguous/contradictory/shitty?

My argument is exactly as I stated: you cannot say "this is optional" and
"this is mandatory" at the same time. This is a problem; at least, I think
most people view contradictions in technical standards as a problem.
Perhaps you do not. The solution is as I stated: pick one and stick with
it, including all the implications that go with it.

>If the former, please state so. If the latter, please suggest some text 
>for one or more of the documents that you feel is technically sound.

Before we can write text we must make the choice. To do otherwise is a
complete waste of time. Writing text and THEN justifying it is backwards,
and it is why we are spending so much time now. The mandate has been 
removed from the draft and people are busy trying to pretend it is still 
there or should simply be reinstated without justification. And yet, there 
must have been a reason it was removed.

We have two (sane) options:

1. RFC1036 was correct. Being able to identify "followups" is so important 
that the header that allows this merits RFC2119 mandates. The implication 
from this is that References MUST appear in followups and MUST NOT appear 
elsewhere, since any other combination of RFC2119 language would not 
accomplish the goal of allowing identification of followups. A further 
implication of this choice is that we MUST document the difference from 
RFC2822 where the header and article format are defined. That's in USEFOR.
Another implication is that we have to define followups. That's not really 
that hard; I've provided a proposed definition that meets current 
practice.  

2. RFC1036 was incorrect. Being able to identify "followups" is not
important enough to merit RFC2119 language and the header that allows this
is thus optional. Since the only method of identifying a followup has been
made optional, the concept of followup is thus not worth defining since
nobody can detect them when they appear anyway. At least, not detect them
with any certainty. (Do you have a mole on your left shoulder? Your
messages don't contain any way of telling; fortunately, I don't care,
although someone else might.)

Once we have chosen the path we wish to take, we can look at how to 
accomplish it.

Unfortunately for position 1, we've already removed the mandate. USEFOR
defers to RFC2822, RFC2822 says "SHOULD" and "may contain". Putting a
mandate BACK after deciding to remove it should, in a correctly operating
system, require justification.  We were getting by with "historical
reasons" when it was a mandate copied from RFC1036; if that wasn't
sufficient to keep it in place, then it won't be enough to put it back now
that it is gone. I've seen no interoperability issues; do you have 
something to report?

So, looking at RFC2119, it appears the only remaining option is 2. USEFOR 
already makes the header optional. The best that USEPRO can do is say "IF 
you include the header, it must contain X". Most of the text Charles just 
proposed meets that criterion. Remove the parts that refer to the specious 
mandate. 

How do we deal with "duties of a followup agent"? Well, RFC2822 talks 
about "replies". That's a pretty common english word, why is it 
unavailable to us? USEPRO can simply talk about what headers are 
available for a reply and what the headers that DO get used in the reply 
contain. E.g., "If a References header is included, it MUST contain ...". 
That's fine. "You MUST include" this optional header is not. And when you 
talk about headers like Newsgroups and Path, you can say "This MUST be 
present and MUST contain..." because USEFOR already says they are 
mandatory.

Now, I've said it all again. Anyone up for a third pass?

And, if you are keeping score on such things, your demand that I repeat 
myself or provide text, with the implications of that demand, is pretty 
insulting, too. Other people can read it the first time I send it; you 
ought to be able to, too.

---

From: Richard Clayton <[email-masked]>
Date: Wed, 27 Apr 2005 11:43:33 +0100

In message <[email-masked]>, John
Stanley <[email-masked]> writes

>We have two (sane) options:

I come at this as a newsreading client designer, author and implementer

>that the header that allows this merits RFC2119 mandates.

the readers of Usenet believe in things like threads and followups; they
like the ability to split a series of rambling conversations into
disparate piles of articles and understand each of those piles one at a
time...

   of course we normally call the piles "threads" but a thread is always
   an artificial construct because it cannot be reliably constructed
   without a complete set of articles to hand and Usenet never
   guarantees that you have this -- so "piles" is rather closer to
   reality :)

there are flaws with a divide and conquer approach to newsgroup reading,
not least if you check a newsgroup for new material faster than new
articles flow in -- but in general it has been found to work; and we can
see the concept being copied into other media such as web-based forums.
viz: it may be flawed, but it's the best UI model we currently have

However, there seems to be an argument being presented that when you
look deeply at the specifications the notion of followups and threads is
mainly an illusion fostered by the reading tools and one can lawfully
miss out all the headers that allow the illusion to be constructed. But
even if the illusion is built on sand (and my experience tells me how
many heuristics you need to stabilise the sand), it still has
considerable practical value

>is thus optional. 

if there is no standardised way of making the piles of articles in a
consistent manner then as time goes on and posting software diverges
from traditional working (and there is no document to guide them
otherwise) those heuristics I mentioned above will have less and less to
work with.  ie: the piles will be different in different software.

This will lead to considerable confusion amongst the readers of the
newsgroup. Some will believe that a particular article belongs in one
pile, some in another. Some will miss the article altogether because
they have used "kill" features of their software to exclude some of the
piles as uninteresting. They will express this confusion when they write
further articles. Most know not to say "the thread above this one" to
express article relationships -- but they often say "earlier in this
thread" to say where to find other material...

So it seems to me that this is a genuine inter-operability issue -- not
at the protocol levels : the articles get to the far end just fine and
the servers will serve them perfectly well. The issue occurs at the
level at which the readers view the newsgroups.  For that reason I think
that a standardised way of expressing which pile an article belongs in
is of very significant value. Yes at present some software is a bit
flaky, but that doesn't mean we should fail to say what should be done.

>Unfortunately for position 1, we've already removed the mandate.

sounds like a cock-up :(

so best fix it then!  ASAP!!

>Other people can read it the first time I send it; you 

ah....   perhaps I could find it earlier in the "thread" ?  :)

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Apr 2005 15:36:33 GMT

>flaky, but that doesn't mean we should fail to say what should be done.

I agree entirely. RFC 2119 is very badly written, and is wide open to
those who claim that "interoperability" means only that "it must be
delivered correctly at the far end". The WG has long striven to use "MUST"
wording for cases where "it does not behave as it is intended to behave if
you don't say 'MUST'".

Anyway, to deal with this present case, you will find some wording in
USEPRO 7.7 entitled "Duties of a Reading Agent" where it says, in effect,
that the agent MAY take note of the References, Subject and Date headers
in order to display list of articles sensibly. This makes such displaying
an optional part of the protocol, which in turn means that reading agents
are entitled to rely upon the presence and correctness of those headers.

Yes, it is pretty meaningless, and it is only there to satisfy the pedants
who object to the use of MUST and SHOULD when dealing with the behaviour
of followup agents. But if that is the price we have to pay to get this
wording in, ..........

>sounds like a cock-up :(

It WAS a cock-up.

>so best fix it then!  ASAP!!

Exactly.

>Signature by unknown keyid: 0xD8D683E2

Eh? I though I knew all your public keys.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5