Subject: Re: <draft-ietf-usefor-usepro-01.txt>

From: John Stanley <[email-masked]>
Date: Thu, 30 Sep 2004 16:08:32 -0700 (PDT)

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>The alternative text, which you did not quote, is

Didn't need to quote it, I thought.

>No, the poster cannot "define" how articles are to be displayed, but he
>can indicate his intention in the matter ...

If he cannot control it, his intention of controlling it is irrelevant. "I 
want ...". That's nice, but you don't get to say. Presenting it as if 
there was some mechanism, when none exists, just leads to foolish 
expectations. Just like the foolish expectations that copyright notices in 
the body of an article are going to change how Google or any other 
automated news processor is going to work.

>>The alternative I suggested is still better. Why has it never appeared in
>>any form?

>You need to explain _why_ your wording is better.

I have, many times. On the other hand, I've not seen any explanation of 
why your new alternative is better, it just appeared out of nowhere. And
now I've explained why YOUR wording is wrong.

>If you read the definition, some followups come from precursors and some
>come from the poster's display intentions.

Since the "display intentions" are irrelevant, that part of the 
definition is irrelevant.

>Followup agents are only
>designed to handle the first case, and don't have the necessary capability
>to handle the second.

Oh, please. Those people who are fooled into thinking they can control
"this ought to appear next to that" find it trivial to use the "followup
agent" to do that. Of course it has "the necessary capability", because
the "necessary capability" is the main function OF the followup agent.
I.e., copying the message id of what this new article is supposed to
appear next to into the References header (which isn't defined as meaning
"display this next to that" in the first place.) Saying that followup 
agents cannot handle the second case is simply ridiculous (but that 
still doesn't mean they can actually do what they are being told they
can do.)

>>We still have the extraneous "by default".

>If we didn't say "by default", then there would be no latitude for the
>poster to override that default.

Untrue. What the followup agent does is not binding on the user who gets
to modify what that agent does. We are specifying what the FOLLOWUP AGENT
does, not the user, so the fact that we specify ONE ACTION for the
followup agent makes that the DEFAULT action for that agent. It is
specious to say that the ONE thing we say a followup agent does is the
default. The "by default" is extraneous; it adds nothing to the text.

Now, were we talking about the headers as they are injected, yes, by
default the Subject header contains X, because at that point we are 
including the actions of the user and not just talking about what the 
followup agent is supposed to do.

>It says nothing about "giving" the poster anything (that is up to the
>implementor of the followup agent).

So let's see if I understand your intention here. The followup agent
"takes" the subject content of the precursor for the new subject, but
nowhere is it intended or specified that this followup agent is supposed
to allow the user to modify this content after it has done its one
specified action? What malarky. If it doesn't say "the user gets to edit 
the followup agent's proto-article prior to posting" and you think that
by not saying this we somehow prohibit it, then we damn well better say it
just to keep silly interpretations like yours from being implemented in 
code (as if user revolt wouldn't stop it.)

>How the overriding is done is not
>specified (maybe the user edits those headers in the article he is given,

What article is he given? You just said nothing is said about doing that, 
so apparently it is forbidden (as are all other things that are not said, 
which is why we have to say things like "MAY prepend the string..." when 
nothing else prohibits it.)

>All that is clear is that if the poster
>omits to provide those headers, the "default" is what he gets.

Once again, he apparently doesn't "get" anything, since we allegedly don't
say he is to be given anything. Twice now you've said that the user is 
given the article to modify, after telling me that he apparently isn't 
supposed to be given the article.

The truth is, the draft says:

   However, posters MAY
   then override that default before posting if they so wish.

This is AFTER the "default" (really, ONLY action specified) is taken
by the followup agent. So, if the poster is not "given" anything to 
modify, how can he modify this "default" before posting? (You do 
understand that "then" indicates a chronological relationship between 
actions, don't you? You do understand that saying "do A then B" really 
does mean A gets done first, yes?)

Further, a followup agent is a special case of a posting agent, which is
clearly intended to assist the poster in creating a valid proto-article.  
If the followup/posting agent does not allow the poster to edit the
Subject header, it cannot do that. While the subject header may be 
syntactically valid, it may not be contextually valid. And since the 
context may change long after any "default taking" of the precursor's
subject content, this editing has to occur long after the taking.

>OK, that is new wording, and it may yet disappear entirely. 

Why is entirely new wording appearing first in the drafts sent to the IETF 
and not in proposals discussed here? When did you say "I propose we put 
strict limits on how articles are to be displayed such that they MUST be 
displayed as they were posted"? Have I not brought this problem of 
surprise changes to your attention before?

>>A moderator receives submissions by some means, not necessarily by email.
>>Certainly someone has come up with, or will come up with, a web-based
>>moderation system, ...

>No, the rule is that the injection agent MUST mail articles to the
>moderator (at his published submission address).

Well, YOUR version of the draft says that, but that may not be the final 
version. I know of NO reason that email MUST be the means of forwarding. 
What interoperability issue is there that mandates such RFC2119 language?
If a specific hierarchy decides that submissions are printed out on the
company line printer and delivered to the moderator via FedEx, does news
break?

But this avoids the actual problem with the text I quoted. Your injection 
agent may mail what it wants to some address, but that address may be a
pipeline into a web-based submission mechanism. The moderator never sees
the submission until it hits the web. If there is a moderation panel, for 
example, they may all work via a web-based system of approval.

Furthermore, there is no law of nature that says that all submissions to
moderated groups must pass through an injection agent. I've mailed
submissions directly, and many moderators publish submission addresses.  
Where do we get off trying to tell them they cannot accept material from a
website or any other method of getting submissions they feel appropriate,
including snail mail or cuniform tablets?

>If the moderator chooses
>to put all sorts of web sites and whatever behind that address, then that
>is up to him.

Yes, and it means that he is not receiving things by email.

>Essentially, the boundary of "moderator" starts anywhere on
>his side of the submission address.

I've never seen software listed as the proposed moderator of a group. I 
have seen a person who intends to run software listed as the moderator, 
but that's not the same thing.

Date: Thu, 30 Sep 2004 16:08:32 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: <draft-ietf-usefor-usepro-01.txt>
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>The alternative text, which you did not quote, is

Didn't need to quote it, I thought.

>No, the poster cannot "define" how articles are to be displayed, but he
>can indicate his intention in the matter ...

If he cannot control it, his intention of controlling it is irrelevant. "I 
want ...". That's nice, but you don't get to say. Presenting it as if 
there was some mechanism, when none exists, just leads to foolish 
expectations. Just like the foolish expectations that copyright notices in 
the body of an article are going to change how Google or any other 
automated news processor is going to work.

>>The alternative I suggested is still better. Why has it never appeared in
>>any form?

>You need to explain _why_ your wording is better.

I have, many times. On the other hand, I've not seen any explanation of 
why your new alternative is better, it just appeared out of nowhere. And
now I've explained why YOUR wording is wrong.

>If you read the definition, some followups come from precursors and some
>come from the poster's display intentions.

Since the "display intentions" are irrelevant, that part of the 
definition is irrelevant.

>Followup agents are only
>designed to handle the first case, and don't have the necessary capability
>to handle the second.

Oh, please. Those people who are fooled into thinking they can control
"this ought to appear next to that" find it trivial to use the "followup
agent" to do that. Of course it has "the necessary capability", because
the "necessary capability" is the main function OF the followup agent.
I.e., copying the message id of what this new article is supposed to
appear next to into the References header (which isn't defined as meaning
"display this next to that" in the first place.) Saying that followup 
agents cannot handle the second case is simply ridiculous (but that 
still doesn't mean they can actually do what they are being told they
can do.)

>>We still have the extraneous "by default".

>If we didn't say "by default", then there would be no latitude for the
>poster to override that default.

Untrue. What the followup agent does is not binding on the user who gets
to modify what that agent does. We are specifying what the FOLLOWUP AGENT
does, not the user, so the fact that we specify ONE ACTION for the
followup agent makes that the DEFAULT action for that agent. It is
specious to say that the ONE thing we say a followup agent does is the
default. The "by default" is extraneous; it adds nothing to the text.

Now, were we talking about the headers as they are injected, yes, by
default the Subject header contains X, because at that point we are 
including the actions of the user and not just talking about what the 
followup agent is supposed to do.

>It says nothing about "giving" the poster anything (that is up to the
>implementor of the followup agent).

So let's see if I understand your intention here. The followup agent
"takes" the subject content of the precursor for the new subject, but
nowhere is it intended or specified that this followup agent is supposed
to allow the user to modify this content after it has done its one
specified action? What malarky. If it doesn't say "the user gets to edit 
the followup agent's proto-article prior to posting" and you think that
by not saying this we somehow prohibit it, then we damn well better say it
just to keep silly interpretations like yours from being implemented in 
code (as if user revolt wouldn't stop it.)

>How the overriding is done is not
>specified (maybe the user edits those headers in the article he is given,

What article is he given? You just said nothing is said about doing that, 
so apparently it is forbidden (as are all other things that are not said, 
which is why we have to say things like "MAY prepend the string..." when 
nothing else prohibits it.)

>All that is clear is that if the poster
>omits to provide those headers, the "default" is what he gets.

Once again, he apparently doesn't "get" anything, since we allegedly don't
say he is to be given anything. Twice now you've said that the user is 
given the article to modify, after telling me that he apparently isn't 
supposed to be given the article.

The truth is, the draft says:

   However, posters MAY
   then override that default before posting if they so wish.

This is AFTER the "default" (really, ONLY action specified) is taken
by the followup agent. So, if the poster is not "given" anything to 
modify, how can he modify this "default" before posting? (You do 
understand that "then" indicates a chronological relationship between 
actions, don't you? You do understand that saying "do A then B" really 
does mean A gets done first, yes?)

Further, a followup agent is a special case of a posting agent, which is
clearly intended to assist the poster in creating a valid proto-article.  
If the followup/posting agent does not allow the poster to edit the
Subject header, it cannot do that. While the subject header may be 
syntactically valid, it may not be contextually valid. And since the 
context may change long after any "default taking" of the precursor's
subject content, this editing has to occur long after the taking.

>OK, that is new wording, and it may yet disappear entirely. 

Why is entirely new wording appearing first in the drafts sent to the IETF 
and not in proposals discussed here? When did you say "I propose we put 
strict limits on how articles are to be displayed such that they MUST be 
displayed as they were posted"? Have I not brought this problem of 
surprise changes to your attention before?

>>A moderator receives submissions by some means, not necessarily by email.
>>Certainly someone has come up with, or will come up with, a web-based
>>moderation system, ...

>No, the rule is that the injection agent MUST mail articles to the
>moderator (at his published submission address).

Well, YOUR version of the draft says that, but that may not be the final 
version. I know of NO reason that email MUST be the means of forwarding. 
What interoperability issue is there that mandates such RFC2119 language?
If a specific hierarchy decides that submissions are printed out on the
company line printer and delivered to the moderator via FedEx, does news
break?

But this avoids the actual problem with the text I quoted. Your injection 
agent may mail what it wants to some address, but that address may be a
pipeline into a web-based submission mechanism. The moderator never sees
the submission until it hits the web. If there is a moderation panel, for 
example, they may all work via a web-based system of approval.

Furthermore, there is no law of nature that says that all submissions to
moderated groups must pass through an injection agent. I've mailed
submissions directly, and many moderators publish submission addresses.  
Where do we get off trying to tell them they cannot accept material from a
website or any other method of getting submissions they feel appropriate,
including snail mail or cuniform tablets?

>If the moderator chooses
>to put all sorts of web sites and whatever behind that address, then that
>is up to him.

Yes, and it means that he is not receiving things by email.

>Essentially, the boundary of "moderator" starts anywhere on
>his side of the submission address.

I've never seen software listed as the proposed moderator of a group. I 
have seen a person who intends to run software listed as the moderator, 
but that's not the same thing.

Date: Thu, 30 Sep 2004 18:51:53 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
Sender: [email-masked]

On Thu, 30 Sep 2004, Bill Davidsen wrote:
> > Note that history-based loop detection does not work if you don't verify
> > that the date of the article is within your history list.
> 
> Unless you assume some evil entity editing the Path header loop 
> detection will work anyway, I would assume.

The Path-header check is just an optimization; loop detection is done by
message ID and history list.  Yes, mangled or reinitialized Path headers
have been known to happen, especially when gateways get involved. 

> ...the worst that could happen is that I 
> would get something so old it expired out of history.

Correct... and it is important that you recognize that case and discard
the article, because loop detection is not reliable for it.  Yes, loops
with propagation delays longer than history-list retention times have been
known to happen, especially when gateways or server outages get involved. 

> And if I've seen 
> it I don't expect to have my peers offer it again.

This isn't a safe assumption unless local feed topology and relayer
software are sufficiently tightly controlled that you can be sure that
either (a) there are no feed loops, or (b) all loops contain at least one
host which does full history/date-based loop detection. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: draft-ietf-usefor-usepro-01
Date: Thu, 30 Sep 2004 09:53:57 -0400
Organization: TMR Associates, Inc
Lines: 23
Sender: [email-masked]

Henry Spencer wrote:
> On Tue, 28 Sep 2004, Bill Davidsen wrote:
> 
>>And date? Not my exchange servers, I don't know your policy on age and 
>>it takes less time to send something old than make a decision on it...
> 
> 
> Note that history-based loop detection does not work if you don't verify
> that the date of the article is within your history list.
> 
> Not doing loop detection is acceptable only in tightly controlled
> situations. 

Unless you assume some evil entity editing the Path header loop 
detection will work anyway, I would assume. I certainly won't send 
anything to a site in the Path, so the worst that could happen is that I 
would get something so old it expired out of history. And if I've seen 
it I don't expect to have my peers offer it again.

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20181
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: <draft-ietf-usefor-usepro-01.txt>
Date: Wed, 29 Sep 2004 18:07:36 GMT
Lines: 183
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Under definitions:

>]   A "followup" is an article containing a response to the contents of
>]   an earlier article (the followup's "precursor").

>And no, the alternative text is not how followup is defined, since the
>ability of the poster to define how articles are displayed does not exist.
>Implying it does is incorrect.

The alternative text, which you did not quote, is

   A "followup" is an article containing a response to the contents of
   an earlier article (its "precursor"), or which is otherwise intended
   to be grouped with that article for purposes of display (e.g. as part
   of a multipart posting such as a FAQ).

No, the poster cannot "define" how articles are to be displayed, but he
can indicate his intention in the matter by creating a suitable
References-header, which is all that it says. The particular wording is
based on a suggestion by Forrest, followed by various tweaks that were
discussed here. It may yet get tweaked some more.

The alternative text in a-6.10 is also due to be rewritten with similar
wording. It is still an open issue as to which alternative wording gets
used.

>The alternative I suggested is still better. Why has it never appeared in
>any form?

You need to explain _why_ your wording is better.

>Under "defining the architecture":

>   A "followup agent" is a combination of reading agent and posting
>   agent that aids in the preparation and posting of a followup.
>[Alternative definition, to be used if the alternative definition for
>"followup" is used:]

>   A "followup agent" is a combination of reading agent and posting
>   agent that aids in the preparation and posting of a response intended
>   as a followup to a precursor.

>The latter is a meaningless extension of the former. From what else does
>a followup come but a precursor? There is no reason for the 'alternative'.

If you read the definition, some followups come from precursors and some
come from the poster's display intentions. Followup agents are only
designed to handle the first case, and don't have the necessary capability
to handle the second.

>Under 7.6, we are still presented with this stentorian "to be taken from"
>wording, instead of the correct "copied from". Plus we have a lots of 
>words in the initial paragraph that tell us that "taken from means copied
>from". If that is true, just say "copied from" and get rid of the 
>extraneous redefinition.

No, "copy" would mean "copy the whole header". "Taken" means "copy the
content of the header".

>We still have the extraneous "by default".

If we didn't say "by default", then there would be no latitude for the
poster to override that default.

> We are talking about the duties of the followup agent, which
>INCLUDES giving the article to the poster for his changes. There is no
>other action we specify for the followup agent, so there is nothing
>but "default".

It says nothing about "giving" the poster anything (that is up to the
implementor of the followup agent). How the overriding is done is not
specified (maybe the user edits those headers in the article he is given,
maybe he is obliged to choose them before composing his article, maybe he
is not given any article at all, because providing the precursor as a
quotation is not obligatory). All that is clear is that if the poster
omits to provide those headers, the "default" is what he gets.

>We also still have the extraneous "MAY be prepended" regarding "Re: " in
>the Subject. Since there is no prohibition, of course it MAY be prepended.
>And so may anything else. We need not say what MAY be prepended when the
>truth of the matter is ANYTHING MAY be prepended.

NO! The whole point of that wording, and of the compromise thrashed out
between those on my left and those on my right, was that in the *default*
anything other than "Re: " SHOULD NOT be prepended. Remember Seth's four
points?

>Para. 4: "MUST be formed by...". Formed how? I think you mean something 
>about it ought to contain the message id of the precursor. Perhaps you 
>meant to say:

>   4. If the precursor did not have a References-header (a-6.10), the
>      followup's References-content MUST be the MessageID-content of
>      the precursor.

OK, I'll buy that.

   4. If the precursor did not have a References-header (a-6.10), the
      followup's References-content MUST be copied from the Message-ID-
      content of the precursor. A followup to an article which already
      had a References-header MUST have a References-header comprising
      the precursor's References-content (subject to trimming as
      described below) followed by CFWS and the Message-ID-content of
      the precursor.

>Para. 5: The followup agent MUST mail the copy to the "copy-addr"? What if 
>the USER decides to mail a copy somewhere else instead?

Well the whole thing is already within the scope of "(and subject to
manual override by the poster)", so that is covered. The various "SHOULD"s
are meant to cover what the followup agent does in the absence of poster
overrides (like mailing to the poster regardless without taking any notice
of the header or asking the poster). They are SHOULD rather than MUST
because the harm that ensues does not actually break anything - it just
violates the whole intent of the header. But I take your point that a
perverse followup agent which mails to the poster in spite of being told
to use the copy-addr hardly merits a MUST, so I have changed it to SHOULD.

 Is the user to be 
>told "FOAD"? If not, then the MUST is ridiculous. In fact, MUST is not
>appropriate for this, as RFC2119 tells us: there is no "interoperability" 
>issue if the poster decides to send a copy somewhere by email other than
>the copy-addr. In fact, I would say it is not unreasonable for someone to
>always mail themselves a copy of anything they post; that would violate
>the MUST of this section.

>Similarly, Para. 1, the "MUST NOT" be posted is incorrect. How does a 
>user override this prohibition? By saying "post this". How does he post
>an article without this?

Again, you have failed to distinguish when the poster chooses to override
the default and when the followup agent (or rather its implementor) does
something stupid. In this case, if the poster takes no action to override
the default, then the agent MUST mail it to the poster and NOT post it.

>Section 7.7:

>   A reading agent downloads articles from a serving agent, as directed
>   by the reader, and displays them (or processes them in some other
>   manner).  The article as displayed MUST be identical to the article
>   as originally posted, subject only to limitations of the display
>   device (such as availability of charsets, etc.). 

>Whoa, doggies. Do you really intend to say that the article cannot have
>irrelevant or undesired headers left un-displayed? Do you really intend
>that the display of the PATH header MUST be as it was originally posted?
>Can the reading agent NOT rearrange headers so they are in a specific 
>order, can it NOT change the date into the local reader's timezone for his 
>convenience? Is displaying the article as originally posted the only thing
>we are going to allow reading agents to do? That's going to make a lot of
>existing code non-conforming.

OK, that is new wording, and it may yet disappear entirely. But the "MUST
be identical" is indeed too severe. I have made a note to review it.

>7.8.  Duties of a Moderator

>   A Moderator receives news articles by email, 

>A moderator receives submissions by some means, not necessarily by email.
>Certainly someone has come up with, or will come up with, a web-based
>moderation system, ...

No, the rule is that the injection agent MUST mail articles to the
moderator (at his published submission address). If the moderator chooses
to put all sorts of web sites and whatever behind that address, then that
is up to him. Essentially, the boundary of "moderator" starts anywhere on
his side of the submission address.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked], [email-masked]
Subject: Re: Usefor/usepro conflicts.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Sep 2004 13:48:41 -0700
Sender: [email-masked]

Sebastian Brocks <[email-masked]> writes:

> And now for the whole mailinglist (sorry Russ)

No problem.

> Russ Allbery schrieb:

>> I strongly disagree.  Everywhere that I've seen poster used on Usenet
>> in a context that drew the distinction between author and transmitter,
>> poster referred to the person who injected the article, not the author.

> In the case of "Followup-to: poster", certainly the author is meant, not
> the person who injected the article, isn't it?

Yeah, that's a very good point.  That would indeed be an exception.
(Well, more precisely speaking, the person in the From header is what's
meant, but per RFC 2822 that should be the author, not the sender.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Wed, 29 Sep 2004 21:43:29 +0200
From: Sebastian Brocks <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Usefor/usepro conflicts.
Lines: 34
Sender: [email-masked]

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig0E0A4C1771E301790B1A8B87
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

And now for the whole mailinglist (sorry Russ)

Russ Allbery schrieb:

> I strongly disagree.  Everywhere that I've seen poster used on Usenet in a
> context that drew the distinction between author and transmitter, poster
> referred to the person who injected the article, not the author.

In the case of "Followup-to: poster", certainly the author is meant, not
the person who injected the article, isn't it?

greetings, Sebastian

--------------enig0E0A4C1771E301790B1A8B87
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

--------------enig0E0A4C1771E301790B1A8B87--

Date: Wed, 29 Sep 2004 15:00:28 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: control propagation (was Re: draft-ietf-usefor-usepro-01)
Sender: [email-masked]

On Wed, 29 Sep 2004, Russ Allbery wrote:
> ...I'm surprised that it's not in son-of-1036 and doesn't appear
> to be in CNews, but perhaps both predate the understanding that this was
> the right thing to do [1991]...
>                 /* Newgroup being sent to a group that doesn't exist.
>                  * Assume it is being sent to the group being created,
>                  * and send the group to all sites that would get the
>                  * group if it were created. */

C News is mostly a late-80s creation and was pretty much in maintenance
mode by 1991, so if this wasn't prominently brought to our attention it
could easily have been missed.  And son-of-1036 reflected C News
experience more than anything else.  I don't recall this issue being
raised when son-of-1036 came out, but there are probably some son-of-1036
responses in my files that never got dealt with properly, as I was most
definitely in news burnout by that time. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 29 Sep 2004 14:54:36 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: Re: relay checking
Sender: [email-masked]

On Wed, 29 Sep 2004, Russ Allbery wrote:
> I thought Henry and I just agreed on language, and I'm curious as to why
> you decided to go with this language instead...
> I don't see a lot of purpose served in distinguishing between existence
> and syntactic validity of headers.  I think that makes the logic of the
> standard slightly more complex for no particularly clear reason.

To clarify my own position on this...

+ I see a small benefit from insisting on presence of mandatory headers,
but only a small one.  It's unobjectionable but not very important. 

+ I'm reluctant to demand full checking by relay agents which otherwise
don't have reason to do it.  Even SHOULD seems too strong a word here. 

+ I would *like* to see relay agents not only permitted, but explicitly
encouraged, to do as much checking as they feel they can. 

+ I don't care deeply about the issue, and could live with any wording
that permits full checking but doesn't require it. 

                                                          Henry Spencer
                                                       [email-masked]

Date: Wed, 29 Sep 2004 14:37:49 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: Re: relay checking
Sender: [email-masked]

On Wed, 29 Sep 2004, Charles Lindsey wrote:
>    3.   It SHOULD reject any article that does not have the correct
>         mandatory headers (section a-5) present.

Why "correct"?  That seems redundant, and it is bound to make people
wonder whether it has some special significance.  I'd take that word out.

Otherwise, this seems okay.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked], [email-masked]
Subject: Re: Usefor/usepro conflicts.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Sep 2004 11:37:09 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> John Stanley <[email-masked]> writes:

>> USEPRO: claims that "poster" is synonymous with RFC2822 "author", which
>> it is not. "Poster" in USEPRO says "composes and submits", while
>> RFC2822 limits "author" to the composer and clearly delineates between
>> "author" and "sender".

> OK, I now have:

>    A "poster" is the person or software that composes a possibly
>    compliant article for submission to a posting agent. The poster is
>    synonymous with [RFC 2822]'s author.

I believe this is wrong.  Poster is synonymous with sender.

>> The word in standard usage implies "the person who posts", which is
>> different than "the person who writes", so as a synonym, it belongs to
>> "sender".

> But the term as widely used throughout Usenet is as I have defined it.

I strongly disagree.  Everywhere that I've seen poster used on Usenet in a
context that drew the distinction between author and transmitter, poster
referred to the person who injected the article, not the author.

> OK, I now have:

>    Posting agents meant for use by ordinary posters SHOULD reject any
>    attempt to post an article which cancels or Supersedes another
>    article of which the poster is not the author or sender.

That works.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked], [email-masked]
Subject: Re: relay checking
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Sep 2004 11:35:12 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Yes, that is more or less the position I had come to following this
> exchange. My draft text now reads as follows:

>    3.   It SHOULD reject any article that does not have the correct
>         mandatory headers (section a-5) present.

>    4.   It MAY reject any article whose headers do not have legal
>         contents.

> Any advance on that?

I thought Henry and I just agreed on language, and I'm curious as to why
you decided to go with this language instead, from an article that
predates the conclusion that Henry and I reached.  Maybe it would be
better to see if Bill agrees with Henry and I and if so, go with that
language since it has more of a consensus?

I don't see a lot of purpose served in distinguishing between existence
and syntactic validity of headers.  I think that makes the logic of the
standard slightly more complex for no particularly clear reason.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked], [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 29 Sep 2004 11:32:30 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> which is not specific to control messages.  (This is interesting to me,
>> since I thought that special propagation of newgroup and rmgroup
>> control messages was long-standing established practice, but apparently
>> it isn't as much as I thought it was.)

> No, that has never been raised in this group before, to my knowledge.

I've raised it in this group in the past, but generally as part of other
discussions of control message handling.

> We did discuss the text that says control messages need to have a
> Newsgroups-header that will steer them to the proper places, and the
> text you see contains some tweaks arising from that.

Control messages should not, in general, be crossposted to other
newsgroups than they one that they affect.  David Lawrence was the one who
first taught me the problems caused by doing so.  For example, when people
form a habit of crossposting control messages to alt.config, someone
limiting what hierarchies in alt.* they receive, they still ends up
getting control messages for the hierarchies they don't want.

Unfortunately, this bad habit is sufficiently established that I doubt we
can really get people to stop (although relay agents could be configured
to ignore the Newsgroups header entirely).  I don't think it's a good idea
to encourage it, though (but I remember the previous discussion and
recognize that documenting it is to a degree documenting existing
practice).

However, relay agents that check the Newsgroups header against a list of
valid groups *do* need to propagate control messages as if the group they
were affecting existed.  *That* behavior does need to be specified in the
standard.  This is independent of the question of whether one should
ignore the Newsgroups header on newgroup and rmgroup control messages
entirely.  I'm surprised that it's not in son-of-1036 and doesn't appear
to be in CNews, but perhaps both predate the understanding that this was
the right thing to do.  This has been implemented in INN since the 1.0
release in 1991.

            if (IsNewgroup && Approved) {
                /* Newgroup being sent to a group that doesn't exist.
                 * Assume it is being sent to the group being created,
                 * and send the group to all sites that would get the
                 * group if it were created. */
                ARTsendnewgroup(*groups);
                Accepted = TRUE;
            }

(By INN 1.4 in 1993, and possibly earlier, this logic had been corrected
to include rmgroups as well.)

If relay agents don't implement this behavior, booster rmgroups are
futile because they won't propagate past the first host that already
removed the group and newgroups won't propagate through relay agents that
don't act on them immediately and require that newsgroups exist before
propagating messages.  This is clearly wrong.

> Now if INN has implemented some optimizations that will further improve
> the propagation of control messages, then good luck to it, but it has
> never been suggested that they should form part of this standard.

Well, I don't believe that's correct, but there's no point in arguing
about past history.  I'm making that statement now.  This should be part
of the standard.

> I wrote it, and it was intended to say what I said in the previous
> paragraph. But its meaning is not clear, so I have rewritten it as
> follows (I have also included the preceding paragraph so you get the
> full context).

>    The descriptions below set out REQUIREMENTS to be followed by sites
>    that receive control messages and choose to honour them. However,
>    nothing in these descriptions should be taken as overriding the right
>    of any such site, in accordance with its local policy, to refuse to
>    honour any particular control message, or to refer it to an
>    administrator for approval (either as a class or on a case-by-case
>    basis).

>    Even if relaying agents do not, as a matter of local policy, intend
>    to honour certain control messages, they MUST still propagate them in
>    the normal manner.
> [The inclusion of that paragraph still remains under discussion.]

This is still not correct.  This doesn't help with the issue addressed
above in propagation of newgroup and rmgroup control messages and simply
contradicts other portions of the document in a fashion that leaves one
wondering what it's supposed to mean.  Please, remove this language and
just explain that newgroup and rmgroup control messages should always be
propagated as if the group they're affecting exists, even if it does not.

> Sure, but there is a general Caveat somewhere covering that, so we do
> not need to repeat it every time we say "Relaying agents MUST ...".

Given that relaying agents clearly do *not* have such a restriction, I
think that we should view anywhere that the document states that relaying
agents MUST propagate some article with a great deal of suspicion.  At the
least, every occurance of such language is an internal contradiction.

If I recall correctly from my reading, there are rather few places where
this contradiction occurs.  This may be one of the only ones.

> It was a long way back, but there was certainly a consensus at the time
> to try and make Distributions work.

Well, again, it's futile to argue about what happened years ago, but I
don't believe a consensus continues to exist in this area.

>> I have no idea how CNews works in this area, but INN allows you to list
>> Path identities that, if found in incoming articles, should cause those
>> articles to be rejected.  This configuration option is entirely
>> independent of your own Path identity and is intended specifically for
>> aliasing out sites or pseudosites.  This facility has been in INN since
>> 1.0, I believe.

> Ah! I had always understood, since the matter was first raised on the
> news.admin.net-abuse groups, that it came out of the normal processing
> of the Path header rather than on special code in transport agents.

Nope.  It was possible because of a pre-existing feature in INN (the path
aliasing concept predates pseudo-sites by quite some time), but path
aliasing is not really part of the normal processing of the Path header.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Wed, 29 Sep 2004 11:06:54 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Re: Usefor/usepro conflicts.
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

I'm fascinated by a web page that contains explicit mailto links for
deliberately invalid email addresses.

>OK, I now have:

>   A "poster" is the person or software that composes a possibly
>   compliant article for submission to a posting agent. The poster is
>   synonymous with [RFC 2822]'s author.

Nope. Wrong action.

>> The word in standard usage
>>implies "the person who posts", which is different than "the person who
>>writes", so as a synonym, it belongs to "sender".

>But the term as widely used throughout Usenet is as I have defined it.

No. It may appear that way since USUALLY the person who writes the article
is then the one that posts it, but the action of posting is clearly
different than the action of writing, and the term "poster" means "one who
posts", not "one who writes". When someone refers to the "original
poster", they are correct, in most cases, only because the "original
author" and "original poster" are the same person. I cannot speak for most
people, but I expect that most people are able to differentiate the
actions, but are simply loose in their usage of the term. This does not
mean we ought to be.

In fact, I think it would remove confusion were we to use the terms both
as they are defined in simple engish and to isolate the actions of
authoring and posting. For example, it makes the concept of "what happens
in a moderated group" much clearer. It also makes the idea of what a 
"posting agent" does clearer. A "posting agent" need not be a text editor,
only something that can take the result of a text editor and assist in
posting it.

Date: Wed, 29 Sep 2004 09:51:58 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-usefor <[email-masked]>
Organization: Bruce Lilly
To: Alexey Melnikov <[email-masked]>
CC: ietf-usefor <[email-masked]>
Subject: Re: Issues with MIME-style parameters
Sender: [email-masked]

Charles Lindsey wrote:

[...]
> So the
> possibility that there may one day be parameters in the Path header has
> changed nothing.
[...]
> why should it be used as an argument with
> Injection-Date?

The WG Chair declared the matter of so-called "MIME" parameters
a closed issue on August 23, a full week before Charles' message
was composed, and more than a month before it was posted to the
mailing list.  This is outrageous.  Mr. Chairman, can we please
settle this issue once and for all.

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20177
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-01
Date: Wed, 29 Sep 2004 14:19:03 GMT
Lines: 136
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I am very leery of statements like "parse to the extent necessary for
>proper operation" in Usenet standards, since that concept cannot be
>tightly specified.

Yes, I would be too. It would need careful wording. But it may not be
necessary, since the protocol requires you to do various things which
obviously involve looking at the Path and Newsgroups headers, and if you
find that those headers are unintelligible as regards what you need to do
with them, then you are going to barf anyway without being told to.

>Currently, INN can optionally ignore the Newsgroups header is ignored in
>newgroup and rmgroup control messages by INN; such messages are treated as
>if they were posted only to the affected newsgroup (and then propagated
>using the special propagation rules for control messages, which ignore
>newsgroup existence).  I'm trying to remember previous discussions we had
>on this topic; did we decide that was wrong?  It doesn't make a lot of
>sense to me right now that it's an option in INN, so it would be nice to
>standardize one type of behavior or another.

>RFC 1036 is silent on the subject and Son-of-1036 addresses it only in a
>note:

>         NOTE:  This check is significant because information on what
>         newsgroups a relayer wishes to receive is often stored at its
>         neighbors, who may not have up-to-date information or may
>         simplify the rules for implementation reasons.  As a hedge
>         against the possibility of missed or delayed newgroup control
>         messages, relayers may wish to observe a notion of a newsgroup
>         subscription that is independent of the list of newsgroups
>         actually known to the relayer.  This would permit reception and
>         relaying of articles in newsgroups that the relayer is not (yet)
>         aware of, subject to more general criteria indicating that they
>         are likely to be of interest.

That NOTE has never made its way into our drafts, and it is not clear
quite what it is suggesting. I think everyone accepts that a message
posted to a group two minutes after that group was newgrouped is going to
fail to appear in many places, though the flooding algorithm should get it
to most of them eventually.

>which is not specific to control messages.  (This is interesting to me,
>since I thought that special propagation of newgroup and rmgroup control
>messages was long-standing established practice, but apparently it isn't
>as much as I thought it was.)

No, that has never been raised in this group before, to my knowledge.

We did discuss the text that says control messages need to have a
Newsgroups-header that will steer them to the proper places, and the text
you see contains some tweaks arising from that.

Now if INN has implemented some optimizations that will further improve the
propagation of control messages, then good luck to it, but it has never
been suggested that they should form part of this standard.

>> Now the other paragraph from draft-13, which now appears in USEPRO
>> section 6, says:

>>    Relaying Agents MUST propagate all control messages regardless of
>>    whether or not they are recognized or processed locally.

>> That was intended to say that, even if your local policy is to ignore
>> all newgroup message and all cancels, you are still supposed to
>> propagate control messages normally for the benefit of neighbouring
>> sites who are not so fussy. It certainly was not meant to say that you
>> were to propagate them to peers who did not take the affected
>> hierarchies at all.

>I don't know where this sentence came from and it's never made any sense
>to me.  It's on my list of language that I believe should be striken from
>the USEPRO document.

I wrote it, and it was intended to say what I said in the previous
paragraph. But its meaning is not clear, so I have rewritten it as follows
(I have also included the preceding paragraph so you get the full
context).

   The descriptions below set out REQUIREMENTS to be followed by sites
   that receive control messages and choose to honour them. However,
   nothing in these descriptions should be taken as overriding the right
   of any such site, in accordance with its local policy, to refuse to
   honour any particular control message, or to refer it to an
   administrator for approval (either as a class or on a case-by-case
   basis).

   Even if relaying agents do not, as a matter of local policy, intend
   to honour certain control messages, they MUST still propagate them in
   the normal manner.
[The inclusion of that paragraph still remains under discussion.]

Hopefully, it is now clearer what it means, so we can proceed to discuss
whether we need it at all. Comments?

>> If it is agreed that what I have just said is what we intend,

>Certainly not.  Relay agents are never required to propagate anything
>whatsoever.

Sure, but there is a general Caveat somewhere covering that, so we do not
need to repeat it every time we say "Relaying agents MUST ...".

>> We agreed long ago to attempt to make the Distribution header more
>> effective by making sure it got looked at at the proper places.

>Note that I do not consider myself to be part of this "we" at this time
>and do not agree that we have consensus to do this.

It was a long way back, but there was certainly a consensus at the time to
try and make Distributions work.

>I have no idea how CNews works in this area, but INN allows you to list
>Path identities that, if found in incoming articles, should cause those
>articles to be rejected.  This configuration option is entirely
>independent of your own Path identity and is intended specifically for
>aliasing out sites or pseudosites.  This facility has been in INN since
>1.0, I believe.

Ah! I had always understood, since the matter was first raised on the
news.admin.net-abuse groups, that it came out of the normal processing of
the Path header rather than on special code in transport agents.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20180
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Usefor/usepro conflicts.
Date: Wed, 29 Sep 2004 14:57:53 GMT
Lines: 64
Sender: [email-masked]

In <[email-masked]> John Stanley <[email-masked]> writes:

>Bravo to USEFOR for having reached the correct style and length.

>Boo to USEPRO for contradicting USEFOR.

>USEPRO: redefines the term "sender" from what RFC2822 says, although the
>Sender header is supposed to be identical to that from RFC2822. There is
>no reason to define 'sender' in our draft. 

Currently, USEPRO is gathering together all the definitions that may be
needed in both documents. When things have settled down, we will check
which terms are still being used and which document (or even both) the
definition belongs in.

Your point about "sender" is noted.

>USEPRO: claims that "poster" is synonymous with RFC2822 "author", which
>it is not. "Poster" in USEPRO says "composes and submits", while RFC2822
>limits "author" to the composer and clearly delineates between "author"
>and "sender".

OK, I now have:

   A "poster" is the person or software that composes a possibly
   compliant article for submission to a posting agent. The poster is
   synonymous with [RFC 2822]'s author.

> The word in standard usage
>implies "the person who posts", which is different than "the person who
>writes", so as a synonym, it belongs to "sender".

But the term as widely used throughout Usenet is as I have defined it.

>This points out a contradiction in 7.5:

>   Posting agents meant for use by ordinary posters SHOULD reject any
>   attempt to post an article which cancels or Supersedes another
>   article of which the poster is not the author.

OK, I now have:

   Posting agents meant for use by ordinary posters SHOULD reject any
   attempt to post an article which cancels or Supersedes another
   article of which the poster is not the author or sender.

>USEFOR tells us that "Compliant software MUST support headers of at least
>998 octets."

I think it needs to say that it MUST NOT generate longer than that, but
MAY accept more. I shall look into that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20179
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Thank you
Date: Wed, 29 Sep 2004 14:28:21 GMT
Lines: 19
Sender: [email-masked]

In <[email-masked]> Bill Davidsen <[email-masked]> writes:

>To whoever got my mail feed going. I have been told by majordome several 
>time that I was subscribed, but until the 26th I wasn't getting anything 
>unless it was sent by hand or perhaps to the old address.

Is anyone else still having problems reading or posting to the new list?
If not, I will stop posting to both.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20178
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: relay checking
Date: Wed, 29 Sep 2004 14:26:38 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Bill Davidsen <[email-masked]> writes:

>Another case where SHOULD is too strong and MAY is too weak. Perhaps we 
>can split this hair and say SHOULD check that mandatory headers are 
>present and MAY check the validity of any recognized header?

Yes, that is more or less the position I had come to following this
exchange. My draft text now reads as follows:

   3.   It SHOULD reject any article that does not have the correct
        mandatory headers (section a-5) present.

   4.   It MAY reject any article whose headers do not have legal
        contents.

Any advance on that?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20161
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
Date: Tue, 28 Sep 2004 18:50:57 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>4. a) During fragmentation, which message header fields are to be copied
>   to the enclosing message header, and which are to be preserved in
>   the enclosed first part message/partial header?  b) Which fields
>   need to be copied to the headers of each of the subsequent message
>   fragments?
>5. During reassembly, which fields of the first part message header
>   are to be retained, which are to be discarded; likewise for the
>   fields in the header of the enclosed first part?

>Specifically regarding question 5, I expect that Approved, Newsgroups,
>Distribution, Path, Followup-To, Expires, Summary, Organization,
>Injection-Date, Injector-Info, and User-Agent can be safely copied to
>the header of the first part and at least Approved, Newsgroups,
>Distribution, Path, Injection-Date, and Expires should be copied to
>the headers of subsequent parts.  Copying these fields to the first
>part is expected behavior when fragmenting per RFC 2046 as amended by
>RFC 3798.  I believe there may be difficulties with Control,
>Supersedes, Archive, Posted-And-Mailed, Mail-Copies-To, and possibly
>Complaints-To.

Having thought about this some more, I have concluded that nearly all
headers about which RFC 2046 says nothing specific should be copied to the
headers of the first fragment (and mostly to the subsequent fragments
too), because the reassembly process will then put them back in the
article proper upon reassembly. The only ones which need to be placed in
the enclosed header in the first fragment (in addition to those mentioned
in RFC 2046) are those that might otherwise have been different in the
various fragments. Lines is an obvious candidate here, and also
User-Agent (since injectors are allowed to add bits to it). I see no
reason to make exception for any others (and hence the changes we need to
make to RFC 2046 are rather slight).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: Usefor/usepro conflicts.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 17:49:07 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> USEPRO: redefines the term "sender" from what RFC2822 says, although the
> Sender header is supposed to be identical to that from RFC2822. There is
> no reason to define 'sender' in our draft.

> USEPRO: claims that "poster" is synonymous with RFC2822 "author", which
> it is not. "Poster" in USEPRO says "composes and submits", while RFC2822
> limits "author" to the composer and clearly delineates between "author"
> and "sender". If we want to use "poster" as a synonym, fine, but to call
> it a synonym when it is not is incorrect. The word in standard usage
> implies "the person who posts", which is different than "the person who
> writes", so as a synonym, it belongs to "sender". This matches the
> actions of a "posting agent", and references in the drafts to the
> assistance posting agents give in posting the article (presumably given
> to the poster).

This makes sense to me.  I think "poster" should in practice be used as a
synonym for "sender" (and in fact we may not need the term at all; we
could just use "sender" with the same definition as RFC 2822).

> This points out a contradiction in 7.5:

>    Posting agents meant for use by ordinary posters SHOULD reject any
>    attempt to post an article which cancels or Supersedes another
>    article of which the poster is not the author.

> Using RFC2822's example, if John asks Michael to cancel the article he
> had Michael post, our posting agent is being told it SHOULD reject that
> action, since the poster (Michael) is not the author (John). This same
> section would have the operator of an incoming gateway unable to cancel
> articles from his gateway.

Yes, I believe the last clause should instead be something more like
"which cancels or Supersedes an article from a different poster."

> USEFOR tells us that "Compliant software MUST support headers of at
> least 998 octets." USEPRO tells us "[relaying agents] MUST NOT refold
> any header (i.e. they must pass on the folding as received), even to
> remove FWS from a Newsgroups-header;". What does a relaying agent which
> has been written to support the minimum do when given an article by an
> agent written to support more than the minimum, and it has? I.e., A
> gives B an article with a header line of more than 998 octets? Both A
> and B are prohibited from refolding it. Does it get truncated or
> discarded?

It would have to be discarded (in other words, the article would be
rejected by that agent), as truncation is also modification of the
article.

> Would it be better to specify a maximum that is supported, so that
> everyone can know ahead of time what that maximum is and follow it?

The supported maximum SHOULD be unlimited, but I'm not sure that we can
flatly require that.  Site policy on maximum article length will, of
course, put a practical limit on header size, but it's hard to clearly
state that in the standard.  I don't believe it makes sense to specify any
magic number other than 998.

This is probably a situation where a posting agent SHOULD ensure that it
generates no headers longer than 998 octets for maximum propagation.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Subject: Re: Usefor/usepro conflicts.
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 17:43:48 -0700
Sender: [email-masked]

John Stanley <[email-masked]> writes:

> Bravo to USEFOR for having reached the correct style and length.

I'd like to second this, btw, since I've not yet said this in public.  The
current USEFOR draft is exactly the sort of document that I feel we've
needed all along, and I'm delighted to see it.  This will be a valuable
and helpful contribution to Usenet going forward.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 28 Sep 2004 17:15:50 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: Usefor/usepro conflicts.
Sender: [email-masked]

Bravo to USEFOR for having reached the correct style and length.

Boo to USEPRO for contradicting USEFOR.

USEPRO: redefines the term "sender" from what RFC2822 says, although the
Sender header is supposed to be identical to that from RFC2822. There is
no reason to define 'sender' in our draft. 

USEPRO: claims that "poster" is synonymous with RFC2822 "author", which
it is not. "Poster" in USEPRO says "composes and submits", while RFC2822
limits "author" to the composer and clearly delineates between "author"
and "sender". If we want to use "poster" as a synonym, fine, but to call
it a synonym when it is not is incorrect. The word in standard usage
implies "the person who posts", which is different than "the person who
writes", so as a synonym, it belongs to "sender". This matches the actions 
of a "posting agent", and references in the drafts to the assistance 
posting agents give in posting the article (presumably given to the 
poster).

This points out a contradiction in 7.5:

   Posting agents meant for use by ordinary posters SHOULD reject any
   attempt to post an article which cancels or Supersedes another
   article of which the poster is not the author.

Using RFC2822's example, if John asks Michael to cancel the article he
had Michael post, our posting agent is being told it SHOULD reject
that action, since the poster (Michael) is not the author (John). This 
same section would have the operator of an incoming gateway unable to 
cancel articles from his gateway.

USEFOR tells us that "Compliant software MUST support headers of at least
998 octets." USEPRO tells us "[relaying agents] MUST NOT refold any header
(i.e. they must pass on the folding as received), even to remove FWS from
a Newsgroups-header;". What does a relaying agent which has been written
to support the minimum do when given an article by an agent written to
support more than the minimum, and it has? I.e., A gives B an article with
a header line of more than 998 octets? Both A and B are prohibited from
refolding it. Does it get truncated or discarded? Would it be better to 
specify a maximum that is supported, so that everyone can know ahead of 
time what that maximum is and follow it?

Date: Tue, 28 Sep 2004 14:58:14 -0700 (PDT)
From: John Stanley <[email-masked]>
To: [email-masked]
Subject: <draft-ietf-usefor-usepro-01.txt>
Sender: [email-masked]

Let's see if the list is now open ...

Under definitions:

]   A "followup" is an article containing a response to the contents of
]   an earlier article (the followup's "precursor").
][Alternative definition, to bu ised if similar woprding is not added to
]the description of the References-header (a-6.10):]

What is "to bu ised"?

And no, the alternative text is not how followup is defined, since the
ability of the poster to define how articles are displayed does not exist.
Implying it does is incorrect.

The alternative I suggested is still better. Why has it never appeared in
any form?

It took a bit of looking, but I finally found the meaning of a-6.10. It 
would have been much simpler had you just used [ARTICLE ...] as the 
reference.

Under "defining the architecture":

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of a followup.
[Alternative definition, to be used if the alternative definition for
"followup" is used:]

   A "followup agent" is a combination of reading agent and posting
   agent that aids in the preparation and posting of a response intended
   as a followup to a precursor.

The latter is a meaningless extension of the former. From what else does
a followup come but a precursor? There is no reason for the 'alternative'.

Under 7.6, we are still presented with this stentorian "to be taken from"
wording, instead of the correct "copied from". Plus we have a lots of 
words in the initial paragraph that tell us that "taken from means copied
from". If that is true, just say "copied from" and get rid of the 
extraneous redefinition.

We still have the extraneous "by default". If the only action 
specified is to copy, I mean, "take", the content from the precursor's
similar header, then that IS the default, and saying "by default" is
meaningless. We are talking about the duties of the followup agent, which
INCLUDES giving the article to the poster for his changes. There is no
other action we specify for the followup agent, so there is nothing
but "default".

We also still have the extraneous "MAY be prepended" regarding "Re: " in
the Subject. Since there is no prohibition, of course it MAY be prepended.
And so may anything else. We need not say what MAY be prepended when the
truth of the matter is ANYTHING MAY be prepended.

Para. 4: "MUST be formed by...". Formed how? I think you mean something 
about it ought to contain the message id of the precursor. Perhaps you 
meant to say:

   4. If the precursor did not have a References-header (a-6.10), the
      followup's References-content MUST be the MessageID-content of
      the precursor.

Para. 5: The followup agent MUST mail the copy to the "copy-addr"? What if 
the USER decides to mail a copy somewhere else instead? Is the user to be 
told "FOAD"? If not, then the MUST is ridiculous. In fact, MUST is not
appropriate for this, as RFC2119 tells us: there is no "interoperability" 
issue if the poster decides to send a copy somewhere by email other than
the copy-addr. In fact, I would say it is not unreasonable for someone to
always mail themselves a copy of anything they post; that would violate
the MUST of this section.

Similarly, Para. 1, the "MUST NOT" be posted is incorrect. How does a 
user override this prohibition? By saying "post this". How does he post
an article without this? By saying "post this". Since both cases involve
the same action, the MUST NOT is specious, and in any case does not 
involve any interoprability issue necessary for such language.

Section 7.7:

   A reading agent downloads articles from a serving agent, as directed
   by the reader, and displays them (or processes them in some other
   manner).  The article as displayed MUST be identical to the article
   as originally posted, subject only to limitations of the display
   device (such as availability of charsets, etc.). 

Whoa, doggies. Do you really intend to say that the article cannot have
irrelevant or undesired headers left un-displayed? Do you really intend
that the display of the PATH header MUST be as it was originally posted?
Can the reading agent NOT rearrange headers so they are in a specific 
order, can it NOT change the date into the local reader's timezone for his 
convenience? Is displaying the article as originally posted the only thing
we are going to allow reading agents to do? That's going to make a lot of
existing code non-conforming.

                                                   It MUST provide
   facilities for decoding any Content-Transfer-Encodings, encoded-
   words, etc., but SHOULD also have the capability to show the article
   exactly as received.

No, you just said it MUST display the article "as originally posted",
which is not the same as "exactly as received". How can it "SHOULD"
do one thing when you've just said it MUST do another? 

7.8.  Duties of a Moderator

   A Moderator receives news articles by email, 

A moderator receives submissions by some means, not necessarily by email.
Certainly someone has come up with, or will come up with, a web-based
moderation system, and this definition would claim that the person we 
would normally call the moderator is not, because he did not receive the 
articles as specified.

The next paragraph continues the error. If you want to list possible means 
of doing something, ok, but to make an exclusive list is wrong.

Date: Tue, 28 Sep 2004 17:38:54 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
Sender: [email-masked]

On Tue, 28 Sep 2004, Bill Davidsen wrote:
> And date? Not my exchange servers, I don't know your policy on age and 
> it takes less time to send something old than make a decision on it...

Note that history-based loop detection does not work if you don't verify
that the date of the article is within your history list.

Not doing loop detection is acceptable only in tightly controlled
situations. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Thank you
Date: Tue, 28 Sep 2004 17:29:36 -0400
Organization: TMR Associates, Inc
Lines: 8
Sender: [email-masked]

To whoever got my mail feed going. I have been told by majordome several 
time that I was subscribed, but until the 26th I wasn't getting anything 
unless it was sent by hand or perhaps to the old address.

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: relay checking
Date: Tue, 28 Sep 2004 17:28:15 -0400
Organization: TMR Associates, Inc
Lines: 20
Sender: [email-masked]

Henry Spencer wrote:

> I'm a little reluctant to use SHOULD for any of this, because that keyword
> is generally understood to mean that the action in question is nearly
> mandatory, that almost everyone is expected to do it but there are special
> situations where it is impractical.  This doesn't seem right for something
> where incomplete implementations are common and expected to remain so.
> 
> I think the right choice is either to say MAY with some accompanying words
> urging people to do it, or to say SHOULD but accompany it with an escape
> hatch like "to the extent that performance requirements permit".

Another case where SHOULD is too strong and MAY is too weak. Perhaps we 
can split this hair and say SHOULD check that mandatory headers are 
present and MAY check the validity of any recognized header?

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: draft-ietf-usefor-usepro-01
Date: Tue, 28 Sep 2004 17:21:21 -0400
Organization: TMR Associates, Inc
Lines: 41
Sender: [email-masked]

Charles Lindsey wrote:

>>A relaying agent MUST check the legality of all headers that it needs to
>>use to do its job, namely at least Path, Date, Message-ID, Control if
>>present, Injection-Date if present, and probably also Newsgroups.
>>Demoting the rest to a SHOULD seems reasonable to me.  Thinking about it,
>>I see no reason to distinguish between mandatory and optional headers
>>here, although perhaps I'm missing something.

Depending on the relaying agent, I would expect the newsgroups and path 
headers to be checked, since those are needed for delivery. Beyond that 
anything is optional, control means nothing since group names need not 
be validated beyond seeing if we accept them and some peer wants them. 
And date? Not my exchange servers, I don't know your policy on age and 
it takes less time to send something old than make a decision on it. The 
correct age of most posts today is measured in ms, anything older is 
down in the noise.

> I could easily be persuaded that the second paragraph and NOTE are
> supefluous, but I would still be interested to hear whether relaying
> agents actually do it or not.

I could be convinced that checking that required headers are present, 
but the number of cases where that is not the case is vanishingly small.
> 
> But the 1st paragraph seems to be an important part of the protocol, and
> part of the machinery for avoiding loops.
> 

In many ways an exchange server is like a router, doing no more checking 
that is required. Some do much more, some are not just doing exchange. I 
would rather not see a lot of mandated checking. Clearly a relaying 
agent may be more than an exchange server, and may do many thing beyond 
routing posts, like handling readers, spam filtering, etc. But it need 
not be, and I don't see any reason the standard should be changed to 
suggest otherwise.

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 14:07:20 -0700
Sender: [email-masked]

Bill Davidsen <[email-masked]> writes:

> Clearly some agent is responsible for generating the required headers,
> beyond the enumeration of mandatory headers need anything else be
> mentioned?

It's probably worthwhile to say something about the breakdown of duties
between the posting agent and the injecting agent, but we do already do
that (in those sections).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 28 Sep 2004 17:01:30 -0400
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
To: Russ Allbery <[email-masked]>
CC: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
Sender: [email-masked]

Russ Allbery wrote:

>>3. I have added a section on "Duties of a Reading Agent". The prime
>>reason for this was to find a home for the bit in draft-13:6.6 that
>>"... reading agents MAY be configured so that unwanted distributions do
>>not get displayed" - this was part of our grand attempt to make the
>>Distribution-header more effective by encouraging it to be checked by
>>both senders and receivers. I have padded this out with other obvious
>>duties for the reading agent.
> 
> 
> I think both that comment and really this entire section should simply be
> striken in its entirety.  It seems completely unnecessary to me.

Clearly some agent is responsible for generating the required headers, 
beyond the enumeration of mandatory headers need anything else be mentioned?

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: draft-ietf-usefor-usepro-01
Date: Tue, 28 Sep 2004 17:01:30 -0400
Organization: TMR Associates, Inc
Lines: 21
Cc: [email-masked]
To: Russ Allbery <[email-masked]>
Sender: [email-masked]

Russ Allbery wrote:

>>3. I have added a section on "Duties of a Reading Agent". The prime
>>reason for this was to find a home for the bit in draft-13:6.6 that
>>"... reading agents MAY be configured so that unwanted distributions do
>>not get displayed" - this was part of our grand attempt to make the
>>Distribution-header more effective by encouraging it to be checked by
>>both senders and receivers. I have padded this out with other obvious
>>duties for the reading agent.
> 
> 
> I think both that comment and really this entire section should simply be
> striken in its entirety.  It seems completely unnecessary to me.

Clearly some agent is responsible for generating the required headers, 
beyond the enumeration of mandatory headers need anything else be mentioned?

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: Usefor Mailing List <[email-masked]>, [email-masked]
Subject: Re: relay checking
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 13:16:16 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:

> I think the right choice is either to say MAY with some accompanying
> words urging people to do it, or to say SHOULD but accompany it with an
> escape hatch like "to the extent that performance requirements permit".

Hm.  Okay, yeah, that works for me.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 28 Sep 2004 15:31:45 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: Re: relay checking
Sender: [email-masked]

On Tue, 28 Sep 2004, Russ Allbery wrote:
> > My feeling is that we should *encourage* relaying agents to do as much
> > checking -- beyond the minimum they need to function -- as they think
> > they can manage, but given that they historically have serious
> > performance constraints, we should refrain from *demanding* extra work
> > from them unless it is clearly vital...
> 
> That sounds like an argument for SHOULD (that's the encouragement) check
> the syntactical validity of the article, without distinctions between
> mandatory and optional headers.  Does that match what you're thinking?

I'm a little reluctant to use SHOULD for any of this, because that keyword
is generally understood to mean that the action in question is nearly
mandatory, that almost everyone is expected to do it but there are special
situations where it is impractical.  This doesn't seem right for something
where incomplete implementations are common and expected to remain so.

I think the right choice is either to say MAY with some accompanying words
urging people to do it, or to say SHOULD but accompany it with an escape
hatch like "to the extent that performance requirements permit".

                                                          Henry Spencer
                                                       [email-masked]

To: Usefor Mailing List <[email-masked]>, [email-masked]
Subject: Re: relay checking
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 12:04:06 -0700
Sender: [email-masked]

Henry Spencer <[email-masked]> writes:

> The feedback we got, way back when, was that this distinction *was*
> valuable -- that merely demanding the presence of the mandatory headers
> eliminated a class of problems.  Since I never knew the details of what
> those problems were, I can't say whether they're still relevant.  On the
> whole, I rather doubt it.

Oh, okay.

> My feeling is that we should *encourage* relaying agents to do as much
> checking -- beyond the minimum they need to function -- as they think
> they can manage, but given that they historically have serious
> performance constraints, we should refrain from *demanding* extra work
> from them unless it is clearly vital.

> If for no other reason, I think this is preferable because it avoids the
> whole question of exactly which things need to be checked and how
> thoroughly.

That sounds like an argument for SHOULD (that's the encouragement) check
the syntactical validity of the article, without distinctions between
mandatory and optional headers.  Does that match what you're thinking?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 28 Sep 2004 14:58:19 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: [email-masked]
Subject: relay checking (was Re: draft-ietf-usefor-usepro-01)
Sender: [email-masked]

On Tue, 28 Sep 2004, Russ Allbery wrote:
> That attitude is incompatible with what Henry was saying about making sure
> that the article is at least basically formed, I think.  I don't see how
> it makes any sense to be sure mandatory headers are present without making
> sure that they contain usable data (and the only definition of usable data
> that makes any sense to me is compliant and parsable data).

The feedback we got, way back when, was that this distinction *was*
valuable -- that merely demanding the presence of the mandatory headers
eliminated a class of problems.  Since I never knew the details of what
those problems were, I can't say whether they're still relevant.  On the
whole, I rather doubt it.

My feeling is that we should *encourage* relaying agents to do as much
checking -- beyond the minimum they need to function -- as they think they
can manage, but given that they historically have serious performance
constraints, we should refrain from *demanding* extra work from them
unless it is clearly vital. 

If for no other reason, I think this is preferable because it avoids the
whole question of exactly which things need to be checked and how
thoroughly. 

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked], [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 28 Sep 2004 11:31:17 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> Right.  Those are the two mandatory headers that have no real impact on
>> relaying.  Henry's point is well-taken in that it might be worthwhile
>> doing some sort of cursory check just to keep from propagating trash,
>> but on the other hand, major servers don't do this now and Usenet
>> hasn't broken.  To me, that argues against a MUST and towards a SHOULD.

> Two points, arising from Henry:
> 1. MUST/SHOULD check for presence of mandatory headers.
> 2. MUST/SHOULD check for legality of listed headers, but only to the
> extent needed to ensure they work. For example, you told us recently that
> you only parse the Date header until you have seen enough to construct a
> valid date; any trailing rubbish is not looked at. Again, for newsgroups,
> you are only concerned to see whether anything in your active file is
> included; you don't care if some malformed names of non-existent groups
> are present, so long as your parser can manage to skip over them.

> Is that OK, assuming I can invent a wording for "parse to the extent
> necessary for proper operation". And in that case, please choose between
> MUST and SHOULD.

I am very leery of statements like "parse to the extent necessary for
proper operation" in Usenet standards, since that concept cannot be
tightly specified.

I don't believe there's any reasonable middle point.  Either we're going
to require that relay agents detect and discard some set of invalid
messages, or we're not going to do that and leave it up to each individual
relay agent.  Currently, existing practice is the latter -- relay agents
accept and propagate all sorts of syntactically broken messages.  If
that's acceptable to people, then I think we should just say SHOULD check
or MAY check headers and leave it at that.

That attitude is incompatible with what Henry was saying about making sure
that the article is at least basically formed, I think.  I don't see how
it makes any sense to be sure mandatory headers are present without making
sure that they contain usable data (and the only definition of usable data
that makes any sense to me is compliant and parsable data).

>> No, see the special propagation rules for newgroup and rmgroup control
>> messages.  Relaying agents have to know about that.

> Yes, but that does not impose any requirement to check the Control
> header for legality, which is what we are discussing here.

It requires that the Control message be parsed to the degree that the type
of control message and its parameters be recognized, which inherently
requires validating its syntax to some degree.  Of course, validating the
syntax doesn't mean that one has to reject invalid messages; one can just
treat them as if they don't have a Control header at all.  Is that what
you're saying?

> At most, it means recognizing that a Control header is present,

You have to at least know whether it's a newgroup or rmgroup control
message.

> However, upon looking at that wording, it does seem to say more than I
> intended. In draft-13 it said (and the new USEFOR needs to say) that the
> Newsgroups header of a control message needs to include the
> newsgroup-name affected, and anything else that will help to steer it to
> where it is wanted.

Currently, INN can optionally ignore the Newsgroups header is ignored in
newgroup and rmgroup control messages by INN; such messages are treated as
if they were posted only to the affected newsgroup (and then propagated
using the special propagation rules for control messages, which ignore
newsgroup existence).  I'm trying to remember previous discussions we had
on this topic; did we decide that was wrong?  It doesn't make a lot of
sense to me right now that it's an option in INN, so it would be nice to
standardize one type of behavior or another.

RFC 1036 is silent on the subject and Son-of-1036 addresses it only in a
note:

    Third, the relayer MUST determine whether any of the article's
    newsgroups are "subscribed to" by the host, i.e. fit a description of
    what hierarchies or newsgroups the site wants to receive.

         NOTE:  This check is significant because information on what
         newsgroups a relayer wishes to receive is often stored at its
         neighbors, who may not have up-to-date information or may
         simplify the rules for implementation reasons.  As a hedge
         against the possibility of missed or delayed newgroup control
         messages, relayers may wish to observe a notion of a newsgroup
         subscription that is independent of the list of newsgroups
         actually known to the relayer.  This would permit reception and
         relaying of articles in newsgroups that the relayer is not (yet)
         aware of, subject to more general criteria indicating that they
         are likely to be of interest.

which is not specific to control messages.  (This is interesting to me,
since I thought that special propagation of newgroup and rmgroup control
messages was long-standing established practice, but apparently it isn't
as much as I thought it was.)

> Now the other paragraph from draft-13, which now appears in USEPRO
> section 6, says:

>    Relaying Agents MUST propagate all control messages regardless of
>    whether or not they are recognized or processed locally.

> That was intended to say that, even if your local policy is to ignore
> all newgroup message and all cancels, you are still supposed to
> propagate control messages normally for the benefit of neighbouring
> sites who are not so fussy. It certainly was not meant to say that you
> were to propagate them to peers who did not take the affected
> hierarchies at all.

I don't know where this sentence came from and it's never made any sense
to me.  It's on my list of language that I believe should be striken from
the USEPRO document.

> If it is agreed that what I have just said is what we intend,

Certainly not.  Relay agents are never required to propagate anything
whatsoever.  Instead, the special propagation rules for newgroup and
rmgroup control messages need to be specifically described.

>> Yes, but beware of backward compatibility here; not everyone will use
>> Injection-Date by preference.  Plus, it's more complicated to say that;
>> it's easier to just check both of them.

> OK, I shall not bother if it uses too much verbiage.

It will add unnecessary complexity even if there's a short way of saying
it.  Please, don't bother.  :)

>> INN does, yes.  Remember that for a relaying agent, if it accepts the
>> message, it also passes it on to other sites.  I think that one SHOULD
>> avoid propagating known-broken messages.

> Yes, but if you don't look, it isn't "known". My inclination is to take
> Henry's lead and demote to MAY. I am pretty sure that transit relayers
> will not be making such checks.

INN does, as I've said several times, although in a somewhat limited
capacity (in that it's somewhat particular about what it cares about and
what it doesn't care about).  I'm hoping to clean that up over time.

> We agreed long ago to attempt to make the Distribution header more
> effective by making sure it got looked at at the proper places.

Note that I do not consider myself to be part of this "we" at this time
and do not agree that we have consensus to do this.

> So am I right in thinking that if I don't want to see cancels from all
> the regular spam cancellers, I have got to instruct my upstream peers to
> register 'cybercancel' as an alias for my site in their sys files?

I have no idea how CNews works in this area, but INN allows you to list
Path identities that, if found in incoming articles, should cause those
articles to be rejected.  This configuration option is entirely
independent of your own Path identity and is intended specifically for
aliasing out sites or pseudosites.  This facility has been in INN since
1.0, I believe.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: LIST: [email-masked], [email-masked];
Xref: clerew local.usefor:20160
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-01
Date: Tue, 28 Sep 2004 11:24:15 GMT
Lines: 131
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>> OK, an explicit list makes sense (leaves From and Subject out of it).

>Right.  Those are the two mandatory headers that have no real impact on
>relaying.  Henry's point is well-taken in that it might be worthwhile
>doing some sort of cursory check just to keep from propagating trash, but
>on the other hand, major servers don't do this now and Usenet hasn't
>broken.  To me, that argues against a MUST and towards a SHOULD.

Two points, arising from Henry:
1. MUST/SHOULD check for presence of mandatory headers.
2. MUST/SHOULD check for legality of listed headers, but only to the
extent needed to ensure they work. For example, you told us recently that
you only parse the Date header until you have seen enough to construct a
valid date; any trailing rubbish is not looked at. Again, for newsgroups,
you are only concerned to see whether anything in your active file is
included; you don't care if some malformed names of non-existent groups
are present, so long as your parser can manage to skip over them.

Is that OK, assuming I can invent a wording for "parse to the extent
necessary for proper operation". And in that case, please choose between
MUST and SHOULD.

>> I am not so sure about Control; surely that is for serving agents, not
>> relaying agents (and the serving agent will probably just hand it over
>> to whatever subsystem deals with control messages and let that check as
>> it sees fit)?

>No, see the special propagation rules for newgroup and rmgroup control
>messages.  Relaying agents have to know about that.

Yes, but that does not impose any requirement to check the Control header
for legality, which is what we are discussing here. At most, it means
recognizing that a Control header is present, and I think the existing
wording (now in USEPRO section 6) says what needs to be said.

However, upon looking at that wording, it does seem to say more than I
intended. In draft-13 it said (and the new USEFOR needs to say) that
the Newsgroups header of a control message needs to include the
newsgroup-name affected, and anything else that will help to steer it to
where it is wanted. But if I only take the big-8 plus uk.* on my system, I
don't want to be flooded out with newgroup messages for jp.*.

Now the other paragraph from draft-13, which now appears in USEPRO section
6, says:

   Relaying Agents MUST propagate all control messages regardless of
   whether or not they are recognized or processed locally.

That was intended to say that, even if your local policy is to ignore all
newgroup message and all cancels, you are still supposed to propagate
control messages normally for the benefit of neighbouring sites who are
not so fussy. It certainly was not meant to say that you were to propagate
them to peers who did not take the affected hierarchies at all.

If it is agreed that what I have just said is what we intend, then I need
to review that wording, especially as the two paragraphs which were once
together are now going to be in different documents.

>> Also, one could omit Date if Injection-Date is present.

>Yes, but beware of backward compatibility here; not everyone will use
>Injection-Date by preference.  Plus, it's more complicated to say that;
>it's easier to just check both of them.

OK, I shall not bother if it uses too much verbiage.

>> As for the rest, why not demote to MAY rather than SHOULD? Do current
>> relaying agents routinely check all the lesser known headers, especially
>> if they are just transit agents?

>INN does, yes.  Remember that for a relaying agent, if it accepts the
>message, it also passes it on to other sites.  I think that one SHOULD
>avoid propagating known-broken messages.

Yes, but if you don't look, it isn't "known". My inclination is to take
Henry's lead and demote to MAY. I am pretty sure that transit relayers
will not be making such checks.

>>    Reading agents SHOULD NOT, unless explicitly configured otherwise,
>>    act automatically on Application types which could change the state
>>    of that agent (e.g. by writing or modifying files), except in the
>>    case of those prescribed for use in control messages (7.2.1.2 and
>>    7.2.4.1).

>Yes, I think this is a security consideration and would rather see it
>there than in a separate duties section.  This isn't a statement about
>what a reading agent should do, but rather what a reading agent shouldn't
>do.

Agreed.

>I really think the Distribution thing is pointless.  Is anyone aware of a
>reading agent that actually does that?  Would anyone reading this list
>actually use that feature, as opposed to just reading the relevant
>regional group?

We agreed long ago to attempt to make the Distribution header more
effective by making sure it got looked at at the proper places. So that
action by reading agents was something we wanted to encourage to happen,
rather than something that was happening currently (hence the MAY). But I
am happy to move that particular bit to USEAGE.

>> I could easily be persuaded that the second paragraph and NOTE are
>> supefluous, but I would still be interested to hear whether relaying
>> agents actually do it or not.

>INN doesn't.

And Henry says CNews doesn't, and I have just tried it on my own CNews and
it didn't.

So am I right in thinking that if I don't want to see cancels from all the
regular spam cancellers, I have got to instruct my upstream peers to
register 'cybercancel' as an alias for my site in their sys files?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked], [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 27 Sep 2004 13:08:07 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> A relaying agent MUST check the legality of all headers that it needs
>> to use to do its job, namely at least Path, Date, Message-ID, Control
>> if present, Injection-Date if present, and probably also Newsgroups.
>> Demoting the rest to a SHOULD seems reasonable to me.  Thinking about
>> it, I see no reason to distinguish between mandatory and optional
>> headers here, although perhaps I'm missing something.

> OK, an explicit list makes sense (leaves From and Subject out of it).

Right.  Those are the two mandatory headers that have no real impact on
relaying.  Henry's point is well-taken in that it might be worthwhile
doing some sort of cursory check just to keep from propagating trash, but
on the other hand, major servers don't do this now and Usenet hasn't
broken.  To me, that argues against a MUST and towards a SHOULD.

> I am not so sure about Control; surely that is for serving agents, not
> relaying agents (and the serving agent will probably just hand it over
> to whatever subsystem deals with control messages and let that check as
> it sees fit)?

No, see the special propagation rules for newgroup and rmgroup control
messages.  Relaying agents have to know about that.

> Also, one could omit Date if Injection-Date is present.

Yes, but beware of backward compatibility here; not everyone will use
Injection-Date by preference.  Plus, it's more complicated to say that;
it's easier to just check both of them.

> As for the rest, why not demote to MAY rather than SHOULD? Do current
> relaying agents routinely check all the lesser known headers, especially
> if they are just transit agents?

INN does, yes.  Remember that for a relaying agent, if it accepts the
message, it also passes it on to other sites.  I think that one SHOULD
avoid propagating known-broken messages.

However, I don't feel strongly enough about this to argue it at length and
will happily go along with whatever the group consensus is.

> However, I have now found another bit in draft-13 which really belongs
> in USEPRO. In 6.21.2 it said:

>    Reading agents SHOULD NOT, unless explicitly configured otherwise,
>    act automatically on Application types which could change the state
>    of that agent (e.g. by writing or modifying files), except in the
>    case of those prescribed for use in control messages (7.2.1.2 and
>    7.2.4.1).

> So that reads like a duty of a reading agent, though maybe it is a
> security consideration.

Yes, I think this is a security consideration and would rather see it
there than in a separate duties section.  This isn't a statement about
what a reading agent should do, but rather what a reading agent shouldn't
do.

I really think the Distribution thing is pointless.  Is anyone aware of a
reading agent that actually does that?  Would anyone reading this list
actually use that feature, as opposed to just reading the relevant
regional group?

> I could easily be persuaded that the second paragraph and NOTE are
> supefluous, but I would still be interested to hear whether relaying
> agents actually do it or not.

INN doesn't.

> But the 1st paragraph seems to be an important part of the protocol, and
> part of the machinery for avoiding loops.

Oh, certainly, I have no objections at all to the first paragraph and
indeed believe it should stay.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 27 Sep 2004 12:36:13 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-01
Sender: [email-masked]

On Mon, 27 Sep 2004, Charles Lindsey wrote:
> >A relaying agent MUST check the legality of all headers that it needs to
> >use to do its job, namely at least Path, Date, Message-ID, Control if
> >present, Injection-Date if present, and probably also Newsgroups...
> 
> OK, an explicit list makes sense (leaves From and Subject out of it).

In the C News days, persistent pressure eventually persuaded us that our
relaying stuff really should check for at least the presence of *all*
mandatory headers -- that it served nobody's interests for blatantly
malformed articles to propagate.  If memory serves, we did not actually do
any checking on the From content beyond verifying its presence. 

Moreover, we did not really do a syntax check on the Path content.  Date
and Message-ID had to be parsable, as did Newsgroups (although our parsing
of that was very simple and would not catch some subtle goofs).

> As for the rest, why not demote to MAY rather than SHOULD? Do current
> relaying agents routinely check all the lesser known headers, especially
> if they are just transit agents?

Certainly C News didn't.

> >>    A relaying agent MAY decline to accept an article if its own path-
> >>    identity is already present in the Path-content...
> 
> ...I would still be interested to hear whether relaying
> agents actually do it or not.

If I recall correctly, C News examined Path only on transmission, not on
reception.

> But the 1st paragraph seems to be an important part of the protocol, and
> part of the machinery for avoiding loops.

Indeed, the first paragraph remains necessary.  It's primarily an efficiency
hack rather than loop prevention -- the message ID check is what really
breaks loops -- but it's an important one.

                                                          Henry Spencer
                                                       [email-masked]

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20157
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: draft-ietf-usefor-usepro-01
Date: Mon, 27 Sep 2004 11:58:20 GMT
Lines: 99
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> 2. In 7.3 Steps 3 and 4 it said that Relaying agents MUST check the
>> legality of all mandatory headers and SHOULD check the legality of the
>> optional ones as well. I doubt that all relaying agents do such full
>> checks (surely the injecting and serving sites are the ones that should
>> check everything), and I suspect something nearer to SHOULD and MAY
>> would be better. Comments?

>A relaying agent MUST check the legality of all headers that it needs to
>use to do its job, namely at least Path, Date, Message-ID, Control if
>present, Injection-Date if present, and probably also Newsgroups.
>Demoting the rest to a SHOULD seems reasonable to me.  Thinking about it,
>I see no reason to distinguish between mandatory and optional headers
>here, although perhaps I'm missing something.

OK, an explicit list makes sense (leaves From and Subject out of it). I am
not so sure about Control; surely that is for serving agents, not relaying
agents (and the serving agent will probably just hand it over to whatever
subsystem deals with control messages and let that check as it sees fit)?
Also, one could omit Date if Injection-Date is present.

As for the rest, why not demote to MAY rather than SHOULD? Do current
relaying agents routinely check all the lesser known headers, especially
if they are just transit agents?

>> 3. I have added a section on "Duties of a Reading Agent". The prime
>> reason for this was to find a home for the bit in draft-13:6.6 that
>> "... reading agents MAY be configured so that unwanted distributions do
>> not get displayed" - this was part of our grand attempt to make the
>> Distribution-header more effective by encouraging it to be checked by
>> both senders and receivers. I have padded this out with other obvious
>> duties for the reading agent.

>I think both that comment and really this entire section should simply be
>striken in its entirety.  It seems completely unnecessary to me.

Well the bit about distributions was in all our earlier drafts, so I can't
just throw it away without some WG discussion. I could be persuaded that
it belongs in USEAGE.

However, I have now found another bit in draft-13 which really belongs in
USEPRO. In 6.21.2 it said:

   Reading agents SHOULD NOT, unless explicitly configured otherwise,
   act automatically on Application types which could change the state
   of that agent (e.g. by writing or modifying files), except in the
   case of those prescribed for use in control messages (7.2.1.2 and
   7.2.4.1).

So that reads like a duty of a reading agent, though maybe it is a
security consideration.

Anyway, I will leave that section in the draft for now until I hear more
opinions from this list.

>> 4. In draft-13:5.6.1 (the Path-header) it said:

>>    A relaying agent SHOULD NOT pass an article to another relaying agent
>>    whose path-identity (or some known alias thereof) already appears in
>>    the Path-content. ...

>>    A relaying agent MAY decline to accept an article if its own path-
>>    identity is already present in the Path-content or if the Path-
>>    content contains some path-identity whose articles the relaying agent
>>    does not want, as a matter of local policy.

>>         NOTE: This last facility is sometimes used to detect and decline
>>         control messages (notably cancel messages) which have been
>>         deliberately seeded with a path-identity to be "aliased out" by
>>         sites not wishing to act upon them.

>> The first para reflects current practice AIUI, but is the 2nd para
>> correct?

>The second paragraph and the NOTE are pointless; relaying agents can
>reject anything they want based on local policy, and there's no reason to
>single this out in particular since it doesn't do anything particularly
>useful.  I'd drop both entirely.

I could easily be persuaded that the second paragraph and NOTE are
supefluous, but I would still be interested to hear whether relaying
agents actually do it or not.

But the 1st paragraph seems to be an important part of the protocol, and
part of the machinery for avoiding loops.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: draft-ietf-usefor-usepro-01
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 26 Sep 2004 17:12:33 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> 2. In 7.3 Steps 3 and 4 it said that Relaying agents MUST check the
> legality of all mandatory headers and SHOULD check the legality of the
> optional ones as well. I doubt that all relaying agents do such full
> checks (surely the injecting and serving sites are the ones that should
> check everything), and I suspect something nearer to SHOULD and MAY
> would be better. Comments?

A relaying agent MUST check the legality of all headers that it needs to
use to do its job, namely at least Path, Date, Message-ID, Control if
present, Injection-Date if present, and probably also Newsgroups.
Demoting the rest to a SHOULD seems reasonable to me.  Thinking about it,
I see no reason to distinguish between mandatory and optional headers
here, although perhaps I'm missing something.

> 3. I have added a section on "Duties of a Reading Agent". The prime
> reason for this was to find a home for the bit in draft-13:6.6 that
> "... reading agents MAY be configured so that unwanted distributions do
> not get displayed" - this was part of our grand attempt to make the
> Distribution-header more effective by encouraging it to be checked by
> both senders and receivers. I have padded this out with other obvious
> duties for the reading agent.

I think both that comment and really this entire section should simply be
striken in its entirety.  It seems completely unnecessary to me.

> 4. In draft-13:5.6.1 (the Path-header) it said:

>    A relaying agent SHOULD NOT pass an article to another relaying agent
>    whose path-identity (or some known alias thereof) already appears in
>    the Path-content. ...

>    A relaying agent MAY decline to accept an article if its own path-
>    identity is already present in the Path-content or if the Path-
>    content contains some path-identity whose articles the relaying agent
>    does not want, as a matter of local policy.

>         NOTE: This last facility is sometimes used to detect and decline
>         control messages (notably cancel messages) which have been
>         deliberately seeded with a path-identity to be "aliased out" by
>         sites not wishing to act upon them.

> The first para reflects current practice AIUI, but is the 2nd para
> correct?

The second paragraph and the NOTE are pointless; relaying agents can
reject anything they want based on local policy, and there's no reason to
single this out in particular since it doesn't do anything particularly
useful.  I'd drop both entirely.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usepro-01.txt
Date: Tue, 21 Sep 2004 15:39:14 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: News Article Architecture and Protocols
	Author(s)	: C. Lindsey
	Filename	: draft-ietf-usefor-usepro-01.txt
	Pages		: 50
	Date		: 2004-9-21
	
This Draft, together with its companion draft [USEFOR], are
   intended as standards track documents, together obsoleting RFC
   1036, which itself dates from 1987.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-usefor-usepro-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usepro-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usepro-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20155
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: draft-ietf-usefor-usepro-01
Date: Tue, 21 Sep 2004 11:33:26 GMT
Lines: 761
Sender: [email-masked]

There was some discussion a while back as to whether various bits or
protocol material specific to particular headers should remain with those
headers after the Usefor/Usepro split, or be extracted and placed in
Usepro. Our Chair has now asked that they be moved to Usepro, and this
draft implements that change.

The draft is now (or soon will be) on
www.landfield.com/usenet-format/drafts/draft-ietf-usefor-usepro-01.txt
www.landfield.com/usenet-format/drafts/draft-ietf-usefor-usepro-01.unpaged
and is also on its way to the drafts editor.

It makes no changes wrt the earlier drafts apart from the items listed
below, though the wording is revised to suit the new environment. Full
diffs are at the end of this message.

The (possible) changes are as follows:

1. In 7.3, I have demoted the MUST in the first para of draft-13:5.5 which
seemed a bit limiting on those relaying agents which tend to fire off
articles whether on not the receiving site wanted them. Surely SHOULD is
sufficient, but I could be persuaded I have gone too far.

2. In 7.3 Steps 3 and 4 it said that Relaying agents MUST check the
legality of all mandatory headers and SHOULD check the legality of the
optional ones as well. I doubt that all relaying agents do such full
checks (surely the injecting and serving sites are the ones that should
check everything), and I suspect something nearer to SHOULD and MAY would
be better. Comments?

3. I have added a section on "Duties of a Reading Agent". The prime reason
for this was to find a home for the bit in draft-13:6.6 that "... reading
agents MAY be configured so that unwanted distributions do not get
displayed" - this was part of our grand attempt to make the
Distribution-header more effective by encouraging it to be checked by both
senders and receivers. I have padded this out with other obvious duties
for the reading agent.

4. In draft-13:5.6.1 (the Path-header) it said:

   A relaying agent SHOULD NOT pass an article to another relaying agent
   whose path-identity (or some known alias thereof) already appears in
   the Path-content. ...

   A relaying agent MAY decline to accept an article if its own path-
   identity is already present in the Path-content or if the Path-
   content contains some path-identity whose articles the relaying agent
   does not want, as a matter of local policy.

        NOTE: This last facility is sometimes used to detect and decline
        control messages (notably cancel messages) which have been
        deliberately seeded with a path-identity to be "aliased out" by
        sites not wishing to act upon them.

The first para reflects current practice AIUI, but is the 2nd para
correct? Surely it would be the serving agent that might do that? For
example, if someone "aliases out" some site (e.g. the $alz or cybercancel
conventions used by spam cancellers, is it the final serving agent that
notices that the pseudo-site 'cybercancel' is present (supposing it does
not want to honour those cancels), or is it the final relaying site that
does not send the article to it? I suspect the former, and have moved the
2nd para and the NOTE to serving agents (with copious pseudo comments to
draw attention to the matter). Please advise me what the correct current
practice is.

So finally, here are the diffs:

*** draft-ietf-usefor-usepro-00.unpaged	Thu Sep  2 18:26:27 2004
--- draft-ietf-usefor-usepro-01.unpaged	Mon Sep 20 17:18:31 2004
***************
*** 1,10 ****
  
  INTERNET-DRAFT                               Charles H. Lindsey
  Usenet Format Working Group                  University of Manchester
!                                              August 2004
  
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-00.txt>
  
  Status of this Memo
  
--- 1,10 ----
  
  INTERNET-DRAFT                               Charles H. Lindsey
  Usenet Format Working Group                  University of Manchester
!                                              September 2004
  
                  News Article Architecture and Protocols
!                    <draft-ietf-usefor-usepro-01.txt>
  
  Status of this Memo
  
***************
*** 30,36 ****
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.
  
!    This Internet-Draft will expire in February 2005.
  
  Abstract
  
--- 30,36 ----
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.
  
!    This Internet-Draft will expire in March 2005.
  
  Abstract
  
***************
*** 112,123 ****
    7.2.  Duties of an Injecting Agent ..............................    0
      7.2.1.  Proto-articles ........................................    0
      7.2.2.  Procedure to be followed by Injecting Agents ..........    0
!   7.3.  Procedure for Forwarding to a Moderator ...................    0
!   7.4.  Duties of a Relaying Agent ................................    0
!     7.4.1.  Example ...............................................    0
!   7.5.  Duties of a Serving Agent .................................    0
!   7.6.  Duties of a Posting Agent .................................    0
!   7.7.  Duties of a Followup Agent ................................    0
    7.8.  Duties of a Moderator .....................................    0
    7.9.  Duties of a Gateway .......................................    0
      7.9.1.  Duties of an Outgoing Gateway .........................    0
--- 112,124 ----
    7.2.  Duties of an Injecting Agent ..............................    0
      7.2.1.  Proto-articles ........................................    0
      7.2.2.  Procedure to be followed by Injecting Agents ..........    0
!     7.2.3.  Procedure for Forwarding to a Moderator ...............    0
!   7.3.  Duties of a Relaying Agent ................................    0
!     7.3.1.  Path-Header Example ...................................    0
!   7.4.  Duties of a Serving Agent .................................    0
!   7.5.  Duties of a Posting Agent .................................    0
!   7.6.  Duties of a Followup Agent ................................    0
!   7.7.  Duties of a Reading Agent .................................    0
    7.8.  Duties of a Moderator .....................................    0
    7.9.  Duties of a Gateway .......................................    0
      7.9.1.  Duties of an Outgoing Gateway .........................    0
***************
*** 1301,1308 ****
     matter of policy to be determined by the administrators of each
     injecting agent, who have the responsibility to ensure that no harm
     arises. In all other circumstances, unintented reinjection is to be
!    avoided (see 7.8).  Nevertheless, in order to preserve the integrity
!    of the network in these special cases, this standard sets out the
     correct way to reinject.
  
     It is usual for an injecting agent to be closely associated with a
--- 1301,1308 ----
     matter of policy to be determined by the administrators of each
     injecting agent, who have the responsibility to ensure that no harm
     arises. In all other circumstances, unintented reinjection is to be
!    avoided (see 7.9).  Nevertheless, in order to preserve the integrity
!    of the network in these special cases, this standard does set out the
     correct way to reinject.
  
     It is usual for an injecting agent to be closely associated with a
***************
*** 1372,1379 ****
        the mandatory headers other than Injection-Date), or which
        contains any header that does not have legal contents.  It SHOULD
        reject any article which contains any header deprecated for
!       Netnews (a-4.2.1).
  
     5. If the article is rejected (for reasons given above, or for other
        formatting errors or matters of site policy) the posting agent
        SHOULD be informed (such as via an NNTP 44x response code) that
--- 1372,1391 ----
        the mandatory headers other than Injection-Date), or which
        contains any header that does not have legal contents.  It SHOULD
        reject any article which contains any header deprecated for
!       Netnews (a-4.2.1).  It SHOULD reject any article whose
!       Newsgroups-header does not contain at least one newsgroup-name for
!       an existing group (as listed by its associated serving agent) and
!       it MAY reject any newsgroup-name which, although syntactically
!       correct, violates a policy restriction established, for some
!       (sub-)hierarchy, by an agency with the appropriate authority
!       (1.2).  Observe that crossposting to unknown newsgroups is not
!       precluded provided at least one of those in the Newsgroups-header
!       is listed.
  
+         NOTE: This ability to reject newsgroup-names in breach of
+         established policy does not extend to relaying agents, though it
+         might be reasonable for posting agents to do it.
+ 
     5. If the article is rejected (for reasons given above, or for other
        formatting errors or matters of site policy) the posting agent
        SHOULD be informed (such as via an NNTP 44x response code) that
***************
*** 1382,1395 ****
  
     6. The Message-ID, Date and From-headers (and their contents) MUST be
        added when not already present (but that situation could not arise
!       during reinjection).
  
     7. The injecting agent MUST NOT alter the body of the article in any
!       way. It MAY, except when reinjecting, add other headers not
!       already provided by the poster, but SHOULD NOT alter, delete, or
!       reorder any existing header, with the specific exception of
!       "tracing" headers such as Injection-Info and Complaints-To, which
!       are to be removed as already mentioned.
  
     8. If the Newsgroups-header contains one or more moderated groups and
        the article does NOT contain an Approved-header, the injecting
--- 1394,1414 ----
  
     6. The Message-ID, Date and From-headers (and their contents) MUST be
        added when not already present (but that situation could not arise
!       during reinjection).  A User-Agent-header MAY be added (or an
!       already present User-Agent-header MAY be augmented) so as to
!       identify the software (e.g. "INN/1.7.2") used by the injecting
!       agent.
  
     7. The injecting agent MUST NOT alter the body of the article in any
!       way (including any change of Content-Transfer-Encoding). It MAY,
!       except when reinjecting, add other headers not already provided by
!       the poster, but SHOULD NOT alter, delete, or reorder any existing
!       header, with the specific exception of "tracing" headers such as
!       Injection-Info and Complaints-To, which are to be removed as
!       already mentioned.  It MAY also, as an interim measure pending
!       widespread adoption of the newly introduced (a-5.5) folding
!       whitespace, reformat the Newsgroups- and any Followup-To-header by
!       removing any such whitespace inserted by the posting agent.
  
     8. If the Newsgroups-header contains one or more moderated groups and
        the article does NOT contain an Approved-header, the injecting
***************
*** 1403,1431 ****
     10.It MUST then prepend the path-identity of the injecting agent and
        a '%' path-delimiter (which serves to separate the pre-injection
        and post-injection regions of the Path-content) to the Path-
!       content; moreover, that path-identity MUST be an FQDN mailable
!       address. This could result in more that one '%' path-delimiter in
        the case of reinjection. See a-5.6.4 for the significance of the
        various path-delimiters.
  
     11.An Injection-Info-header (a-6.19) SHOULD be added, identifying the
        trusted source of the article, and a suitable Complaints-To-header
!       (a-6.20) MAY be added.
  
!    12.The injecting agent MUST then add an An Injection-Date-header (a-
!       5.7) if one is not already present, but it MUST NOT alter, or
!       remove, an already present Injection-Date-header (and likewise
!       SHOULD NOT alter, or remove, an already present NNTP-Posting-
!       Date-header).  Finally, it forwards the article to one or more
!       relaying or serving agents, and the injection process is to be
!       considered complete.
  
! 7.3.  Procedure for Forwarding to a Moderator
  
     An injecting agent forwards ar article to a moderator as follows:
  
     1. It MUST forward it to the moderator of the first (leftmost)
!       moderated group listed in the Newsgroups-header via email (see 7.7
        for how that moderator may forward it to further moderators).
        There are two possibilities for doing this:
  
--- 1423,1467 ----
     10.It MUST then prepend the path-identity of the injecting agent and
        a '%' path-delimiter (which serves to separate the pre-injection
        and post-injection regions of the Path-content) to the Path-
!       content; this SHOULD then be followed by CRLF and WSP if it would
!       otherwise result in a line longer than 79 characters.  The
!       prepended path-identity MUST be an FQDN mailable address (a-
!       5.6.2).  This could result in more that one '%' path-delimiter in
        the case of reinjection. See a-5.6.4 for the significance of the
        various path-delimiters.
  
     11.An Injection-Info-header (a-6.19) SHOULD be added, identifying the
        trusted source of the article, and a suitable Complaints-To-header
!       (a-6.20) MAY be added.  Each injecting agent SHOULD use a
!       consistent form of the Injection-Info-header for all articles
!       emanating from the same or similar origins.
  
!         NOTE: The step above is the only place in which an Injection-
!         Info- or Complaints-To-header is to be created. It follows that
!         these headers MUST NOT be created, replaced, changed or deleted
!         by any other agent (except during reinjection, in which case
!         they will always relate to the latest injection and can, to that
!         extent, be regarded as variant headers).
  
!    12.The injecting agent MUST then add an Injection-Date-header (a-5.7)
!       if one is not already present, but it MUST NOT alter, or delete,
!       an already present Injection-Date-header (and likewise SHOULD NOT
!       alter, or delete, an already present NNTP-Posting-Date-header).
!       Finally, it forwards the article to one or more relaying or
!       serving agents, and the injection process is to be considered
!       complete.
  
+         NOTE: The step above is the only place where an Injection-Date-
+         header is to be created It follows that it MUST NOT subsequently
+         be replaced, changed or deleted by any other agent, even during
+         reinjection.
+ 
+ 7.2.3.  Procedure for Forwarding to a Moderator
+ 
     An injecting agent forwards ar article to a moderator as follows:
  
     1. It MUST forward it to the moderator of the first (leftmost)
!       moderated group listed in the Newsgroups-header via email (see 7.8
        for how that moderator may forward it to further moderators).
        There are two possibilities for doing this:
  
***************
*** 1440,1446 ****
  
        (b)  The article is sent as an email as it stands, with the
             addition of such extra headers (e.g. a To-header) as are
!            necessary for an email.
  
        Although both of these methods have seen use in the past, the
        preponderance of current usage on Usenet has been for method (b)
--- 1476,1483 ----
  
        (b)  The article is sent as an email as it stands, with the
             addition of such extra headers (e.g. a To-header) as are
!            necessary for an email. The existing Message-ID-header SHOULD
!            be retained.
  
        Although both of these methods have seen use in the past, the
        preponderance of current usage on Usenet has been for method (b)
***************
*** 1462,1468 ****
             "news.announce.important" would be emailed to "news-
             [email-masked]".
  
! 7.4.  Duties of a Relaying Agent
  
     A Relaying Agent accepts injected articles from injecting and other
     relaying agents and passes them on to relaying or serving agents
--- 1499,1505 ----
             "news.announce.important" would be emailed to "news-
             [email-masked]".
  
! 7.3.  Duties of a Relaying Agent
  
     A Relaying Agent accepts injected articles from injecting and other
     relaying agents and passes them on to relaying or serving agents
***************
*** 1469,1487 ****
     according to mutually agreed policy. Relaying agents SHOULD accept
     articles ONLY from trusted agents.
  
     A relaying agent processes articles as follows:
  
     1. It MUST establish the trusted identity of the source of the
        article and compare it with the leftmost path-identity of the
        Path-content. If it matches it MUST then prepend its own path-
!       identity and a '/' path-delimiter to the Path-header.  If it does
!       not match then it prepends instead two entries to the Path-
!       content; firstly the true established path-identity of the source
!       followed by a '?'  path-delimiter, and then, to the left of that,
!       its own path-identity followed by a '/' path-delimiter as usual.
!       This prepending of two entries SHOULD NOT be done if the provided
!       and established identities match.  See a-5.6.4 for the
!       significance of the various path-delimiters.
  
          NOTE: In order to prevent overloading, relaying agents should
          not routinely query an external entity (such as a DNS-server) in
--- 1506,1564 ----
     according to mutually agreed policy. Relaying agents SHOULD accept
     articles ONLY from trusted agents.
  
+    An article SHOULD NOT be relayed unless the sending agent has been
+    configured to supply and the receiving agent to receive at least one
+    of the newsgroup-names in its Newsgroups-header and at least one of
+    the distributions in its Distribution-header, if any.  Exceptionally,
+    ALL relaying agents are deemed willing to supply or accept the
+    distribution "world", and NO relaying agent should supply or accept
+    the distribution "local".
+ [That SHOULD has been demoted from a MUST in draft-13. Any objections?]
+ 
+         NOTE: Although it would seem redundant to filter out unwanted
+         distributions at both ends of a relaying link (and it is clearly
+         more efficient to do so at the sending end), many sending sites
+         have been reluctant, historically speaking, to apply such
+         filters (except to ensure that distributions local to their own
+         site or cooperating subnet did not escape); moreover they tended
+         to configure their filters on an "all but those listed" basis,
+         so that new and hitherto unheard of distributions would not be
+         caught. Indeed many "hub" sites actually wanted to receive all
+         possible distributions so that they could feed on to their
+         clients in all possible geographical (or organizational)
+         regions.
+ 
+         Therefore, it is desirable to provide facilities for rejecting
+         unwanted distributions at the receiving end. Indeed, it may be
+         simpler to do so locally than to inform each sending site of
+         what is required, especially in the case of specialized
+         distributions (for example for control messages, such as cancels
+         from certain issuers) which might need to be added at short
+         notice.  The possibility for reading agents to filter
+         distributions has been provided (7.7) for the same reason.
+ 
+    An article SHOULD NOT be relayed if the path-identity of the
+    receiving agent (or some known alias thereof) appears in its Path-
+    header, and the receiving agent MAY detect whether its own path-
+    identity is already present in the Path-content so as to avoid
+    further unnecessary relaying.
+ [See related remarks under serving agents.]
+ 
     A relaying agent processes articles as follows:
  
     1. It MUST establish the trusted identity of the source of the
        article and compare it with the leftmost path-identity of the
        Path-content. If it matches it MUST then prepend its own path-
!       identity and a '/' path-delimiter to the Path-content; this SHOULD
!       then be followed by CRLF and WSP if it would otherwise result in a
!       line longer than 79 characters.  If it does not match then it
!       prepends instead two entries to the Path-content; firstly the true
!       established path-identity of the source followed by a '?'  path-
!       delimiter, and then, to the left of that, its own path-identity
!       followed by a '/' path-delimiter as usual. This prepending of two
!       entries SHOULD NOT be done if the provided and established
!       identities match.  See a-5.6.4 for the significance of the various
!       path-delimiters.
  
          NOTE: In order to prevent overloading, relaying agents should
          not routinely query an external entity (such as a DNS-server) in
***************
*** 1499,1504 ****
--- 1576,1584 ----
  
     4. It SHOULD reject any article whose optional headers (section a-6)
        do not have legal contents.
+ [Is that too strong? Are relaying agents really expected to check
+ headers in that detail? I suggest s/SHOULD/MAY/. Even the MUST in Step 4
+ for mandatory headers might be demoted to SHOULD.]
  
     5. It SHOULD reject any article that has already been sent to it (a
        database of message identifiers of recent messages is usually kept
***************
*** 1508,1526 ****
        cancel message (or an equivalent Supersedes-header) issued by its
        poster or by some other trusted entity.
  
     7. It MAY reject any article without an Approved-header posted to
        newsgroups known to be moderated (this practice is strongly
        recommended, but the information necessary to do so may not be
        available to all agents).
  
!    8. Finally, it passes articles which match mutually agreed criteria
!       on to neighbouring relaying and serving agents. However, it SHOULD
!       NOT forward articles to sites whose path-identity is already in
!       the Path-header.
  
!         NOTE: It is usual for relaying and serving agents to restrict
!         the Newsgroups, Distributions, age and size of articles that
!         they wish to receive.
  
     If the article is rejected as being invalid, unwanted or unacceptable
     due to site policy, the agent that passed the article to the relaying
--- 1588,1604 ----
        cancel message (or an equivalent Supersedes-header) issued by its
        poster or by some other trusted entity.
  
     7. It MAY reject any article without an Approved-header posted to
        newsgroups known to be moderated (this practice is strongly
        recommended, but the information necessary to do so may not be
        available to all agents).
  
!    8. It MAY delete any Xref-header that is present.
  
!    9. Finally, it passes the articles on to neighbouring relaying and
!       serving agents.
  
     If the article is rejected as being invalid, unwanted or unacceptable
     due to site policy, the agent that passed the article to the relaying
***************
*** 1530,1542 ****
     external entity that an article was not relayed UNLESS that external
     entity has explicitly requested that it be informed of such errors.
  
     Relaying agents MUST NOT alter, delete or rearrange any part of an
!    article expect for headers designated as variant (a-4.2.5.3).
  
! 7.4.1.  Example
  
        Path: foo.isp.example/
           foo-server/bar.isp.example?10.123.12.2/old.site.example!
           barbaz/baz.isp.example%dialup123.baz.isp.example!not-for-mail
--- 1608,1633 ----
     external entity that an article was not relayed UNLESS that external
     entity has explicitly requested that it be informed of such errors.
  
     Relaying agents MUST NOT alter, delete or rearrange any part of an
!    article expect for headers designated as variant (a-4.2.5.3).  In
!    particular
  
!      o they MUST NOT create or augment a User-Agent-header in order to
!        identify themselves;
!      o they MUST NOT rewrite the Newsgroups-header in any way, even if
!        some supposedly non-existent newsgroup is included;
!      o they MUST NOT refold any header (i.e. they must pass on the
!        folding as received), even to remove FWS from a Newsgroups-
!        header;
!      o they MUST NOT alter the Date-header or the Injection-Date-header;
!      o they MUST NOT delete any unrecognized header whose header-name is
!        syntactically correct (whether or not it is registered with IANA
!        [RFC 3864]);
!      o they MUST NOT change the Content-Transfer-Encoding of the body or
!        any body part.
  
+ 7.3.1.  Path-Header Example
+ 
        Path: foo.isp.example/
           foo-server/bar.isp.example?10.123.12.2/old.site.example!
           barbaz/baz.isp.example%dialup123.baz.isp.example!not-for-mail
***************
*** 1576,1582 ****
          fold the line, on the grounds that it seemed to be getting a
          little too long.
  
! 7.5.  Duties of a Serving Agent
  
     A Serving Agent takes an article from a relaying or injecting agent
     and files it in a "news database". It also provides an interface for
--- 1667,1673 ----
          fold the line, on the grounds that it seemed to be getting a
          little too long.
  
! 7.4.  Duties of a Serving Agent
  
     A Serving Agent takes an article from a relaying or injecting agent
     and files it in a "news database". It also provides an interface for
***************
*** 1585,1594 ****
     article-locator (usually in the form of a decimal number - see a-
     6.16).
  
!    A serving agent MUST maintain a list showing the moderation status
!    (see 6.2.1) of the newsgroups it stores in the news database, and
!    SHOULD include in that list all groups likely to be crossposted to
!    from those groups (e.g. all other groups in the same hierarchy(ies)).
  
          NOTE: Since control messages are often of interest, but should
          not be displayed as normal articles in regular newsgroups, it is
--- 1676,1686 ----
     article-locator (usually in the form of a decimal number - see a-
     6.16).
  
!    A serving agent MUST maintain a list of the newsgroups it stores in
!    its news database showing the moderation status of each one (see
!    6.2.1), and SHOULD include in that list all groups likely to be
!    crossposted to from those groups (e.g. all other groups in the same
!    hierarchy(ies)).
  
          NOTE: Since control messages are often of interest, but should
          not be displayed as normal articles in regular newsgroups, it is
***************
*** 1596,1604 ****
          newsgroup named "control" or in a pseudo-newsgroup in a sub-
          hierarchy under "control." (e.g. "control.cancel").
  
     A serving agent processes articles as follows:
  
!    1.  It MUST establish the trusted identity of the source of the
        article and modify the Path-header as for a relaying agent (7.3).
  
     2. It MUST examine the Injection-Date-header (or, if that is absent,
--- 1688,1715 ----
          newsgroup named "control" or in a pseudo-newsgroup in a sub-
          hierarchy under "control." (e.g. "control.cancel").
  
+    A serving agent MAY decline to accept an article if its own path-
+    identity is already present in the Path-content or if the Path-
+    content contains some path-identity whose articles the serving agent
+    does not want, as a matter of local policy.
+ [That has been changed from a-5.6.1 in previous drafts, where it said "A
+ relaying agent MAY decline...". That seemed plain wrong to me. It is
+ fine for a relaying agent to ignore articles which have apparently
+ already passed through it (and I still say that), but surely it is for
+ serving agents to "reject" for policy reasons, and to which the NOTE
+ below would apply.  Comments?]
+ 
+         NOTE: This last facility is sometimes used to detect and decline
+         control messages (notably cancel messages) which have been
+         deliberately seeded with a path-identity to be "aliased out" by
+         sites not wishing to act upon them.
+ [Again, is this "aliasing out" usually detected by the serving agent, or
+ does it more usually work because the previous relaying agent will never
+ have sent it in the first place?]
+ 
     A serving agent processes articles as follows:
  
!    1. It MUST establish the trusted identity of the source of the
        article and modify the Path-header as for a relaying agent (7.3).
  
     2. It MUST examine the Injection-Date-header (or, if that is absent,
***************
*** 1612,1618 ****
        header that does not have legal contents.
  
     4. It SHOULD reject any article that has already been sent to it (a
!       database of message identifiers of recent messages is usually kept
        and matched against).
  
     5. It SHOULD reject any article that matches an already received
--- 1723,1729 ----
        header that does not have legal contents.
  
     4. It SHOULD reject any article that has already been sent to it (a
!       database of message identifiers of recent articles is usually kept
        and matched against).
  
     5. It SHOULD reject any article that matches an already received
***************
*** 1627,1634 ****
  
     8. Finally, it stores the article in its news database.
  
! 7.6.  Duties of a Posting Agent
  
     A Posting Agent is used to assist the poster in creating a valid
     proto-article and forwarding it to an injecting agent.
  
--- 1738,1753 ----
  
     8. Finally, it stores the article in its news database.
  
!    Serving agents MUST NOT create new newsgroups simply because an
!    unrecognized newsgroup-name occurs in a Newsgroups-header (see a-
!    7.2.1 for the correct method of newsgroup creation).
  
+    Serving agents MUST NOT alter, delete or rearrange any part of an
+    article in any other way. The list of particular cases given for
+    relaying agents (7.3) applies here also.
+ 
+ 7.5.  Duties of a Posting Agent
+ 
     A Posting Agent is used to assist the poster in creating a valid
     proto-article and forwarding it to an injecting agent.
  
***************
*** 1641,1648 ****
     attempt to post an article which cancels or Supersedes another
     article of which the poster is not the author.
  
- 7.7.  Duties of a Followup Agent
  
     A Followup Agent is a special case of a posting agent, and as such is
     bound by all the posting agent's requirements. Followup agents MUST
     create valid followups and are subject to special requirements
--- 1760,1771 ----
     attempt to post an article which cancels or Supersedes another
     article of which the poster is not the author.

+ 7.6.  Duties of a Followup Agent
+ 
     A Followup Agent is a special case of a posting agent, and as such is
     bound by all the posting agent's requirements. Followup agents MUST
     create valid followups and are subject to special requirements
***************
*** 1709,1714 ****
--- 1832,1856 ----
        Followup agents SHOULD NOT attempt to send email to any address
        ending in ".invalid".
  
+ 7.7.  Duties of a Reading Agent
+ 
+    A reading agent downloads articles from a serving agent, as directed
+    by the reader, and displays them (or processes them in some other
+    manner).  The article as displayed MUST be identical to the article
+    as originally posted, subject only to limitations of the display
+    device (such as availability of charsets, etc.). It MUST provide
+    facilities for decoding any Content-Transfer-Encodings, encoded-
+    words, etc., but SHOULD also have the capability to show the article
+    exactly as received.
+ 
+    It MAY present lists of articles available for display, and MAY
+    structure those lists so as to show the relationships between the
+    articles, as determined by the References-, Subject-, Date- and
+    other-headers (see [USEAGE] for some usual methods of doing this). It
+    MAY also be configured so that unwanted distributions (a-6.6) are
+    ignored.
+ 
+ 
  7.8.  Duties of a Moderator
  
     A Moderator receives news articles by email, decides whether to
***************
*** 1898,1903 ****
--- 2039,2049 ----
     4. Netnews control messages should not be gated to another medium
        unless they would somehow be meaningful in that medium.
  
+    5. Changes MAY be made to the Content-Transfer-Encoding of some or
+       all parts of the body, and even to the charsets specified in
+       encoded-words or in Content-Type-headers, but such changes SHOULD
+       NOT be made unless absolutely necessary.
+ 
  7.9.2.  Duties of an Incoming Gateway
  
     The incoming gateway has the serious responsibility of ensuring that
***************
*** 2297,2302 ****
--- 2447,2455 ----
     [RFC 2822] P. Resnick, "Internet Message Format", RFC 2822, April
          2001.
  
+    [RFC 3864] G. Klyne, M. Nottingham, and J. Mogul, "Registration
+         procedures for message header fields", RFC 3864.
+ 
     [RFC 850] Mark R. Horton, "Standard for interchange of Usenet
          messages", RFC 850, June 1983.
  
***************
*** 2341,2348 ****
     Comments on this draft should preferably be sent to the mailing list
     of the Usenet Format Working Group at
  
-         [email-masked]
-         shortly to be replaced by
          [email-masked].
  
  Appendix A.1 - A-News Article Format
--- 2494,2499 ----
***************
*** 2494,2496 ****
--- 2644,2673 ----
  Appendix C - Change Log
  
  [This Appendix is to be removed prior to final publication.]
+ 
+    For version 01
+ 
+    1    Numerous texts describing protocol features related to
+         particular headers in parts of [ARTICLE] which were destined to
+         become part of [USEFOR] have been moved to appropriate locations
+         within section 7 of this document. Such revised texts will be
+         found in sections
+         7.2.2 Steps 4, 6, 7, 10, 11, 12;
+         7.2.3 Step 1(b);
+         7.3 introductory paragraphs, Steps 1, 4, 8, 9, and some final
+         paragraphs;
+         7.4 introductory and final paragraphs;
+         7.9.1 Step 5.
+ 
+    2    A section on "Duties of a Reading Agent" (7.8) has been added.
+ 
+    3    Some demotions MUST -> SHOULD -> MAY, as noted in pseudo-
+         comments, have been made or proposed in sections
+         7.3
+         7.3 Step 4.
+ 
+    4    Part of the procedure for examining Path-headers by relaying
+         agents has been moved to serving agents, as explained in
+         pseudo-comments in section 7.4.
+ 
+    5    Some renumbering of sections and minor textual clarifications.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20154
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: The great mailing list muddle
Date: Fri, 17 Sep 2004 13:34:47 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> Bill Davidsen <[email-masked]> writes:

>Charles Lindsey wrote:
>> 
>That probably explains why your posts are the only ones I am getting any 
>more. I tried again to subscribe, but it told me I was already 
>subscribed. If that's true you are the only one posting, which I doubt.

Paul Hoffman tells me that anyone still having problems may contact him at
Paul Hoffman / IMC <[email-masked]>. It seems that he has a lot of spam
to wade through and sometimes he misses some messages relating to
management of his lists.

Is anyone other than Bill still having problems? If so, please report here
(on either list, since I still read and post to both).

I hope shortly to be transferring all the documents and old list archives
to the new site, since Paul has given me ftp access.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 17 Sep 2004 01:27:58 -0400 (EDT)
From: Henry Spencer <[email-masked]>
To: Usefor Mailing List <[email-masked]>
cc: Usefor Mailing List <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-01.txt
Sender: [email-masked]

On Thu, 16 Sep 2004, Frank Ellermann wrote:
> | When an article is posted to more than one newsgroup, it is
> | said to be "crossposted"; note that this differs from posting
> | the same text as part of each of several articles, one per 
> | newsgroup.
> 
> Should we say [...] "articles with different Message-Ids" here,
> instead of "one per newsgroup" ?

I think "per newsgroup" is the important issue here and shouldn't be
omitted.

                                                          Henry Spencer
                                                       [email-masked]

Date: Thu, 16 Sep 2004 09:36:47 -0400
From: Bill Davidsen <[email-masked]>
To: Charles Lindsey <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: The great mailing list muddle
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Bill Davidsen <[email-masked]> writes:
> 
> 
>>Thanks for the heads-up, it explains why I no longer get any mail for 
>>this list. I've sent the subscribe several times, and even gotten ack 
>>the first time IIRC, but clearly the new list is more bozo'd than the 
>>old. Pity it couldn't have been moved to somewhere competent, like 
>>lists.debian.org or such.
> 
> 
>>Nice to know this has become a closed list, rather than dead.
> 
> 
> Well I am now cleanly on the new list, but it took a direct appeal to Paul
> Hoffman to fix it. He apoloigised that my previous request had "somehow
> got lost" and subscribed me by hand.
> 
> I think we had better continue to post articles to both lists for a while
> longer.
> 
That probably explains why your posts are the only ones I am getting any 
more. I tried again to subscribe, but it told me I was already 
subscribed. If that's true you are the only one posting, which I doubt.

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-usefor-usefor-01.txt
Date: Thu, 16 Sep 2004 08:24:51 +0200
Organization: <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

[email-masked] wrote:
 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-01.txt

| When an article is posted to more than one newsgroup, it is
| said to be "crossposted"; note that this differs from posting
| the same text as part of each of several articles, one per 
| newsgroup.

Should we say [...] "articles with different Message-Ids" here,
instead of "one per newsgroup" ?

        Bye, Frank (testing the write access via gmane)

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20150
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: The great mailing list muddle
Date: Wed, 15 Sep 2004 14:24:42 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Bill Davidsen <[email-masked]> writes:

>Thanks for the heads-up, it explains why I no longer get any mail for 
>this list. I've sent the subscribe several times, and even gotten ack 
>the first time IIRC, but clearly the new list is more bozo'd than the 
>old. Pity it couldn't have been moved to somewhere competent, like 
>lists.debian.org or such.

>Nice to know this has become a closed list, rather than dead.

Well I am now cleanly on the new list, but it took a direct appeal to Paul
Hoffman to fix it. He apoloigised that my previous request had "somehow
got lost" and subscribed me by hand.

I think we had better continue to post articles to both lists for a while
longer.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usefor-01.txt
Date: Wed, 15 Sep 2004 15:58:24 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: News Article Format
	Author(s)	: C. Lindsey
	Filename	: draft-ietf-usefor-usefor-01.txt
	Pages		: 24
	Date		: 2004-9-15
	
This document specifies the syntax of network news articles in the
   context of the "Internet Message Format" (RFC 2822) and "Multipurpose
   Internet Mail Extensions (MIME)" (RFC 2045).  This document
   supersedes RFC 1036, updating it to reflect current practice and
   incorporating incremental changes specified in other documents.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-usefor-usefor-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usefor-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usefor-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usefor-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20146
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
Date: Tue, 14 Sep 2004 13:41:32 GMT
Lines: 75
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Bruce Lilly <[email-masked]> writes:

>> Briefly (this was discussed on ietf-822 in June 2003) the DSN RFC
>> introduced header fields which need to be treated similarly to
>> Message-ID w.r.t. first part message header and enclosed message header
>> for fragmentation and reassembly.  Several of the news-specific fields
>> will also need to be treated differently than what is the case per RFC
>> 2046 rules in lieu of any amendments to those rules.  To take one
>> particular example, the Control field par RFC 2046 rules would have to
>> be copied to the first part message header in order to survive
>> fragmentation/reassembly (only specific fields in the enclosed message
>> header from the original message header which is encapsulated in the
>> first part are copied during reassembly).  That means that that first
>> part will be interpreted as if it were a (complete) control message.  If
>> RFC 2046 rules were to be amended (as they were for DSN-specific fields
>> by RFCs 2298/3798) then the Control field could remain encapsulated
>> within the first part, not copied to the first part message header, the
>> partial messages would not be treated as control messages (only the
>> reassembled whole would be treated as such).

>Ah, I see.  Yes, that makes sense.  I don't personally find work on
>message/partial important enough to spend time on it, but if you come up
>with language to deal with these issues, I'd be willing to review it and
>say whether I think it makes sense.

Well it would be stupid to split a control message using message/partial,
but in the event that it did happen, it would be better to have the
Control-header in the first part (I don't care whether it is also included
in the encapsulated article, though it would be illogical not to do so).

The point is that most agents receiving the message, and particularly
serving agents which are the ones that would implement any control action,
are most unlikely to possess the capability of reassembling the various
parts, or even to have any concept of message/partial at all. Therefore it
is essential that they see the first part as an ordinary control message
and act upon it accordingly. The subsequent parts would then appear in the
relevant groups as ordinary articles, which might be confusing but would
not be likely to do any harm.

In the event that some serving agent did have reassembly capability, then
either

1) It would notice the Control-header first and act upon it. It should
then be smart enough not to bother with reassembly or, if it did
reassemble, to know that it had already done the Control stuff. But even
if it tried to act upon the Control message twice, that would not actually
cause any harm. Or:

2) It would notice the Content-Type: message partial first, in which case
it would then reassemble the message and insert it back into the system,
in which case it would be acted upon as a control message.

>Splitting text messages that small is obnoxious and anti-social and is
>something that I'd filter out on my news server.  General consensus in
>news.software.nntp among news administrators (this came up as well) pretty
>much matched that feeling.  Nothing smaller than 1MB should be split on
>Usenet, and preferrably as far as I'm concerned nothing smaller than 10MB
>(although 1MB is the more common metric).

And indeed there is a NOTE in draft-13 which makes that point. I don't
think it has found its way into the new drafts yet, but it should.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: Message fragmentation via message/partial and news-specific header fields
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Tue, 14 Sep 2004 11:03:29 -0700
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> This message, posted to the old list, is now reposted to the new list
> for the benefit of anyone no longer reading the old one.

This is the first time I've seen this message, so either the old list is
no longer working or I've been silently unsubscribed from it for some
reason.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 14 Sep 2004 14:22:32 +0100 (BST)
From: Charles Lindsey <[email-masked]>
To: [email-masked]
Subject: Message fragmentation via message/partial and news-specific header fields
Sender: [email-masked]

This message, posted to the old list, is now reposted to the new list for
the benefit of anyone no longer reading the old one.

>From:  "Charles Lindsey" <[email-masked]>
>Subject: Re: Message fragmentation via message/partial and news-specific header fields
>Message-ID: <[email-masked]>
>X-Newsreader: NN version 6.5.2 (NOV)
>References: <[email-masked]>
>Date: Thu, 2 Sep 2004 16:05:18 GMT

In <[email-masked]> Bruce Lilly <[email-masked]> writes:

>We had had some discussion about message fragmentation via the MIME media type
>message/partial,, and there was some text in earlier drafts, however the
>matter is conspicuously absent from the recently-produced usefor-usepro-00
>draft.

I think it is a matter for usefor rather than usepro, and yes it needs
some mention there.

>First there are some questions that need to be considered:
>1. a) Where is message fragmentation expected to occur (e.g. during transport
>   between transfer agents when it is recognized that the receiving agent
>   has some message size limit), and b) by what criteria (presumably some
>   site-dependent size limit, but how is that communicated to that site's
>   feeding agents)?

At the injecting agent, at the latest.

>2. Where does reassembly take place?

At each recipients reading agent, or later (i.e. he manually feeds it into
a reassembly program, since many reading agents are unlikely to provide the
facility).

>3. How is multiple fragmentation to be handled? [e.g. a 1GB original
>   message might be fragmented into 100 MB pieces at some point, which
>   might then be further fragmented into 10 MB pieces, etc.  That means
>   that a given article may appear multiple times at various places in
>   the network; unfragmented, and broken into various different-sized
>   pieces -- the original message and the various pieces each having
>   separate, unique message identifiers.)

It MUST NOT happen. Relaying agents MUST NOT modify articles in this
manner as they pass through them. Either they pass them unchanged, or they
drop them on the floor (and it it floods around via some slower path, then
well and good). But it is a fundamental principle of Usenet that all copies
of a particular article are supposed to be identical, apart from variant
headers (though if different copies have passed via different gateways,
that might not be possible).

As far as transport and serving within Usenet is concerned, each part
is a distinct article with its own distinct msg-id.

>4. a) During fragmentation, which message header fields are to be copied
>   to the enclosing message header, and which are to be preserved in
>   the enclosed first part message/partial header?  b) Which fields
>   need to be copied to the headers of each of the subsequent message
>   fragments?
>5. During reassembly, which fields of the first part message header
>   are to be retained, which are to be discarded; likewise for the
>   fields in the header of the enclosed first part?

>Specifically regarding question 5, I expect that Approved, Newsgroups,
>Distribution, Path, Followup-To, Expires, Summary, Organization,
>Injection-Date, Injector-Info, and User-Agent can be safely copied to
>the header of the first part and at least Approved, Newsgroups,
>Distribution, Path, Injection-Date, and Expires should be copied to
>the headers of subsequent parts.  Copying these fields to the first
>part is expected behavior when fragmenting per RFC 2046 as amended by
>RFC 3798.  I believe there may be difficulties with Control,
>Supersedes, Archive, Posted-And-Mailed, Mail-Copies-To, and possibly
>Complaints-To.

No, you certainly don't copy Path. It grows as the various parts propagate
through the network, and different parts may arrive at a given host via
different routes. OTOH, any pre-injection portion of the Path, including
the tail entry, present before the split might well be retained in the
Paths of all the parts.

Likewise, Injection-Date and Injection-Info could well be different for
the various parts, depending on where the splitting was done.

The rest of your suggestions seem about right.

Incidentally, what happens with Received headers in email? Presumably the
final reassembled message sees the Received headers from the first part,
and all the rest of them get lost.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: The great mailing list muddle
Date: Tue, 14 Sep 2004 07:00:19 -0400
Organization: TMR Associates, Inc
Lines: 55
Sender: [email-masked]

Thanks for the heads-up, it explains why I no longer get any mail for 
this list. I've sent the subscribe several times, and even gotten ack 
the first time IIRC, but clearly the new list is more bozo'd than the 
old. Pity it couldn't have been moved to somewhere competent, like 
lists.debian.org or such.

Nice to know this has become a closed list, rather than dead.

Charles Lindsey wrote:
> If seems that our Chair has switched to the new mailing list without at
> the same time posting a warning message to the old list for the benefit of
> those who had omitted, or more likely who had tried and failed, to
> subscribe to the new.
> 
> So here is a heads up that the submission address for the Usefor Working
> Group list is now [email-masked] and requests to subscribe should be
> sent to [email-masked] with the single word 'subscribe' in
> the body of the message. Well, actually, it is a majordomo system, so the
> usual majordomo commands will work.
> 
> But, if you try to do anything fancy, like having a different subscription
> address (such as a gateway to a local mailing list) but still want to post
> as from your usual mail address, then you will get a reply like
> 
> Your request to [email-masked]:
>  
> subscribe ietf-usefor [email-masked]
>  
> has been forwarded to the owner of the "ietf-usefor" list for approval. 
> 
> and after that, you will hear nothing even if, as I did, you ask our Chair
> to ensure that your subscription is processed promptly and he assures you
> that he has done so.
> 
> Sadly, this seems to be a common situation with mailing lists hosted at
> imc.org.
> 
> So now we have the ridiculous situation of a Working Group whose Editor is
> not on the mailing list. And our Chair has gone on holiday until September
> 21st.
> 
> For the moment, I am sending messages to both lists, and I suggest that
> others do the same.
> 
> And on top of all that, as you may have observed, the automatic archiving
> process at landfield.com stopped working on August 13th. I phoned Kent as
> soon as I noticed it and he promised to fix it, but evidently did not do
> so.
> 

-- 
bill davidsen <[email-masked]>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

To: [email-masked], [email-masked]
Xref: clerew local.usefor:20137
Newsgroups: local.usefor
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: The great mailing list muddle
Date: Mon, 13 Sep 2004 12:57:36 GMT
Lines: 50
Sender: [email-masked]

If seems that our Chair has switched to the new mailing list without at
the same time posting a warning message to the old list for the benefit of
those who had omitted, or more likely who had tried and failed, to
subscribe to the new.

So here is a heads up that the submission address for the Usefor Working
Group list is now [email-masked] and requests to subscribe should be
sent to [email-masked] with the single word 'subscribe' in
the body of the message. Well, actually, it is a majordomo system, so the
usual majordomo commands will work.

But, if you try to do anything fancy, like having a different subscription
address (such as a gateway to a local mailing list) but still want to post
as from your usual mail address, then you will get a reply like

Your request to [email-masked]:
 
subscribe ietf-usefor [email-masked]
 
has been forwarded to the owner of the "ietf-usefor" list for approval. 

and after that, you will hear nothing even if, as I did, you ask our Chair
to ensure that your subscription is processed promptly and he assures you
that he has done so.

Sadly, this seems to be a common situation with mailing lists hosted at
imc.org.

So now we have the ridiculous situation of a Working Group whose Editor is
not on the mailing list. And our Chair has gone on holiday until September
21st.

For the moment, I am sending messages to both lists, and I suggest that
others do the same.

And on top of all that, as you may have observed, the automatic archiving
process at landfield.com stopped working on August 13th. I phoned Kent as
soon as I noticed it and he promised to fix it, but evidently did not do
so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: ietf-usefor <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 03 Sep 2004 11:11:46 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> Briefly (this was discussed on ietf-822 in June 2003) the DSN RFC
> introduced header fields which need to be treated similarly to
> Message-ID w.r.t. first part message header and enclosed message header
> for fragmentation and reassembly.  Several of the news-specific fields
> will also need to be treated differently than what is the case per RFC
> 2046 rules in lieu of any amendments to those rules.  To take one
> particular example, the Control field par RFC 2046 rules would have to
> be copied to the first part message header in order to survive
> fragmentation/reassembly (only specific fields in the enclosed message
> header from the original message header which is encapsulated in the
> first part are copied during reassembly).  That means that that first
> part will be interpreted as if it were a (complete) control message.  If
> RFC 2046 rules were to be amended (as they were for DSN-specific fields
> by RFCs 2298/3798) then the Control field could remain encapsulated
> within the first part, not copied to the first part message header, the
> partial messages would not be treated as control messages (only the
> reassembled whole would be treated as such).

Ah, I see.  Yes, that makes sense.  I don't personally find work on
message/partial important enough to spend time on it, but if you come up
with language to deal with these issues, I'd be willing to review it and
say whether I think it makes sense.

> No, it's useful for any message whose size exceeds some threshold,
> including large text documents (such as, oh, maybe
> draft-ietf-usefor-article-13.txt for example).

Splitting text messages that small is obnoxious and anti-social and is
something that I'd filter out on my news server.  General consensus in
news.software.nntp among news administrators (this came up as well) pretty
much matched that feeling.  Nothing smaller than 1MB should be split on
Usenet, and preferrably as far as I'm concerned nothing smaller than 10MB
(although 1MB is the more common metric).

>> and 99.99% of the binary posters are never going to use it

> If the functionality is built into UAs and/or injection agents and/or
> transport agents is can be transparent to the users.

Yes, I know.  The authors of the many different software packages used in
binary newsgroups, which is by and large the only place that news
administrators want to ever see fragmented posts, are uninterested in
implementing it.  This was literally a two year long discussion, involving
a much larger number of people than have ever participated in this working
group.  You can see the conclusion in effect on Usenet right now, namely
widespread implementation of yEnc.

>> based on the *extensive* discussions that have been had with the
>> authors of that software and with news administrators on
>> news.software.nntp.

> Exactly which "that software" are you talking about -- some NNTP
> implementation?

No, software used in binary newsgroups (for posting, serving, and
reading).

> If the discussion was only in a narrow special-interest area for NNTP
> software, it likely missed UA authors as well as implementors of
> transport methods other than NNTP.

Quite possibly.  It won't make any difference, but hey, until I actually
took part in the two year long discussion, I didn't believe it either.

You can trust me or you can tilt at the windmill, your choice.  Given how
long I spent tilting at the windmill, I'm certainly not going to begrudge
anyone else the opportunity.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 02 Sep 2004 14:44:49 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
CC: ietf-usefor <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
Sender: [email-masked]

Russ Allbery wrote:
> Bruce Lilly <[email-masked]> writes:
> 
>>Russ Allbery wrote:
> 
> 
>>>Is there really any point in talking about this in our drafts?
> 
> 
>>Yes, for the same reason that the issue was addressed in RFC 3798 (and
>>2298 before it).
> 
> 
> What was that reason?  I'm not familiar with either of those RFCs.

Briefly (this was discussed on ietf-822 in June 2003) the DSN RFC
introduced header fields which need to be treated similarly to
Message-ID w.r.t. first part message header and enclosed message
header for fragmentation and reassembly.  Several of the news-
specific fields will also need to be treated differently than
what is the case per RFC 2046 rules in lieu of any amendments
to those rules.  To take one particular example, the Control
field par RFC 2046 rules would have to be copied to the first
part message header in order to survive fragmentation/reassembly
(only specific fields in the enclosed message header from the
original message header which is encapsulated in the first
part are copied during reassembly).  That means that that first
part will be interpreted as if it were a (complete) control
message.  If RFC 2046 rules were to be amended (as they were
for DSN-specific fields by RFCs 2298/3798) then the Control
field could remain encapsulated within the first part, not
copied to the first part message header, the partial messages
would not be treated as control messages (only the reassembled
whole would be treated as such).

> This is all nice and good and my point is that message fragmentation is
> something that's only useful on Usenet for binary messages

No, it's useful for any message whose size exceeds some threshold,
including large text documents (such as, oh, maybe
draft-ietf-usefor-article-13.txt for example).

> and 99.99% of
> the binary posters are never going to use it

If the functionality is built into UAs and/or injection agents
and/or transport agents is can be transparent to the users. Especially
bearing in mind the fact of common mail/news UAs not to mention
web browser/messaging UAs -- MIME message/partial applies to mail,
news, and HTTP, so support for message/partial by any one component
of such common-use UAs becomes available to the others; provided
the news-specific issues are properly addressed.

> based on the *extensive*
> discussions that have been had with the authors of that software and with
> news administrators on news.software.nntp.

Exactly which "that software" are you talking about -- some
NNTP implementation?

> So while this might be useful to someone, it is an extreme edge case and
> one with which we have little to no implementation experience, which makes
> me doubt in the extreme that this working group is going to be able to do
> an adequate job of handling the problem.

This group is likely the only group which can determine how its
news-specific header fields should be handled during fragmentation
and reassembly.

>>Clear to whom and from what perspective?  Since the discussion didn't
>>take place here, can you summarize the discussion or point to an
>>archived summary?
> 
> 
> As mentioned above, the outcome of the discussion was a complete and utter
> lack of interest in anything MIME-related for message fragmentation,
> despite several of us pushing it pretty hard.

If the discussion was only in a narrow special-interest area for
NNTP software, it likely missed UA authors as well as implementors
of transport methods other than NNTP.

>  What is being used is not
> something that we can or should standardize.

Then we should stick to the implications for message/partial.

To: [email-masked], [email-masked]
Path: not-for-mail
From: Bill Davidsen <[email-masked]>
Newsgroups: mail.usenet-format
Subject: Re: Message fragmentation via message/partial and news-specific  header fields
Date: Thu, 02 Sep 2004 14:30:36 -0400
Organization: TMR Associates, Inc
Lines: 64
Sender: [email-masked]

Russ Allbery wrote:
> Bruce Lilly <[email-masked]> writes:
> 
>>Russ Allbery wrote:
> 
> 
>>>Is there really any point in talking about this in our drafts?
> 
> 
>>Yes, for the same reason that the issue was addressed in RFC 3798 (and
>>2298 before it).
> 
> 
> What was that reason?  I'm not familiar with either of those RFCs.
> 
> 
>>1. Message/partial fragmentation has already been used on Usenet.
>>2. Fragmented messages via message/partial may appear at mail-to-news
>>   gateways.
>>3a. Message/partial is, AFAIK, the only standardized method for
>>   message fragmentation and reassembly.  And since it is part of
>>   MIME, it is widely applicable (e.g. to HTTP as well as Internet
>>   Message Format).
>>3b. If there is some other method of fragmentation/reassembly, the same
>>   sort of issues need to be discussed
> 
> 
> This is all nice and good and my point is that message fragmentation is
> something that's only useful on Usenet for binary messages and 99.99% of
> the binary posters are never going to use it based on the *extensive*
> discussions that have been had with the authors of that software and with
> news administrators on news.software.nntp.
> 
> So while this might be useful to someone, it is an extreme edge case and
> one with which we have little to no implementation experience, which makes
> me doubt in the extreme that this working group is going to be able to do
> an adequate job of handling the problem.
> 
> 
>>>There have been extensive discussions in news.software.nntp about this
>>>and the outcome was quite clear.
> 
> 
>>Clear to whom and from what perspective?  Since the discussion didn't
>>take place here, can you summarize the discussion or point to an
>>archived summary?
> 
> 
> As mentioned above, the outcome of the discussion was a complete and utter
> lack of interest in anything MIME-related for message fragmentation,
> despite several of us pushing it pretty hard.  What is being used is not
> something that we can or should standardize.
> 
Hell, yENC is even standardized in practice, about 20% of all postings I 
am asked to exampne for inappropriate content defy any decode to which I 
have access. If there's one for UNIXen which will handle these posts, 
which don't follow the "standard" at yenc.org, I haven't found it.

People will do what they choose, and to date they choose hacks.

-- 
    -bill davidsen ([email-masked])
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

To: ietf-usefor <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 02 Sep 2004 10:54:20 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:
> Russ Allbery wrote:

>> Is there really any point in talking about this in our drafts?

> Yes, for the same reason that the issue was addressed in RFC 3798 (and
> 2298 before it).

What was that reason?  I'm not familiar with either of those RFCs.

> 1. Message/partial fragmentation has already been used on Usenet.
> 2. Fragmented messages via message/partial may appear at mail-to-news
>    gateways.
> 3a. Message/partial is, AFAIK, the only standardized method for
>    message fragmentation and reassembly.  And since it is part of
>    MIME, it is widely applicable (e.g. to HTTP as well as Internet
>    Message Format).
> 3b. If there is some other method of fragmentation/reassembly, the same
>    sort of issues need to be discussed

This is all nice and good and my point is that message fragmentation is
something that's only useful on Usenet for binary messages and 99.99% of
the binary posters are never going to use it based on the *extensive*
discussions that have been had with the authors of that software and with
news administrators on news.software.nntp.

So while this might be useful to someone, it is an extreme edge case and
one with which we have little to no implementation experience, which makes
me doubt in the extreme that this working group is going to be able to do
an adequate job of handling the problem.

>> There have been extensive discussions in news.software.nntp about this
>> and the outcome was quite clear.

> Clear to whom and from what perspective?  Since the discussion didn't
> take place here, can you summarize the discussion or point to an
> archived summary?

As mentioned above, the outcome of the discussion was a complete and utter
lack of interest in anything MIME-related for message fragmentation,
despite several of us pushing it pretty hard.  What is being used is not
something that we can or should standardize.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 02 Sep 2004 12:01:32 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-usefor <[email-masked]>
Organization: Bruce Lilly
To: Russ Allbery <[email-masked]>
CC: ietf-usefor <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
Sender: [email-masked]

Russ Allbery wrote:

> Is there really any point in talking about this in our drafts?

Yes, for the same reason that the issue was addressed in RFC 3798 (and
2298 before it).

> It seems highly unlikely to me that we're going to get the people who
> fragment Usenet messages to adopt MIME to do so, even if it would be the
> right thing.

1. Message/partial fragmentation has already been used on Usenet.
2. Fragmented messages via message/partial may appear at mail-to-news
   gateways.
3a. Message/partial is, AFAIK, the only standardized method for
   message fragmentation and reassembly.  And since it is part of
   MIME, it is widely applicable (e.g. to HTTP as well as Internet
   Message Format).
3b. If there is some other method of fragmentation/reassembly, the same
   sort of issues need to be discussed

> There have been extensive discussions in news.software.nntp
> about this and the outcome was quite clear.

Clear to whom and from what perspective?  Since the discussion didn't
take place here, can you summarize the discussion or point to an
archived summary?

To: ietf-usefor <[email-masked]>
Subject: Re: Message fragmentation via message/partial and news-specific header fields
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 02 Sep 2004 01:27:31 -0700
Sender: [email-masked]

Bruce Lilly <[email-masked]> writes:

> We had had some discussion about message fragmentation via the MIME
> media type message/partial,, and there was some text in earlier drafts,
> however the matter is conspicuously absent from the recently-produced
> usefor-usepro-00 draft.  The rules for message fragmentation and
> assembly based on standard header fields are discussed in RFC 2046 as
> amended by RFC 3798.  There was some discussion of the matter on the
> ietf-822 mailing list in June 2003 (q.v.).  I also expect some relevant
> considerations to be reintroduced into ietf-822 mailing list discussion
> in the near future.

Is there really any point in talking about this in our drafts?

It seems highly unlikely to me that we're going to get the people who
fragment Usenet messages to adopt MIME to do so, even if it would be the
right thing.  There have been extensive discussions in news.software.nntp
about this and the outcome was quite clear.  Given that, I think this
would mostly be an exercise in protocol design for its own sake, and I'm
not sure that it's worth spending the time and effort on it given the
unlikeliness of widespread use.

I'd rather just not mention it and expend our resources on more pressing
problems, unless there's some significant need to attack this problem and
some reasonable belief that people will adopt a standardized solution in
any significant numbers.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Wed, 01 Sep 2004 11:33:54 -0400
From: Bruce Lilly <[email-masked]>
Reply-To: ietf-usefor <[email-masked]>
Organization: Bruce Lilly
To: [email-masked]
CC: USEFOR <[email-masked]>
Subject: Message fragmentation via message/partial and news-specific header fields
Sender: [email-masked]

We had had some discussion about message fragmentation via the MIME media type
message/partial,, and there was some text in earlier drafts, however the
matter is conspicuously absent from the recently-produced usefor-usepro-00
draft.  The rules for message fragmentation and assembly based on standard
header fields are discussed in RFC 2046 as amended by RFC 3798.  There was
some discussion of the matter on the ietf-822 mailing list in June 2003 (q.v.).
I also expect some relevant considerations to be reintroduced into ietf-822
mailing list discussion in the near future.

There are several implications for message fragmentation and reassembly
which appear not to have been adequately considered.  I plan to mention
several of these implications.

First there are some questions that need to be considered:
1. a) Where is message fragmentation expected to occur (e.g. during transport
   between transfer agents when it is recognized that the receiving agent
   has some message size limit), and b) by what criteria (presumably some
   site-dependent size limit, but how is that communicated to that site's
   feeding agents)?
2. Where does reassembly take place?
3. How is multiple fragmentation to be handled? [e.g. a 1GB original
   message might be fragmented into 100 MB pieces at some point, which
   might then be further fragmented into 10 MB pieces, etc.  That means
   that a given article may appear multiple times at various places in
   the network; unfragmented, and broken into various different-sized
   pieces -- the original message and the various pieces each having
   separate, unique message identifiers.)
4. a) During fragmentation, which message header fields are to be copied
   to the enclosing message header, and which are to be preserved in
   the enclosed first part message/partial header?  b) Which fields
   need to be copied to the headers of each of the subsequent message
   fragments?
5. During reassembly, which fields of the first part message header
   are to be retained, which are to be discarded; likewise for the
   fields in the header of the enclosed first part?

Specifically regarding question 5, I expect that Approved, Newsgroups,
Distribution, Path, Followup-To, Expires, Summary, Organization,
Injection-Date, Injector-Info, and User-Agent can be safely copied to
the header of the first part and at least Approved, Newsgroups,
Distribution, Path, Injection-Date, and Expires should be copied to
the headers of subsequent parts.  Copying these fields to the first
part is expected behavior when fragmenting per RFC 2046 as amended by
RFC 3798.  I believe there may be difficulties with Control,
Supersedes, Archive, Posted-And-Mailed, Mail-Copies-To, and possibly
Complaints-To.