Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 30 Jan 2007 19:26:08 GMT

In <[email-masked]> "Forrest J. Cavalier III" <forrest@xxxxxxxxxxxxxxx> writes:

>Charles Lindsey wrote:

>>Reinjection similarly can be assured of not creating loops simply
>>by preserving the identity of the article (both Message-ID and Date header
>>fields) when reinjecting.

>>Hence it follows that if we were to make the rules for Injection-Date
>>exactly the same as the current rules for Date, it would result in a
>>system no different from the present situation, except for an increase in
>>the staleness margin for late-injected articles.

>Sure, after the flag day.

What flag day? The whole idea is that, in the interim period until
Injection-Date is widely acted upon, agents will calculate staleness using
Date. Since that is exactly what happens at present, the situation will be
no different/worse than at present. But, as Injection-Date comes to be be
recognized, things will get better, insofar as some articles that
currently fail to propagate well will propagate better. The improvement is
gradual, but nevertheless monotonic.

>Since that won't happen, re-injectors would rely on the false statements in
>USEPRO that tell them Injection-Date is honored, when the non-upgraded servers
>still look at Date.

USEPRO will make no such statement (well, Usepro-06 did not, because it
explicitly said what I have just said above).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Wed, 31 Jan 2007 21:21:50 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I'm not sure what Mailman does.

> Your message to which I am replying arrived with

>     Date: Tue, 30 Jan 2007 14:27:35 -0800

> Is that identical to what you submitted? I believe this lists uses
> Mailman.

Uh, I don't know what Mailman's mail to news gateway does, I mean.  Which
I don't think this list checks.

I do know that it always replaces the message ID and generates a new one
for the newsgroup, though.  (And then doesn't change References.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24322
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Wed, 31 Jan 2007 17:59:52 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Harald Tveit Alvestrand <[email-masked]> writes:

>> what's current practice for email gateways on incoming messages? Do they:

>> - keep the Date: header and accept rejection for old articles?
>> - modify the Date: header if the article is rejected on submission?
>> - modify the Date: header in all cases?

>I think it's hard to get a read on what all gateways do, since there are
>so many different implementations and they're not really discussed in a
>common place.  All of mine keep the Date header, though.

Well that's what I had always assumed gateways (I think we are thinking
mainly of mailing list expanders here) would do (read: I have never
spotted one changing anything, though I have not looked that hard).

>I'm not sure what Mailman does.

Your message to which I am replying arrived with

    Date: Tue, 30 Jan 2007 14:27:35 -0800

Is that identical to what you submitted? I believe this lists uses
Mailman.

I have carefully set the Date header on this message to be:

    Date: Wed, 31 Jan 2007 17:59:52 GMT

Will people please report if it arrives exactly like that?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24313
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Tue, 30 Jan 2007 19:26:08 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <forrest@xxxxxxxxxxxxxxx> writes:

>Charles Lindsey wrote:

>>Reinjection similarly can be assured of not creating loops simply
>>by preserving the identity of the article (both Message-ID and Date header
>>fields) when reinjecting.

>>Hence it follows that if we were to make the rules for Injection-Date
>>exactly the same as the current rules for Date, it would result in a
>>system no different from the present situation, except for an increase in
>>the staleness margin for late-injected articles.

>Sure, after the flag day.

What flag day? The whole idea is that, in the interim period until
Injection-Date is widely acted upon, agents will calculate staleness using
Date. Since that is exactly what happens at present, the situation will be
no different/worse than at present. But, as Injection-Date comes to be be
recognized, things will get better, insofar as some articles that
currently fail to propagate well will propagate better. The improvement is
gradual, but nevertheless monotonic.

>Since that won't happen, re-injectors would rely on the false statements in
>USEPRO that tell them Injection-Date is honored, when the non-upgraded servers
>still look at Date.

USEPRO will make no such statement (well, Usepro-06 did not, because it
explicitly said what I have just said above).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Tue, 30 Jan 2007 14:27:35 -0800
Sender: [email-masked]

Harald Tveit Alvestrand <[email-masked]> writes:

> While email gateways are outside of the scope of the WG, it might be
> relevant to ask...

> what's current practice for email gateways on incoming messages? Do they:

> - keep the Date: header and accept rejection for old articles?
> - modify the Date: header if the article is rejected on submission?
> - modify the Date: header in all cases?

> If they're all currently keeping the Date: header and accept rejection,
> then I don't see any reason to bend over backwards to do something
> different; if they all mangle Date: before submission, that's a clear
> indication that they've learned to do so by bad experiences.

I think it's hard to get a read on what all gateways do, since there are
so many different implementations and they're not really discussed in a
common place.  All of mine keep the Date header, though.

I'm not sure what Mailman does.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 30 Jan 2007 14:01:38 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Question:

While email gateways are outside of the scope of the WG, it might be 
relevant to ask...

what's current practice for email gateways on incoming messages? Do they:

- keep the Date: header and accept rejection for old articles?
- modify the Date: header if the article is rejected on submission?
- modify the Date: header in all cases?

If they're all currently keeping the Date: header and accept rejection, 
then I don't see any reason to bend over backwards to do something 
different; if they all mangle Date: before submission, that's a clear 
indication that they've learned to do so by bad experiences.

                Harald

--On 22. januar 2007 09:18 -0800 Russ Allbery <[email-masked]> wrote:

>
> Forrest J Cavalier <[email-masked]> writes:
>
>> Here's a thought....
>
>> USEFOR should have defined "Authoring-Date:", or some such thing, not
>> "Injection-Date:"  This would be completely backwards compatible, and
>> put the work to resolve it on reading agents.
>
> Yeah, that was my fourth proposal.  It's possible to argue that it doesn't
> require any change to USEFOR.
>
> --
> Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>
>
>

Date: Mon, 29 Jan 2007 21:24:06 -0800
From: Harald Tveit Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Re: #1417: USEPRO 3.4: Injecting-agent modification of message-ID
Sender: [email-masked]

Speaking as a contributor, I'm happy with this proposal for resolution.

Speaking as chair, I'll add this resolution proposal to the tracker, and 
assume that it represents an acceptable solution based on the positive 
comments + the lack of comments against it.

            Harald

--On 18. januar 2007 20:24 -0800 Russ Allbery <[email-masked]> wrote:

>
> Proposed wording:  Change the current text:
>
>    6.   The injecting agent MUST NOT alter the body of the article in
>         any way (including any change of Content-Transfer-Encoding).  It
>         MAY add other header fields not already provided by the poster,
>         but injecting agents are encouraged to use the Injection-Info
>         header for such information and to minimize the addition of
>         other headers.  It SHOULD NOT alter, delete, or reorder any
>         existing header field except the Path header.
>
> to:
>
>    6.   The injecting agent MUST NOT alter the body of the article in
>         any way (including any change of Content-Transfer-Encoding).  It
>         MAY add other header fields not already provided by the poster,
>         but injecting agents are encouraged to use the Injection-Info
>         header for such information and to minimize the addition of
>         other headers.  It SHOULD NOT alter, delete, or reorder any
>         existing header field except the Path header field.  It MUST NOT
>         alter or delete any existing Message-ID header field.
>
> Note the minor s/Path header/Path header field/ change at the end of the
> current text as well.  I noticed that while looking at this.
>
> I'd prefer that discussion about changing the general SHOULD NOT about all
> header fields to a MUST NOT be a separate discussion, since the Message-ID
> has a distinguished protocol status and an interoperability use case that
> the other header fields don't have.
>
> --
> Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>
>
>

To: [email-masked]
Xref: clerew local.usefor:24302
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1083: USEPRO 5.3: Rules for generating message-ID
Date: Mon, 29 Jan 2007 12:41:02 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Frank Ellermann <[email-masked]> writes:

>> But if an injecting agent creates Message-IDs it can't afford
>> to create collisions.  It has to work if it generates more than
>> one Message-ID per second, and it has to work if its local time
>> sometimes jumps backwards into the "past".  It has to work at
>> the end of daylight savings time, and after a reboot.

>I think that RFC 2822 and USEFOR already adequately address this.

Yes, RFC 2822 covers it, but not with the emphasis that perhaps it
dererves (no use of the phrase "for all time", for example). And all
USEFOR does is to add that the uniqueness applies across the whole of
Email, Netnews and any other protocols which might be involved, rather
than just across Emails as RFC 2822 implied.

So yes, maybe S-o-1036 expressed it better, and I would have no problem
with language in USEPRO which re-emphasised this point once again, for the
benefit of casual readers. But such language would not be normative. A
NOTE in the duties of Injecting Agents would suffice. I tend to agree that
giving reasons for why things are as they are is always a Good Thing.

>> Your concept "Message-ID + Injection-Date = id" is IMO wrong. 
>> The id of an article is the Message-ID, as the name suggests.

It is an approximation which was adequate for explaining the points that
Russ was making, but I do not think that terminology should appear in
USEPRO.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sun, 28 Jan 2007 20:01:45 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Back in 2002 there was no "Injection-Date", trying GMaNe's
> search I found it, apparently you invented this 2004-02-01:

> http://article.gmane.org/gmane.ietf.usenet.format/24867/

Ah, yes.  Looks like you and Charles are correct and it was originally
related to compatibility with RFC 2822's Date header.  I'd completely
forgotten about that part of the discussion.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date:  Mon, 29 Jan 2007 04:46:08 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

Russ Allbery wrote:

 [why was Injection-Date invented]
> (I thought you were here for the previous discussions?)

No.  I found the odd "TLD invalid" invention in a draft, and
that inspired me to check what the hell this WG is planning.

That was near the last-last-last usefor-07 WGLC in 2002, and
after a few articles and Ralph's "no consensus" poll I forgot
about it, expecting that it will be ready soon (minus UTF-8).

Later I sometimes looked, but for some time I refrained from
getting involved into the debates about "Re:" and references,
or the "OUGHT-ification" phase.

In May 2004 that changed, one ASRG Chair resigned, Caller-ID
was added to the MARID agenda, and I wrote my first mail to
anything@ietf, in that case ietf-ipr@ietf, in only three days.
It convinced me that something is decisively rotten with the
standards process, and that waiting for good RFCs is no plan.

Back in 2002 there was no "Injection-Date", trying GMaNe's
search I found it, apparently you invented this 2004-02-01:

http://article.gmane.org/gmane.ietf.usenet.format/24867/

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1083: USEPRO 5.3: Rules for generating message-ID
Organization: The Eyrie
Date: Sun, 28 Jan 2007 19:10:12 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> But if an injecting agent creates Message-IDs it can't afford
> to create collisions.  It has to work if it generates more than
> one Message-ID per second, and it has to work if its local time
> sometimes jumps backwards into the "past".  It has to work at
> the end of daylight savings time, and after a reboot.

I think that RFC 2822 and USEFOR already adequately address this.

I certainly don't object to someone resurrecting the message ID generation
draft that was previously discussed in this working group as an
information RFC on how one can go about generating message IDs with the
desired properties, but RFC 2822 is reasonably clear about the required
properties and USEFOR already reiterates that a message identifier must be
unique forever.

> Your concept "Message-ID + Injection-Date = id" is IMO wrong. 
> The id of an article is the Message-ID, as the name suggests.

You apparently completely misunderstood my message if that's what you got
out of it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1083: USEPRO 5.3: Rules for generating message-ID
Date:  Mon, 29 Jan 2007 03:47:40 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 49
Sender: [email-masked]

Russ Allbery wrote:
 
> I propose closing this ticket with no further changes.

I disagree.  Not talking about spam cancel conventions is okay.
Not talking about Message-IDs is not.  For UAs it can be okay
if they only try to get it right based on the text in RFC 2822.

But if an injecting agent creates Message-IDs it can't afford
to create collisions.  It has to work if it generates more than
one Message-ID per second, and it has to work if its local time
sometimes jumps backwards into the "past".  It has to work at
the end of daylight savings time, and after a reboot.

Your concept "Message-ID + Injection-Date = id" is IMO wrong. 
The id of an article is the Message-ID, as the name suggests.

The Injection-Date is a kludge dealing with incomplete history
files.  It's impossible and/or undesirable to keep a complete
list of all seen Message-IDs in practice, but it's the ideal
for worldwide unique Message-IDs forever.  

This is a core feature of s-o-1036, worldwide unique forever.

It took me very long to get this, from my Fidonet POV, where
the path is everything, and the Message-ID only optional.  I
tried weird compromises like "if two years isn't enough, then
maybe five years, or ten years".  But of course the s-o-1036
evangelists finally convinced me that "unique forever" is not
two, five, or ten years, but much longer.  And that this idea
of the Message-ID is the essence of NetNews (taking groups as
given, that was similar in FTN echomail).

Any text intending to obsolete s-o-1036 has to get this idea
to all readers (or at least to the implementors reading it).
For outsiders with a 2822-POV (Message-ID not mandatory) or
my former FTN POV it's not obvious.  It's also not obvious
why there is a history in addition to the path.  

Not talking about it (as a trick to avoid the question of a 
reasonable minimum) is no option.  S-o-1036 has this right,
and if USEPRO refuses to outline the principles of operation
it is pointless to continue with it.  Readers need to know
*_why_* something is a MUST NOT or SHOULD or MAY.  For that
they need to know how stuff works beyond their own server or
implementation.

Frank

Date: Thu, 25 Jan 2007 12:38:44 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Charles Lindsey wrote:

>> Reinjection similarly can be assured of not creating loops simply
>>by preserving the identity of the article (both Message-ID and Date header
>>fields) when reinjecting.
> 
> 
> Hence it follows that if we were to make the rules for Injection-Date
> exactly the same as the current rules for Date, it would result in a
> system no different from the present situation, except for an increase in
> the staleness margin for late-injected articles.
> 

Sure, after the flag day.

Since that won't happen, re-injectors would rely on the false statements in 
USEPRO that tell them Injection-Date is honored, when the non-upgraded servers 
still look at Date.

Don't you think that is a problem?

Date: Thu, 25 Jan 2007 12:35:57 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Sender: [email-masked]

Charles Lindsey wrote:

>>I don't see any way to preserve Path except by renaming before reinjection. 
>>Reinjection is a POST through an injecting agent, after all.
> 
> 
> So, injection inserts a "POSTED" and reinjection inserts a second "POSTED"
> which, if the article then appears "iffy" or otherwise spamish, directs
> your attention immediately to the point to which your suspicions should be
> directed.
> 
> 
>>Either injecting agents are responsible for adding trustable headers to the 
>>proto articles (after removing any that are present) or they are not.
> 
> 
> Destruction of evidence can only hinder forensics.
> 

That's not the only goal, especially since forensics is not the primary purpose 
of Path.  (Unlike "Received" headers.)

Do you have any position on the mischief that can be achieved with Path pre-loading?

To: [email-masked]
Xref: clerew local.usefor:24282
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Thu, 25 Jan 2007 17:21:58 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Most discussions would have moved on between when they authored and when they 
>injected days later.  It is rude to think everything you write is timeless.

The late injection may or may not be for reasons beyond your control. But
the advantage of having Date tied to the moment of authorship (as clearly
set out in RFC 2822) means that if some reader thinks "what is this guy
talking about - where has he been these last 3 days" he can then look at
the Date header and realise "Ah! he wrote it before all the momentous
events of the last 3 days happened".

>And if you read all the intervening messages before you late posted, just
>to ensure that what you wrote days ago is still applicable, why should the
>date header be a record of when you typed it, and not when you affirm the 
>message to contain what you want to send?  It's splitting hairs.

No, the moment when you pressed 'send' is the moment when you considered
your article complete (see RFC 2822 again). It is not the moment of
injection.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24280
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Thu, 25 Jan 2007 12:18:34 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:

>> Yes, my preference is to preserve all the Path header, even early parts of
>> it that recorded its progress through some private system before getting
>> injected into Usenet. It will do litle harm, and may help unravel some
>> complicated cases where things have gone wrong. That is why Usepro-06 said
>> SHOULD preserve the old Path during reinjection. Then you get to see the
>> full story of exactly where it has been.
>> 

>You are comingling two points of argument, I think.  I do not consider
>the Path header as trace information.

Eh? I regard Path very much as trace information, our equivalent to the
email Received header (but much less verbose). It is certinaly widely used
for that purpose, for trying to find out where spam and other malpractice
originated - which requires some estimate of which parts of the Path are
bogus, which is the primary reason we have added diagnostic information to
it.

>I don't see any way to preserve Path except by renaming before reinjection. 
>Reinjection is a POST through an injecting agent, after all.

So, injection inserts a "POSTED" and reinjection inserts a second "POSTED"
which, if the article then appears "iffy" or otherwise spamish, directs
your attention immediately to the point to which your suspicions should be
directed.

>Either injecting agents are responsible for adding trustable headers to the 
>proto articles (after removing any that are present) or they are not.

Destruction of evidence can only hinder forensics.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24281
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Thu, 25 Jan 2007 17:09:55 GMT
Lines: 334
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>First, let's go back to the Netnews protocol as it exists today and look
>at how injection and reinjection work without Injection-Date.

OK, so we stop firing obscure (but possible) scenarios back and forth
while we examine these possibilities.

>Every article has a unique identity in its message identifier, but
>...........  Therefore, an article's functional identity is
>the Date and message identifier pair.

Yes, Modulo some "fuzziness" of the role of the Date, as Forrest has
pointed out.

Agreed paragraphs snipped ......

>Note that for proto-articles ........ it's best practice for a posting
> agent injecting an article at multiple injection agents to provide the
> complete identity of the article (both Message-ID and Date header fields)
.....

More than that, both Uespro-06 and -07 REQUIRE the posting agent to do
that before multiple injection.

>  Reinjection similarly can be assured of not creating loops simply
>by preserving the identity of the article (both Message-ID and Date header
>fields) when reinjecting.

Hence it follows that if we were to make the rules for Injection-Date
exactly the same as the current rules for Date, it would result in a
system no different from the present situation, except for an increase in
the staleness margin for late-injected articles.

>....  So currently, reinjection breaks down if the
>reinjection cannot happen within the staleness period of the Date header
>....  The only difference is that reinjecting
>agents are generally the sole path to reach a particular site, so if the
>reinjection is delayed, articles go missing.

Eh? ITYM "reinjecting agents are generally the sole path for injected
articles from a particular site to reach the wider Usenet, so if ...".

>  Because of this, some
>reinjecting agents change the identity of the message by regenerating the
>Date header.

OK, let us coin the term "Magic Exception" for that practice. Clearly it
is an "Exception" to the laid-down rules, but why "Magic"? Because:

. It is hard to define exactly when the practice is safe, beyond "you
would know one of these situations when you see it".
. It is certainly only for those who "really REALLY" know what they are
doing, and certainly "Kiddies, don't try this at home" applies.
. It may not be documented in any standard, but people "in the know" will
recognize and condone it so long as it is wisely used - a bit like we
discussed for the $alz convention.

>  More on the risks of that in a moment.

>Now, the problem with this is that the Date header field carries two
>meanings.  It's part of the identity of the article, and it's also
>human-readable information about when the message was composed.  Some
>posting agents always treat it as the former and either let the injecting
>agent generate it or only generate it at injection time.  Some posting
>agents treat it as the latter and generate it at message composition time.

And we want to encourage the latter, for conformity with RFC 2822, and
contrary to the advice in s-o-1036, so explicit words of encouragement
would be in order.

>The working group therefore introduced a new header, Injection-Date, which
>serves *only* the protocol function and separates the protocol function
>from the user presentation function.  Date is now of interest only to
>humans and contains the composition time, which serves no protocol
>function.  Injection-Date is part of the identity of the message.

Right!

>However, as part of this change, we also changed who was responsible for
>establishing the identity of a message.  Now, the injecting agent *always*
>establishes the identity of a proto-article, and the posting agent isn't
>permitted to do so.

Yes, Usepro-06 did not permit that but, in the light of this discussion, I
am thinking it went too far. I have no problem with letting the posting
agent do it in appropriate circumstances.

>  Only when a proto-article is injected does it acquire
>a unique identity in practice; prior to that, it will only have a message
>identifier, which is not sufficient alone to prevent duplication of the
>message.

Look at it this way. It is the "act of injection" which requires an
Injection-Date to be created. Whether it is created by the injector (the
posting agent) or by the injectee (the injecting agent) is a secondary
matter, so long as one of them does it. Of course, there are good reasons
why normal practice should be for the injecting agent to do it, but
special cases such as multiple- or re-injection could be different.

But, from the posting agent's POV, the "act of injection" comes later than
the "act of authorship". Typical current posting agents work as follows.
The poster writes an article and presses "send". At that point, a Date
header is created and the posting agent attempts to make an NNTP
connection to a server. If that fails (because the server cannot be
reached), it puts the article into an "Outbox" where it sits until a
server can be reached. Maybe it tries every 5 minutes, or maybe there is a
mechanism for it to be activated when next the user dials in. Either way,
only then should the Injection-Date be added (assuming the posting agent
is gong to add it, which would probably not be the usual case anyway).

>This isn't a problem only for reinjection.  It's a problem for every case
>where an article is touched by multiple injecting agents.  In particular,
>since the introduction of Injection-Date, a proto-article that's injected
>at multiple injecting agents will be assigned a slightly different
>identity by each one.  In the normal case, these identities will only vary
>by seconds or at most minutes, which in practice is highly unlikely to
>cause problems, but theoretically we still broke the identity model.

So if posting agents are permitted to add Injection-Date, you might
recommend them to do so when multi-injecting, though it is hardly
necessary with those short delays (delays of several days are a different
matter of course, meriting at least a warning in the document). But
typical current posting agents don't do multi-injecting, so this is only
likely to arise with savy users who have written their own injecting
scripts (or, more likely, are running their own private news servers).

>Since the introduction of Injection-Date, time-delayed multiple injection
>is no longer safe against duplication. ... Prior to Injection-Date,
>it could make sure that all copies had the same identity and the later
>injection would never duplicate, just possibly suffer from poor
>propagation because it's stale.

Unless it took advantage of the Magic Exception, which you say some of
them did.

>Of course, serial multiple injection, reinjection, suffers the most, since
>it is the most likely to happen some time later.  So in usepro-06,
>reinjection was distinguished from injection and only in the reinjection
>case (as determined by the injecting agent) was the article permitted to
>retain its original Injection-Date.

Actually, it did say that an existing Injection-Date (however arising)
MUST NOT be altered. But it was never my intention that reinjection would
require prior agreement from the injecting agent (which is not to say that
an injecting agent has not the usual right to refuse articles which it
detects to be reinjections).

>So, we have some competing goals:

Though actually the areas of competition competition are not all that
great, leading us to hope that they can be bridged.

........

>So far, we have a couple of different solutions.

> * The current draft, my approach, basically takes the stance of "well, we
>   asked for it, so we take the consequences."........ the agent
>   doing the reinjection is required to verify as well as possible that it
>   won't create the chance of duplicates.  In practice, the drawbacks are
>   hopefully limited.

The chief drawback is the difficulty for the reinjecting agent to do that
verification.

>  There's no way for that check to be perfect, but
>   hopefully it's no worse than the other case of multiple injection that
>   we're already dealing with.  As a nice side effect, this also means
>   that delays in reinjection don't cause articles to be lost.

> * usepro-06 makes reinjection a special case with a different set of
>   rules that require that the identity of the article be preserved.  In
>   other to do this, negotiation between the posting agent and the
>   injecting agent is required .....

No, there was no necessity for negotiation.

>  ..........  The
>   core point is that reinjection preserves article identity, but in the
>   process maintains the current drawback that delays in reinjection may
>   cause articles to be lost.  There may be a bit of an attractive
>   nuisance here, the same one that's present in the existing protocol,
>   for reinjecting agents to drop the Injection-Date header anyway so as
>   to not lose articles, thereby reintroducing the problems the current
>   draft causes.

Which would be a Magic Exception, neither better nor worse than that
practised by current Magicians.

>Forrest has proposed only allowing reinjection between disjoint networks.

Apparently he did not intend to propose that. Anyway, it would be
dangerous and unenforceable.

>...........  The
>difficulty here, as Charles correctly points out, is that there's no
>general way of establishing that two networks are disjoint.  One can
>sometimes determine for individual articles that they passed through a
>host on network A and on network B and therefore the networks are not
>disjoint, but to be sure would require full knowledge of at least one of
>the networks.  There are some common cases where this is possible, but
>it's not possible in general, and it's possible to think that you know the
>networks are disjoint and be wrong.

Exactly! That is an excellent summary of my position. It is even possible
not to be aware that you are reinjecting.

>There is another solution that require modifying USEFOR:

> * Drop Injection-Date entirely and go back to the current protocol.  This
>   solves this problem and reintroduces the problem that we were trying to
>   fix originally.

No. If, as a minimum, Injection-Date follows exactly the same rules as the
existing Date, then you are never any worse than the present situation.
OTOH, you are sometimes better because you can gain some extra time before
the staleness check hits you.

>There is another solution that arguably requires modifying USEFOR:

Actually, I think the existing USEFOR works here just fine. What it says
is:

   "This header field MUST be inserted whenever an article is injected."

No mention of whether it is the injector or the injectee that is
responsible for that, so we are free to specify the mechanism as seems
best [1]. So the first sentence in what follows is not needed.

> * Change the definition of the Injection-Date header field so that the
>   posting agent can provide it, and indeed MUST provide it if they wish
>   Date to be treated as a comment rather than a protocol element.
>   Require posting agents doing multiple injection to include either a
>   Date or (preferrably) a Posting-Date header.

ITYM s/Posting-Date/Injection-Date/. Note that we already require posting
agents to provide Date and Message-ID when multi-injecting. We might
choose to add Injection-Date to that.

>                                                Don't allow injecting
>   agents to set Injection-Date at all if Date was provided.

No No! "Don't allow injecting agents to set Injection-Date at all if
Injection-Date was already provided". If Date alone was already provided,
the injecting agent should presume it was an authorship date and add a
proper Injection-Date (even if it was only a few seconds later). That
satisfies the MUST in USEFOR.

>  This has the
>   advantage of solving the multiple injection problem (present in both my
>   draft and in usepro-06), and allows reinjecting agents to assert the
>   same article identity in the same way as is possible with the current
>   protocol.  Basically, this would make Injection-Date *entirely* the
>   same as the protocol purpose of the current Date header, rather than
>   mostly the same.  It has the same difficulties with delayed
>   reinjection.

Which brings us back to the Magic Exception.

Yes, with the above provisos, I would support that proposal.

>One could make the argument that we could do this now, that nothing in the
>current USEFOR definition says that the posting agent can't provide this
>header.  The current text implies it, but doesn't say so outright.

Nor could it, because the terms "posting agent" and "injecting agent" are
not even defined in USEFOR.

>  The
>header name is confusing if the posting agent is providing it, but we
>could live with that.

Not confusing at all if you regard "injection" as an interaction between
an injector and an injectee.

>The drawback of this approach, of course, is that it still doesn't deal
>with the issue of delayed reinjection.

Right! So now we need to take a serious look at the "Magic Exception".
Firstly, we need to understand in which circumstances it would be safe.
For sure, it is safe in the case of a user who runs a private news server
on his own machine and never interacts with users outside that machine
except via proper Usenet injecting agents. Fortunately, that is the
commonest case in which reinjection is likely to arise (next most common
will be exotic gatewaying situations, and after that I am not sure beyond
people who are reinjecting without realising it).

Are there any other situation where it is guaranteed safe? Perhaps a small
private network which is a 'peninsula' to the rest of Usenet, with the
reinjecting server on the 'isthmus'. But can you be sure it is really a
'peninsula'? Perhaps you can if it is all behind a firewall that blocks
all unauthorized outgoing NNTP. But the further you go beyond that, the
more "iffy" it becomes.

So, if/when we feel we understand that, do we then write some form of
Magic Exception into the draft, and if so how? Or do we leave it as a
piece of Magic only to be practised by Magicians who "really REALLY" know
what they are up to? Because, for sure, such magicians will indeed exist
and do it whatever we say (merely claiming, if asked, that their "private
server" is really just an exotic user agent from the POV of the rest of
Usenet).

>  Some reinjecting agents are going
>to want to change the identity of a message so that it will still be
>accepted even though it's old.  Doing so inherently runs the risk of
>creating duplicates, but doing so is going to be a common desire --
>gateways go down or off-line for a while, networks go out, people want to
>pull down older news to bootstrap or seed a new Netnews network, etc.  If
>we just outlaw it, we have no control over how people do it if they choose
>to break the protocol.  If we can come up with something useful to say
>about *how* to change the identity of the message, maybe we can reduce the
>risk.

Yes, I think that is a good summary of what we would need to look at. But
all those issues already arise, in much the same form, on the existing
network, and the predicted "Death of Usenet" still has not happened.

[1] OK, there is a hint that Injection-Date is done by news servers in an
example in the definition of "generate", but it would be a minor tweak to
fix that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 23 Jan 2007 10:25:01 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:
> 
> 
>>Russ Allbery wrote:
> 
> 
>>>We need to say *what* other header fields it will rename.  I think we also
>>>have to allow for drop as well as rename; for example, if I'm posting to a
>>>disconnected INN server and then gatewaying those articles to Usenet, my
>>>local trace information is entirely irrelevant and there's no reason to
>>>retain it.
> 
> 
>>Isn't that permitted by the last SHOULD?  Why is the trace information
>>entirely irrelevant?  It isn't any more irrelevant than opaque trace
>>information from all the other servers.
> 
> 
> Yes, my preference is to preserve all the Path header, even early parts of
> it that recorded its progress through some private system before getting
> injected into Usenet. It will do litle harm, and may help unravel some
> complicated cases where things have gone wrong. That is why Usepro-06 said
> SHOULD preserve the old Path during reinjection. Then you get to see the
> full story of exactly where it has been.
> 

You are comingling two points of argument, I think.  I do not consider
the Path header as trace information.

I don't see any way to preserve Path except by renaming before reinjection. 
Reinjection is a POST through an injecting agent, after all.

Either injecting agents are responsible for adding trustable headers to the 
proto articles (after removing any that are present) or they are not.

That question is already answered: we live with what existing injecting agents do.

To: [email-masked]
Xref: clerew local.usefor:24263
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Mon, 22 Jan 2007 23:35:53 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Frank Ellermann <[email-masked]> writes:

>> The incompability of the 2822 Date and the s-o-1036 Date is obvious, is
>> that the reason why you invented the new Injection-Date here ?

>No.  (I thought you were here for the previous discussions?)  We invented
>Injection-Date to solve a specific problem with off-line posts.  Avoiding
>having to redefine an RFC 2822 header field was an additional advantage,
>but wasn't the initial motivation.

No, I think Frank is largely right. The wording in RFC 2822 was a
significant part of the thinking that led to Injection-Date (and, after
all, that RFC 2822 wording explicitly talks about off-line preparation of
emails).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24262
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date: Mon, 22 Jan 2007 23:32:48 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Okay, I took a long walk to think about this (which I should have done
>before sending the last message -- I'm sorry, Charles, I'm confused by all
>the specific examples but after thinking about this on a walk, I can see
>the general drive of what you're getting at).  Let's take a step back and
>look at the theory.  I think it's the specific use cases that are mixing
>things up.

OK, there's a lot of good stuff in there, but it is late at night, and
needs a lot of thinking about. So I have printed it out, and shall study
it carefully before coming back, which may not be for a day or so.

But just one immediate reaction. On reading it through, I kept saying to
myself "Ah But! There is more to be said on that". And then a few further
paragraphs down you indeed "said more on that". Which means, perhaps, that
there is some convergence afoot.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24261
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Mon, 22 Jan 2007 23:27:26 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Russ Allbery wrote:

>> We need to say *what* other header fields it will rename.  I think we also
>> have to allow for drop as well as rename; for example, if I'm posting to a
>> disconnected INN server and then gatewaying those articles to Usenet, my
>> local trace information is entirely irrelevant and there's no reason to
>> retain it.

>Isn't that permitted by the last SHOULD?  Why is the trace information
>entirely irrelevant?  It isn't any more irrelevant than opaque trace
>information from all the other servers.

Yes, my preference is to preserve all the Path header, even early parts of
it that recorded its progress through some private system before getting
injected into Usenet. It will do litle harm, and may help unravel some
complicated cases where things have gone wrong. That is why Usepro-06 said
SHOULD preserve the old Path during reinjection. Then you get to see the
full story of exactly where it has been.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Mon, 22 Jan 2007 12:18:36 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:
> Russ Allbery wrote:
>> Forrest J Cavalier <[email-masked]> writes:

>>> Here's a thought....

>>> USEFOR should have defined "Authoring-Date:", or some such thing, not
>>> "Injection-Date:"  This would be completely backwards compatible, and
>>> put the work to resolve it on reading agents.

>> Yeah, that was my fourth proposal.  It's possible to argue that it doesn't
>> require any change to USEFOR.

> I didn't understand that as your fourth proposal.  I am saying that we
> don't modify servers at all.  UA can generate and look at Authoring-Date,
> and servers look at Date, just as they do now.

Oh!  Sorry, yes, that's something different.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Mon, 22 Jan 2007 20:59:34 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 16
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> Aren't you arguing a logical fallacy: "Good enough for me, so must be
> good enough for everyone."?

No, it's just an observation.  I got the numbers wrong, 3*365*5000 is
more like "not one mail to postmaster@ in 5,000,000", not only 500,000.

> Let 2142 pretend what it wants.  We don't need to go along.

Most likely we're going nowhere, and keep s-o-1036 with its newsmaster@
and usenet@.  S-o-1036 got all "principles of operation" right, so it's
not too bad.

Frank

Date: Mon, 22 Jan 2007 13:09:26 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:
> Forrest J Cavalier <[email-masked]> writes:
> 
> 
>>Here's a thought....
> 
> 
>>USEFOR should have defined "Authoring-Date:", or some such thing, not
>>"Injection-Date:"  This would be completely backwards compatible, and
>>put the work to resolve it on reading agents.
> 
> 
> Yeah, that was my fourth proposal.  It's possible to argue that it doesn't
> require any change to USEFOR.

I didn't understand that as your fourth proposal.  I am saying that we
don't modify servers at all.  UA can generate and look at Authoring-Date,
and servers look at Date, just as they do now.

I still want to see statistics about how often this late posting business
happens.  People shouldn't be posting stale articles to active discussions.
Most discussions would have moved on between when they authored and when they 
injected days later.  It is rude to think everything you write is timeless.

And if you read all the intervening messages before you late posted, just
to ensure that what you wrote days ago is still applicable, why should the
date header be a record of when you typed it, and not when you affirm the 
message to contain what you want to send?  It's splitting hairs.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Mon, 22 Jan 2007 09:18:52 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Here's a thought....

> USEFOR should have defined "Authoring-Date:", or some such thing, not
> "Injection-Date:"  This would be completely backwards compatible, and
> put the work to resolve it on reading agents.

Yeah, that was my fourth proposal.  It's possible to argue that it doesn't
require any change to USEFOR.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 22 Jan 2007 08:37:00 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:

>>>If we disallow late injection in general, both injection and
>>>reinjection, then we don't need Injection-Date.
> 
> 
>>Yes, but that's not realistic.
> 
> 
> It should be a warning sign for a train of logic if the thing that one
> declares not realistic is how all of Netnews currently works and what RFC
> 1036 requires.
> 
> It's realistic; Netnews has worked that way for decades.  The question is
> rather whether the benefits from changing it are worth the costs, and
> proceduraly, whether it's worth reopening the topic rather than working
> with what we've got now in USEFOR.

Why should we treat USEFOR as sacrosanct?  I mean we don't want to
descend into any more tarpits, but why ignore hindsight?

> 
> 
>>Users can poll their server, and then go offline for some time creating
>>articles with different days, waiting in their outbound for better
>>times.  Then they go online again and try to post their articles.
>>That's one scenario where Injection-Date could be better than Date.
> 
> 
> Or said off-line posting agent can rewrite Date when the message is
> actually injected.  That's what they have to do now.  You lose information
> for humans, but the protocol works.

Here's a thought....

USEFOR should have defined "Authoring-Date:", or some such thing, not 
"Injection-Date:"  This would be completely backwards compatible, and put the 
work to resolve it on reading agents.

Let's go back and fix USEFOR.

Date: Mon, 22 Jan 2007 08:30:44 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1093
Sender: [email-masked]

Frank Ellermann wrote:
> postmaster@, webmaster@, and abuse@ are among the local parts I consider
> as "potentially valid", therefore it won't directly go to a junk folder.
> 
> The spam to these addresses is very rare, I don't recall a single case
> of postmaster@ in the last 500,000 mails.  For webmaster@ it's not so
> good, but a smart harvester could find this address if it follows the
> <meta rev="made" href="http://putl.net/xyzzy/mailto/webmaster" /> links
> on my pages, and if it also knows the %40 trick.

Aren't you arguing a logical fallacy: "Good enough for me, so must be good
enough for everyone."?

Today's batch of email:
Checked: 1139 BCC: 250 NewUCE: 520 SpamCatch: 886 RepeatUCE: 395 Disposed: 915

At least 38 of those were to webmaster@.  Which to my knowledge appears
nowhere.  So as far as I know, it wasn't harvested.

Let 2142 pretend what it wants.  We don't need to go along.

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sun, 21 Jan 2007 13:53:36 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:
 
>> It should be a warning sign for a train of logic if the thing that one
>> declares not realistic is how all of Netnews currently works and what
>> RFC 1036 requires.

> The incompability of the 2822 Date and the s-o-1036 Date is obvious, is
> that the reason why you invented the new Injection-Date here ?

No.  (I thought you were here for the previous discussions?)  We invented
Injection-Date to solve a specific problem with off-line posts.  Avoiding
having to redefine an RFC 2822 header field was an additional advantage,
but wasn't the initial motivation.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date:  Sun, 21 Jan 2007 22:37:26 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 34
Sender: [email-masked]

Russ Allbery wrote:
 
> It should be a warning sign for a train of logic if the thing that one
> declares not realistic is how all of Netnews currently works and what 
> RFC 1036 requires.

The incompability of the 2822 Date and the s-o-1036 Date is obvious, is
that the reason why you invented the new Injection-Date here ?

  [s-o-1036 explanation of "history"]
>> If it's really not yet there USEPRO needs a similar section with the
>> same RECOMMENDED N = 7 days minimum.
 
> We already discussed this years ago and reached a working group 
> consensus to not document any specific retention time, but instead to
> document the restraint on retention time

The reason for N = 7 given in s-o-1036 is convincing, that's also long 
enough to tolerate plausible "late injections" by implementations of an
RFC 2822 Date.  Maybe it's unnecessary after a "transition period" for
the introduction of Injection-Date, but that's years away.

> people use all sorts of different retention times, ranging from three
> days to a year (and in some specialized cases, even shorter than three
> days -- I've used a retention time as short as a day before).

news.clara.net also tried shorter times for their binary groups, with
many unhappy users wanting their money back, because they didn't get all
pieces of the binaries.  I don't care much about attempts to distribute
complete CD images with NNTP.  But combined mail+news UAs implementing
the RFC 2822 Date instead of a "posting date" make sense.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Sun, 21 Jan 2007 22:04:24 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Russ Allbery wrote:

>> or a variation of this scheme in s-o-1036.

> Not a standard at all.

| A draft update to RFC 1036 ( "Son of RFC 1036" ) was released by Henry
| Spencer in June 1994 but this was not further pursued and is now itself
| out of date. Currently a combination of this and RFC 1036 are regarded
| as the de-facto standard.
         ^^^^^^^^^^^^^^^^^
S-o-1036 has the "how stuff works" clear, even if some of its ideas like
See-Also didn't make it.

> Please don't mix the protocol standard for Netnews with a standard for
> e-mail addresses.

count ,mail, all => 138 occurences in 133 lines (of s-o-1036)

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1093
Organization: The Eyrie
Date: Sun, 21 Jan 2007 09:27:10 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> postmaster@, webmaster@, and abuse@ are among the local parts I consider
> as "potentially valid", therefore it won't directly go to a junk folder.

> The spam to these addresses is very rare, I don't recall a single case
> of postmaster@ in the last 500,000 mails.  For webmaster@ it's not so
> good, but a smart harvester could find this address if it follows the
> <meta rev="made" href="http://putl.net/xyzzy/mailto/webmaster" /> links
> on my pages, and if it also knows the %40 trick.

Lucky you.

I'm [email-masked] as well as postmaster at quite a few other
systems here, and I can assure you that from where I'm sitting, you look
like an exception.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sun, 21 Jan 2007 09:25:39 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> if we want to use the normal arrival-based expiration, we need to
>> require it somewhere.

>> If we disallow late injection in general, both injection and
>> reinjection, then we don't need Injection-Date.

> Yes, but that's not realistic.

It should be a warning sign for a train of logic if the thing that one
declares not realistic is how all of Netnews currently works and what RFC
1036 requires.

It's realistic; Netnews has worked that way for decades.  The question is
rather whether the benefits from changing it are worth the costs, and
proceduraly, whether it's worth reopening the topic rather than working
with what we've got now in USEFOR.

> Users can poll their server, and then go offline for some time creating
> articles with different days, waiting in their outbound for better
> times.  Then they go online again and try to post their articles.
> That's one scenario where Injection-Date could be better than Date.

Or said off-line posting agent can rewrite Date when the message is
actually injected.  That's what they have to do now.  You lose information
for humans, but the protocol works.

> Either I'm too clumsy to get this right, or both USEPRO drafts don't
> discuss the function of a history file yet.  S-o-1036 9.2 explains how
> that's supposed to work, proposing a minimum of N = 7 days for "seen",
> with a Date older than now - N considered as "stale".

> If it's really not yet there USEPRO needs a similar section with the
> same RECOMMENDED N = 7 days minimum.

We already discussed this years ago and reached a working group consensus
to not document any specific retention time, but instead to document the
restraint on retention time (namely that you have to reject any article
with a Date older than your retention time).  I'd prefer not to revisit
that decision again.

In practice, people use all sorts of different retention times, ranging
from three days to a year (and in some specialized cases, even shorter
than three days -- I've used a retention time as short as a day before).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Date:  Sun, 21 Jan 2007 09:57:31 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 97
Sender: [email-masked]

Russ Allbery wrote:

> if we want to use the normal arrival-based expiration, we need to
> require it somewhere.

Or allow it somewhere.  I don't get the difference if articles older
than "X" determined by a header field are anyway rejected as too old.

> For example, we're theoretically breaking our uniqueness model with
> multiple injection a minute apart.  This opens the door to duplicates.
> However, to get a duplicate in practice, the later copy of the article
> would have to arrive less than a minute after the earlier copy had
> expired, something that in practice is hideously unlikely.

Hitting one critical minute of 4321 for 4320=3*24*60 isn't impossible,
but finding a route to this server needing 3 days might be difficult.

> If we disallow late injection in general, both injection and reinjection,
> then we don't need Injection-Date.

Yes, but that's not realistic.  Users can poll their server, and then go
offline for some time creating articles with different days, waiting in
their outbound for better times.  Then they go online again and try to
post their articles.  That's one scenario where Injection-Date could be
better than Date.

Another scenario is a xyz2news gateway where the xyz comes with a Date,
but may take some time (fido2news, mail2news).  It's then desirable to 
preserve the original Date.  Unless there's another xyz2news gateway
injecting the same article with a very different earlier Injection-Date.

Either I'm too clumsy to get this right, or both USEPRO drafts don't
discuss the function of a history file yet.  S-o-1036 9.2 explains how
that's supposed to work, proposing a minimum of N = 7 days for "seen",
with a Date older than now - N considered as "stale".

If it's really not yet there USEPRO needs a similar section with the
same RECOMMENDED N = 7 days minimum.

> I think Injection-Date introduces more problems than it solves and
> the complexity isn't worth it.

The reinjection part was somewhat odd, I never got it.  Based on your
explanation I finally begin to understand the USEPRO-06 idea.  Your
proposal to treat reinjection as special case of gatewaying offered a
way to ignore these oddities.

But the underlying concepts (history + dupes + late injection) have
to be clear.  Going back to no Injection-Date doesn't help for late
injections, and that's no irrelevant corner case like reinjection.

>> How many posters per day on Usenet will be posting days later than
>> authoring?  

Poll Friday, write Saturday, and inject Monday could be a plausible
scenario.  The USEFOR decision to stick to RFC 2822 where that's at
all possible was intentional.  2822 defines:

 The origination date specifies the date and time at which the creator
 of the message indicated that the message was complete and ready to
 enter the mail delivery system.  For instance, this might be the time
 that a user pushes the "send" or "submit" button in an application
 program.  In any case, it is specifically not intended to convey the
 time that the message is actually transported, but rather the time at
 which the human or other creator of the message has put the message
 into its final form, ready for transport.  (For example, a portable
 computer user who is not connected to a network might queue a message
 for delivery.  The origination date is intended to contain the date
 and time that the user queued the message, not the time when the user
 connected to the network to send the message.)

However s-o-1036 states:

 The Date header contains the date and time when the article was 
 submitted for transmission

That's obviously incompatible, and the reason why USEFOR adds the new
beast Injection-Date for the function related to history files.  Just
removing Injection-Date would also cause havoc for combined UAs (mail
or news), implementors can't get incompatible Date semantics right...
or maybe they could, but they won't bother, and that's the problem.
 
> I think that if we're introducing Injection-Date, we need to provide
> a transition during which it's clear that Injection-Date still isn't
> going to be a panacea, and in that transition period, I don't think
> we're doing posters any favors accepting articles that most people
> will keep ignoring.

If injecting agents behave like relaying agents, and use the Date as
surrogate for the missing Injection-Date, then they'll reject a Date
older than N = 7 days.  That should work to some degree.  Or there's
a smaller window, less than N days, for this purpose.  Declaring when
the transition period is finished is the job for another draft in the
late 40s of this century.  Maybe 2048+, then it's in sync with BCP 18.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Sun, 21 Jan 2007 08:00:24 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Russ Allbery wrote:
 
>> Except that "[email-masked]" is not likely to be visibly
>> plastered all over Usenet and the Web, ready to be scraped by spammers
>> (as "[email-masked]" might well be).
 
> Right, that's why postmaster so rarely gets spam.
> (In case it's not obvious, I'm being extremely sarcastic.)

postmaster@, webmaster@, and abuse@ are among the local parts I consider
as "potentially valid", therefore it won't directly go to a junk folder.

The spam to these addresses is very rare, I don't recall a single case
of postmaster@ in the last 500,000 mails.  For webmaster@ it's not so
good, but a smart harvester could find this address if it follows the
<meta rev="made" href="http://putl.net/xyzzy/mailto/webmaster" /> links
on my pages, and if it also knows the %40 trick.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sat, 20 Jan 2007 21:41:42 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:
> Russ Allbery wrote:

>> Forrest has proposed only allowing reinjection between disjoint networks.

> Well, no.  I was coerced.  My original text said little.  Then I upped
> it to handle leaf nodes.  Then I was coerced to consider the disjoint
> network multi-injection.  This WG is a broadening experience, doncha
> know.

*heh*.  Sorry if I mis-stated there!  :)

> If you are a private leaf node, you know you want to reinject articles
> posted locally and no others.  In this case, you KNOW articles did not
> pass through A nor B, because they originated at your node and you
> haven't relayed them yet, since you don't relay.  QED.

> You can implement this many different ways.  I don't think we need to
> standardize that implementation.

That makes sense.  Basically, if we say that you're only allowed to do
reinjection that changes the identity of the article (dropping
Injection-Date) when you *know* that the networks are disjoint, and leave
the problem of knowing that to the implementation, it handles the simple
leaf-node case while leaving the question of how to determine this in more
complicated cases to the individual.

We can still get problems if that knowledge is wrong, of course.

> Let those with other cases read Duties of Gateways.  They don't get a
> standard formula, because this WG doesn't have a consensus about their
> needs and prevalance, and the Injection-Date scheme that was proposed is
> now consensed to be faulty (although different WG members think it is
> flawed for different reasons.)

There is that -- the Gateways section allows for Netnews-to-Netnews
gateways with more general descriptions of what sorts of care one has to
use, and is available as a fallback description of what has to be done for
weird and difficult cases.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sat, 20 Jan 2007 21:36:57 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Russ, I think your lengthy analysis is accurate, except for two points.

> 1. The article identity (for preventing duplicates) is not actually a
> 2-tuple of Message-ID, Date.  It is Message-ID + fuzzy arrival date.
> Every date within the past 3 days is probably as good as any other date
> in that range.  The header fields do not always set the roll-off from
> message-id history.  It is arrival order.  That changes your conclusions
> a bit too, I think.

> Overconstraining the solutions to guaranteeing (Message, Date) is not
> strictly necessary, and I think you throw out some potentially rewarding
> arrangements.

This is, of course, entirely correct, and as soon as I read this, I went
"doh, yes."  However, after poking at this idea for a few minutes, I
couldn't come up with an effect of the distinction that would change the
conclusions.  Could you elaborate a bit more?  Having history expiration
be by arrival rather than by header means that in practice servers will
keep a longer history than strictly necessary and could check some older
articles, but if the Date header (by whatever name) is stale, the article
still loses.

(Plus, I think it's actually conforming to our current specification to
write a news server that expires history entries based on the
Injection-Date header of the article rather than by arrival time.  INN
optionally can do this, and I think such an implementation still fulfills
the requirements of the standard.  That, of course, doesn't change your
point, but does mean that if we want to use the normal arrival-based
expiration, we need to require it somewhere.)

> 2. I think you present all branches of your fault tree as if they were
> all equally likely in practice.  I disagree with that approach when we
> admit we don't have a solution for all cases for all people.

Yeah, that was partly a side effect of trying to look at this at a
theoretical level rather than analyzing practical examples.  I'm worried
that we're not agreeing on the overall theoretical analysis structure
before discussing what we can do for individual cases.

It's true that not all of these problems are equal.  For example, we're
theoretically breaking our uniqueness model with multiple injection a
minute apart.  This opens the door to duplicates.  However, to get a
duplicate in practice, the later copy of the article would have to arrive
less than a minute after the earlier copy had expired, something that in
practice is hideously unlikely.

> I prefer to take the most likely of situations, standardize something
> that works, and then ignore the least likely branches.  They fall
> outside our standard, or they can go look at Duties of a Gateway and
> figure out something.

> If we disallow "late (re)injection", don't most of the problems go away?

If we disallow late injection in general, both injection and reinjection,
then we don't need Injection-Date.  If we can drop Injection-Date, we get
back the current Netnews protocol, which we know works and which we can
make statements about based on current practice.  In this case, we can add
a requirement that posting agents doing multiple injection always provide
the same Message-ID and Date header fields in all copies and that
reinjection always preserves both of those headers, and everything works
reliably and without duplicates except for late injection and reinjection.

We may or may not want to say something about how to work around late
reinjection, or we could just leave it to the gateways section for someone
who really knows what they're doing.

This, however, reverses a decision made in USEFOR, which we were treating
as sacrosanct, which is why I was trying not to push that route.  (It's
been my preference for a while.  I think Injection-Date introduces more
problems than it solves and the complexity isn't worth it.)

> Drop late injection.  It almost requires a flag day, not because we get
> duplicates (we would), but because the contents of a newsgroup is going
> to be different at different sites, depending on whether they
> implemented expiry/incoming relay on Injection Date, or Date.  And isn't
> that is the problem that it was intending to prevent?  If an article is
> re-injected late, we have the amusing situation of an article appearing,
> not appearing, and appearing as a duplicate on various servers.

> And all this complexity about late injections for what?  A handful of
> posters per day on Usenet?  Hours later is not going to matter.    How
> many posters per day on Usenet will be posting days later than authoring?
> Does anyone have an estimate?

> Aren't we knocking ourselves out over mere annoyance?  Just tell those
> posters that their message had better have a current Date: header if
> they want it to propagate, (that's reality, and telling them different
> is not helpful) and if a late Date header seems like deception, they can
> note it in the body of their message. If they want a sooner expiry, add
> a header field.  Voila!  Backwards compatible too.

This is basically the line of reasoning under why I put Date staleness
checking back in for injecting agents.  I think that if we're introducing
Injection-Date, we need to provide a transition during which it's clear
that Injection-Date still isn't going to be a panacea, and in that
transition period, I don't think we're doing posters any favors accepting
articles that most people will keep ignoring.  It may be that checking at
the injecting agent so that we can return an error message isn't the best
transitional plan, but it feels better to me than not giving people any
warning at all.

One point on which I think Charles is definitely right, though, is that
Injection-Date is a different thing than Injection-Info and the other
non-proto-article headers like Xref.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Sun, 21 Jan 2007 00:00:15 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:
> Forrest has proposed only allowing reinjection between disjoint networks.

Well, no.  I was coerced.  My original text said little.  Then I upped it
to handle leaf nodes.  Then I was coerced to consider the disjoint network 
multi-injection.  This WG is a broadening experience, doncha know.

> If the reinjecting agent is the only possible path between two otherwise
> disjoint networks, then the reinjecting agent can do article identity
> verification for the second network and it doesn't matter if the article
> acquires a new identity when crossing that network boundary.  The
> difficulty here, as Charles correctly points out, is that there's no
> general way of establishing that two networks are disjoint.  One can
> sometimes determine for individual articles that they passed through a
> host on network A and on network B and therefore the networks are not
> disjoint, but to be sure would require full knowledge of at least one of
> the networks.  There are some common cases where this is possible, but
> it's not possible in general, and it's possible to think that you know the
> networks are disjoint and be wrong.

I don't think we can solve the general case.  Are we going to try to standardize 
"Daves the resurrectors"?  Yikes!

I fully admit I provided text that applied for the narrow case, but I think it
is the extremely most common case.

If you are a private leaf node, you know you want to reinject articles posted 
locally and no others.  In this case, you KNOW articles did not pass through A 
nor B, because they originated at your node and you haven't relayed them yet,
since you don't relay.  QED.

You can implement this many different ways.  I don't think we need to 
standardize that implementation.

Sure, I understand there are people who will want to reinject who are not
private leaf nodes.  They are not my concern unless they can fit the description 
in the text we provide.

Let those with other cases read Duties of Gateways.  They don't get
a standard formula, because this WG doesn't have a consensus about
their needs and prevalance, and the Injection-Date scheme that was
proposed is now consensed to be faulty (although different WG members
think it is flawed for different reasons.)

Date: Sat, 20 Jan 2007 23:42:42 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:
>   I think it's the specific use cases that are mixing
> things up.

I think that indicates that no matter what we do, someone is going
to screw it up sometime, somewhere.

Russ, I think your lengthy analysis is accurate, except for two points.

1. The article identity (for preventing duplicates) is not actually a 2-tuple
of Message-ID, Date.  It is Message-ID + fuzzy arrival date.  Every date within
the past 3 days is probably as good as any other date in that range.  The
header fields do not always set the roll-off from message-id history.  It is 
arrival order.  That changes your conclusions a bit too, I think.

Overconstraining the solutions to guaranteeing (Message, Date) is not
strictly necessary, and I think you throw out some potentially rewarding
arrangements.

2. I think you present all branches of your fault tree as if they were all
equally likely in practice.  I disagree with that approach when we
admit we don't have a solution for all cases for all people.

I prefer to take the most likely of situations, standardize something that 
works, and then ignore the least likely branches.  They fall outside our
standard, or they can go look at Duties of a Gateway and figure out something.

If we disallow "late (re)injection", don't most of the problems go away?

Sure, I'd like late reinjection to work too, but it doesn't happen enough to 
work to standardize.  Let some future standard tackle it.

So, after all this time, no one seems to have proposed something that we have 
consensus that works.  Too bad.  It isn't our problem that the people who 
proposed Injection-Date as a solution to this problem didn't get it right.  If 
they want it, they can propose something that gets consensus.  I don't care to
fix it for them.

My opinion: Standardize reinjection from private leaf nodes (it's common).
Standardize multi-point injection.

Drop late injection.  It almost requires a flag day, not because we get
duplicates (we would), but because the contents of a newsgroup is going to be 
different at different sites, depending on whether they implemented 
expiry/incoming relay on Injection Date, or Date.  And isn't that is the problem 
that it was intending to prevent?  If an article is re-injected late, we have
the amusing situation of an article appearing, not appearing, and appearing
as a duplicate on various servers.

And all this complexity about late injections for what?  A handful of posters 
per day on Usenet?  Hours later is not going to matter.    How many posters per 
day on Usenet will be posting days later than authoring?  Does anyone have an 
estimate?

Aren't we knocking ourselves out over mere annoyance?  Just tell those posters 
that their message had better have a current Date: header if they want it to 
propagate, (that's reality, and telling them different is not helpful) and if a 
late Date header seems like deception, they can note it in the body of their 
message. If they want a sooner expiry, add a header field.  Voila!  Backwards
compatible too.

Date: Sat, 20 Jan 2007 20:03:17 -0500
From: Dan Schlitt <[email-masked]>
cc: [email-masked]
Subject: Re: #1093
Sender: [email-masked]

Judging from the "Unknown Users" list I see in my log files I conclude
that the spammers have a list of likely users that they use. They don't
have any relation to addresses that they find out in the Internet. Some
even look like variations that might be used by some places to avoid
spam to more obvious users.

One mailbox or three, it doesn't seem like something that merits
spending time on. Is it a protocol issue? Does it need to be
standardized? No and no.

/dan

-- 

Dan Schlitt
[email-masked]

On Sat, 20 Jan 2007, Russ Allbery wrote:

>
> Charles Lindsey <[email-masked]> writes:
>
> > Except that "[email-masked]" is not likely to be visibly
> > plastered all over Usenet and the Web, ready to be scraped by spammers
> > (as "[email-masked]" might well be).
>
> Right, that's why postmaster so rarely gets spam.
>
> (In case it's not obvious, I'm being extremely sarcastic.)
>
> > So if a spammer sends to [email-masked] he must have read RFC
> > 2142, and then looked at all the domains he found in Path headers and in
> > Injection-Info and so tried putting "news@" in front of them.
>
> No, they just send mail to news@ every single host that they run across in
> any context.  Why bother going to the work of limiting it down?
>
>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sat, 20 Jan 2007 16:57:15 -0800
Sender: [email-masked]

Okay, I took a long walk to think about this (which I should have done
before sending the last message -- I'm sorry, Charles, I'm confused by all
the specific examples but after thinking about this on a walk, I can see
the general drive of what you're getting at).  Let's take a step back and
look at the theory.  I think it's the specific use cases that are mixing
things up.

First, let's go back to the Netnews protocol as it exists today and look
at how injection and reinjection work without Injection-Date.

Every article has a unique identity in its message identifier, but we
don't expect agents to retain a database of every message identifier.
Therefore, in practice, articles are uniquely identified by the
combination of message identifier and Date header field.  If the Date
header field is recent, we can rely on a database lookup for the message
identifier to know if we've seen this article before.  If the Date header
field is old, we may not know whether we've seen this article before or
not, so we assume we have.  Therefore, an article's functional identity is
the Date and message identifier pair.  Uniquely identifying an article is
how we prevent duplicating that article on the network.

Now, in the current Netnews protocol, an injecting agent is responsible
for ensuring that every article has a unique identity by adding
Message-ID and Date header fields if they're not present in a
proto-article.  However, the posting agent MAY assert the identity of the
article by filling out the Message-ID and Date header fields in advance,
in which case the injecting agent MUST use the existing identity of the
article.

In any case in which multiple injection happens, either in parallel
(submitting the same proto-article to multiple injecting agents) or in
serial (reinjection), the identity of the article must be preserved to
avoid the possibility of duplicating it on the network.  So long as the
identity of the article is preserved, every agent on the network can
independently be assured that it won't accept a duplicate of an article.
This independent verification is important because it allows for multiple
independent paths of propagation.

Note that for proto-articles, this guarantee only holds if the posting
agent provides the identity of the article.  Therefore, in the existing
Netnews protocol, it's best practice for a posting agent injecting an
article at multiple injection agents to provide the complete identity of
the article (both Message-ID and Date header fields) so that every copy of
the article has the same identity and no agent will accept multiple copies
of it.  Reinjection similarly can be assured of not creating loops simply
by preserving the identity of the article (both Message-ID and Date header
fields) when reinjecting.

Note, however, that this means that the full propagation of a message is
constrained to only those sites that it can reach prior to the expiration
of its Date header field.  So currently, reinjection breaks down if the
reinjection cannot happen within the staleness period of the Date header
(and it breaks down differently at different sites, so some sites in a
network may accept a delayed reinjected article and others may reject it).
This is the same as for relaying.  The only difference is that reinjecting
agents are generally the sole path to reach a particular site, so if the
reinjection is delayed, articles go missing.  Because of this, some
reinjecting agents change the identity of the message by regenerating the
Date header.  More on the risks of that in a moment.

Now, the problem with this is that the Date header field carries two
meanings.  It's part of the identity of the article, and it's also
human-readable information about when the message was composed.  Some
posting agents always treat it as the former and either let the injecting
agent generate it or only generate it at injection time.  Some posting
agents treat it as the latter and generate it at message composition time.
This means that proto-articles that are held for a time before injection
will either lose information about when the message was composed or will
run the risk of being rejected by servers because their Date header field
is stale.

The working group therefore introduced a new header, Injection-Date, which
serves *only* the protocol function and separates the protocol function
from the user presentation function.  Date is now of interest only to
humans and contains the composition time, which serves no protocol
function.  Injection-Date is part of the identity of the message.

However, as part of this change, we also changed who was responsible for
establishing the identity of a message.  Now, the injecting agent *always*
establishes the identity of a proto-article, and the posting agent isn't
permitted to do so.  Only when a proto-article is injected does it acquire
a unique identity in practice; prior to that, it will only have a message
identifier, which is not sufficient alone to prevent duplication of the
message.

This isn't a problem only for reinjection.  It's a problem for every case
where an article is touched by multiple injecting agents.  In particular,
since the introduction of Injection-Date, a proto-article that's injected
at multiple injecting agents will be assigned a slightly different
identity by each one.  In the normal case, these identities will only vary
by seconds or at most minutes, which in practice is highly unlikely to
cause problems, but theoretically we still broke the identity model.
Since the introduction of Injection-Date, time-delayed multiple injection
is no longer safe against duplication.  If a posting agent injects an
proto-article at one injecting agent immediately and then at another
injecting agent some days later, it may introduce a duplicate article into
the network, and there's nothing the posting agent can do except not do
time-delayed multiple injection to prevent this.  Prior to Injection-Date,
it could make sure that all copies had the same identity and the later
injection would never duplicate, just possibly suffer from poor
propagation because it's stale.

Of course, serial multiple injection, reinjection, suffers the most, since
it is the most likely to happen some time later.  So in usepro-06,
reinjection was distinguished from injection and only in the reinjection
case (as determined by the injecting agent) was the article permitted to
retain its original Injection-Date.  In other words, normal posting agents
were still not permitted to assert the complete article identity, but a
special type of posting agent was if an injecting agent chose to let it.

So, we have some competing goals:

 * We want every copy of an article to have the same identity so that
   duplicates can be effectively suppressed at any node of the network
   without requiring reliance on prior nodes to do verification in any
   particular way.  This is robust against multiple propagation paths.

 * We can't rely on the Date provided by a posting agent to be part of
   the article's identity because then we'll give some articles too old
   of a date and fail to propagate them.  In other words, we have to
   treat a Date header field provided by a posting agent as a comment
   rather than a protocol element.

 * We want to support passing the same proto-article to multiple injecting
   agents and reprocessing an article through an injecting agent because
   both are useful in practice in some specific situations and both have
   historically been supported by Netnews.

So far, we have a couple of different solutions.

 * The current draft, my approach, basically takes the stance of "well, we
   asked for it, so we take the consequences."  Multiply injected articles
   simply have different identities -- that's the consequence of how we
   defined Injection-Date.  Each time an article passes through an
   injecting agent, its identity changes, and in the case where it's most
   likely that identity change will be serious (reinjection), the agent
   doing the reinjection is required to verify as well as possible that it
   won't create the chance of duplicates.  In practice, the drawbacks are
   hopefully limited.  There's no way for that check to be perfect, but
   hopefully it's no worse than the other case of multiple injection that
   we're already dealing with.  As a nice side effect, this also means
   that delays in reinjection don't cause articles to be lost.

 * usepro-06 makes reinjection a special case with a different set of
   rules that require that the identity of the article be preserved.  In
   other to do this, negotiation between the posting agent and the
   injecting agent is required and software has to know whether it's doing
   injection or reinjection.  There are some follow-on effects for trace
   headers, but this could be separated out and handled differently.  The
   core point is that reinjection preserves article identity, but in the
   process maintains the current drawback that delays in reinjection may
   cause articles to be lost.  There may be a bit of an attractive
   nuisance here, the same one that's present in the existing protocol,
   for reinjecting agents to drop the Injection-Date header anyway so as
   to not lose articles, thereby reintroducing the problems the current
   draft causes.

Forrest has proposed only allowing reinjection between disjoint networks.
If the reinjecting agent is the only possible path between two otherwise
disjoint networks, then the reinjecting agent can do article identity
verification for the second network and it doesn't matter if the article
acquires a new identity when crossing that network boundary.  The
difficulty here, as Charles correctly points out, is that there's no
general way of establishing that two networks are disjoint.  One can
sometimes determine for individual articles that they passed through a
host on network A and on network B and therefore the networks are not
disjoint, but to be sure would require full knowledge of at least one of
the networks.  There are some common cases where this is possible, but
it's not possible in general, and it's possible to think that you know the
networks are disjoint and be wrong.

There is another solution that require modifying USEFOR:

 * Drop Injection-Date entirely and go back to the current protocol.  This
   solves this problem and reintroduces the problem that we were trying to
   fix originally.

There is another solution that arguably requires modifying USEFOR:

 * Change the definition of the Injection-Date header field so that the
   posting agent can provide it, and indeed MUST provide it if they wish
   Date to be treated as a comment rather than a protocol element.
   Require posting agents doing multiple injection to include either a
   Date or (preferrably) a Posting-Date header.  Don't allow injecting
   agents to set Injection-Date at all if Date was provided.  This has the
   advantage of solving the multiple injection problem (present in both my
   draft and in usepro-06), and allows reinjecting agents to assert the
   same article identity in the same way as is possible with the current
   protocol.  Basically, this would make Injection-Date *entirely* the
   same as the protocol purpose of the current Date header, rather than
   mostly the same.  It has the same difficulties with delayed
   reinjection.

One could make the argument that we could do this now, that nothing in the
current USEFOR definition says that the posting agent can't provide this
header.  The current text implies it, but doesn't say so outright.  The
header name is confusing if the posting agent is providing it, but we
could live with that.

The drawback of this approach, of course, is that it still doesn't deal
with the issue of delayed reinjection.  Some reinjecting agents are going
to want to change the identity of a message so that it will still be
accepted even though it's old.  Doing so inherently runs the risk of
creating duplicates, but doing so is going to be a common desire --
gateways go down or off-line for a while, networks go out, people want to
pull down older news to bootstrap or seed a new Netnews network, etc.  If
we just outlaw it, we have no control over how people do it if they choose
to break the protocol.  If we can come up with something useful to say
about *how* to change the identity of the message, maybe we can reduce the
risk.

That's the summary of the current situation as I see it.  I think it's the
most complex problem we have left to resolve before we have a publishable
draft.  Every approach to this problem that I can see introduces risk
somewhere, including sticking with the current protocol and thereby
tacitly encouraging people to drop the Date header when they need to.  My
approach in the current draft is a mess, but so is every other approach
that I can think of.  It doesn't, however, deal with multiple injection,
which would be valuable.

If we want to pursue the last option mentioned above, namely requiring the
posting agent to provide Injection-Date if it wants Date to be ignored
(ideally renaming and slightly redefining the header in the process), I
can propose some text, but we still need to decide what, if anything, we
want to say about delayed reinjection.  I think this route is cleaner than
my current draft, and it takes an important step back towards Charles's
draft in that we would then require Injection-Date not be changed by the
injecting agent if already present (modulo whatever we wanted to say about
intentionally changing article identity).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Organization: The Eyrie
Date: Sat, 20 Jan 2007 14:35:54 -0800
Sender: [email-masked]

I'm going to bow out of any further discussion of Charles's proposal for
how reinjection should be dealt with unless someone else weighs in and
supports it in a way that I can make head or tails of.  I think we're now
at the point where posting agents are sending messages directly to serving
agents, which is so confused that I can't even work out what model of
Netnews is being applied.

So far, other than the changes and clarifications from Forrest's proposal
(which I want to continue discussing and which I think is roughly in the
same spirit as my text but more rigorous), I haven't seen anything
proposed which in my opinion is technically better than what's currently
in the draft.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1093
Organization: The Eyrie
Date: Sat, 20 Jan 2007 14:27:14 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> That's the status quo wrt news@ and usenet@ in RFC 2142,

A standard that I feel no particular urge to link to our work.  It can
hang out there and succeed or fail on its own merits.  There's no
compelling need for us to get involved.  If you want to propose a
follow-on to RFC 2412 that addresses flaws in that standard, there's a
whole IETF process to do so and this isn't the working group that would
handle such work.

> or a variation of this scheme in s-o-1036.

Not a standard at all.

Please don't mix the protocol standard for Netnews with a standard for
e-mail addresses.  Those are two separate protocols and there's no need to
link them.

Anyway, this is a pointless conversation; everyone already knows what I
think.  If other folks than Frank and Charles want to chime in, you
probably should; otherwise, I intend to ignore this issue until the WG
chairs discuss resolution.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1093
Organization: The Eyrie
Date: Sat, 20 Jan 2007 14:20:56 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Except that "[email-masked]" is not likely to be visibly
> plastered all over Usenet and the Web, ready to be scraped by spammers
> (as "[email-masked]" might well be).

Right, that's why postmaster so rarely gets spam.

(In case it's not obvious, I'm being extremely sarcastic.)

> So if a spammer sends to [email-masked] he must have read RFC
> 2142, and then looked at all the domains he found in Path headers and in
> Injection-Info and so tried putting "news@" in front of them.

No, they just send mail to news@ every single host that they run across in
any context.  Why bother going to the work of limiting it down?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Sat, 20 Jan 2007 21:56:57 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 39
Sender: [email-masked]

Russ Allbery wrote:

>> "a mailbox news@<path-identity> SHOULD exist for the most commonly
>>  used <path-identity> of a news server".
 
> I don't think "if you run a Netnews server, you have to be spammable"
> is really going to fly in the real world.

That's the status quo wrt news@ and usenet@ in RFC 2142, or a variation
of this scheme in s-o-1036.  With "my" catchall vanity domain I get tons
of spam (several thousands daily), and role accounts like postmaster@
don't get much of this flood.

Most spam local parts are based on something that I've actually used,
e.g. bugzilla@ or the RHS of Message-IDs, or wild guesses like "joe",
or pieces of Message-IDs.  Apparently spammers stay to some degree 
away from abuse@, but otherwise they don't care.  I could modify my
script to create a statistics for some days if you're interested in
the bloody details wrt news@ + usenet@ + newsmaster@

How admins with role accounts or similar constructs (info@, nobody@)
solve their spam problem is irrelevant for this WG, it affects all
mail users and all domains with an MX alike.

What's relevant is to pick a single default instead of the two or
three today.  No default at all makes no sense - if somebody is
determined to get no mails they publish no MX, or reject mails to 
undesired local parts, or forward it to /dev/nul.

News admins can't read the admin groups of hundreds of TLHs.  They
need a way to contact them in the case of problems requiring manual
intervention.  Some socipathic admins could decide that existing
abuse@ or Tech-C whois-addresses are good enough for this purpose,
but I don't want a standards track document for sociopaths.

If USEPRO-07 stays worse than s-o-1036 it shouldn't be published.

Frank

To: [email-masked]
Xref: clerew local.usefor:24228
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date (was: Re: ISSUE: Reinjection and Injection-Date)
Date: Fri, 19 Jan 2007 21:56:15 GMT
Lines: 152
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> There are essentially two options:

>> Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

>> Option 2: Injection-Date headers, once written, are NEVER altered.

>Just to try to be painfully clear here, Injection-Date using the method
>laid out in the current USEPRO draft is *not* rewritten by injecting
>agents.  The whole point of that option is that only the gateways who are
>doing the reinjection make this transformation.

Indeed, whether it is the injecting agent that rewrites the header, or
whether it simply creates a new header because the
poster/gayewayer/reinjector had removed the old one, makes little
difference, since the observed effect is the same.

>> Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

>> We are concerned with an article that is being reinjected. ...

>Lack of safety can only occur if the networks are not disjoint *and* there
>is a delay in gatewaying longer than the history retention of whatever
>path causes the networks to not be disjoint (whether a relaying agent or
>another gateway) but shorter than the staleness check in the reinjecting
>gateway (*and* no staleness checks are applied to the Date header field,
>but we can all agree on that as a long-term goal).

Agreed, which is why you need something rather odd, like the Anytown
Squash Club, that creates a longish delay for things to go wrong. And to
cause an actual loop you need two separate longish delays somewhere
(though that is not impossible, just requires something odder than that
Squash Club).

>> But the real problems arise when FIRST is injected in a small private
>> network (even a private news server maintained by an individual poster).
>> .... Such small private neworks
>> are notorious for "leaking" (and it is not necessarily the "owner" of
>> the network who will be the REINJECTOR that perpetrates the leak).

>Note that *relaying* leaks from such private networks are quite rare
>(although not non-existent).  Leaks are almost always via reinjection
>(suck/rpost feeds).

And usually caused by someone far removed from the first injector, so the
two of them are unknown to each other, and neither of them believes he is
doing anything out of the ordinary. The problem, in essence, is that I do
not believe in this concept of "disjoint networks", since it is almost
impossible to prove that two networks are truly disjoint.

>> Option 2: Injection-Date headers, once written, are NEVER altered.

>> This clearly removes the risks inherent in Option 1. It is inherently
>> much safer, and it is 'fail-safe' in the sense that the only thing that
>> can go wrong is that an article whose propagation was genuinely delayed
>> at some point might fail to propagate beyond a relaying agent with a
>> short history file expiry. But better to lose a few articles than run
>> the risk of loops.

>As you note, this approach makes it impossible for a gateway that is down
>for longer than the retention period of its most immediate upstream
>relaying agent to ever go back and gate the articles that it missed during
>its period of downtime.

Right. And that is the problem we need to fix. It is only really fixable
if the gateway that is down can be Absolutely Sure that he is leak-proof
(e.g. he is a stand-alone server). In that case, it is perfectly
reasonable for him to rewrite the Injection-Date when he come back online,
or to remove it and let the injecting agent create a new one. The only
question is whether and how to write that into the draft as a "truly
exceptional" situation.

>This is exactly the sort of problem that caused us to add Injection-Date
>in the first place.

Yes, it is the man who carries his prepared article, or even a complete
private server, around on his laptop. But I still want the Date header to
reflect the time of composition, since that is what is clearly stated by
RFC 2822, which we are following in this regard (that was also part of the
reason to add Injection-Date).

> If this is acceptable, I move that we drop
>Injection-Date entirely and go back to the well-understood and
>already-implemented staleness checking on Date and make people who hold
>posts for a while work around this in their posting agent.  It would save
>a ton of complexity.

No, because whatever scenario we come up with that causes problems with
Injection-Date, you can write a similar scenarion that breaks with Date.

>> But since B wants the articles for its own use (not for relaying back
>> into Usenet), it is presumably a serving agent, in which case its expiry
>> time is likely to be more than a week, in which case the staleness check
>> is unlikely to be triggered. But even if that is not so, this is a case
>> where all B's admin needs to do is to temporarily increase the expiry
>> time of his history file. This is always safe; even if some site on his
>> local network tries to relay an article back into Usenet, it is still
>> protected by its _original_ Injection-Date.

>No, this is *unsafe* if that host is part of a Netnews network because it
>means that articles that the relaying agent had already accepted it may
>accept again because it now thinks that its staleness cutoff is longer
>than it really was and will therefore think that articles that it had
>expired out of its history would be present in history had it already seen
>it.

Not so. This is not a relaying agent we are talking about (if a relaying
agent goes down for some length of time, then articles will still flood
around it, albeit perhaps more slowly).

We are talking about a serving agent that has been down for some period,
and has therefore received no articles during that time. Therefore, the
only articles that will get accepted twice are those that were in transit
at or around the time of the breakdown, such that there is some
uncertainty about exactly which articles were missed, and for safety it
might have asked for a few more than it actually needed (e.g. by a
slightly pessimistic date-time in a NEWNEWS).

And it is only his direct clients who will see those few messages again.
If, having accepted articles that he has already seen before, he then
tries to relay them out to the rest of Usenet, they will still be
protected by their original Injection-Date. So any damage/inconvenience
will be restricted to himself and his immediate clients. That seems a
reasonable price to pay after such a major disruption.

>Please remember that one of the very common cases of suck/rpost feeds is
>*from* a standalone server *to* Usenet, so assuming that the destination
>of such reinjected articles is always a private stand-alone server is
>clearly incorrect.

Eh? The cases I am concerned about are when the destination of some
articles is the wider Usenet (and possibly via several routes, and
possibly unintentionally so). So I am not at all assuming or concerned
with articles sent to a stand-alone server. So could you please explain
your point there more clearly?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24227
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Fri, 19 Jan 2007 20:52:14 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Forrest J Cavalier <[email-masked]> writes:

>> There is no concise way to describe a "proper" re-injection.  There are
>> too many special cases and exceptions.  The role of such software is not
>> clear.  Is it a relay?  Posting agent?  Gateway?

>This is actually exactly the question that I think we can answer, by
>saying that it's a gateway.  That's the closest analogy, since a gateway
>is already a sort of posting agent.

But a posting agent is not so obviously a sort of gateway.

I think the answer is that it may be a Posting Agent and it may be a
Gatway. but in fact it will often be a confused situation where the four
parties involved (first injector, second injector and the two injecting
agents) all have a different view of what role they are playing, whether
by accident, by misunderstanding, or by deliberate intent.

>  It's definitely not a relaying agent

True, though one of the parties might believe he was relaying.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24226
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1417 USEPRO 3.4: Injecting-agent modification of message ID
Date: Fri, 19 Jan 2007 20:30:33 GMT
Lines: 62
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>In <[email-masked]> Russ Allbery <[email-masked]> writes:

>>Charles Lindsey <[email-masked]> writes:

>>> No, I think SHOULD is sufficient. As usual, it means "there needs to be
>>> an exceptional reason to break it". I can't think of such a reason
>>> offhand, but that is not to say it cannot arise. OTOH, there appears to
>>> be no interoperability problem, unless you want to count the existence
>>> of two articles with the same text as interoperability.

>>There is a significant interoperability problem in that we support
>>simultaneous injection with multiple injecting agents and rewriting the
>>message ID breaks that.  This is, in practice, not an uncommon thing to
>>want to do.

>>Message-ID here is special because of that.

>A valid point. I still mildly prefer remaining with SHOULD, but accept
>that I seem to be outvoted.

I have been thinking further about this, and the consequences if various
headers DO get changed by injecting agents and the article is then
injected in two places.

Message-ID:

If that case you have, technically speaking, two distinct articles which
just happen to be otherwise identical, and they will both propagate around
Usenet quite happily causing no technical harm whatsoever. Of course, it
will annoy people (and indeed it often happens currently, usually because
someone thought his article had failed to get out, and pressed 'send'
again - usually resulting in mild flames in his direction).

Date:, Path:, Sender:, User-Agent (and anything else an injecting agent
might be tempted to change)

In that case you have two different versions off what is supposed to be
the same article (i.e. they have the same messagg-id). Some sites will see
one and some the other. That is not supposed to happen in a well-regulated
network. OTOH, no huge harm arises, so "SHOULD NOT alter" (as in the
present text) may be sufficient. It just strikes me as slightly odd that
we are proposing a "MUST NOT alter" for the case that causes no technical
harm (but much annoyance, because people will notice, which they would not
in the other cases).

Xref:, newly added bits of Path:, Injection-Info (and maybe Injection-Date:
according to the outcome of #1416)

And these, of course, are actually intended to be different.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24225
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.2.1 - Number of path entries per site
Date: Fri, 19 Jan 2007 20:08:17 GMT
Lines: 55
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> The essential difference in what I am proposing is the replacement
>> of "the same news server" by "the same site", thus allowing more 
>> flexibility to omit identities that are not serving any useful purpose

>Okay, I missed that, distracted by the removal of a MAY in your version.
>The "same site" idea might make sense, I can't judge it.  Whenever Russ
>talked about "slaved servers" all I can say that it sounds interesting,
>and he obviously knows what this really means ... ;-)

What I am suggesting is not necessarily related to 'slaved servers',
though it might well apply there.

>>>> However, the last (leftmost) such <path-identity> so appended
>>>> MUST be one that is expected by the destination site when it
>>>> in turn comes to apply Step 3 above.
> 
>>> Can that fly if different peers expect different leftmost identities ?
>>> If not you can only say SHOULD.
> 
>> That would be an unusual situation. Why should it happen?

>> if MISMATCH problems are to be avoided, I think that MUST is needed,
>> and if ingenious site admins dream up bizarre schemes which still obey
>> that MUST, then they are welcome to do so.

>If they can.  "Swap the order of your path identities depending on the
>peer" could be tricky.

It would, which is a good reason why admins should avoid putting
themselves in a position where that might be needed. All I am saying is
that they MUST put in their leftmost <path-identity> something that the
downstream agent can readily use for MISMATCH checking. If they want to
make life unnecessarily complicated for themselves in order to achieve
that, then that is their problem :-( .

And note that MISMATCH checking does not necessarily have to be related to
the IP address from which the relayed article was received, It can just as
well be based on the SASL authentication identity that was used when the
relaying NNTP connection was set up in the first place.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24224
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1093
Date: Fri, 19 Jan 2007 19:59:57 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I don't think "if you run a Netnews server, you have to be spammable" is
>really going to fly in the real world.  It works so wonderfully well for
>e-mail, after all.

Except that "[email-masked]" is not likely to be visibly
plastered all over Usenet and the Web, ready to be scraped by spammers (as
"[email-masked]" might well be).

So if a spammer sends to [email-masked] he must have read RFC
2142, and then looked at all the domains he found in Path headers and in
Injection-Info and so tried putting "news@" in front of them. And that
situation already exists today (RFC 2142 is already out there), but I have
not heard that excessive amounts of spam are currently hitting news@
addresses (indeed, I would expect them to be rather quiet spamwise
compared to more visible addresses, and I don't see how putting "news@"
into USEPRO would change that).

Which is just to point out a flaw in your counter-argument, and not to say
that I necessarily support Frank's attempts to reopen this issue.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24229
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Fri, 19 Jan 2007 22:00:21 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>How about standardizing reinjection ONLY for disjoint networks, letting
>other uses of reinjection be out of scope?

But truly disjoint networks are not the problem.

The poblem is netowrks that are _not_ disjoint (even though somebody
thought they were). You cannot just declare them "out of scope" - you have
to define a protocol that works in spite of them.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 19 Jan 2007 12:30:02 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:

> Forrest J Cavalier <[email-masked]> writes:
> 
>>Russ Allbery wrote:
> 
> 
>>>Since the normal case for disjoint Netnews networks is that they'll
>>>have disjoint sets of articles as well.  (Also, one doesn't need to
>>>limit it to one in each network necessarily.  I was trying to avoid
>>>making that limitation in my language.)
> 
> 
>>Simple changes.  Make it conditional... "To cause an article to appear
>>on disjoint networks a posting agent SHOULD..."
> 
> 
>>And "s/one/at least one/"
> 
> 
> Yeah, that fixes my concerns there.
> 
> 
>>>Charles has a valid point that it's really beyond the ability of a
>>>posting agent to determine whether two Netnews networks are truly
>>>disjoint.
> 
> 
>>Yes, this point bothered me as I was going to bed.  Relays ought to
>>check Path and would rewording indicate that?  Would this be adequate.
> 
> 
> What sort of checking of Path do you have in mind?  I can't figure out
> exactly what that would entail.
> 
> 
>>>We need to say *what* other header fields it will rename.  I think we
>>>also have to allow for drop as well as rename; for example, if I'm
>>>posting to a disconnected INN server and then gatewaying those articles
>>>to Usenet, my local trace information is entirely irrelevant and
>>>there's no reason to retain it.
> 
> 
>>Isn't that permitted by the last SHOULD?  Why is the trace information
>>entirely irrelevant?  It isn't any more irrelevant than opaque trace
>>information from all the other servers.
> 
> 
> It often is.  The hostname is localhost, the IP address is 127.0.0.1, etc.
> It's allowed by SHOULD, but I don't think we need a SHOULD-level decision
> to decide to throw away useless tracking information.  Although I guess I
> don't feel strongly about it.

Yes, it is often 127.0.0.1, but it is not necessarily 127.0.0.1.  It may
include other information that is not useless. It's opaque information.

A corporate private leaf node site or mom-and-pop-and-mickey-mouse ISP (if there 
are any left) may have enough users internally that they have use for
preservation.

I'm don't feel strongly about this either.

>>>If this is the complete section, we're not saying that the gatewaying
>>>rules apply anywhere, which I think we still need to do since there may
>>>be bidirectional gatewaying at work (among other things).
> 
> 
>>Unnecessary complexity.  Why make people at private leaf nodes consider
>>the requirements of bidirectional gatewaying, discarding half of those
>>requirements?
> 
> 
> Er, I don't understand.  I'm not proposing that one-directional gateways
> be subject to the constraints of bidirectional gateways, of course.  I'm
> saying that this *is* a gateway and therefore needs to follow the rules of
> gateways, including such things as not gatewaying a message that it had
> already gatewayed, marking messages it has gatewayed, etc.  This is needed
> particularly in the case of bidirectional gatewaying, which you may not
> know is happening.  All that is spelled out in the gateway section, and I
> don't want to copy it all into this section again.
> 
> All those cautions and caveats are applicable here, so I'd like to just
> cite that section and be done with it.

I'm almost OK with that.  The trouble I see is that 3.9 Duties of
Gateways is almost all about mailing lists.  (And most of the trouble is
there too.)  Despite the 3.9 paragraph that says "reinjection is handled
separately below", "proto-article" is mentioned exactly once thereafter,
(just to say it must be valid.)

More seriously The outgoing gateway description in 3.9.1 seems to even 
contradict the RFC2119 language I wrote for the special case (handling local 
posts from a private leaf node.)  I'm not thrilled with the awkwardness and lack 
of clarity in 3.9. I can live with it because it is probably wordy enough that 
clueful people do the right thing when they read it.  I don't count the typical 
  private leaf node operators in that group....not a slur, just a statement of 
skill level.

My idea was to standardize the special case, and then provide enough detail that 
it works.  If people fall outside that special case, then they are gateways.  If
people can fit themselves into the special case, it is better to have the 
RFC2119 imperatives rather than the wishy washy 3.9.1.

If the Path checking and idea is sound, do we really gain anything by having
them keep lists of previously gatewayed articles? Tagging them?  Are those
really PROTOCOL requirements, necessary to get the end result?

I appreciate your concerns that there is lots to consider in the general case. 
Can we "have it both ways" and say, under reinjection....

    "Requirements and precautions to avoid problems are described in detail
    at 3.9 Duties of Gateways."

Does that work?

Date: Fri, 19 Jan 2007 12:00:50 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: #1416: USEPRO 3.9: Reinjection and Injection-Date
Sender: [email-masked]

Russ Allbery wrote:

> Forrest J Cavalier <[email-masked]> writes:
> 
>>Russ Allbery wrote:
>>Yes, this point bothered me as I was going to bed.  Relays ought to
>>check Path and would rewording indicate that?  Would this be adequate.
> 
> 
> What sort of checking of Path do you have in mind?  I can't figure out
> exactly what that would entail.

 From 3.5 Duties of a Relaying agent.
    In order to avoid unnecessary relaying attempts, an article SHOULD
    NOT be relayed if the <path-identity> of the receiving agent (or some
    known alias thereof) appears as a <path-identity> (excluding within
    the <tail-entry> or following a "POSTED" <diag-keyword>) in its Path
    header field.

(Need to modify it for reinjection.)  An article posted locally on a
private leaf node can easily be distinguished from others received
via suck by checking the Path.

Also I don't know that we need RFC2119 imperatives for the reinjection
case.  There is more than one way of determining that an article was
posted locally.

From: Russ Allbery <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: #1416: USEPRO 3.9: Reinjection and Injection-Date (was: Re: Injection-Date and reinjection)
Organization: The Eyrie
Date: Fri, 19 Jan 2007 07:12:10 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:
> Russ Allbery wrote:

>> Since the normal case for disjoint Netnews networks is that they'll
>> have disjoint sets of articles as well.  (Also, one doesn't need to
>> limit it to one in each network necessarily.  I was trying to avoid
>> making that limitation in my language.)

> Simple changes.  Make it conditional... "To cause an article to appear
> on disjoint networks a posting agent SHOULD..."

> And "s/one/at least one/"

Yeah, that fixes my concerns there.

>> Charles has a valid point that it's really beyond the ability of a
>> posting agent to determine whether two Netnews networks are truly
>> disjoint.

> Yes, this point bothered me as I was going to bed.  Relays ought to
> check Path and would rewording indicate that?  Would this be adequate.

What sort of checking of Path do you have in mind?  I can't figure out
exactly what that would entail.

>> We need to say *what* other header fields it will rename.  I think we
>> also have to allow for drop as well as rename; for example, if I'm
>> posting to a disconnected INN server and then gatewaying those articles
>> to Usenet, my local trace information is entirely irrelevant and
>> there's no reason to retain it.

> Isn't that permitted by the last SHOULD?  Why is the trace information
> entirely irrelevant?  It isn't any more irrelevant than opaque trace
> information from all the other servers.

It often is.  The hostname is localhost, the IP address is 127.0.0.1, etc.
It's allowed by SHOULD, but I don't think we need a SHOULD-level decision
to decide to throw away useless tracking information.  Although I guess I
don't feel strongly about it.

>> If this is the complete section, we're not saying that the gatewaying
>> rules apply anywhere, which I think we still need to do since there may
>> be bidirectional gatewaying at work (among other things).

> Unnecessary complexity.  Why make people at private leaf nodes consider
> the requirements of bidirectional gatewaying, discarding half of those
> requirements?

Er, I don't understand.  I'm not proposing that one-directional gateways
be subject to the constraints of bidirectional gateways, of course.  I'm
saying that this *is* a gateway and therefore needs to follow the rules of
gateways, including such things as not gatewaying a message that it had
already gatewayed, marking messages it has gatewayed, etc.  This is needed
particularly in the case of bidirectional gatewaying, which you may not
know is happening.  All that is spelled out in the gateway section, and I
don't want to copy it all into this section again.

All those cautions and caveats are applicable here, so I'd like to just
cite that section and be done with it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Fri, 19 Jan 2007 09:50:07 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Injection-Date and reinjection
Sender: [email-masked]

Russ Allbery wrote:

> Since the normal case for disjoint Netnews networks is that they'll have
> disjoint sets of articles as well.  (Also, one doesn't need to limit it to
> one in each network necessarily.  I was trying to avoid making that
> limitation in my language.)

Simple changes.  Make it conditional... "To cause an article to appear on
disjoint networks a posting agent SHOULD..."

And "s/one/at least one/"

> 
> 
>>When it cannot be avoided, a posting agent which converts existing
>>articles back to proto-articles for reinjection to a disjoint network
>>MUST NOT reinject articles already accepted on that network.  It MUST
>>perform the same staleness tests of Injection-Date or Date header fields
>>as would be performed by a relaying agent.  (This helps prevent
>>reappearance of expired articles.)  It MUST NOT alter header fields
>>permitted in proto-articles, especially Message-ID.  (This helps prevent
>>loops.)  To make the article acceptable to an injecting agent, it SHOULD
>>rename other header fields to preserve information, but MAY remove them.
> 
> 
> Charles has a valid point that it's really beyond the ability of a posting
> agent to determine whether two Netnews networks are truly disjoint.

Yes, this point bothered me as I was going to bed.  Relays ought to check Path
and would rewording indicate that?  Would this be adequate.

> 
> Beyond that, the bit about the Message-ID is a good addition that I want
> to keep, regardless of how else we word this.  The gateway stuff already
> says this sort of, but we should emphasize it.
> 
> We need to say *what* other header fields it will rename.  I think we also
> have to allow for drop as well as rename; for example, if I'm posting to a
> disconnected INN server and then gatewaying those articles to Usenet, my
> local trace information is entirely irrelevant and there's no reason to
> retain it.

Isn't that permitted by the last SHOULD?  Why is the trace information
entirely irrelevant?  It isn't any more irrelevant than opaque trace
information from all the other servers.

> 
> If this is the complete section, we're not saying that the gatewaying
> rules apply anywhere, which I think we still need to do since there may be
> bidirectional gatewaying at work (among other things).

Unnecessary complexity.  Why make people at private leaf nodes consider
the requirements of bidirectional gatewaying, discarding half of those
requirements?

Can't bidirectional gatewaying be discussed under gatewaying, or let
implementors assume it is just two unidirectional reinjections?  The bit about
using the Path header may be important for this case.  What concerns
do you have?

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Organization: The Eyrie
Date: Thu, 18 Jan 2007 23:20:43 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> Normally, articles are transferred between news servers by relaying
> agents.  In the case of disjoint Netnews networks, such as a private
> server with no relaying peers, a posting agent SHOULD cause the
> identical proto-article with Message-Id header field to be transmitted
> to multiple injecting agents, one in each disjoint network.

"If one article is to be injected into disjoint Netnews networks, such as
both Usenet and a private server with no relaying peers, a posting agent
..."

Since the normal case for disjoint Netnews networks is that they'll have
disjoint sets of articles as well.  (Also, one doesn't need to limit it to
one in each network necessarily.  I was trying to avoid making that
limitation in my language.)

> When it cannot be avoided, a posting agent which converts existing
> articles back to proto-articles for reinjection to a disjoint network
> MUST NOT reinject articles already accepted on that network.  It MUST
> perform the same staleness tests of Injection-Date or Date header fields
> as would be performed by a relaying agent.  (This helps prevent
> reappearance of expired articles.)  It MUST NOT alter header fields
> permitted in proto-articles, especially Message-ID.  (This helps prevent
> loops.)  To make the article acceptable to an injecting agent, it SHOULD
> rename other header fields to preserve information, but MAY remove them.

Charles has a valid point that it's really beyond the ability of a posting
agent to determine whether two Netnews networks are truly disjoint.

Beyond that, the bit about the Message-ID is a good addition that I want
to keep, regardless of how else we word this.  The gateway stuff already
says this sort of, but we should emphasize it.

We need to say *what* other header fields it will rename.  I think we also
have to allow for drop as well as rename; for example, if I'm posting to a
disconnected INN server and then gatewaying those articles to Usenet, my
local trace information is entirely irrelevant and there's no reason to
retain it.

If this is the complete section, we're not saying that the gatewaying
rules apply anywhere, which I think we still need to do since there may be
bidirectional gatewaying at work (among other things).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Fri, 19 Jan 2007 02:07:55 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Sender: [email-masked]

How about standardizing reinjection ONLY for disjoint networks, letting
other uses of reinjection be out of scope?

Taking a shot at that.....

Normally, articles are transferred between news servers by relaying agents.  In 
the case of disjoint Netnews networks, such as a private server with no relaying 
peers, a posting agent SHOULD cause the identical proto-article with Message-Id 
header field to be transmitted to multiple injecting agents, one in each 
disjoint network.

When it cannot be avoided, a posting agent which converts existing articles back 
to proto-articles for reinjection to a disjoint network MUST NOT reinject 
articles already accepted on that network.  It MUST perform the same staleness 
tests of Injection-Date or Date header fields as would be performed by a 
relaying agent.  (This helps prevent reappearance of expired articles.) It MUST 
NOT alter header fields permitted in proto-articles, especially Message-ID. 
(This helps prevent loops.)  To make the article acceptable to an injecting 
agent, it SHOULD rename other header fields to preserve information, but MAY 
remove them.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1417: USEPRO 3.4: Injecting-agent modification of message-ID
Date:  Fri, 19 Jan 2007 06:52:56 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Russ Allbery wrote:

> I'd prefer that discussion about changing the general SHOULD NOT about
> all header fields to a MUST NOT be a separate discussion, since the 
> Message-ID has a distinguished protocol status and an interoperability
> use case that the other header fields don't have.

ACK, but is there any controversy about your proposal limited to the
Message-ID ?  IFF not I'd say "make it so".

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1416: USEPRO 3.9: Reinjection and Injection-Date (was: Re: Injection-Date and reinjection)
Organization: The Eyrie
Date: Thu, 18 Jan 2007 21:27:53 -0800
Sender: [email-masked]

Sorry, I messed up the Subject header here.  The parent of this article
is also part of this ticket.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1416: USEPRO 3.9: Reinjection and Injection-Date (was: Re: ISSUE: Reinjection and Injection-Date)
Organization: The Eyrie
Date: Thu, 18 Jan 2007 21:27:01 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> There are essentially two options:

> Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

> Option 2: Injection-Date headers, once written, are NEVER altered.

Just to try to be painfully clear here, Injection-Date using the method
laid out in the current USEPRO draft is *not* rewritten by injecting
agents.  The whole point of that option is that only the gateways who are
doing the reinjection make this transformation.

The current draft doesn't define a "reinjecting agent" exactly, but such
an agent would be a gateway and posting agent, *not* any sort of injecting
agent.

> Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

> We are concerned with an article that is being reinjected. Let us refer
> to its two incarnartions as FIRST and SECOND and the person/entity doing
> the reinjection as the REINJECTOR. For this option to be safe, it is
> ESSENTIAL that FIRST and SECOND were injected into totally disjoint
> networks, with absolutely no possibility of leaks between them, and it
> is the absolute responsibility of the REINJECTOR to ensure that this
> condition is met.

Lack of safety can only occur if the networks are not disjoint *and* there
is a delay in gatewaying longer than the history retention of whatever
path causes the networks to not be disjoint (whether a relaying agent or
another gateway) but shorter than the staleness check in the reinjecting
gateway (*and* no staleness checks are applied to the Date header field,
but we can all agree on that as a long-term goal).

Observe that even in this situation, the problem can only happen once per
article.  In other words, the worse case scenario is for a host to see the
same article twice and twice only.  At that point, the requirements of a
gateway cut off further reinjection of the same article.

> But the real problems arise when FIRST is injected in a small private
> network (even a private news server maintained by an individual poster).
> Although it is argued that such a network should be regarded as
> gatewaying into the network where SECOND is to be propagated, it is far
> from certain that the owner of such a network or such an individual
> poster will regard himself as a "gatewayer", or that he will believe
> that section 3.9 applies in any way to him. Such small private neworks
> are notorious for "leaking" (and it is not necessarily the "owner" of
> the network who will be the REINJECTOR that perpetrates the leak).

Note that *relaying* leaks from such private networks are quite rare
(although not non-existent).  Leaks are almost always via reinjection
(suck/rpost feeds).

> Option 2: Injection-Date headers, once written, are NEVER altered.

> This clearly removes the risks inherent in Option 1. It is inherently
> much safer, and it is 'fail-safe' in the sense that the only thing that
> can go wrong is that an article whose propagation was genuinely delayed
> at some point might fail to propagate beyond a relaying agent with a
> short history file expiry. But better to lose a few articles than run
> the risk of loops.

As you note, this approach makes it impossible for a gateway that is down
for longer than the retention period of its most immediate upstream
relaying agent to ever go back and gate the articles that it missed during
its period of downtime.

This is exactly the sort of problem that caused us to add Injection-Date
in the first place.  If this is acceptable, I move that we drop
Injection-Date entirely and go back to the well-understood and
already-implemented staleness checking on Date and make people who hold
posts for a while work around this in their posting agent.  It would save
a ton of complexity.

> But since B wants the articles for its own use (not for relaying back
> into Usenet), it is presumably a serving agent, in which case its expiry
> time is likely to be more than a week, in which case the staleness check
> is unlikely to be triggered. But even if that is not so, this is a case
> where all B's admin needs to do is to temporarily increase the expiry
> time of his history file. This is always safe; even if some site on his
> local network tries to relay an article back into Usenet, it is still
> protected by its _original_ Injection-Date.

No, this is *unsafe* if that host is part of a Netnews network because it
means that articles that the relaying agent had already accepted it may
accept again because it now thinks that its staleness cutoff is longer
than it really was and will therefore think that articles that it had
expired out of its history would be present in history had it already seen
it.

Extending the staleness cutoff is *not* safe unless it's done at least
that length of time after increasing the expiration time on the history
file so that the history file really does contain that many days of
history.

Please remember that one of the very common cases of suck/rpost feeds is
*from* a standalone server *to* Usenet, so assuming that the destination
of such reinjected articles is always a private stand-alone server is
clearly incorrect.  Please also remember that you cannot assume that the
person running the gateway and the person running its first upstream
relaying agent are the same person or that the latter cares about the
concerns of the former.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Organization: The Eyrie
Date: Thu, 18 Jan 2007 21:01:18 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> There is no concise way to describe a "proper" re-injection.  There are
> too many special cases and exceptions.  The role of such software is not
> clear.  Is it a relay?  Posting agent?  Gateway?

This is actually exactly the question that I think we can answer, by
saying that it's a gateway.  That's the closest analogy, since a gateway
is already a sort of posting agent.  It's definitely not a relaying agent
(if so, there's no need for reinjection) or an injecting agent (since an
injecting agent could talk to a relaying agent directly and hence preserve
the existing injection fields).  If it were in a position to be either of
those things, the message could just be relayed instead.

> I find 3.3.2 poorly written and redundant.

Quite possible.  :)

> I consider the mention of "special exceptions" an attractive nuisance.
> An implementor looking for justification will always consider themselves
> "special."  That's just psychology.

> I think mentioning reinjection will entice novices to try something that
> only experts can be expected to do without causing trouble.

A Netnews to Netnews gateway is probably an easier piece of software to
write than a full serving agent.  suck/rpost and newsx aren't horribly
complex.

I generally agree with what you're saying, but this is common enough that
I don't want to remain silent about it.  Plus, there are various ways in
which people do this incorrectly, so we can improve the situation by
explaining clearly how to do it correctly and thus prevent harm.

The goal with the "exceptional" language is to try to point out to people
that there is usually a different, better solution to the problem.  I'd
like to try to say this somehow.  If it's possible to set up a relaying
relationship or to do multiple injection, that should be done instead.
Reinjection should not be the first choice, since it adds complications.

> I have annotated the draft 3.3.2 below, but the end result is that would
> replace 3.3.2 with....

> 3.3.2 Reinjection of Articles

>    Converting an already injected article into a proto-article and
>    forwarding it to an injecting agent, "reinjecting", causes problems
>    such as loops, loss of trace information, and re-appearance of
>    expired articles, unless handled very carefully.

Note that we have a definition of reinjecting in 1.5:

   "Reinjecting" an article is passing an already-injected article to an
   injection agent.

We may want to reconsider this definition as well as part of this
discussion.

I'm worried about this start of the section from an editorial concept,
since it leaps into converting an injected article into a proto-article
without saying how that idea would even come up.  I think it's worthwhile
to say something like:

    Normally, articles are transferred between news servers by relaying
    agents.  To reach news servers not reachable by relaying agents with
    an article, the best approach is to give the identical proto-article
    to injecting agents in each disconnected Netnews network.  If,
    however, neither of these approaches are possible, it may be necessary
    to convert an existing article back to a proto-article so that it can
    be injected into another Netnews network.  This action, called
    reinjection, may cause problems such as loops, loss of trace
    information, and reappearance of expired articles unless handled
    carefully and SHOULD be avoided when possible.

That's a bit of an expansion over what you say and what's already in the
section, but makes the alternatives clearer.  What do people think of that
sort of language?

>    A posting agent that does reinjection is a limited type of gateway
>    and is also subject to all of the requirements of an incoming
>    gateway.  If the header must be altered before forwarding to an
>    injecting agent, renaming header fields is preferred over removing.
>    Checks using those header fields (that would otherwise be done by the
>    injecting agent) MUST be accomplished before the alteration.

I want to explicitly say how you convert an article back to a
proto-article so that it's clear that we're not talking about removing or
changing Message-ID.  So:

    A posting agent that does reinjection is a limited type of gateway and
    is also subject to all of the requirements of an incoming gateway.  It
    MUST also perform the same checks against the existing Injection-Date
    or Date header fields as would be performed by a relaying agent.  It
    then MUST convert the article to a proto-article by renaming or
    removing any existing Injection-Info, Injection-Date, and Xref header
    fields.  Renaming is preferrable in most cases for Injection-Info and
    Injection-Date to preserve trace information.  Such a posting agent
    may need to rename or remove other, non-standard trace headers (such
    as NNTP-Posting-Host or X-Trace) to make the article acceptable to an
    injecting agent.

Note that it's actually the date checks of a *relaying agent* that we
should be performing; the checks of an injecting agent don't take
Injection-Date into account.  I had that wrong in the current text.

I'd agree with a rewritten section including just the two paragraphs I
propose above.  That would contain all the important parts of the current
section, I think.

> 3.3.2.  Reinjection of Articles

>    A given article SHOULD be processed by an injecting agent once and
>    only once.

> [Recommend: moving to Duties of an injecting agent.]

This is only true in the presence of multiple injection if you squint at
it properly.  I'm not sure that this sentence is really adding anything
useful once we explain the normal article propagation rules.

>    The Injection-Date or Injection-Info header fields are
>    added by an injecting agent and are not permitted in a proto-article.
> [Repeats something from 3.3.1, which repeats something in 3.3 para 2.
> Strike it, and remove the repeat in 3.3.1.]

This is part of the definition of a proto-article, so the one place it
should be stated is in the proto-article section.  Instead of removing it
from there, I'd change the text in 3.3 para 2 to:

   Posting agents MUST ensure that proto-articles they create are valid
   according to Section 3.3.1, [USEFOR] and any other applicable policies.

I'm not sure why I said SHOULD here originally.  Surely, complying with
the basic requirements of the standard is a MUST.  I guess I said SHOULD
because the primary responsibility lies with an injecting agent, but we
already say that explicitly elsewhere.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1417: USEPRO 3.4: Injecting-agent modification of message-ID
Organization: The Eyrie
Date: Thu, 18 Jan 2007 20:24:07 -0800
Sender: [email-masked]

Proposed wording:  Change the current text:

   6.   The injecting agent MUST NOT alter the body of the article in
        any way (including any change of Content-Transfer-Encoding).  It
        MAY add other header fields not already provided by the poster,
        but injecting agents are encouraged to use the Injection-Info
        header for such information and to minimize the addition of
        other headers.  It SHOULD NOT alter, delete, or reorder any
        existing header field except the Path header.

to:

   6.   The injecting agent MUST NOT alter the body of the article in
        any way (including any change of Content-Transfer-Encoding).  It
        MAY add other header fields not already provided by the poster,
        but injecting agents are encouraged to use the Injection-Info
        header for such information and to minimize the addition of
        other headers.  It SHOULD NOT alter, delete, or reorder any
        existing header field except the Path header field.  It MUST NOT
        alter or delete any existing Message-ID header field.

Note the minor s/Path header/Path header field/ change at the end of the
current text as well.  I noticed that while looking at this.

I'd prefer that discussion about changing the general SHOULD NOT about all
header fields to a MUST NOT be a separate discussion, since the Message-ID
has a distinguished protocol status and an interoperability use case that
the other header fields don't have.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1083: USEPRO 5.3: Rules for generating message-ID
Organization: The Eyrie
Date: Thu, 18 Jan 2007 20:17:22 -0800
Sender: [email-masked]

I propose closing this ticket with no further changes.

There seems to be a rough consensus in the working group that the $alz
convention of issuing spam cancels with message IDs formed by adding
"cancel." to the beginning of the LHS of the original message ID to allow
for intentional collisions between multiple cancels for the same message
is the sort of weird stunt that can be left as an intentional protocol
violation.

The rest of the ticket is talking about namespace issues in deciding what
message IDs to use for messages.  USEFOR now defers to RFC 2822 for this,
and I think the treatment in RFC 2822 is sufficient.  This is a tricky
area, so if we can avoid having to say more about it and use the consensus
of the RFC 2822 working group, that saves us a lot of work and tricky
wording discussion.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: #1413: USEPRO 5.5: ihave/sendme syntax
Organization: The Eyrie
Date: Thu, 18 Jan 2007 20:06:57 -0800
Sender: [email-masked]

I think we have an agreed resolution for this one.  Namely, change the
syntax of Ihave-arguments to:

       Ihave-arguments     = 1*WSP *( msg-id 1*WSP ) relayer-name

Replace this language from the second paragraph after:

   If <relayer-name> is not given, it is determined from the origin of
   the control message.

with:

   Contrary to [RFC1036], the relayer-name MUST be given as the last
   argument in the Control header field.

I think Charles and I are both happy with this.  It brings our syntax in
line with Son-of-1036.  Are there any objections to this resolution, or
anything that I'm missing?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: USEAGE
Organization: The Eyrie
Date: Thu, 18 Jan 2007 17:09:02 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:
 
>> My point is that we need to not say things that are directly in
>> contradiction with reality unless we can reasonably expect reality to
>> change.

> If the reality is just messy we should try to get it right.  That can be
> very harmless, like specifying ONE admin address, or outline ONE typical
> procedure for moderators, where we trust that it shouldn't cause havoc
> if folks actually adopt it.

I believe that we have already done this for moderated groups, and that
you are now asking for us to take a step farther beyond that into saying
that all other procedures are wrong.

> "Reasonably expect reality to change" is far stronger, and for the path
> diagnosis I wouldn't bet that it flies.

Which is, as has been previously noted, a strong argument for removing the
path diagnosis.  However, that argument is closed, since it's in USEFOR
and USEFOR is closed.  That doesn't mean it's not a valid argument, or
that it won't apply to other discussions in the future.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: USEAGE
Date:  Fri, 19 Jan 2007 01:51:04 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 25
Sender: [email-masked]

Russ Allbery wrote:
 
> My point is that we need to not say things that are directly in
> contradiction with reality unless we can reasonably expect reality
> to change.

If the reality is just messy we should try to get it right.  That can
be very harmless, like specifying ONE admin address, or outline ONE
typical procedure for moderators, where we trust that it shouldn't
cause havoc if folks actually adopt it.

"Reasonably expect reality to change" is far stronger, and for the
path diagnosis I wouldn't bet that it flies.  At least it's better
than conflicting inventions for this fifth wheel.

In an unrelated article Forest wrote: 
| Feelings are a fine place to start, but we aren't writing a novel.

If my gut feeling is that moderated articles should always work as
expected if moderators stay away from modifying the body and the
Message-ID it's not meant as novel.  My maybe naive concept of how
I'd write a mod-cancel-bot was really "note approved Message-IDs".

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: #1093
Organization: The Eyrie
Date: Thu, 18 Jan 2007 16:41:51 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> That's another reason why 2142 has to be updated.  Maybe something in
> the direction of "a mailbox news@<path-identity> SHOULD exist for the
> most commonly used <path-identity> of a news server".

I don't think "if you run a Netnews server, you have to be spammable" is
really going to fly in the real world.  It works so wonderfully well for
e-mail, after all.

If individual Netnews networks want to require this, that's another
matter, but there's no requirement that an arbitrary Netnews installation
support e-mail at all.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Fri, 19 Jan 2007 01:21:42 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 60
Sender: [email-masked]

Charles Lindsey wrote:

> Hey! Part of the rules for the Martians was that they had copies of all
> the standards track RFCs, and that inlcudes 2142 :-) .

<evil> So if we find that two addresses in a PS are messy, then I could
try to convince you that it should be fixed on standards track, and not
in a BCP... </evil>

>>> Please update ticket #1093 to say "specify one and only one default
>>> admin address (updating RFC 2142)".  Obviously this can't be a MUST
>>> for any <path-identity>, it won't work without MX (or A/AAAA).

> Then you have to say which one it should be. So you might say generate
> news@, but SHOULD still accept usefor@, or maybe vice versa.

IMO just news@ is good enough for everybody.  The usenet@ and newsmaster@
could be mentioned (JFTR) in an appendix about differences from s-o-1036.

If an appendix about such differences makes sense - if the implementors
here say that such differences won't affect existing implementations we
don't need an explicit list in USEPRO.

> you can't go beyond that because currently there is no preference between
> them, and people use whichever they feel like.

That's precisely the reason why we should go beyond that.  No MUST, it
is only a useful convention.

> the real point is that you would also have to specify how to find the
> domain to use with it, and RFC 2142 tried to deduce it from the Path in
> methods that were muddled and unnecessarily complicated.

That's another reason why 2142 has to be updated.  Maybe something in the
direction of "a mailbox news@<path-identity> SHOULD exist for the most
commonly used <path-identity> of a news server".

> In earlier drafts we had wording that some/all of the Path domains had
> to be "mailable"

Yes, but I think that was all far too convoluted.  News admins can figure
out how to arrange news@<path-identity>.  USEPRO doesn't need to specify
details about MX, CNAME, A, AAAA, etc., folks can read RFC 2821 for that.

> I still hold an opinion against having <path-identity>s which do not
> resolve, at least to an MX, in the DNS

A <path-nodot> legacy name has no MX.  And if admins change their setup
using multiple path identities they could use old "dead" FQDNs in a path.

Or they use additional "gbl" FQDNs relevant only in their own LAN (until
ICANN convinces them that they are idiots, but that's not our problem ;-)

> that is an issue I will raise at some point, or we can take it as part
> of this #1093 if #1093 is to remain open).

It's related, but maybe not exactly the same issue (?)

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: #1093
Date:  Fri, 19 Jan 2007 00:46:57 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Harald Alvestrand wrote:

> Are you arguing that there should be a single admin address, AND that
> it should be documented in USEPRO?

YES and yes.  For the latter I've no clue if a USEAGE BCP can update a
rather old PS 2142.  There's of course also the issue that I'm not sure
when (or if) USEAGE will be published, and if admins would read it.

> I can see some people agreeing with the first part, but not agreeing
> with the second, and vice versa.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: USEPRO 3.2.1 - Number of path entries per site
Date:  Fri, 19 Jan 2007 00:36:24 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

Charles Lindsey wrote:

  [multiple path identities]
>> I fail to see the difference from the MAY.
 
> The essential difference in what I am proposing is the replacement
> of "the same news server" by "the same site", thus allowing more 
> flexibility to omit identities that are not serving any useful purpose

Okay, I missed that, distracted by the removal of a MAY in your version.
The "same site" idea might make sense, I can't judge it.  Whenever Russ
talked about "slaved servers" all I can say that it sounds interesting,
and he obviously knows what this really means ... ;-)

>>> However, the last (leftmost) such <path-identity> so appended
>>> MUST be one that is expected by the destination site when it
>>> in turn comes to apply Step 3 above.
 
>> Can that fly if different peers expect different leftmost identities ?
>> If not you can only say SHOULD.
 
> That would be an unusual situation. Why should it happen?

You example was about "upstream" vs. "downstream" peers.  These terms
aren't yet defined, one scenario could be that the "downstream" peers
are on the other side of a firewalll and see a non-public IP.  In that
case it would be more important for "!!" to match the path-identity on
the "upstream" side with a public IP.

> in any case, it is a matter for negotiation between the peers concerned

That doesn't sound like a MUST, or does it ?

> if MISMATCH problems are to be avoided, I think that MUST is needed,
> and if ingenious site admins dream up bizarre schemes which still obey
> that MUST, then they are welcome to do so.

If they can.  "Swap the order of your path identities depending on the
peer" could be tricky.  That's a question for folks who have actually
implemented such schemes or at least looked into sources - I didn't.

Frank

Date: Thu, 18 Jan 2007 13:49:09 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID
Sender: [email-masked]

Charles Lindsey wrote:
> 
> A valid point. I still mildly prefer remaining with SHOULD, but accept
> that I seem to be outvoted.
> 

That's a fallacy that you were "outvoted."

You were "out-argued."

You produced no _reason_ for preferring SHOULD, not even a contrived special
case.  You provided a preference, a mere feeling.  Then, you concede a "valid 
point", but it seems to have no effect on what you think, beyond stating its 
validity.

If you expect people to continue to offer patient explanations, can you at least 
provide an illusion that your opinion is open to good reasoning?

Feelings are a fine place to start, but we aren't writing a novel.

To: [email-masked]
Xref: clerew local.usefor:24203
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID
Date: Thu, 18 Jan 2007 15:14:17 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> No, I think SHOULD is sufficient. As usual, it means "there needs to be
>> an exceptional reason to break it". I can't think of such a reason
>> offhand, but that is not to say it cannot arise. OTOH, there appears to
>> be no interoperability problem, unless you want to count the existence
>> of two articles with the same text as interoperability.

>There is a significant interoperability problem in that we support
>simultaneous injection with multiple injecting agents and rewriting the
>message ID breaks that.  This is, in practice, not an uncommon thing to
>want to do.

>Message-ID here is special because of that.

A valid point. I still mildly prefer remaining with SHOULD, but accept
that I seem to be outvoted.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24202
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1093
Date: Thu, 18 Jan 2007 15:13:12 GMT
Lines: 67
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Just to be sure what you are arguing:

>Are you arguing that there should be a single admin address, AND that it 
>should be documented in USEPRO?

>I can see some people agreeing with the first part, but not agreeing 
>with the second, and vice versa.

>Frank Ellermann wrote:
>> Harald Alvestrand wrote:
>>
>>   
>>> #1093: USEPRO 2.3: Supported email addresses
>>>    This has a recorded consensus not to say MUST or SHOULD in USEPRO
>>>    about the addresses news@, usenet@ and abuse@.
>>>    -07 seems to mention none of them.
>>>    Unless new technical information comes to light, I'll close it.
>>>     
>>
>> The abuse@ part of this issue is covered by Injection-Info parameter
>> "mail-complaints-to".  
>>
>> But the mess of not less than three admin addresses specified in 2142
>> and s-o-1036, with an "obscure" (= I didn't find it) justification in
>> 2142 based on an obsoleted RFC, has to be cleaned up.
>>
>> If those Martian implementors install a server to test their software
>> they're not going to read both 2142 and s-o-1036.  And if they read it
>> they could decide that two or three role addresses for one admin does
>> not pass Martian giggle tests.

Hey! Part of the rules for the Martians was that they had copies of all
the standards track RFCs, and that inlcudes 2142 :-) .
>>
>> Please update ticket #1093 to say "specify one and only one default
>> admin address (updating RFC 2142)".  Obviously this can't be a MUST 
>> for any <path-identity>, it won't work without MX (or A/AAAA).

Then you have to say which one it should be. So you might say generate
news@, but SHOULD still accept usefor@, or maybe vice versa. But you can't
go beyond that because currently there is no preference between them, and
people use whichever they feel like. If one has to choose one, I suppose
it would be whichever one is used in the Injection-Info.

But the real point is that you would also have to specify how to find the
domain to use with it, and RFC 2142 tried to deduce it from the Path in
methods that were muddled and unnecessarily complicated. In earlier drafts
we had wording that some/all of the Path domains had to be "mailable", and
the consensus to which Harald refers was to take that out.

But I remain neutral on whether to reopen this issue (though I still hold
an opinion against having <path-identity>s which do not resolve, at least
to an MX, in the DNS, and that is an issue I will raise at some point, or
we can take it as part of this #1093 if #1093 is to remain open).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID
Organization: The Eyrie
Date: Wed, 17 Jan 2007 09:01:07 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> No, I think SHOULD is sufficient. As usual, it means "there needs to be
> an exceptional reason to break it". I can't think of such a reason
> offhand, but that is not to say it cannot arise. OTOH, there appears to
> be no interoperability problem, unless you want to count the existence
> of two articles with the same text as interoperability.

There is a significant interoperability problem in that we support
simultaneous injection with multiple injecting agents and rewriting the
message ID breaks that.  This is, in practice, not an uncommon thing to
want to do.

Message-ID here is special because of that.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24191
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID (was: Re: Multi-point injection)
Date: Wed, 17 Jan 2007 11:38:53 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Forrest J Cavalier <[email-masked]> writes:

>> In a quick review, I don't see anything under "Duties of an Injection
>> Agent" that a Message ID field present in a proto-article MUST be
>> preserved.  (This is required for multi-point injection.)

>> What am I missing?

>Well, there's:

>   6.   The injecting agent MUST NOT alter the body of the article in
>        any way (including any change of Content-Transfer-Encoding).  It
>        MAY add other header fields not already provided by the poster,
>        but injecting agents are encouraged to use the Injection-Info
>        header for such information and to minimize the addition of
>        other headers.  It SHOULD NOT alter, delete, or reorder any
>        existing header field except the Path header.

Yes, that is the wording intended to cover the matter. Just to recall the
history, those words were written as a compromise after a long battle as
to whether in injecting agent was allowed to alter (NO) or add (YES) a
Sender header, but they were intended to cover all headers.

>but I think you're right that message ID is special and needs a stronger
>constraint than SHOULD.

No, I think SHOULD is sufficient. As usual, it means "there needs to be an
exceptional reason to break it". I can't think of such a reason offhand,
but that is not to say it cannot arise. OTOH, there appears to be no
interoperability problem, unless you want to count the existence of two
articles with the same text as interoperability.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24188
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: USEAGE
Date: Wed, 17 Jan 2007 11:07:06 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> Forrest is right here. USEAGE is largely based on GNKSA, and the GNKSA
>> was very much concerned with improving the quality of news servers.

>IIRC the N in GNKSA was for "newsreader" (UA), not for "news server".
>All I ever did with it was to evaluate my UA.  Based on that I consider
>USEAGE as a kind of Emily Postnews for geeks.

No, the "GN" stood for "Good-Netkeeping", and was a play on the "Good
Housekeeping Seal of Approval" published (maybe it still is) by Good
Housekeeping magazine.

But yes, now I have re-read it, it was primarily concerned with user
agents.

But USEAGE was always intended to deal with all agents, and much of it
consists of texts pulled from the earlier 'article' series of drafts after
we had decided to make that split.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24189
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Wed, 17 Jan 2007 11:18:23 GMT
Lines: 42
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>There is no concise way to describe a "proper" re-injection.  There
>are too many special cases and exceptions.  The role of such software
>is not clear.  Is it a relay?  Posting agent?  Gateway?

>I find 3.3.2 poorly written and redundant.

I tend to agree, but am not sure your rewrite is correct either. I think
we really need to resolve the Injection-Date Issue (#1416), and then come
back to exactly what we say here.

>I consider the mention of "special exceptions" an attractive nuisance.
>An implementor looking for justification will always consider themselves
>"special."   That's just psychology.

Which is why our drafts sometimes needs to say "Don't try to do this at
home, kiddies", but of course the problem is to find a polite way of
saying that. Essentially, the draft needs to be written so that not only
does it specify the correct tecnical details, but also so that it gives the
correct _impression_ of how things are _intended_ to work

>I think mentioning reinjection will entice novices to try something that only 
>experts can be expected to do without causing trouble.

Indeed. Reinjection needs to happen in a few cases which are hard to
specify precisely in advance. But the impression to be given is definitely
that "you would recognize those cases when you see them, but it takes
great knowledge and skill to recognize them", though of course you can't
word it like that. But saying less rather than more is probably a move in
that direction.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24190
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.2.1 - Number of path entries per site
Date: Wed, 17 Jan 2007 11:32:28 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>>    Except possibly when relaying to other hosts on the same site, every
>>    injecting, relaying, or serving agent that accepts an article MUST
>>    update the Path header field ....

>An s/MAY/SHOULD/ in the -07 text might also do what you want.  If that's
>not your intention I fail to see the difference from the MAY.

The essential difference in what I am proposing is the replacement of "the
same news server" by "the same site", thus allowing more flexibility to
omit identities that are not serving any useful purpose. But no, I don't
waht to say you SHOULD omit them. It is just a mater of allowing some
discretion to the site.

>>        However, the last (leftmost) such <path-identity> so appended
>>        MUST be one that is expected by the destination site when it
>>        in turn comes to apply Step 3 above.

>Can that fly if different peers expect different leftmost identities ?
>If not you can only say SHOULD.

That would be an unusual situation. Why should it happen? But in any case,
it is a matter for negotiation between the peers concerned. It might even
be necessary to add diferent identities to the Path according to which
peer you were relaying it to, which would be horribly messy.

But if MISMATCH problems are to be avoided, I think that MUST is needed,
and if ingenious site admins dream up bizarre schemes which still obey
that MUST, then they are welcome to do so.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24192
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Ticket status, USEPRO
Date: Wed, 17 Jan 2007 11:42:46 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>I have now filed tickets for all the issues raised with ISSUE: in their 
>subject lines, except for "ISSUE: Injecting agents and From", which I do 
>not see any reason to pursue. I'll file it if anyone but Charles wants 
>to argue that a change is needed based on what Charles wrote.

My request for an issue number or that still stands.

>The following tickets are open:

>#1083: USEPRO 5.3: Rules for generating message-ID
>#1093: USEPRO 2.3: Supported email addresses
>   This has a recorded consensus not to say MUST or SHOULD in USEPRO 
>about the addresses news@, usenet@ and abuse@.
>   -07 seems to mention none of them.
>   Unless new technical information comes to light, I'll close it.

Agreed. The matter of a NOTE referring to the matter and directing you to
USEAGE still remains, however.

>#1412: USEPRO 5.3: Cancel newsgroups: Matching
>#1413: USEPRO 5.5: ihave/sendme syntax
>#1414: USEPRO 3.2.1: delimiter for multiple Path identities
>#1415: USEPRO 3.2.1: Number of path entries per site
>#1416: USEPRO 3.9: Reinjection and Injection-Date
>#1417: USEPRO 3.4: Injecting-agent modification of message-ID

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 17 Jan 2007 11:17:12 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: #1093
Sender: [email-masked]

In message <[email-masked]>, Harald Alvestrand
<[email-masked]> writes
>
>Just to be sure what you are arguing:
>
>Are you arguing that there should be a single admin address, AND that it 
>should be documented in USEPRO?
>
>I can see some people agreeing with the first part, but not agreeing 
>with the second, and vice versa.

Apart from tidiness what is gained by saying that usenet/newsmaster/news
whatever is not to be used ?   Everyone just aliases them all together
anyway -- and you can hardly dish out "usenet@" for some other purpose
because its historical baggage means that it may still get some useful
email from the unreconstructed, and a lot of spam from being on lists.

I'm all for tidiness, but this horse is long out of the stable.

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Wed, 17 Jan 2007 11:55:08 +0100
From: Harald Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: #1093
Sender: [email-masked]

Just to be sure what you are arguing:

Are you arguing that there should be a single admin address, AND that it 
should be documented in USEPRO?

I can see some people agreeing with the first part, but not agreeing 
with the second, and vice versa.

Frank Ellermann wrote:
> Harald Alvestrand wrote:
>
>   
>> #1093: USEPRO 2.3: Supported email addresses
>>    This has a recorded consensus not to say MUST or SHOULD in USEPRO
>>    about the addresses news@, usenet@ and abuse@.
>>    -07 seems to mention none of them.
>>    Unless new technical information comes to light, I'll close it.
>>     
>
> The abuse@ part of this issue is covered by Injection-Info parameter
> "mail-complaints-to".  
>
> But the mess of not less than three admin addresses specified in 2142
> and s-o-1036, with an "obscure" (= I didn't find it) justification in
> 2142 based on an obsoleted RFC, has to be cleaned up.
>
> If those Martian implementors install a server to test their software
> they're not going to read both 2142 and s-o-1036.  And if they read it
> they could decide that two or three role addresses for one admin does
> not pass Martian giggle tests.
>
> Please update ticket #1093 to say "specify one and only one default
> admin address (updating RFC 2142)".  Obviously this can't be a MUST 
> for any <path-identity>, it won't work without MX (or A/AAAA).
>
> Frank
>
>
>
>   

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  #1093 (was: Ticket status, USEPRO)
Date:  Wed, 17 Jan 2007 11:43:06 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Harald Alvestrand wrote:

> #1093: USEPRO 2.3: Supported email addresses
>    This has a recorded consensus not to say MUST or SHOULD in USEPRO
>    about the addresses news@, usenet@ and abuse@.
>    -07 seems to mention none of them.
>    Unless new technical information comes to light, I'll close it.

The abuse@ part of this issue is covered by Injection-Info parameter
"mail-complaints-to".  

But the mess of not less than three admin addresses specified in 2142
and s-o-1036, with an "obscure" (= I didn't find it) justification in
2142 based on an obsoleted RFC, has to be cleaned up.

If those Martian implementors install a server to test their software
they're not going to read both 2142 and s-o-1036.  And if they read it
they could decide that two or three role addresses for one admin does
not pass Martian giggle tests.

Please update ticket #1093 to say "specify one and only one default
admin address (updating RFC 2142)".  Obviously this can't be a MUST 
for any <path-identity>, it won't work without MX (or A/AAAA).

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID
Date:  Wed, 17 Jan 2007 11:25:01 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 15
Sender: [email-masked]

Russ Allbery wrote:
 
> I think you're right that message ID is special and needs a stronger
> constraint than SHOULD.

+1

Not limited to the Message-ID.  If they don't like a header field they
should reject the message.  

An "add missing magic SP on the fly" strategy could make sense, but in
the times of DKIM even that isn't necessarily okay.

Frank

Date: Tue, 16 Jan 2007 21:03:08 +0100
From: Harald Alvestrand <[email-masked]>
To: [email-masked]
Subject: Ticket status, USEPRO
Sender: [email-masked]

I have now filed tickets for all the issues raised with ISSUE: in their 
subject lines, except for "ISSUE: Injecting agents and From", which I do 
not see any reason to pursue. I'll file it if anyone but Charles wants 
to argue that a change is needed based on what Charles wrote.

The following tickets are open:

#1083: USEPRO 5.3: Rules for generating message-ID
#1093: USEPRO 2.3: Supported email addresses
   This has a recorded consensus not to say MUST or SHOULD in USEPRO 
about the addresses news@, usenet@ and abuse@.
   -07 seems to mention none of them.
   Unless new technical information comes to light, I'll close it.
#1412: USEPRO 5.3: Cancel newsgroups: Matching
#1413: USEPRO 5.5: ihave/sendme syntax
#1414: USEPRO 3.2.1: delimiter for multiple Path identities
#1415: USEPRO 3.2.1: Number of path entries per site
#1416: USEPRO 3.9: Reinjection and Injection-Date
#1417: USEPRO 3.4: Injecting-agent modification of message-ID
   The 2 people discussing seem to agree that document should say "MUST 
NOT modify a message-ID already present", or
   words to that effect.

When following up, please place the ticket number in the subject line, 
and follow up ONE ticket per message, unless the message proposes that 
two tickets should be merged.

                      Harald

From: Russ Allbery <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: ISSUE: USEPRO 3.4: Injecting-agent modification of message ID (was: Re: Multi-point injection)
Organization: The Eyrie
Date: Tue, 16 Jan 2007 11:33:53 -0800
Sender: [email-masked]

Forrest J Cavalier <[email-masked]> writes:

> In a quick review, I don't see anything under "Duties of an Injection
> Agent" that a Message ID field present in a proto-article MUST be
> preserved.  (This is required for multi-point injection.)

> What am I missing?

Well, there's:

   6.   The injecting agent MUST NOT alter the body of the article in
        any way (including any change of Content-Transfer-Encoding).  It
        MAY add other header fields not already provided by the poster,
        but injecting agents are encouraged to use the Injection-Info
        header for such information and to minimize the addition of
        other headers.  It SHOULD NOT alter, delete, or reorder any
        existing header field except the Path header.

but I think you're right that message ID is special and needs a stronger
constraint than SHOULD.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: USEPRO 3.2.1 - Number of path entries per site
Date:  Tue, 16 Jan 2007 20:18:07 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Charles Lindsey wrote:

>    Except possibly when relaying to other hosts on the same site, every
>    injecting, relaying, or serving agent that accepts an article MUST
>    update the Path header field ....

An s/MAY/SHOULD/ in the -07 text might also do what you want.  If that's
not your intention I fail to see the difference from the MAY.

> it is _essential_ that the leftmost <path-identity> that is added
> be the one known to the upstreams (case 1); otherwise, the MISMATCH
> checking is not going to work.

Yes.

> the present USEPRO wording suggests the exact opposite of that

Yes, that could need a note, or a similar minor tweak.  It's to some
degree obvious.  In your example there's no unique way to get it right
for all peers.  Picking a legacy identity as leftmost path identity is
probably a bad idea (wrt simple MISMATCH checks).

>        However, the last (leftmost) such <path-identity> so appended
>        MUST be one that is expected by the destination site when it
>        in turn comes to apply Step 3 above.

Can that fly if different peers expect different leftmost identities ?
If not you can only say SHOULD.

Frank

Date: Tue, 16 Jan 2007 14:12:19 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Multi-point injection
Sender: [email-masked]

In a quick review, I don't see anything under "Duties of an Injection Agent" 
that a Message ID field present in a proto-article MUST be preserved.
(This is required for multi-point injection.)

What am I missing?

Date: Tue, 16 Jan 2007 14:10:38 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Injection-Date and reinjection
Sender: [email-masked]

There is no concise way to describe a "proper" re-injection.  There
are too many special cases and exceptions.  The role of such software
is not clear.  Is it a relay?  Posting agent?  Gateway?

I find 3.3.2 poorly written and redundant.

I consider the mention of "special exceptions" an attractive nuisance.
An implementor looking for justification will always consider themselves
"special."   That's just psychology.

I think mentioning reinjection will entice novices to try something that only 
experts can be expected to do without causing trouble.

I have annotated the draft 3.3.2 below, but the end result is that would
replace 3.3.2 with....

3.3.2 Reinjection of Articles

    Converting an already injected article into a proto-article and forwarding
    it to an injecting agent, "reinjecting", causes problems such as loops,
    loss of trace information, and re-appearance of expired articles, unless
    handled very carefully.

    A posting agent that does reinjection is a limited type of gateway and is
    also subject to all of the requirements of an incoming gateway.  If the
    header must be altered before forwarding to an injecting agent,
    renaming header fields is preferred over removing.  Checks using those
    header fields (that would otherwise be done by the injecting agent) MUST be
    accomplished before the alteration.

Annotating the draft, here is how and why I would make changes.

3.3.2.  Reinjection of Articles

    A given article SHOULD be processed by an injecting agent once and
    only once.

[Recommend: moving to Duties of an injecting agent.]

    The Injection-Date or Injection-Info header fields are
    added by an injecting agent and are not permitted in a proto-article.
[Repeats something from 3.3.1, which repeats something in 3.3 para 2.
Strike it, and remove the repeat in 3.3.1.]

    Their presence (or the presence of other unstandardized or obsolete
    trace headers such as NNTP-Posting-Host, NNTP-Posting-Date, or
    X-Trace) indicates that the proto-article is instead an article and
    has already been processed by an injecting agent.

[Delete it or Move it to Duties of an injecting agent.]

   A posting agent
    SHOULD normally reject such articles.

[How does a posting agent get to reject an article?  The Posting
agent is what is creating a valid proto-article, is it not?  An
injecting agent already MUST reject such articles, as I read the draft.
Strike it.]

    In the exceptional case that an article needs to be reinjected for
    some reason (such as transferring an article from one Netnews to
    another where those networks have no relaying agreement), the posting
    agent doing the reinjection MUST convert the article back into a
    proto-article before passing it to an injecting agent (such as by
    renaming the Injection-Info and Injection-Date header fields and
    removing any Xref header field) and MUST perform the date checks on
    the existing Injection-Date or Date header fields that would
    otherwise be done by the injecting agent.

[I would say strike it all, but the advice to do the date checks and
rename is possibly useful, if it appears at the end of the next paragraph...]

    Reinjecting articles may cause loops, loss of trace information, and
    other problems and should only be done with care and when there is no
    available alternative.  A posting agent that does reinjection is a
    limited type of gateway and as such is subject to all of the
    requirements of an incoming gateway in addition to the requirements
    of a posting agent.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Serving agents and duplicates
Date:  Tue, 16 Jan 2007 19:59:40 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Charles Lindsey wrote:
 
> What is being discussed is whether is is intended/sensible/whatever for
> relayers to refuse to relay the cancelled article itself, on the grounds
> that they have seen and chosen to honour the cancel message.

s-o-1036 has a long explanation how to handle cancels, and it proposes
"reject and discrad" after "not arrived yet".  But it's apparently based
on a "MUST honour cancel" rule.

> So all that we are discussing now is whether to include a NOTE pointing
> this out.

Of course differences from s-o-1036 need explicit justifications.  It
has
an obscure newsmaster@, so now it's three together with usenet@ and
news@.
Is that finally bad enough to get rid of this chaos once and for all ?

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: USEAGE
Organization: The Eyrie
Date: Tue, 16 Jan 2007 10:51:56 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> Nobody's forced to follow RFCs, Russ' "we can't enforce this" line of
> arguments isn't compelling.  He can say "I can't implement requirement
> xy, the xy part of the protocol is typically handled by a human and not
> by a piece of software".  That's not the same as "necessarily no part of
> the protocol" (if the system can't work as expected without xy).

You're misportraying my argument.  My point is that we need to not say
things that are directly in contradiction with reality unless we can
reasonably expect reality to change.  Otherwise, our document is not
useful for writing an interoperable Netnews implementation, thus defeating
the entire point of this exercise.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: USEAGE
Date:  Tue, 16 Jan 2007 19:40:16 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 26
Sender: [email-masked]

Charles Lindsey wrote:
 
> Forrest is right here. USEAGE is largely based on GNKSA, and the GNKSA
> was very much concerned with improving the quality of news servers.

IIRC the N in GNKSA was for "newsreader" (UA), not for "news server".
All I ever did with it was to evaluate my UA.  Based on that I consider
USEAGE as a kind of Emily Postnews for geeks.

Nobody's forced to follow RFCs, Russ' "we can't enforce this" line of
arguments isn't compelling.  He can say "I can't implement requirement
xy, the xy part of the protocol is typically handled by a human and not
by a piece of software".  That's not the same as "necessarily no part
of the protocol" (if the system can't work as expected without xy).

> If a Martian landed in his spaceship and all he had available was the
> set of standards-track RFCs (and a Babefish in his ear so he could
> understand English), then he ought to be able to write an 
> implementation that would interoperate successfully with the rest of
> Usenet.

Including simple modbots as far as I'm concerned, supporting policies
considered as good way to approve articles automatically by Martians.

Frank

To: [email-masked]
Xref: clerew local.usefor:24176
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: Reinjection and Injection-Date
Date: Tue, 16 Jan 2007 12:22:37 GMT
Lines: 172
Sender: [email-masked]

ISSUE Number requested.

This ISSUE concerns whether injecting agents should remove or preserve an
already present Injection-Date header (it is agreed that they should
replace any pre-existing Injection-Info).

There is a separate issue concerning whether injecting agents should
reject articles that are more than 72 hours stale according to the Date
header, but I will raise a separate ISSUE on that one.

There are essentially two options:

Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

Option 2: Injection-Date headers, once written, are NEVER altered.

I prefer Option 2, on the grounds that Option 1 is unsafe in the presence of
unanticipated leaks.

Further details follow:

----------------------------------------------------------------------------

The purpose of the Injection-Date header is to allow relaying and serving
agents to detect articles that may have passed through them previously,
but which may have expired from their history file (some relaying agents
expire their history file in as little as 3 days). This check is important
because, if not made, there are scenarios where the same article might
then be propagated with further delays and then return to the same agent
again and again.

In the absence of an Injection-Date header, the Date-header is to be used
(for interoperability with existing software). However, the purpose of
introducing Injection-Date was to allow Date to be used for the time of
composition of the article, which might predate its injection by many days.

It is to be noted that Usenet propagation is nowadays so rapid that only
exceptionally will this check be called into play. Nevertheless, unusual
setups with delayed paths can exist in a variety of ways (gatewaying,
private networks, etc). If the examples shown below appear improbable and
far fetched, remember that in a network of hundreds of thousands of
servers and millions of end nodes even the most outrageous scenario is
bound to occur sooner or later.

Option 1: Injection-Dates are ALWAYS rewritten by (re-)injecting agents.

We are concerned with an article that is being reinjected. Let us refer to
its two incarnartions as FIRST and SECOND and the person/entity doing the
reinjection as the REINJECTOR. For this option to be safe, it is ESSENTIAL
that FIRST and SECOND were injected into totally disjoint networks, with
absolutely no possibility of leaks between them, and it is the absolute
responsibility of the REINJECTOR to ensure that this condition is met.

Where the REINJECTOR is a conventional incoming gateway (mail2news,
fido2news, etc) it is argued (see 3.9.2) that they already MUST be taking
separate precautions (e.g. tagging) to prevent loops, and hence loss of
the Injection-Date when reinjected into Usenet can be tolerated (though,
oddly, 3.9.1 still recommends preservation of Injection-Date - wherever it
is possible - for outgoing gateways). However, it is also arguable that
the consequences of looping are so severe that having more than one
precaution against it is always a Good Idea.

But the real problems arise when FIRST is injected in a small private
network (even a private news server maintained by an individual poster).
Although it is argued that such a network should be regarded as gatewaying
into the network where SECOND is to be propagated, it is far from certain
that the owner of such a network or such an individual poster will regard
himself as a "gatewayer", or that he will believe that section 3.9 applies
in any way to him. Such small private neworks are notorious for "leaking"
(and it is not necessarily the "owner" of the network who will be the
REINJECTOR that perpetrates the leak).

The only really safe situation is where the REINJECTOR is an individual
poster who has total control of the one and only machine on which the
private network exists. As soon as you escalate to a small organization
with several users, and then to a loose confederation of cooperating
hosts, the possibility of the owner or the REINJECTOR being in a position
to give the required absolute guarantee rapidly recedes.

Consider the following scenario. Suppose the Anytown Squash Club sets up a
private network with a newsgroup anytown.squash, for discussing meetings,
matches, court bookings, and general chat. It is totally informal, uses
dialup copnnections and UUCP, and is barely reliable (propagation can be
slow, especially when some machine is down, or its owner forgets to dialup
as often as he is supposed to). So propagation times of three days are not
uncommon. Who cares?  Everybody knows it is a private network, and that
the 'anytown' hierarchy does not exist anywhere else, so nobody bothers
about security or leaks.  Maybe one or two of the members are also users
of Usenet (how else did they get the idea of using Netnews, and they only
used UUCP because nobody had access to a permanently connected machine to
run an NNTP server). Amateurish and shambolic.

And then, one day, one of the members wanted to raise an issue that went
beyond the mere politics of squash in Anytown, and so he had a bright
idea. "I will crosspost this article to rec.sport.squash". And three days
later his article reaches another member who also had a usenet connection
and was subscribed to rec.sport.squash. Need I say any more?

Option 2: Injection-Date headers, once written, are NEVER altered.

This clearly removes the risks inherent in Option 1. It is inherently much
safer, and it is 'fail-safe' in the sense that the only thing that can go
wrong is that an article whose propagation was genuinely delayed at some
point might fail to propagate beyond a relaying agent with a short
history file expiry. But better to lose a few articles than run the risk
of loops.

Moreover, we MIGHT allow exceptions to this rule in carefully defined
circumstances (whether we define them explicitly in the draft, or leave it
to those who really REALLY know what they are doing to break the rules
when they know it is safe). The only situation I can think of is where
there is a private server run on a single machine whose owner writes an
article and then fails to (re)inject it for several days (like the server
was actually on his laptop). So he is indeed a REINJECTOR who can provide
that absolute guarantee. Moreover, it will always be the case that the
owner of such a private server can in any case make it appear to his
injecting agent that it is really just a user agent (even though it is
internally organized otherwise).

The only scenario that has been suggested which causes problems with this
option is where some (small and local) server/network B, which takes
articles from a well-connected server A, is offline for a week following
some power failure and then seeks to catch up by reading all the missed
articles from A. Note that it only requires these articles for local use -
any role it might normally have had in relaying articles onwards to other
Usenet sites is irrelevant, since these articles would already have
propagated to those other sites by other means.

So it sucks the missing articles and reinjects them into itself,
preserving their injection dates as is required by this option, and hence
possibly triggering the local staleness check.

But since B wants the articles for its own use (not for relaying back into
Usenet), it is presumably a serving agent, in which case its expiry time
is likely to be more than a week, in which case the staleness check is
unlikely to be triggered. But even if that is not so, this is a case where
all B's admin needs to do is to temporarily increase the expiry time of
his history file. This is always safe; even if some site on his local
network tries to relay an article back into Usenet, it is still protected
by its _original_ Injection-Date. The worst that can happen is that a few
articles that had already been received just before his power failure
might be received again (some small error in the time given in his NEWNEWS
command, perhaps); that seems like an acceptably small blip in what is
likely to be a pretty traumatic recovery process.

So what we have to consider is whether the problems encountered in
recovering B's server (which will be localized to his own local users) are
worse or better than the problems caused by the poor design of the Anytown
Squash Club's setup (which may be evident throughout Usenet).

And also whether a reasonable service can be provided to a user who
carries an article around on his laptop for a week before injecting it
into Usenet, and whether it is reasonable for him to omit or "fiddle" his
Injection-Date in that sort of situation (and also whether we would
explicitly mention such small "cheats" in out draft - for sure, that user
has to bear the responsibility if he gets it wrong).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24177
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Empty backlog
Date: Tue, 16 Jan 2007 12:24:58 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Re: Injection-Date and reinjection
>> <[email-masked]> Wed, 3 Jan 2007 16:04:51 GMT

>I think it would be useful for someone else to weigh in here, as you
>apparently find my analysis unpersuasive and I find your analysis
>unpersuasive....

>> Re: Path header field
>> <[email-masked]> Mon, 1 Jan 2007 20:52:37 GMT

>I don't agree.  I'm not sure that I have anything more specific to say.

>> Re: Approved header field content (was: Protocol changes in draft-allbery-usefor-usepro-00) 
>> <[email-masked]> Tue, 2 Jan 2007 19:42:21 GMT

>I don't have anything more to say on this one that I haven't already said.
>I'd just be repeating myself.

I have now raised issues on the first two of these, and may well do so on
the third presently.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24171
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: USEPRO 3.2.1 - Number of path entries per site
Date: Mon, 15 Jan 2007 20:48:46 GMT
Lines: 123
Sender: [email-masked]

ISSUE Number requested. This was #18 and #19 in my list or protocol problems.

In 3.2.1 we have:

   If a relaying or serving agent receives an article from an injecting
   or serving agent that is part of the same news server, it MAY leave
   the Path header field of the article unchanged.  Otherwise, every
   injecting, relaying, or serving agent that accepts an article MUST
   update the Path header field ....

That allows prepending a <path-identity> to be omitted only when the
relaying/serving/injecting agents are all part of the same news server and
any relaying that is done is essentially in the internals of that server.
So you only expect to see one entry in the Path.

But the original intention, when we discussed this in connection with the
Path header in USEFOR (and as reflected in Usepro-06) went further than
this. Essentially, we did not want to REQUIRE more than one
<path-identity> per site (even though the article might get relayed
between several hosts at that site because the admins considered it more
efficient to split things up that way).

One quite often sees that, for example
	!fu-berlin.de!uni-berlin.de!individual.net!
which are all the University of Berlin in various guises, and
	!feed4.jnfs.ja.net!jnfs.ja.net!
and, unless it is really helpful for the site concerned for sorting out
problems later, does not need to be seen by the rest of the net.

I therefore wish to see a wording which gives the site discretion to omit
some of those surplus entries if they are not otherwise useful.

The wording I had in Usepro-06 was "Except possibly when relaying to other
hosts on the same site, it then MUST ....". That could be adapted to the
present text by writing:

   Except possibly when relaying to other hosts on the same site, every
   injecting, relaying, or serving agent that accepts an article MUST
   update the Path header field ....

which also has the advantage of being shorter. It does also suggest that
it is the final agent before it leaves the site that is the one to
include.

Which brings me to my next point, which is where a site, far from reducing
the number of identities it adds, chooses to add some extra ones.
Currently, that is covered by:

   4.  The agent MUST then prepend its own <path-identity> to the Path
       header field content.

   5.  The agent MAY then prepend additional <path-identity>s for itself
       to the Path header field content, following each <path-identity>
       so added with either "!!" or "!".  This is permitted for agents
       that have multiple <path-identity>s (such as during a transition
       from one to another).  Each of these <path-identity>s MUST meet
       the requirements set out in Section 3.2.

Yes, I know Russ has just changed that text slightly, and his changes
should stand, but my concern is with another aspect of the text.

Essentially, as I indicated in a reply to Richard, a site has need of two
identities (mabye even more):

1. It needs to tell its downstreams
      "Articles that I relay to you will come from the following IP
      address(es)/domain name(s): news1.example.net, news2.example.net
      ..., and those are the name(s) which you will see in our leftmost
      <path-identity>."
and then it's downstream can do MISMATCH checks properly.

2. It needs to tell its upstreams
      "Articles already passed through this site will contain one or more
      of the following <path-identities>: example.net, example_net....
      Please do not relay to us any articles with those already in the
      Path."

Now the <path-identity>s in both those two cases will often be the same,
but it is not necessarily so. Moreover, in case #2 it may be necessary to
add and specify more than one identity (say a domain-name and a
<path-nodot>, as in Richard's 'demon' example).

However, it is _essential_ that the leftmost <path-identity> that is added
be the one known to the upstreams (case 1); otherwise, the MISMATCH
checking is not going to work. So any that are put there for purely case 2
purposes MUST be prepended first (i.e. to the right).

But the present USEPRO wording suggests the exact opposite of that, and
will therefore lead to MISMATCH problems. So I think what you need is a
single step something like:

   4.  The agent MUST then prepend one or more <path-identity>s
       identifying itself (as set out in section 3.2) to the Path header
       field content, following each <path-identity> so added with either
       "!!" or "!" {OK, that needs Russ' revised '!'  wording, which I
       don't have in front of me right now}. However, the last (leftmost)
       such <path-identity> so appended MUST be one that is expected by
       the destination site when it in turn comes to apply Step 3 above.

{That will also require some shuffling around of prepended "!"s in Steps 1, 2
and 3 to ensure that the right number of "!"s get prepended in the right
places.}

That could be followed by the present

       This is permitted for agents that have multiple <path-identity>s
       (such as during a transition from one to another).

although that may already be clear enough from 3.2.

And please, can we have a definition of "expected" as referring to the
"verified source" from which the article had been received.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24170
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: USEAGE
Date: Mon, 15 Jan 2007 19:20:46 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Frank Ellermann
><[email-masked]> writes

>>The protocol should at least offer a default way (= news@) to fix the
>>simple case "missed the status change". 

>Why not expect to use the email address in the Injection-Info ? rather
>than overloading the content of path fields -- which are there for very
>different reasons  (phx.gbl is rather common, for example) and don't
>otherwise need to work for email at all

Not really. That "complaints-to" address is intended to reach the 'abuse'
desk, and abuse-desks are notorious for being clueless ignorebots, and are
certainly not the place to complain about configuration issues.

So yes, the "news@" (or "usenet@") address is the proper place to draw
attention to such things, and should hopefully lead to the admins who
maintain the site (who are also often noted for cluelessness, but not to
the same extent as "abuse@").

But the last time the WG discussed this, we agreed not to make any hard
connection between <path-identity>s and "news@". RFC 2142 tried to do
that, and failed. USEAGE might try to fix it (it is still a Good Idea for
those addresses to exist, and it would be in order to USEPRO to NOTE the
need and direct the reader to USEAGE).

>> It took the moderators of a
>>German NG months to convince Google that the NG is now moderated.

>I don't think putting requirements to support the address
>[email-masked]  into the document will make a lot
>of difference to that :(

The Ultimate Sanction, of course, is the UDP :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24169
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: USEAGE
Date: Mon, 15 Jan 2007 19:11:09 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Okay, that would also fall under the "policy" category.  But for some
>cases it's not absolutely clear what "valid according to the protocol"
>is.  If some server admins think that the status of a WG is moderated,
>and others think it's unmoderated, maybe because one group missed the
>status change, or even intentionally doesn't support it, it's messy.

I think it is clear from USEPRO that a group is evidently MEANT to be
moderated everywhere, or nowhere. But the USEPRO provides no mechanism to
enforce this (though Usepro-06 did at least allude to the existence of
hierarchy administrators to be in charge of such things). But I think we
are agreed that enforcement is not for this document, and that there are
going to be individual maverick sites who deliberately choose to 'go it
alone'. Even USEAGE can do no more than point out that the problem exists,
and that its only solution lies in achieving a concensus amongst admins
and users. At least USEPRO encourages other relaying sites not to
propagate articles in moderated groups without an Approved header.

>The protocol should at least offer a default way (= news@) to fix the
>simple case "missed the status change".  It took the moderators of a
>German NG months to convince Google that the NG is now moderated.

Indeed. The larger the organization, the larger the cluelessness :-( . But
"news@" is no magic solution, and the WG has already decided not to
standardize that. So it's no use complaining here - you have to persuade
the chairs to allow the issue to be reopened.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24168
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: USEAGE (was: Serving agents and duplicates)
Date: Mon, 15 Jan 2007 18:04:55 GMT
Lines: 55
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Forrest J. Cavalier III wrote:

>>> USEAGE is for users and UAs, not for server admins and implementors.
> 
>> You have repeated this recently, but I think it is incorrect.

>Then it's time to update the WG Charter.

Please! No new charter! We have enough to keep us busy without getting
bogged down in Charter Wars.

But Forrest is right here. USEAGE is largely based on GNKSA, and the GNKSA
was very much concerned with improving the quality of news servers.

Essentially, USEAGE is concerned with establishing "Best Practice" in
areas where the protocol standard permits things which, whilst not causing
things to break, could severely disrupt the smooth operation of Usenet.

>Stuff related to body conventions, GNKSA, netiquette, attribution
>lines, and what else.  Not very technical, not standards track.

And a lot of stuff relating to cancels (e.g. 1st, 2nd and 3rd party) and
other stuff related to news servers. What is expected of hierarchy admins.
Moderation. And lots more.

>Checking some old articles posted by Alexey he apparently preferred
>to move all "policy" and "user interface" issues to USEAGE, and he
>also wanted the WG to decide what's eventually going where.

Yes, but that does not mean that things not related to "policy" and "user
interface" don't belong there.

>Whatever we decide, we shouldn't abuse USEAGE as dumping ground for
>controversial USEPRO issues.

I agree with that, and I do discern a slight tendency to push too much
into USEAGE. Each case on its merits. USEPRO must work as a standalone
document (you cannot force implementors to read USEAGE). If a Martian
landed in his spaceship and all he had available was the set of
standards-track RFCs (and a Babefish in his ear so he could understand
English), then he ought to be able to write an implementation that would
interoperate successfully with the rest of Usenet.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24167
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Minor change: Xref wording
Date: Mon, 15 Jan 2007 17:48:15 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Seth Breidbart wrote:

>> I don't know about "(and usually will)"; non-crossposted
>> articles don't need them

>Still nice if you want to create nntp-URLs.  On GMane it's
>good to know the article number for various services like
>spam reporting, get "raw" article with HTTP, and so on.

And another reason why all articles should have an Xref, whether
crossposted or not, is that reading agents usually maintain a ".newsrc"
file, in a widely used format that relies heavily on article numbers.

So reading agents will expect to have access to these numbers (whether
from the article itself or from the overview).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24166
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates
Date: Mon, 15 Jan 2007 17:44:08 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> NOTE: It is less usual for cancels to be honored by relaying agents,
>> since it is serving agents that must ultimately decide whether their
>> clients should see the affected articles. It may, however, sometimes be
>> useful to reduce the load and expedite propagation of other articles.

>> That wording could probably be improved.

>This feels like an implementation decision rather than a protocol issue to
>me.  I can see the appeal of trying to distill these various interesting
>points we've discussed on the mailing list into a document so that people
>can understand more how Netnews software works, but the standards-track
>protocol specification doesn't feel to me to be the right place to do
>that.

The general point at issue here is the extent to which a standards-track
protocol specification should include NOTEs (or other parenthetic remarks)
to explain the reasoning behind requirements (or omission thereof) whose
reasons might not be immediately apparent to the reader (to the "less than
omniscient reader" as Henry Spencer so eloquently put it).

It is sadly true that implementors tend to ignore requirements whose
purpose they do not fully comprehend (and that accounts for a lot or
broken implementations of all sorts of protocols). Standards writers need
to be aware of this and to write documents that not only give the correct
requirements, but are also written with an understanding of how people
might misread them (lawmakers could also learn that lesson too :-( ).

How do you know what is going to be midunderstood? Difficult, but if some
member of this WG misreads/misunderstands some part of the spec, then that
is prima facie evidence that other future readers might well do the same.

For sure, many such cases can be covered by directing attention to USEAGE.
It depends how serious the consequences of the misunderstanding might be.
Whether this particular matter is a case in point is a matter for debate,
but you cannot, in general, turn down all such requests for clarifications
on the grounds that "this document just has to describe the protocol".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24165
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates
Date: Mon, 15 Jan 2007 17:25:30 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>For USEPRO it must be clear that not honouring a Cancel is one
>thing, and refusing to relay it another mostly unrelated thing.

Hey! I think you're attacking the wrong target. Nobody is suggesting
refusing to relay a cancel message. The protocol clearly intends that
cancel messages should progagate just like other articles, so that
everybody gets to see them and can decide for themselves whether to honour
them.

What is being discussed is whether is is intended/sensible/whatever for
relayers to refuse to relay the cancelled article itself, on the grounds
that they have seen and chosen to honour the cancel message.

My view us that they should not be doing that (let serving agents decide
for themselves whether to honour the cancel) and I was not aware that it
was an allowable practice, and hence it never got mentioned in Usepro-06.

Then Russ pointed out that a few relaying agents _did_ do that in order to
avoid the load of propagating vast quantities of spam (of which there
seems to be less around these days, probably because the spammers find it
more profitable to send spam by email). It is agreed that the practice is
not common (and with current spam levels probably unnecessary).

So all that we are discussing now is whether to include a NOTE pointing
this out.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: USEAGE
Organization: The Eyrie
Date: Sun, 14 Jan 2007 18:42:38 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:

>> It's called a newgroup control message, which can be sent multiple times
>> and which clearly states the moderation status at a protocol level.

> If a site ignores the newgroup updating the status for unknown reasons
> sending it again and again is no plan.  Something has to be fixed
> manually.

Which is, hence, outside the scope of the *protocol* and in the arena of
individual choices about configuration.  Marking a group as unmoderated is
not a protocol error.  It is a free choice of the local administrator.
Others are free to argue with them if they believe that choice to be
erroneous.

>> I don't see what standardizing a contact address will do to help.

> It's the same idea as for DNS (hostmaster), SMTP (postmaster), ftp
> (ftp), http (webmaster), and more.  It could help.  Especially for
> underspecified features like moderated newsgroups.

I don't believe that moderated newsgroups are underspecified.  There's
enough information in our current document to implement moderated
newsgroups in an interoperable fashion.  They're just not specified the
way that you would like them to work, with strict limitations on what the
moderator is allowed to do.  Such limitations aren't specified for mailing
lists either.

We could have a long argument about what moderated newsgroups should be
like in an ideal world (and indeed keep having that argument even though
it's pointless), but we have essentially *no* enforcement ability here.
We can specify how moderated newsgroups actually behave (which is largely
at the discretion of the moderator), or we can make something up about how
you would like them to work and have the resulting standard be useless for
implementors because, in the real world, they wouldn't work that way.

> If USEPRO doesn't update RFC 2142 then 2142 is still "officially" state
> of the art wrt NNTP, that can't be what TINW want.

I don't think that's how standards work.  Nothing in NNTP refers to RFC
2142 and compliance with it is not a requirement for NNTP.  It looks to me
like sites can choose whether they wish to comply with RFC 2142 or not
independent of whether they choose to implement NNTP, and I don't see any
serious problems with the current situation that we need to address.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: USEAGE
Date:  Mon, 15 Jan 2007 02:49:03 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Russ Allbery wrote:

> It's called a newgroup control message, which can be sent multiple times
> and which clearly states the moderation status at a protocol level.

If a site ignores the newgroup updating the status for unknown reasons
sending it again and again is no plan.  Something has to be fixed manually.

> I don't see what standardizing a contact address will do to help.

It's the same idea as for DNS (hostmaster), SMTP (postmaster), ftp (ftp),
http (webmaster), and more.  It could help.  Especially for underspecified
features like moderated newsgroups.

If USEPRO doesn't update RFC 2142 then 2142 is still "officially" state of
the art wrt NNTP, that can't be what TINW want.

Frank

Date: Mon, 15 Jan 2007 01:01:41 +0000
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: USEAGE
Sender: [email-masked]

In message <[email-masked]>, Frank Ellermann
<[email-masked]> writes

>If some server admins think that the status of a WG is moderated,
>and others think it's unmoderated, maybe because one group missed the
>status change, 

it doesn't matter very much until they accept a posted article

>or even intentionally doesn't support it, it's messy.

if it's intentional, then all the email in the world won't help

>The protocol should at least offer a default way (= news@) to fix the
>simple case "missed the status change". 

Why not expect to use the email address in the Injection-Info ? rather
than overloading the content of path fields -- which are there for very
different reasons  (phx.gbl is rather common, for example) and don't
otherwise need to work for email at all

> It took the moderators of a
>German NG months to convince Google that the NG is now moderated.

I don't think putting requirements to support the address
[email-masked]  into the document will make a lot
of difference to that :(

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Sun, 14 Jan 2007 16:49:45 -0800
Sender: [email-masked]

Russ Allbery <[email-masked]> writes:
> Charles Lindsey <[email-masked]> writes:

>>      8. It MAY delete any Xref header field already present. It MAY add a
>>         new Xref header field [for its own use/convenience] (but recall
>>         that [USEFOR] permits at most one such field).

>> {observe I still prefer to include "for its own ...", as exaplained to
>> Seth.}

> Works for me.  Anyone have any objections?

>    7.  It MUST (except when specially configured to preserve the
>        <article-locator>s set by the sending site) remove any Xref
>        header field from each article.  It then MAY (and usually will)
>        add a new Xref header field (but recall that [USEFOR] permits at
>        most one such field).

> I'll commit this tomorrow unless anyone has objections.

Applied.  I changed field to header field in the parenthetical for
consistency with the wording used elsewhere.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 14 Jan 2007 16:48:50 -0800 (PST)
Sender: [email-masked]

    Date: Sunday, January 14, 2007 @ 16:48:49
  Author: eagle
Revision: 2243

"header field" not "field" for wording consistency.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-01-15 00:43:01 UTC (rev 2242)
+++ docs/usefor/usepro.xml	2007-01-15 00:48:49 UTC (rev 2243)
@@ -911,7 +911,7 @@
             &lt;article-locator>s set by the sending site) remove any Xref
             header field from each article.  It then MAY (and usually
             will) add a new Xref header field (but recall that <xref
-            target="USEFOR" /> permits at most one such field).</t>
+            target="USEFOR" /> permits at most one such header field).</t>
 
             <t>Finally, it stores the article and makes it available for
             reading agents.</t>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: USEAGE
Organization: The Eyrie
Date: Sun, 14 Jan 2007 16:47:55 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:

> The protocol should at least offer a default way (= news@) to fix the
> simple case "missed the status change".  It took the moderators of a
> German NG months to convince Google that the NG is now moderated.

The protocol does.  It's called a newgroup control message, which can be
sent multiple times and which clearly states the moderation status at a
protocol level.  If a given site declines to honor such control messages,
I don't see what standardizing a contact address will do to help.
(Particularly since Google *has* such a contact address, and it never
helps with getting them to fix such things.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 14 Jan 2007 16:43:02 -0800 (PST)
Sender: [email-masked]

    Date: Sunday, January 14, 2007 @ 16:43:01
  Author: eagle
Revision: 2242

Improve wording of Xref modifications allowed by relaying and serving
agents and reference USEFOR for the requirement that there be at most
one Xref header field.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-01-15 00:39:13 UTC (rev 2241)
+++ docs/usefor/usepro.xml	2007-01-15 00:43:01 UTC (rev 2242)
@@ -832,11 +832,10 @@
             <t>It MUST update the Path header field as described in <xref
             target="path-construct" />.</t>
 
-            <t>It MAY delete any Xref header field present and MAY add a
-            new Xref header field with any valid content.  The Xref header
-            field is not used by relaying agents, but relaying agents that
-            are also serving agents may generate Xref header fields for
-            their own internal purposes.</t>
+            <t>It MAY delete any Xref header field already present.  It
+            MAY add a new Xref header field for its own use (but recall
+            that <xref target="USEFOR" /> permits at most one such
+            header field).</t>
 
             <t>Finally, it passes the article on to other relaying and
             serving agents to which it is configured to send articles.</t>
@@ -911,7 +910,8 @@
             <t>It MUST (except when specially configured to preserve the
             &lt;article-locator>s set by the sending site) remove any Xref
             header field from each article.  It then MAY (and usually
-            will) generate a fresh Xref header field.</t>
+            will) add a new Xref header field (but recall that <xref
+            target="USEFOR" /> permits at most one such field).</t>
 
             <t>Finally, it stores the article and makes it available for
             reading agents.</t>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 14 Jan 2007 16:39:14 -0800 (PST)
Sender: [email-masked]

    Date: Sunday, January 14, 2007 @ 16:39:13
  Author: eagle
Revision: 2241

Clarify the wording of "!!" path-diagnostics and mention that RFC1036
didn't include path-diagnostic and therefore older implementations
will not add them.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-01-12 04:03:11 UTC (rev 2240)
+++ docs/usefor/usepro.xml	2007-01-15 00:39:13 UTC (rev 2241)
@@ -355,7 +355,8 @@
                   <t>If the expected &lt;path-identity> of the source of
                   the article matches the leftmost &lt;path-identity> of
                   the Path header field's content, use "!"
-                  (&lt;diag-match>).</t>
+                  (&lt;diag-match>), resulting in two consecutive
+                  "!"s.</t>
 
                   <t>If the expected &lt;path-identity> of the source of
                   the article does not match, use "!.MISMATCH." followed
@@ -367,7 +368,10 @@
                   followed by the FQDN, IP address, or expected
                   &lt;path-identity> of the source.</t>
                 </list>
-              </t>
+              Be aware that <xref target="RFC1036" /> did not include
+              &lt;path-diagnostic>.  Implementations which predate this
+              specification will add only single "!" characters between
+              &lt;path-identity> strings.</t>
 
               <t>The agent MUST then prepend its own &lt;path-identity> to
               the Path header field content.</t>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: USEAGE
Date:  Mon, 15 Jan 2007 01:19:10 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 29
Sender: [email-masked]

Harald Alvestrand wrote:

> if things can be done 2 ways, both of which are valid according to the
> protocol, and neither of which harms interoperability, but the WG thinks
> that one of the ways is better for the users in many common cases than
> the other, and the world needs to be told that, that thing belongs in
> USEAGE if it is to be stated by the WG at all.

Okay, that would also fall under the "policy" category.  But for some
cases it's not absolutely clear what "valid according to the protocol"
is.  If some server admins think that the status of a WG is moderated,
and others think it's unmoderated, maybe because one group missed the
status change, or even intentionally doesn't support it, it's messy.

The protocol should at least offer a default way (= news@) to fix the
simple case "missed the status change".  It took the moderators of a
German NG months to convince Google that the NG is now moderated.  This
is a really bad situation - in that case for the users trying to post
via Google - and I'm talking about a "newusers questions" group... :-(

> Of recent discussions, I suspect that most words about mangling of
> messages to moderated newsgroups by the moderator is the clearest
> example of such an issue.

If the "moderated" concept is too unclear to work in some predictable
way getting rid of it completely would simplify USEPRO and USEAGE.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Serving agents and duplicates
Date:  Mon, 15 Jan 2007 00:59:32 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 17
Sender: [email-masked]

Harald Alvestrand wrote:

> I say (technical contributor hat) "drop it".

Please post a picture with that hat :-)  How about this note:

| If server admins decide that they don't wish to honour a
| Cancel locally (in the role as serving agent) they would
| typically still relay the Cancel to their peers.

Are "don't relay => reject" and "reject => don't honour" 
obvious, or does this need MUSTard ?  It could be confusing
if they honour the Cancel, but don't relay it - unless it's
the effect of a corresponding Distribution header field.

Frank

Date: Sun, 14 Jan 2007 12:09:04 +0100
From: Harald Alvestrand <[email-masked]>
To: Frank Ellermann <[email-masked]>
Cc: [email-masked]
Subject: Re: USEAGE
Sender: [email-masked]

Frank Ellermann wrote:
> Forrest J. Cavalier III wrote:
>
>   
>>> USEAGE is for users and UAs, not for server admins and implementors.
>>>       
>  
>   
>> You have repeated this recently, but I think it is incorrect.
>>     
>
> Then it's time to update the WG Charter.  Sam wrote that he'd clear
> his [DISCUSS], apparently he forgot to actually do this.  If -12 is
> also good enough for Russ USEFOR should get its approval really soon
> now, and we can tackle the WG Charter based on an approved RFC   
>
>   
>> Do you have a basis for your statement?
>>     
>
> http://article.gmane.org/gmane.ietf.usenet.format/24423/match=useage
>   
This was Charles' first attempt at split, 2003.
> http://article.gmane.org/gmane.ietf.usenet.format/26531/match=useage
>   
This is Alexey talking about the difference between Informational and 
BCP, 2004. He was wrong on some properties of Informational (a LOT of 
info is WG product), but I think our current goal is BCP.
> http://article.gmane.org/gmane.ietf.usenet.format/26684/match=useage
>   
Alexey doesn't say what goes where here.
> Stuff related to body conventions, GNKSA, netiquette, attribution
> lines, and what else.  Not very technical, not standards track.
>
>   
>> Can the chair(s) please state something?
>>     
>
> Checking some old articles posted by Alexey he apparently preferred
> to move all "policy" and "user interface" issues to USEAGE, and he
> also wanted the WG to decide what's eventually going where.
>   
This doesn't mean that USEAGE has nothing outside these 2 groups.
> Whatever we decide, we shouldn't abuse USEAGE as dumping ground for
> controversial USEPRO issues.
>   
I think (chair hat on) that if things can be done 2 ways, both of which 
are valid according to the protocol, and neither of which harms 
interoperability, but the WG thinks that one of the ways is better for 
the users in many common cases than the other, and the world needs to be 
told that, that thing belongs in USEAGE if it is to be stated by the WG 
at all.

Of recent discussions, I suspect that most words about mangling of 
messages to moderated newsgroups by the moderator is the clearest 
example of such an issue.

                          Harald

Date: Sun, 14 Jan 2007 11:57:21 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: Serving agents and duplicates
Sender: [email-masked]

Charles Lindsey wrote:
>
>
> Well we agree at least to the extent that relaying agents are less likely
> to honour cancels than serving agents. I think that point is worth
> mentioning, so would you accept a NOTE, perhaps somethng like this,
> probably in section 5.3?
>
> NOTE: It is less usual for cancels to be honored by relaying agents, since
> it is serving agents that must ultimately decide whether their clients
> should see the affected articles. It may, however, sometimes be useful to
> reduce the load and expedite propagation of other articles.
>
> That wording could probably be improved.
>   
We can live without that wording.

As Russ says, it adds complexity, and makes no difference in what is 
permitted or required.

I say (technical contributor hat) "drop it".

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  USEAGE (was: Serving agents and duplicates)
Date:  Sun, 14 Jan 2007 00:52:18 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 31
Sender: [email-masked]

Forrest J. Cavalier III wrote:

>> USEAGE is for users and UAs, not for server admins and implementors.
 
> You have repeated this recently, but I think it is incorrect.

Then it's time to update the WG Charter.  Sam wrote that he'd clear
his [DISCUSS], apparently he forgot to actually do this.  If -12 is
also good enough for Russ USEFOR should get its approval really soon
now, and we can tackle the WG Charter based on an approved RFC   

> Do you have a basis for your statement?

http://article.gmane.org/gmane.ietf.usenet.format/24423/match=useage
http://article.gmane.org/gmane.ietf.usenet.format/26531/match=useage
http://article.gmane.org/gmane.ietf.usenet.format/26684/match=useage

Stuff related to body conventions, GNKSA, netiquette, attribution
lines, and what else.  Not very technical, not standards track.

> Can the chair(s) please state something?

Checking some old articles posted by Alexey he apparently preferred
to move all "policy" and "user interface" issues to USEAGE, and he
also wanted the WG to decide what's eventually going where.

Whatever we decide, we shouldn't abuse USEAGE as dumping ground for
controversial USEPRO issues.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Sat, 13 Jan 2007 13:07:26 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

>> How about:

>>    8. It MAY delete any Xref header field already present and MAY add a
>>       new Xref header field provided that the resulting article has at
>>       most one Xref header field.

>      8. It MAY delete any Xref header field already present. It MAY add a
>         new Xref header field [for its own use/convenience] (but recall
>         that [USEFOR] permits at most one such field).

> {observe I still prefer to include "for its own ...", as exaplained to
> Seth.}

Works for me.  Anyone have any objections?

>>   7.  It MUST (except when specially configured to preserve the
>>       <article-locator>s set by the sending site) remove any Xref
>>       header field from each article.  It then MAY (and usually will)
>>       add a new Xref header field provided that the resulting article
>>       has at most one Xref header field.

> You could fix this the same way as above.

   7.  It MUST (except when specially configured to preserve the
       <article-locator>s set by the sending site) remove any Xref
       header field from each article.  It then MAY (and usually will)
       add a new Xref header field (but recall that [USEFOR] permits at
       most one such field).

I'll commit this tomorrow unless anyone has objections.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Sat, 13 Jan 2007 14:56:19 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: Usefor Mailing List <[email-masked]>
Subject: Re: Serving agents and duplicates
Sender: [email-masked]

Frank Ellermann wrote:
> Forrest J. Cavalier III wrote:
> 
> 
>>I oppose adding any note about this to USEPRO.  I do not expect
>>USEPRO to anticipate and explain all the ways Netnews software
>>might work.  Maybe USEAGE could be the place for such a note.
> 
> 
> USEAGE is for users and UAs, not for server admins and implementors.

You have repeated this recently, but I think it is incorrect.  I
read the draft abstract of USEAGE quite differently than you do.
Do you have a basis for your statement?

Some sections already in the USEAGE draft cover the implementation
and operation of servers.  How is it not for server admins and
implementors?

It seems the particpants of the working group disagree on how the
documents are to be split.

Can the chair(s) please state something?  It has been quite a while.

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Serving agents and duplicates
Date:  Sat, 13 Jan 2007 18:36:03 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Forrest J. Cavalier III wrote:

> I oppose adding any note about this to USEPRO.  I do not expect
> USEPRO to anticipate and explain all the ways Netnews software
> might work.  Maybe USEAGE could be the place for such a note.

USEAGE is for users and UAs, not for server admins and implementors.

For USEPRO it must be clear that not honouring a Cancel is one
thing, and refusing to relay it another mostly unrelated thing.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Minor change: Xref wording
Date:  Sat, 13 Jan 2007 18:26:04 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Seth Breidbart wrote:

>> It MUST (except when specially configured to preserve the
>> <article-locator>s set by the sending site)
 
> Isn't that again the definition of SHOULD?

No, it's better.  With a SHOULD you need a good excuse to
violate it, and ideally there's a list of common excuses
(like "old app" etc.).  In the end you decide what's good.

With "MUST except" only the listed exceptions are allowed,
a proper subset of the SHOULD definition.

> I don't know about "(and usually will)"; non-crossposted
> articles don't need them

Still nice if you want to create nntp-URLs.  On GMane it's
good to know the article number for various services like
spam reporting, get "raw" article with HTTP, and so on.

Frank

Date: Fri, 12 Jan 2007 23:35:23 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Serving agents and duplicates
Sender: [email-masked]

Russ Allbery wrote:

> Charles Lindsey <[email-masked]> writes:
> 
> 
>>Well we agree at least to the extent that relaying agents are less
>>likely to honour cancels than serving agents. I think that point is
>>worth mentioning, so would you accept a NOTE, perhaps somethng like
>>this, probably in section 5.3?
> 
> 
>>NOTE: It is less usual for cancels to be honored by relaying agents,
>>since it is serving agents that must ultimately decide whether their
>>clients should see the affected articles. It may, however, sometimes be
>>useful to reduce the load and expedite propagation of other articles.
> 
> 
>>That wording could probably be improved.
> 
> 
> This feels like an implementation decision rather than a protocol issue to
> me.  I can see the appeal of trying to distill these various interesting
> points we've discussed on the mailing list into a document so that people
> can understand more how Netnews software works, but the standards-track
> protocol specification doesn't feel to me to be the right place to do
> that.
> 
<METOO>
I oppose adding any note about this to USEPRO.  I do not expect USEPRO to 
anticipate and explain all the ways Netnews software might work.  Maybe
USEAGE could be the place for such a note.
</METOO>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Serving agents and duplicates
Organization: The Eyrie
Date: Fri, 12 Jan 2007 14:25:35 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Well we agree at least to the extent that relaying agents are less
> likely to honour cancels than serving agents. I think that point is
> worth mentioning, so would you accept a NOTE, perhaps somethng like
> this, probably in section 5.3?

> NOTE: It is less usual for cancels to be honored by relaying agents,
> since it is serving agents that must ultimately decide whether their
> clients should see the affected articles. It may, however, sometimes be
> useful to reduce the load and expedite propagation of other articles.

> That wording could probably be improved.

This feels like an implementation decision rather than a protocol issue to
me.  I can see the appeal of trying to distill these various interesting
points we've discussed on the mailing list into a document so that people
can understand more how Netnews software works, but the standards-track
protocol specification doesn't feel to me to be the right place to do
that.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24128
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates
Date: Fri, 12 Jan 2007 21:13:59 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> For sure, the primary intent of an honoured cancel is that readers no
>> longer get to see it, which means that serving agents need to take
>> appropriate action (so we say "SHOULD", both in 5.3 and 3.6).

>> But for relaying agents, it is really an optional extra.

>It's really an optional extra for relaying agents to honor cancels at all,
>since as you point out, it may or may not be that useful depending on the
>configuration of the relaying agent and how fast it propagates messages to
>its peers.

Well we agree at least to the extent that relaying agents are less likely
to honour cancels than serving agents. I think that point is worth
mentioning, so would you accept a NOTE, perhaps somethng like this,
probably in section 5.3?

NOTE: It is less usual for cancels to be honored by relaying agents, since
it is serving agents that must ultimately decide whether their clients
should see the affected articles. It may, however, sometimes be useful to
reduce the load and expedite propagation of other articles.

That wording could probably be improved.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24129
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.2.1: delimiter for multiple Path identities
Date: Fri, 12 Jan 2007 21:22:48 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>>    5.  The agent MAY then prepend additional <path-identity>s for itself
>>        to the Path header field content, normally followed by a
>>        <diag-match> "!" (to form "!!"),

>How would that form "!!"?  There's no provision for adding the first "!"
>in your language at that point in the flow.  Going into step 5, the head
>of the Path header is the first path-identity of the system.  If a "!"
>needs to be added, we need to say so explicitly.

I was relying on the wording further up. Essentially, anyone who adds a
<path-identity> (even the same site doing it several times) has the
opportunity to add suitable diagnostics, and it would be better if one
piece of wording could cover the case whether it was the same site or a
different site that was doing it. I agree that, with the present
structure, my wording does not quite do it.

>> Note, however, that I have some other problems with this step and the
>> one before it, as discussed in another thread.

In particular, Steps 4 and 5 do not achieve quite the effect intended.

>Open an issue.

Yes, I shall raise the Steps 4 and 5 problem as an issue, and then maybe
the wording about getting "!!" in the right places will fall out, or can
be reconsidered, as a result of dealing with that.

-- 
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24131
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Minor change: Xref wording
Date: Fri, 12 Jan 2007 21:56:51 GMT
Lines: 76
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Seth Breidbart <[email-masked]> writes:
>> Frank Ellermann <[email-masked]> wrote:
>>> Seth Breidbart wrote:

>>>> Why not just
>>>>     8. It MAY delete any Xref header field already present and MAY add
>>>>        a new Xref header field.

>>> At some point we've to make sure that there's at most one Xref header
>>> field.  How about s/and MAY/and then/ ?  USEFOR states in chapter 3:

>>> Each of these header fields may occur at most once in a news article.

>> That wouldn't allow creating one if there isn't already one to be
>> removed (unless you define "delete any Xref header field already
>> present" as "do nothing" when there's no such header).

>> Simplest would be just to add that it can't add one if one is already
>> and still present.

>How about:

>    8. It MAY delete any Xref header field already present and MAY add a
>       new Xref header field provided that the resulting article has at
>       most one Xref header field.

     8. It MAY delete any Xref header field already present. It MAY add a
	new Xref header field [for its own use/convenience] (but recall
        that [USEFOR] permits at most one such field).

{observe I still prefer to include "for its own ...", as exaplained to
Seth.}

>Wordy, but all the other wordings I came up with were even more wordy.

>Currently, we have:

>   7.  It MUST (except when specially configured to preserve the
>       <article-locator>s set by the sending site) remove any Xref
>       header field from each article.  It then MAY (and usually will)
>       generate a fresh Xref header field.

>for serving agents.  To clear up the same issue and provide consistent
>wording, how about:

>   7.  It MUST (except when specially configured to preserve the
>       <article-locator>s set by the sending site) remove any Xref
>       header field from each article.  It then MAY (and usually will)
>       add a new Xref header field provided that the resulting article
>       has at most one Xref header field.

You could fix this the same way as above.

I agree that the first part of that is correct. The "(except..." is a
special case for sites which need to keep article numbers synchronized
between several serving agents (it might be reworded to make the reason
clearer, but we don't want to enter into a long discussion here).

And I agree that normal behaviour should be for serving agents to generate
an Xref, even where there is only one article in the Newsgroups header.
But this is really a private matter between the serving agents and its
clients, and I think the "MAY (and uaually will)" covers all that pretty
well.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24130
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Minor change: Xref wording (was: Xref and relaying agents)
Date: Fri, 12 Jan 2007 21:32:57 GMT
Lines: 49
Sender: [email-masked]

In <[email-masked]> Seth Breidbart <[email-masked]> writes:

>Russ Allbery <[email-masked]> wrote:
>> Charles Lindsey <[email-masked]> writes:

>>> If the Xrefs in question are not being added by genuine serving agents,
>>> then it is probably better to omit all mention of serving agents at this
>>> point.
>>
>>>    8.  It MAY delete any Xref header field already present and MAY add a new
>>>        Xref header field for its own use.
>>
>>> s/use/convenience/ if you like, to forestall people who might ask "what
>>> possible use might there be?"
>>

>Why not just

>    8. It MAY delete any Xref header field already present and MAY add
>       a new Xref header field.

>There's no reason to specify further with a MAY.

The point is that for a relaying agent to add an Xref serves no useful
purpose for the protocol, and it only happens (as Russ has now pointed
out) because INN finds it less bother to add it that to spot the
situations where it didn't need to. It is, of course, harmless, but of no
use whatsoever to subsequent agents.

Since I was confused as to why this unnecessary provision had been made,
it follows that other readers may be confused too; hence the suggestion of
some wording such as "for its own use/convenience" just to de-confuse
them, by indicating that it is just a private matter for that relaying
agent.

Of course, for serving agents, things are very different, because their
clients want to see Xrefs to prevent showing the same article in different
groups, and maybe for other reasons.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Empty backlog
Organization: The Eyrie
Date: Thu, 11 Jan 2007 19:57:19 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> I also was also hoping for some response from you to:

> Re: Injection-Date and reinjection
> <[email-masked]> Wed, 3 Jan 2007 16:04:51 GMT

I think it would be useful for someone else to weigh in here, as you
apparently find my analysis unpersuasive and I find your analysis
unpersuasive.  Nothing in your message changed anything about my opinion,
and I'm not sure I have much more to say, but if other people discuss the
parts that they find unclear, perhaps I'll see more to discuss.

You should probably open this as an issue.

> Re: Path header field
> <[email-masked]> Mon, 1 Jan 2007 20:52:37 GMT

I don't agree.  I'm not sure that I have anything more specific to say.

> Re: Approved header field content (was: Protocol changes in draft-allbery-usefor-usepro-00) 
> <[email-masked]> Tue, 2 Jan 2007 19:42:21 GMT

I don't have anything more to say on this one that I haven't already said.
I'd just be repeating myself.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Thu, 11 Jan 2007 19:47:20 -0800
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:
> Russ Allbery <[email-masked]> wrote:
>> Seth Breidbart <[email-masked]> writes:
>>> Russ Allbery <[email-masked]> wrote:

>>>> Currently, we have:

>>>>   7.  It MUST (except when specially configured to preserve the
>>>>       <article-locator>s set by the sending site)

>>> Isn't that again the definition of SHOULD?

>> No.  There is only one specific exception allowed, not any exception
>> that the implementor feels is warranted.  I don't believe that should
>> be weakened.

> "It MUST delete it unless it's configured to keep it."

It's saying more than that.  It's talking about preserving the article
locators from a particular sending site.  The one case where keeping the
Xref header is permitted is when the server is configured to do Xref
slaving to another site.  We're trying to avoid getting into the details
of Xref slaving, but we're trying to capture the idea that this has to be
an explicit configuration choice and should not be the default behavior of
a server because it requires a specific setup to work properly.  It will,
for instance, break if the serving agent receives articles in one group
from more than one source, or doesn't carry exactly the same newsgroups as
the sending server.

> I don't see any strength difference.  (Does requiring "special"
> configuring mean anything, compared with, say, "general" configuring?)

I think so.

> Besides, nothing says it can't delete it and replace it with an
> identical copy.

It would violate a SHOULD in Section 3.2, end of first paragraph, but more
than that, that's a bizarre way of describing an action that is a commonly
supported feature of considerable utility in very specific situations.  I
don't want to go through those sorts of unnatural contortions to avoid
explicitly mentioning such an option.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 11 Jan 2007 22:37:56 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: Minor change: Xref wording
Sender: [email-masked]

Russ Allbery <[email-masked]> wrote:
> Seth Breidbart <[email-masked]> writes:
>> Russ Allbery <[email-masked]> wrote:
>
>>> Currently, we have:
>>> 
>>>   7.  It MUST (except when specially configured to preserve the
>>>       <article-locator>s set by the sending site)
>
>> Isn't that again the definition of SHOULD?
>
> No.  There is only one specific exception allowed, not any exception that
> the implementor feels is warranted.  I don't believe that should be
> weakened.

"It MUST delete it unless it's configured to keep it."  I don't see
any strength difference.  (Does requiring "special" configuring mean
anything, compared with, say, "general" configuring?)

Besides, nothing says it can't delete it and replace it with an
identical copy.  The difference between that and not deleting it at
all is rather hard to detect.

Seth

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Thu, 11 Jan 2007 19:31:36 -0800
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:
> Russ Allbery <[email-masked]> wrote:

>> Currently, we have:
>> 
>>   7.  It MUST (except when specially configured to preserve the
>>       <article-locator>s set by the sending site)

> Isn't that again the definition of SHOULD?

No.  There is only one specific exception allowed, not any exception that
the implementor feels is warranted.  I don't believe that should be
weakened.

> I don't know about "(and usually will)"; non-crossposted articles don't
> need them (which doesn't mean that some software doesn't find it easier
> to generate them for all articles anyway).  I suspect that some (most?) 
> servers always will, and some servers usually won't.

Does any modern server not always generate them?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Minor change: Xref wording
Date:  Fri, 12 Jan 2007 04:21:49 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 43
Sender: [email-masked]

Russ Allbery wrote:
 
> Seth Breidbart <[email-masked]> writes:
>> Frank Ellermann <[email-masked]> wrote:
 
>>>>     8. It MAY delete any Xref header field already present and MAY add
>>>>        a new Xref header field.
 
>>> At some point we've to make sure that there's at most one Xref header
>>> field.  How about s/and MAY/and then/ ?  USEFOR states in chapter 3:
 
>>> Each of these header fields may occur at most once in a news article.
 
>> That wouldn't allow creating one if there isn't already one to be
>> removed (unless you define "delete any Xref header field already
>> present" as "do nothing" when there's no such header).

Yes, of course they can't remove something that's not there... :-)
But Seth's right, my simple "and then" proposal also wasn't clear.

> How about:
>     8. It MAY delete any Xref header field already present and MAY add a
>        new Xref header field provided that the resulting article has at
>        most one Xref header field.
 
> Wordy, but all the other wordings I came up with were even more wordy.

Yes, that will do.

>    7.  It MUST (except when specially configured to preserve the
>        <article-locator>s set by the sending site) remove any Xref
>        header field from each article.  It then MAY (and usually will)
>        add a new Xref header field provided that the resulting article
>        has at most one Xref header field.

> Since we allow specially-configured serving agents to not remove the 
> Xref header field, we have to say the same thing here if we want to
> make sure this is clear.

Okay - took me some time to get it, but now I think it's really better.

Frank

Date: Thu, 11 Jan 2007 22:15:03 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Sender: [email-masked]

Russ Allbery <[email-masked]> wrote:

> Currently, we have:
>
>   7.  It MUST (except when specially configured to preserve the
>       <article-locator>s set by the sending site)

Isn't that again the definition of SHOULD?

7. It SHOULD remove any Xref header field from each article.  It MAY
   add a new Xref header field provided that the resulting article has
   at most on Xref header field.

I don't know about "(and usually will)"; non-crossposted articles
don't need them (which doesn't mean that some software doesn't find it
easier to generate them for all articles anyway).  I suspect that some
(most?) servers always will, and some servers usually won't.

Seth

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Thu, 11 Jan 2007 16:58:14 -0800
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:
> Frank Ellermann <[email-masked]> wrote:
>> Seth Breidbart wrote:

>>> Why not just
>>>     8. It MAY delete any Xref header field already present and MAY add
>>>        a new Xref header field.

>> At some point we've to make sure that there's at most one Xref header
>> field.  How about s/and MAY/and then/ ?  USEFOR states in chapter 3:

>> Each of these header fields may occur at most once in a news article.

> That wouldn't allow creating one if there isn't already one to be
> removed (unless you define "delete any Xref header field already
> present" as "do nothing" when there's no such header).

> Simplest would be just to add that it can't add one if one is already
> and still present.

How about:

    8. It MAY delete any Xref header field already present and MAY add a
       new Xref header field provided that the resulting article has at
       most one Xref header field.

Wordy, but all the other wordings I came up with were even more wordy.

Currently, we have:

   7.  It MUST (except when specially configured to preserve the
       <article-locator>s set by the sending site) remove any Xref
       header field from each article.  It then MAY (and usually will)
       generate a fresh Xref header field.

for serving agents.  To clear up the same issue and provide consistent
wording, how about:

   7.  It MUST (except when specially configured to preserve the
       <article-locator>s set by the sending site) remove any Xref
       header field from each article.  It then MAY (and usually will)
       add a new Xref header field provided that the resulting article
       has at most one Xref header field.

Since we allow specially-configured serving agents to not remove the Xref
header field, we have to say the same thing here if we want to make sure
this is clear.

(I would say that it's safe to just defer to USEFOR here on the general
principle that of course agents aren't allowed to create invalid articles,
except that exactly the problem of generating multiple Xref headers has
been seen with poorly written news servers in the wild and is therefore
probably worth a specific mention.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: news@
Date:  Fri, 12 Jan 2007 01:48:41 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 45
Sender: [email-masked]

Charles Lindsey wrote:
 
> you cannot "update 2142" unless you provide some normative text to
> replace it. And our earlier consensus was to NOT write any such
> normative text.

The Chairs decided to start new USEPRO drafts, old decisions wrt the
drafts up to 06 are not necessarily relevant anymore.  We had very
long deliberations about the precise wording down to details like A
and AAAA vs. MX records, but that was thrown away

1:  I support a news@ default, it's needed and useful.  As a SHOULD.
2:  An additional (or alternative) usenet@ is confusing and bogus,
    this part of RFC 2142 has to be removed (by an update).
3:  For the "mismatch" idea it's necessary that the path-identity 
    has an IP matching the connecting IP, or more IPs including the
    connecting IP, is that correct ?
4:  Putting it all together adding an MX for news@<path-dentity> is
    straight forward, but of course they can also run a SMTP server
    directly at the IP of that server for this purpose.

I see zero reasons why those very simple requirements shouldn't go
into the spec.  
 
> RFC 2142 was merely reflecting the existing confusion as to which
> of those addresses was correct/preferable. That confusion still
> persists

Then let's kill it.  RFC 2142 says that both addresses are specified
in RFC 977, but I don't find it.  Besides RFC 977 is now obsolete,
and I'm almost sure that RFC 3977 doesn't mention it.

> Any mention of 2142 in USEPRO would be informative only

With an "updates 2142" we'd be sure to get rid of the cruft.  If an
admin ignores the control messages informing them about a status
change for a group there must be a simple way to discuss such issues
by mail.  

Alternatively let's eliminate the _concept_ of moderated groups from
this draft because its "specification" is far too inadequate and too
incomplete for an RFC, let alone standards track.

Frank

Date: Thu, 11 Jan 2007 19:11:36 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Sender: [email-masked]

Frank Ellermann <[email-masked]> wrote:
> Seth Breidbart wrote:

>> Why not just
>>     8. It MAY delete any Xref header field already present and MAY add
>>        a new Xref header field.
>
> At some point we've to make sure that there's at most one Xref header
> field.  How about s/and MAY/and then/ ?  USEFOR states in chapter 3:
>
> Each of these header fields may occur at most once in a news article.

That wouldn't allow creating one if there isn't already one to be
removed (unless you define "delete any Xref header field already
present" as "do nothing" when there's no such header).

Simplest would be just to add that it can't add one if one is already
and still present.

Seth

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Minor change: Xref wording
Date:  Fri, 12 Jan 2007 00:51:03 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 13
Sender: [email-masked]

Seth Breidbart wrote:

> Why not just
>     8. It MAY delete any Xref header field already present and MAY add
>        a new Xref header field.

At some point we've to make sure that there's at most one Xref header
field.  How about s/and MAY/and then/ ?  USEFOR states in chapter 3:

Each of these header fields may occur at most once in a news article.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording
Organization: The Eyrie
Date: Thu, 11 Jan 2007 14:37:19 -0800
Sender: [email-masked]

Seth Breidbart <[email-masked]> writes:

> Why not just

>     8. It MAY delete any Xref header field already present and MAY add
>        a new Xref header field.

> There's no reason to specify further with a MAY.

I'm fine with that too.  Charles, what do you think?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 11 Jan 2007 16:52:23 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Minor change: Xref wording (was: Xref and relaying agents)
Sender: [email-masked]

Russ Allbery <[email-masked]> wrote:
> Charles Lindsey <[email-masked]> writes:

>> If the Xrefs in question are not being added by genuine serving agents,
>> then it is probably better to omit all mention of serving agents at this
>> point.
>
>>    8.  It MAY delete any Xref header field already present and MAY add a new
>>        Xref header field for its own use.
>
>> s/use/convenience/ if you like, to forestall people who might ask "what
>> possible use might there be?"
>
> I think I added the other bit based on WG discussion, but I don't remember
> for sure.  I'm happy with taking it out again (or maybe for the first time
> if my memory is failing me), but marking this as a Minor change so that
> people have a chance to object.

Why not just

    8. It MAY delete any Xref header field already present and MAY add
       a new Xref header field.

There's no reason to specify further with a MAY.

Seth

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Minor change: Xref wording (was: Xref and relaying agents)
Organization: The Eyrie
Date: Thu, 11 Jan 2007 13:15:31 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> If the Xrefs in question are not being added by genuine serving agents,
> then it is probably better to omit all mention of serving agents at this
> point.

>    8.  It MAY delete any Xref header field already present and MAY add a new
>        Xref header field for its own use.

> s/use/convenience/ if you like, to forestall people who might ask "what
> possible use might there be?"

I think I added the other bit based on WG discussion, but I don't remember
for sure.  I'm happy with taking it out again (or maybe for the first time
if my memory is failing me), but marking this as a Minor change so that
people have a chance to object.

If there are no objections, I'll make this change on Sunday.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24109
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Xref and relaying agents
Date: Thu, 11 Jan 2007 12:21:21 GMT
Lines: 73
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>Stanford's news system doesn't look anything like either of your models.
>It looks like this:

> <other news peers> <-+-----> relaying agent <--+-> serving/injecting agent
>                       \                       /
>                        ----> relaying agent <-

>Those two relaying agents both have article storage so that they can hold
>articles for a while if a peer is slow, or if the internal serving agent
>is slow.  They both generate Xref headers because INN just always
>generates Xref headers no matter what -- not necessarily ideal, but in
>practice completely harmless.  Neither of those relaying agents accept any
>connections from reading agents and could not serve articles to reading
>agents because they both have overview and nnrpd disabled.

Ah! I had not supposed that an Xref header would ever be constructed in
that situation (where does it get its article number from?). It is clearly
unnecessary, and indeed harmless, but anyway justifies a "MAY create it".

>You're focusing way too much on the removal part.  I know of no agent that
>removes Xref headers except when adding its own Xref header.  The draft
>allows removal without adding a new one, just because there's no reason
>not to, but what happens in practice is that relaying agents remove the
>existing Xref header and add a new one.

Sure, Xrefs put there by another system are harmless, even though
meaningless to other systems. Deleting them is probably a good idea, to
remove clutter and possible confusion, but MAY is quite strong enough for
that.

>> This is evident from the words "relaying agents that are also serving
>> agents" which, according to our model, is actually a contradiction in
>> terms. There are relaying agents and there are serving agents, and some
>> *news-servers* may include both (but that does not cause the relaying
>> agent to also "be" a serving agent).

>I'm happy to consider a minor change wording proposal that rewords that in
>some fashion to avoid the apparent contradiction.  I think the existing
>wording is clear, but I don't mind tweaking it if other people don't feel
>the same.

If the Xrefs in question are not being added by genuine serving agents,
then it is probably better to omit all mention of serving agents at this
point.

   8.  It MAY delete any Xref header field already present and MAY add a new
       Xref header field for its own use.

s/use/convenience/ if you like, to forestall people who might ask "what
possible use might there be?"

That takes care of the argument "but according to the protocol there is no
possible way for there to be an already present Xref to be deleted",
because the wording has now provided one ("an earlier relaying agent did
it" - and who cares if it was actually an earlier serving agent by some
quirk of an actual implementation).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24110
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: news@
Date: Thu, 11 Jan 2007 12:28:35 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> I think the consensus, as recorded in the issue after we had discussed it,
>> was that we did not want to take any normative action either for or
>> against RFC 2142. It is so badly written as to be meaningless anyway.

>That's a good reason or at least an opportunity to do an "updates 2142".

But you cannot "update 2142" unless you provide some normative text to
replace it. And our earlier consensus was to NOT write any such normative
text. By all means petition the chairs to reopen that issue if you want.
> 
>> I agree that having "[email-masked]" and "[email-masked]" is
>> a good idea

>Then we actually disagree because I think it's completely confusing to 
>have two defaults where one (news@) is good enough. 

Yes, but RFC 2142 was merely reflecting the existing confusion as to which
of those addresses was correct/preferable. That confusion still persists
(some people use one, some the other; my own system recognizes both, and
aliases both to me).

>> Therefore it is for USEAGE, but it still needs a _mention_ in USEPRO,
>> with a pointer to USEAGE.

>And I really don't want a normative reference to USEAGE, waiting for 
>USEPRO is already very bad.

Any mention of 2142 in USEPRO would be informative only, just to point out
that it existed and encourage implementors to go look at it. It could well
be in a NOTE.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: USEPRO 3.2.1: delimiter for multiple Path identities
Organization: The Eyrie
Date: Thu, 11 Jan 2007 06:36:19 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I think we have two options.  My preferred option would be to say:

>>  5.  The agent MAY then prepend additional <path-identity>s for itself
>>      to the Path header field content.  Each <path-identity> so added
>>      MUST be followed with either "!!" or "!".  Using "!!" is
>>      RECOMMENDED.  This is permitted for agents that have multiple
>>      <path-identity>s (such as during a transition from one to another).
>>      Each of these <path-identity>s MUST meet the requirements set out in
>>      Section 3.2.

> Yes, I think that is the effect we want, but there is no need for the
> MUSTard. I suggest (even though it is slightly longer):

>    5.  The agent MAY then prepend additional <path-identity>s for itself
>        to the Path header field content, normally followed by a
>        <diag-match> "!" (to form "!!"),

How would that form "!!"?  There's no provision for adding the first "!"
in your language at that point in the flow.  Going into step 5, the head
of the Path header is the first path-identity of the system.  If a "!"
needs to be added, we need to say so explicitly.

>        since it will surely recognize the earlier <path-identity>s as
>        its own.  This is permitted for agents that have multiple
>        <path-identity>s (such as during a transition from one to
>        another).  Each of these <path-identity>s MUST meet the
>        requirements set out in Section 3.2.

I don't understand why we would go out of our way to avoid MUST and SHOULD
here, when MUST and SHOULD are used in the rest of the section.  I think
the wording I proposed above is more consistent with the rest of the
section.  I also don't see the justification for downgrading "!!" from
SHOULD to "normally"; that was the flaw that sparked this issue to start
with.

language doesn't explicitly require any delimiter after the additional
path-identities at all; that might be obvious, but I'd rather not write
language, 

> Note, however, that I have some other problems with this step and the
> one before it, as discussed in another thread.

Open an issue.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Serving agents and duplicates
Organization: The Eyrie
Date: Thu, 11 Jan 2007 06:23:05 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> For sure, the primary intent of an honoured cancel is that readers no
> longer get to see it, which means that serving agents need to take
> appropriate action (so we say "SHOULD", both in 5.3 and 3.6).

> But for relaying agents, it is really an optional extra.

It's really an optional extra for relaying agents to honor cancels at all,
since as you point out, it may or may not be that useful depending on the
configuration of the relaying agent and how fast it propagates messages to
its peers.

*If* that relaying agent chooses to honor cancels, which could be
considered an optional extra for relaying agents, *then* it SHOULD act on
them just like any other agent honoring cancels.

I see no reason to increase the complexity of effect of cancel messages by
making it different for different agents when cancels semantically mean
the same thing to all agents and we already provide for agents choosing
not to honor them.  We already (IMO unecessarily) single out serving
agents, and *ideally* I'd rather not even do that, but I can see where
serving agents are the most common and significant case and could stand
explicit mention.  But let's not go farther down that path.  Any agent
(except possibly a posting agent) can usefully choose to honor a cancel
and it means the same thing to any of them, namely "please withdraw the
named article from circulation and access, including proactively if you
haven't seen it yet."

So, opposed, and I think it's unlikely I'm going to change my mind.

If you still think a change is necessary, you should ask to open an issue
and see if other members of the working group support you.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24108
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates
Date: Thu, 11 Jan 2007 11:56:13 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> So I see now what you are getting at. I think I would therefore want to
>> take the view that serving agents SHOULD (modulo site policy) act on
>> cancels, but that for relaying agents it should be, at most, "MAY".

>Given the current security problems with honoring cancels, honoring
>cancels should not be anything more than a MAY.  I prefer the approach
>currently taken by the draft, where that MAY is even implicit, as it is
>for all control messages.  We simply say "here's what the control message
>means and here are the problems with it; whether or not you want to act on
>it is up to you."

>The language that started this discussion is the language in duties that
>explains what the agent needs to do *if the cancel was honored*.  So I
>think you've accidentally changed topics in the middle of this thread, and
>we should get back to the original topic.

But I do not want to confuse the question of "whether a cancel sould be
honoured", which is a political/policy decision, with the purely technical
question of where and how you actually act upon it once the policy is
decided.

For sure, the primary intent of an honoured cancel is that readers no
longer get to see it, which means that serving agents need to take
appropriate action (so we say "SHOULD", both in 5.3 and 3.6).

But for relaying agents, it is really an optional extra. Some relaying
agents have found it useful to take action so as to reduce the amount of
spam clogging the system, but that action is not _necessary_ from the POV
of achieving that primary intent. I think this is a useful distinction to
make, hence my suggestion to use "MAY" in the case of relaying agents.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: I-D ACTION:draft-ietf-usefor-usefor-12.txt
Date:  Wed, 10 Jan 2007 21:31:29 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 10
Sender: [email-masked]

[email-masked] wrote:

> http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

Here's the diff, apparently fine, thanks:

http://tools.ietf.org/rfcdiff?url1=&url2=http%3A%2F%2Ftools.ietf.org%2Fid%2Fdraft-ietf-usefor-usefor-12.txt&difftype=--hwdiff

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Xref and relaying agents
Organization: The Eyrie
Date: Wed, 10 Jan 2007 11:59:59 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> There are essentially two ways in which things may be implemented:

I think that you're considering anything that has article storage to be a
serving agent, which is causing you to talk at cross-purposes to what the
draft actually says.  That's not the definition of a serving agent.  A
serving agent is an agent that makes articles available to reading
agents.  Full stop.

Relaying agents also have article storage.  That does *not* mean that
they're combined with a serving agent.

Stanford's news system doesn't look anything like either of your models.
It looks like this:

 <other news peers> <-+-----> relaying agent <--+-> serving/injecting agent
                       \                       /
                        ----> relaying agent <-

Those two relaying agents both have article storage so that they can hold
articles for a while if a peer is slow, or if the internal serving agent
is slow.  They both generate Xref headers because INN just always
generates Xref headers no matter what -- not necessarily ideal, but in
practice completely harmless.  Neither of those relaying agents accept any
connections from reading agents and could not serve articles to reading
agents because they both have overview and nnrpd disabled.

> But some no doubt follow the RIGHT scheme in which there are unlikely to
> be any Xref headers for the relaying agent to remove. The Model in
> USEPRO essentially sets out the RIGHT scheme, which then necessitates
> some mention of relayers removing Xrefs which, according to the model,
> should never have been there.

You're focusing way too much on the removal part.  I know of no agent that
removes Xref headers except when adding its own Xref header.  The draft
allows removal without adding a new one, just because there's no reason
not to, but what happens in practice is that relaying agents remove the
existing Xref header and add a new one.

This is a purposeless modification to the article, and back in the mists
of time in the working group, Brad Templeton used to argue for disallowing
it.  It turns out, however, to be much easier to write a multi-function
news server if one just always generates Xref headers, and Netnews has
functioned in that environment for over ten years.  Everything is now used
to that possibility, and I don't see any reason to pick a fight and try to
get software to change over something that's quite minor and unimportant.

> This is evident from the words "relaying agents that are also serving
> agents" which, according to our model, is actually a contradiction in
> terms. There are relaying agents and there are serving agents, and some
> *news-servers* may include both (but that does not cause the relaying
> agent to also "be" a serving agent).

I'm happy to consider a minor change wording proposal that rewords that in
some fashion to avoid the apparent contradiction.  I think the existing
wording is clear, but I don't mind tweaking it if other people don't feel
the same.

> Hence my suggeted wording:

>    Any Xref header, whether present on input or added by an associated
>    local serving agent, MAY be deleted before relaying.

This, however, is too broken to be used.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Serving agents and duplicates
Organization: The Eyrie
Date: Wed, 10 Jan 2007 11:45:12 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> So I see now what you are getting at. I think I would therefore want to
> take the view that serving agents SHOULD (modulo site policy) act on
> cancels, but that for relaying agents it should be, at most, "MAY".

Given the current security problems with honoring cancels, honoring
cancels should not be anything more than a MAY.  I prefer the approach
currently taken by the draft, where that MAY is even implicit, as it is
for all control messages.  We simply say "here's what the control message
means and here are the problems with it; whether or not you want to act on
it is up to you."

The language that started this discussion is the language in duties that
explains what the agent needs to do *if the cancel was honored*.  So I
think you've accidentally changed topics in the middle of this thread, and
we should get back to the original topic.

> But, either way, the wording you have written for relaying agents only
> applies to cancels arriving before the actual message, which is not the
> common case (except for sites which deliberately try to make it the
> common case, as you have said).

Correct.  As mentioned previously, I think that's the only case that
requires any special handling in the duties section.

> So it is odd that 5.3 does not mention relaying agents - but I suppose
> that, if the cancel arrives after the message, it is too late for a
> relaying agent to act.

Not true, as I think should be apparent from my previous message on this
thread.

No specific agents are singled out in 5.3 because the meaning of the
cancel message does not change based on what agent is processing it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24099
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates
Date: Wed, 10 Jan 2007 17:30:03 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Harald Alvestrand <[email-masked]> writes:

>> Because if it is not there, a relaying agent that stops messages for which
>> it has seen a cancel would be nonconformant with the specification. My
>> guess is that combined relaying/serving agents would indeed do this.

Looks like a case for a "MAY". Yes, I agree that a combined
relaying/serving agent might not forward the cancelled message since it
might no longer have a copy available for that purpose.

>> If a relaying agent were not allowed to honor cancels, it would be
>> nonconformant if it did.
>> That would be stupid.

Ah! I had always taken the view that serving agents would be expected to
act upon cancels (subject to site policy) but that relaying agents ought
to forward all articles received on the grounds that to do otherwise was
interfering with the right of downstream sites (notably downstream serving
agents) to decide for themselves, according to their site policy, whether
to honour them. And I still think that is the proper behaviour.

>Also, in the days of spam cancels, people went out of their way to receive
>spam cancels on relaying agents prior to the original message so that they
>could avoid even relaying the spam.  This is less of an issue these days,
>but it may still come up in some situations.

So I see now what you are getting at. I think I would therefore want to
take the view that serving agents SHOULD (modulo site policy) act on
cancels, but that for relaying agents it should be, at most, "MAY".
Perhaps USEAGE could then discuss the ethics of doing so. I really don't
like the idea of doing it for "ordinary" cancels, though I can see the
benefit in the case of spam.

But, either way, the wording you have written for relaying agents only
applies to cancels arriving before the actual message, which is not the
common case (except for sites which deliberately try to make it the common
case, as you have said). So it is odd that 5.3 does not mention relaying
agents - but I suppose that, if the cancel arrives after the message, it
is too late for a relaying agent to act. That is an asymmetry which should
perhaps be mentioned.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24101
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Status of the USEFOR document
Date: Wed, 10 Jan 2007 19:22:19 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Ken Murchison <[email-masked]> writes:

>Alexey Melnikov wrote:
>> 
>> Hi folks,
>> After several discussions between Lisa, Harald and myself it became 
>> clear that making the reference to USEPRO normative would be the 
>> easiest/quickest way to remove remaining IESG DISCUSSes. This change 
>> will delay publication of the USEFOR document until the USEPRO document 
>> is approved by IESG.
>> 
>> Any objections to doing that?

>I submitted -12 last night, which makes USEPRO normative and fixes the 
>other DISCUSSes:

I did an rfcdiff, and it seems all correct.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24100
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Xref and relaying agents
Date: Wed, 10 Jan 2007 18:14:36 GMT
Lines: 94
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Charles Lindsey wrote:
>>
>> If the model is left unchanged, then the confusing wording under relaying
>> needs to be clarified. I have already suggested:
>>
>>    Any Xref header, whether present on input or added by an associated
>>    local serving agent, MAY be deleted before relaying.
>>
>> which is an acceptable alternative to my suggested deffinition change. But
>> one or the other needs doing.
>>   
>You have then abandoned the model that relaying agents talk to each 
>other, and that serving agents only talk to their local relaying agents 
>and reading agents.

There are essentially two ways in which things may be implemented:

        -------------------                -------------------
       | outgoing relayed  |              | outgoing relayed  |
       |     articles      |              |     articles      |
        -------------------                -------------------
                 ^                             ^
                 |                            /
        -------------------                  /
       |   relaying agent  |                /
        -------------------                /
                 ^              -------------------    -------------------
                 |             |   relaying agent  |  |   serving agent   |
        -------------------     -------------------    -------------------
       |   serving agent   |            ^                      ^
        -------------------              \                    /
                 ^                        \                  /
                 |                         \                /
        -------------------                ------------------- 
       | incoming relayed  |              | incoming relayed  |
       |     articles      |              |     articles      |
        -------------------                -------------------

I suspect implementations mostly follow the LEFT scheme, which is likely
to give rise to Xref headers created by the serving agent which the
relaying agent MAY then choose to remove.

But some no doubt follow the RIGHT scheme in which there are unlikely to
be any Xref headers for the relaying agent to remove. The Model in USEPRO
essentially sets out the RIGHT scheme, which then necessitates some
mention of relayers removing Xrefs which, according to the model, should
never have been there.

My objection is to the wording in the draft which says, for relayers:

   8.  It MAY delete any Xref header field present and MAY add a new
       Xref header field with any valid content.  The Xref header field
       is not used by relaying agents, but relaying agents that are also
       serving agents may generate Xref header fields for their own
       internal purposes.

and particularly to the bit "and MAY add a new Xref header field", since
is is clear that that never happens in either of the two schemes, and it
is only put there as an artefact to explain why there might be an Xref to
remove.

This is evident from the words "relaying agents that are also serving
agents" which, according to our model, is actually a contradiction in
terms. There are relaying agents and there are serving agents, and some
*news-servers* may include both (but that does not cause the relaying
agent to also "be" a serving agent).

Hence my suggeted wording:

   Any Xref header, whether present on input or added by an associated
   local serving agent, MAY be deleted before relaying.

which is shorter and avoids that contortion of the model by speaking of
an "associated" serving agent (i.e. one contained in the same
news-server).

>You have increased the amount of coupling in the system model, for no 
>benefit except saving a special-case handling of the Xref: header.

Yes I agree that is undesirable, though not harmful in this case. I would
prefer my suggested rewording.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Turning to issue tracking on the USEPRO document
Date:  Wed, 10 Jan 2007 20:17:00 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 18
Sender: [email-masked]

Seth Breidbart wrote:

>> defined in 3.6.4 of RFC 2822, which we inherit via USEFOR
[...]
> Isn't that precisely what SHOULD is for?  "Don't do this, it can break
> stuff, unless you really understand what you're doing and want stuff
> to break in that specific way."

Yes, but it's not "our" MUSTard, it's in 2822.  For the protocol here we
can ignore these intentional collisions, they're working as designed if
the spam cancellers use the same "old" variant of the $alz convention.

The "new" variant with its need-to-know algorithm is anyway too obscure,
and a preemptive cancel against a $alz cancel should be obvious for
those who really understand what they're doing.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: news@
Date:  Wed, 10 Jan 2007 20:02:50 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Charles Lindsey wrote:
 
> I think the consensus, as recorded in the issue after we had discussed it,
> was that we did not want to take any normative action either for or
> against RFC 2142. It is so badly written as to be meaningless anyway.

That's a good reason or at least an opportunity to do an "updates 2142".
 
> I agree that having "[email-masked]" and "[email-masked]" is
> a good idea

Then we actually disagree because I think it's completely confusing to 
have two defaults where one (news@) is good enough. 

> Therefore it is for USEAGE, but it still needs a _mention_ in USEPRO,
> with a pointer to USEAGE.

Server admins and implementors might not read USEAGE, it's for users 
and UAs.  But admins need to know that they're expected to install a
news@ address as default for obscure problems (in my CfV example for
potential issues with the possible status change of an existing group).

And I really don't want a normative reference to USEAGE, waiting for 
USEPRO is already very bad.

Frank

To: [email-masked]
Xref: clerew local.usefor:24095
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Injecting agents and From
Date: Wed, 10 Jan 2007 16:33:54 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Charles Lindsey wrote:
>> I disagree. Historically (as in the days of BNews and Cnews), as I think I
>> have shown, 'inews' was the mechanism which caused injection to take
>> place, and filling in the From header was part of what it did. As I said,
>> the 'i' in 'inews' stands for 'inject' (just as the 'r' in 'rnews' stood
>> for 'relay').
>>
>> When NNTP became the common mode of intereaction with newsreaders,
>> automatic filling in of the From could not then be done, but that did not
>> alter the situation wrt to existing usage - 'inews' did not suddenly
>> cease to become a means of injecting; rather in NNTP contexts it lost some
>> of its functionality.
>>   
>To put it another way:
>In the days of BNews and CNews, the "inews" program formed part of the 
>posting agent as well as part of the injecting agent.

No, I would put it as "inews" is/was the interface to the injecting function
of the newsserver, as used by those useragents which do not make a direct
TCP connection to port 119 themselves (as modern user agents mostly do). But
such user agents do still exist, and are used.

The interface provided by inews is fairly well documented (see the
O'Reilly book, for example). Where an NNTP connection on port 119 is to be
made regardless, 'inews' is simply a wrapper that makes the TCP
connection, but it still supports the traditional interface.

>In an NNTP context, it is only a part of the posting agent.

>PLEASE don't attempt to require exact match between pieces of the model 
>and pieces of implementations that predate the model.

I don't think the basic model has changed since those early days. There
have always been user/injecting/relaying/serving agents connected together
according to out present model (though not necessarily known by that
terminology).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24094
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: news@ (was: Turning to issue tracking on the USEPRO document)
Date: Wed, 10 Jan 2007 16:21:30 GMT
Lines: 32
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

>> I would like to see some mention. RFC 2142 is a standards track RFC which
>> purports to affect Netnews (it should have been infomational, but wasn't).
>> So a mention is certainly in order (however disdainful), USEAGE is the
>> place for any serious discussion, though.

>We can do an "updates 2142" as needed.  In a recent CfV about a status
>change to moderated for a German newsgroup the proponents mentioned news@,
>it's clearly needed and useful.  Less so if there's a choice of news@ and
>usenet@.  The address SHOULD exist.  Not like postmaster@, but still.

I think the consensus, as recorded in the issue after we had discussed it,
was that we did not want to take any normative action either for or
against RFC 2142. It is so badly written as to be meaningless anyway.

I agree that having "[email-masked]" and "[email-masked]" is a
good idea, but not slavishly following 2142. Therefore it is for USEAGE,
but it still needs a _mention_ in USEPRO, with a pointer to USEAGE.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usefor-12.txt 
Date: Tue, 09 Jan 2007 15:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: Netnews Article Format
	Author(s)	: C. Lindsey, et al.
	Filename	: draft-ietf-usefor-usefor-12.txt
	Pages		: 42
	Date		: 2007-1-9
	
This document specifies the syntax of Netnews articles in the context
   of the "Internet Message Format" (RFC 2822) and "Multipurpose
   Internet Mail Extensions (MIME)" (RFC 2045).  This document obsoletes
   RFC 1036, providing an updated specification to reflect current
   practice and incorporating incremental changes specified in other
   documents.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usefor-12.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-usefor-usefor-12.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usefor-12.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usefor-12.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usefor-12.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Tue, 9 Jan 2007 13:27:48 -0500 (EST)
From: Seth Breidbart <[email-masked]>
To: [email-masked]
Subject: Re: Turning to issue tracking on the USEPRO document
Sender: [email-masked]

Russ Allbery wrote:

> If we document an exception for $alz-convention cancels, we're overriding
> the uniqueness constraints defined in 3.6.4 of RFC 2822, which we inherit
> via USEFOR.  I'm not sure if we want to do this or not.  It makes me very
> uncomfortable; it's quite difficult to describe when this is okay and when
> it isn't.

Isn't that precisely what SHOULD is for?  "Don't do this, it can break
stuff, unless you really understand what you're doing and want stuff
to break in that specific way."

Seth

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  news@ (was: Turning to issue tracking on the USEPRO document)
Date:  Tue, 09 Jan 2007 16:46:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 14
Sender: [email-masked]

Charles Lindsey wrote:

> I would like to see some mention. RFC 2142 is a standards track RFC which
> purports to affect Netnews (it should have been infomational, but wasn't).
> So a mention is certainly in order (however disdainful), USEAGE is the
> place for any serious discussion, though.

We can do an "updates 2142" as needed.  In a recent CfV about a status
change to moderated for a German newsgroup the proponents mentioned news@,
it's clearly needed and useful.  Less so if there's a choice of news@ and
usenet@.  The address SHOULD exist.  Not like postmaster@, but still.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Turning to issue tracking on the USEPRO document
Date:  Tue, 09 Jan 2007 16:31:34 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 22
Sender: [email-masked]

Charles Lindsey wrote:
 
>> A pair of converging drafts could be also interesting
>> if Charles is willing to go to that trouble.
 
> Probably not a good idea, unless things go completely 
> off the rails :-( .

Your list of at the moment 42 differences is also a way
to tackle this, converted into "issues" later, and then
resolve them one after the other.

About the security considerations, apparently RussH liked
the style of your version so much that USEFOR now gets a
normative reference to USEPRO.  Therefore it could be a
good idea to keep this part as is, even if it's not the
way how RussA would say ths same things - but there are
no serious substantial differences I'm aware of, it's 
only a matter of the style.  I also liked your wording.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Serving agents and duplicates
Organization: The Eyrie
Date: Tue, 09 Jan 2007 07:26:26 -0800
Sender: [email-masked]

Harald Alvestrand <[email-masked]> writes:

> Because if it is not there, a relaying agent that stops messages for which
> it has seen a cancel would be nonconformant with the specification. My
> guess is that combined relaying/serving agents would indeed do this.

> If a relaying agent were not allowed to honor cancels, it would be
> nonconformant if it did.
> That would be stupid.

Also, in the days of spam cancels, people went out of their way to receive
spam cancels on relaying agents prior to the original message so that they
could avoid even relaying the spam.  This is less of an issue these days,
but it may still come up in some situations.

Note too that relaying agents don't relay immediately in all situations.
Cancelling a message may still have an effect for slow peers, causing the
message to disappear before it can be relayed to them (particularly for
peers that were *intentionally* slowed by five minutes or so, another
strategy commonly used in the days of spam cancels).

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Xref and relaying agents
Date:  Tue, 09 Jan 2007 16:10:17 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 19
Sender: [email-masked]

Harald Alvestrand wrote:

> If you want to question why increasing the amount of coupling in the
> system model is bad, I can get you some references on system modelling
> and object orientation.

Small interfaces are nice.  I'm not exactly sure what this Xref business
is about, therefore I compare it with a Return-Path in mail:  There MUST
be one and only one after delivery, and it has to reflect the MAIL FROM
as seen by the MDA - and any other Return-Paths inserted earlier have to
be removed by the MDA.

For an Xref it's similar wrt the serving agent, any old Xrefs have to be
removed, a new is inserted.  Ideally MUST + MUST, but SHOULD + SHOULD is
good enough if some serving agents don't support the concept at all (?)
Or no 2119 keywords if it's more complex.

Frank

Date: Tue, 09 Jan 2007 08:23:40 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Alexey Melnikov <[email-masked]>
CC: [email-masked], Harald Alvestrand <[email-masked]>, Lisa Dusseault <[email-masked]>
Subject: Re: Status of the USEFOR document
Sender: [email-masked]

Alexey Melnikov wrote:
> 
> Hi folks,
> After several discussions between Lisa, Harald and myself it became 
> clear that making the reference to USEPRO normative would be the 
> easiest/quickest way to remove remaining IESG DISCUSSes. This change 
> will delay publication of the USEFOR document until the USEPRO document 
> is approved by IESG.
> 
> Any objections to doing that?

I submitted -12 last night, which makes USEPRO normative and fixes the 
other DISCUSSes:

http://tools.ietf.org/rfcdiff?url2=http://www.contrib.andrew.cmu.edu/~murch/draft-ietf-usefor-usefor-12.txt

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Tue, 09 Jan 2007 08:14:31 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: [email-masked]
Subject: Re: Turning to issue tracking on the USEPRO document
Sender: [email-masked]

Russ Allbery wrote:
> Here's a rundown of the current three open issues concerning USEPRO in the
> WG tracker:
> 
>> #1051: USEPRO general: Document changes from RFC 1036
>>
>>   Section 3.1 of USEPRO -03 should be updated to only list protocol
>>   related changes.
> 
> This has now been moved to appendix A, and I believe it only lists
> protocol-related changes now.  My recommendation would be to close this
> issue,

Closed.

>> #1083: USEPRO 7.5: Rules for Generating message-ID
> 
> A question about where we should document rules for generating message IDs
> and the <cancel. $alz convention.  I think the context here concerned
> cancel messages, so the new section affected by this issue is 5.3.
> 
> If we document an exception for $alz-convention cancels, we're overriding
> the uniqueness constraints defined in 3.6.4 of RFC 2822, which we inherit
> via USEFOR.  I'm not sure if we want to do this or not.  It makes me very
> uncomfortable; it's quite difficult to describe when this is okay and when
> it isn't.
> 
> One possibility is that we don't document this at all and leave the
> constraints in RFC 2822 intact, which would mean that $alz-convention
> cancels would be an intentional protocol violation.

+1.  I'm not familiar with the $alz convention but it seems too scary to 
put in a standard track document.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Tue, 09 Jan 2007 14:11:32 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: Xref and relaying agents
Sender: [email-masked]

Charles Lindsey wrote:
> In <[email-masked]> Harald Alvestrand <[email-masked]> writes:
>
>   
>> Charles Lindsey wrote:
>>     
>
>   
>>> Ah! I see! We carefully said that the order of the steps in the various
>>> "Duties" sections could be varied so long as the final effect was the
>>> same, but we omitted to say the same for the order in which serving and
>>> relaying were done.
>>>
>>> You could cover it in the definitions section 1.4 by saying:
>>>
>>> 'A "relaying agent" accepts articles from injecting agents, serving
>>> agents, or other relaying agents and distributes them ...".'
>>>
>>> possibly with corresponding tweaks in the varius "Duties" sections.
>>>
>>>       
>> No.
>>     
>
> Why not? It reflects exactly the ways in which implementations actually do
> things, and removes possible confusions (like the one under discussion)
> when the implementation does things in a different order to the one in our
> model, necessitating wording which is even more confusing if you do not
> appreciate the subtlety behind it.
>
> If the model is left unchanged, then the confusing wording under relaying
> needs to be clarified. I have already suggested:
>
>    Any Xref header, whether present on input or added by an associated
>    local serving agent, MAY be deleted before relaying.
>
> which is an acceptable alternative to my suggested deffinition change. But
> one or the other needs doing.
>   
You have then abandoned the model that relaying agents talk to each 
other, and that serving agents only talk to their local relaying agents 
and reading agents.

You have increased the amount of coupling in the system model, for no 
benefit except saving a special-case handling of the Xref: header.

If you want to question why increasing the amount of coupling in the 
system model is bad, I can get you some references on system modelling 
and object orientation.

                       Harald

Date: Tue, 09 Jan 2007 14:08:38 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: Serving agents and duplicates
Sender: [email-masked]

Charles Lindsey wrote:
>> ITYM serving agents. Relaying agents should always just pass the cancel
>> message on and let serving agents downstream decide for themselves whether
>> to honour it.
>>     
>
> But having read the latest draft, I see that this is not so. Relaying
> agents are invited to honour cancel messages, even though they have no
> articles actually stored which they might want to hide/destroy. And there
> is nothing in 5.3 to suggest that cancel messages require action from
> other than serving agents.
>
> So why this requirement in relaying agents?
>
>   
Because if it is not there, a relaying agent that stops messages for 
which it has seen a cancel would be nonconformant with the 
specification. My guess is that combined relaying/serving agents would 
indeed do this.

If a relaying agent were not allowed to honor cancels, it would be 
nonconformant if it did.
That would be stupid.

                  Harald

Date: Tue, 09 Jan 2007 14:05:11 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: ISSUE: Injecting agents and From
Sender: [email-masked]

Charles Lindsey wrote:
> I disagree. Historically (as in the days of BNews and Cnews), as I think I
> have shown, 'inews' was the mechanism which caused injection to take
> place, and filling in the From header was part of what it did. As I said,
> the 'i' in 'inews' stands for 'inject' (just as the 'r' in 'rnews' stood
> for 'relay').
>
> When NNTP became the common mode of intereaction with newsreaders,
> automatic filling in of the From could not then be done, but that did not
> alter the situation wrt to existing usage - 'inews' did not suddenly
> cease to become a means of injecting; rather in NNTP contexts it lost some
> of its functionality.
>   
To put it another way:
In the days of BNews and CNews, the "inews" program formed part of the 
posting agent as well as part of the injecting agent.

In an NNTP context, it is only a part of the posting agent.

PLEASE don't attempt to require exact match between pieces of the model 
and pieces of implementations that predate the model.
>   
>>  It's a semantic distinction that I think is important in
>> trying to understand which piece of software does what.  C News used
>> without NNTP will be an unusual case.
>>     
>
> USEPRO should not be concerned with which piece of software does what. The
> fact remains that some injection mechanisms did (and still do) support
> automatic deduction of From, even if those mechanisms are now relatively
> uncommon, and USEPRO should recognise this.
>
> The wording in USEPRO-06 did recognize it, but also indicated that it was
> unusual, which seems to reflect the situation as it stands today. The
> actualy wording used, in connection with headers omittable in a
> proto-article, was "... (and even From if the particular injecting agent
> can derive that information from other sources)". I see no reason why we
> should not continue to say something of that nature.
>
> ISSUE raised.
>
>   

To: [email-masked]
Xref: clerew local.usefor:24067
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Mon, 8 Jan 2007 19:00:44 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> "Charles Lindsey" <[email-masked]> writes:

>In <[email-masked]> Russ Allbery <[email-masked]> writes:

>>I now have:

>(Presumably in section 3.6)

>>   4.  It SHOULD reject any article that matches an already-received and
>>       honored cancel message or Supersedes header field following the
>>       same rules as a relaying agent (Section 3.5).

>>which is basically the same text as for relaying agents.

Ah! I did not spot the significance of that ...

>>This would need to be added to relaying agents as well.

>ITYM serving agents. Relaying agents should always just pass the cancel
>message on and let serving agents downstream decide for themselves whether
>to honour it.

But having read the latest draft, I see that this is not so. Relaying
agents are invited to honour cancel messages, even though they have no
articles actually stored which they might want to hide/destroy. And there
is nothing in 5.3 to suggest that cancel messages require action from
other than serving agents.

So why this requirement in relaying agents?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24068
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Xref and relaying agents
Date: Mon, 8 Jan 2007 19:32:37 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>Charles Lindsey wrote:

>>Ah! I see! We carefully said that the order of the steps in the various
>>"Duties" sections could be varied so long as the final effect was the
>>same, but we omitted to say the same for the order in which serving and
>>relaying were done.
>>
>>You could cover it in the definitions section 1.4 by saying:
>>
>>'A "relaying agent" accepts articles from injecting agents, serving
>>agents, or other relaying agents and distributes them ...".'
>>
>>possibly with corresponding tweaks in the varius "Duties" sections.
>>
>No.

Why not? It reflects exactly the ways in which implementations actually do
things, and removes possible confusions (like the one under discussion)
when the implementation does things in a different order to the one in our
model, necessitating wording which is even more confusing if you do not
appreciate the subtlety behind it.

If the model is left unchanged, then the confusing wording under relaying
needs to be clarified. I have already suggested:

   Any Xref header, whether present on input or added by an associated
   local serving agent, MAY be deleted before relaying.

which is an acceptable alternative to my suggested deffinition change. But
one or the other needs doing.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24075
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 3.2.1: delimiter for multiple Path identities (was: Path header field)
Date: Mon, 8 Jan 2007 21:12:57 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I think we have two options.  My preferred option would be to say:

>  5.  The agent MAY then prepend additional <path-identity>s for itself
>      to the Path header field content.  Each <path-identity> so added
>      MUST be followed with either "!!" or "!".  Using "!!" is
>      RECOMMENDED.  This is permitted for agents that have multiple
>      <path-identity>s (such as during a transition from one to another).
>      Each of these <path-identity>s MUST meet the requirements set out in
>      Section 3.2.

Yes, I think that is the effect we want, but there is no need for the
MUSTard. I suggest (even though it is slightly longer):

   5.  The agent MAY then prepend additional <path-identity>s for itself
       to the Path header field content, normally followed by a
       <diag-match> "!" (to form "!!"), since it will surely recognize the
       earlier <path-identity>s as its own.  This is permitted for agents
       that have multiple <path-identity>s (such as during a transition
       from one to another).  Each of these <path-identity>s MUST meet the
       requirements set out in Section 3.2.

Note, however, that I have some other problems with this step and the one
before it, as discussed in another thread.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24076
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Protocol changes in draft-allbery-usefor-usepro-00
Date: Mon, 8 Jan 2007 21:28:34 GMT
Lines: 192
Sender: [email-masked]

The following list summarizes the outcomes of the long list of protocol
differences that I raised. Each begins with [-1], [00] or [+1] to indicate my
own assessment of it (possibly modified since the original list). I regard the
{+1} and [00] as resolved (unless someone else want to take up one of the
[00] ones).

Of the original 44 items (numbered 1-42 because of some duplication), there
remain 18 [-1]s, some of which are the subject of continuing discussions. The
rest may well become ISSUEs in the next few days. A list of them follows:

1
3
4
6
9
11
12
16
18
20
23
30
34
37
38
39
40
42

1. [-1] (2.2) <path-identity> - Possibility to use a FQDN not resolvable
in the DNS.

2. [+1] was [00] (3.2) Russ[-1] now case-insensitive
<path-identity>s are case-sensitive.

3. [-1] (3.3.1,3.4,3.9.2) From not omittable in proto-article ISSUE

4. [-1] (3.2.2,3.4,3.8) Injection-Date can be rewritten.
<[email-masked]> Wed, 3 Jan 2007 16:04:51 GMT Re: Injection-Date and reinjection

5. [+1] was [-1] (3.3.4) Now fixed
SHOULD trim References to keep <= 998: reduced from MUST.

6. [-1] was [00] (3.4)
Injecting agents reporting rejection to user agent; was
SHOULD, now merely "encouraged".

7. [+1] (3.4) Injecting agent SHOULD have access to list of newsgroups.

8. [00] was [-1] (3.4) with revised wording (accepted)
Requirement for injecting agent to forward articles to
moderator groups reduced from MUST to SHOULD.

9. [-1] (3.4,3.5) under current discussion
Prepending of <path-identity> to Path by injecting agent
reduced from MUST to SHOULD.

10. [00] (3.4) Folding of Path header when length becomes excessive
reduced from SHOULD to MAY.

11. [-1] (3.4) Recommendation that each injecting agent SHOULD use a
consistent format for Injection-Info removed.

12. [-1] (3.5) Explanatory text suggested referring to USEPRO 3.2.4
The significance of "world" and "local" in the Distribution
header are no longer mentioned.

13. [+1] (3.5,3.6) Removed possibility of "aliassing out" a site where you
didn't want an article to go by including it in the Path to the right of
POSTED.

14. [+1] (3.5) Relaying agent MUST reject articles without Newgroups or
Message-ID or (injection-Date or Date).

15. {+1} was [-1] (3.5) Relaying agent MUST reject any article already already sent 
to it.

16. [-1] (3.5) under current discussion
Relaying agent SHOULD deal with cancel message (if it
chooses to honour it)

17. [00] (3.5) Relaying agents SHOULD determine verified/expected source
of article and construct <path-diagnostics> accordingly.

18. [-1] (3.5) Russ[-1] Incorrectly fixed
No provision for >1 <path-identity> per relaying agent.
<[email-masked]> Mon, 1 Jan 2007 20:52:37 GMT Re: Path header field

19. [00] (3.5) Russ[+1] Partially fixed
No provision to omit <path-identity> for intermediate
servers on same site.

20. {-1} was [00] CHL[+1] with revised wording (but wording not accepted)
(3.5) Relaying agent MAY add a new Xref header.

Alert! Duplicated issue number #21!

21A. [00] (3.6) Storage of control messages in the (pseudo) hierarchy
control(.*) is a SHOULD.

21B. [00] was [-1] with revised wording (accepted)
(3.6) No mention of processing cancel messages by serving/storage
agents.

22. [00] Frank[-1] Forrest[+1]
(3.7) Removed recommendation that reading agent SHOULD have the
capability to display the raw article 'as received'.

23. [-1] Frank[-1]
(3.8) Moderators merely "encouraged" (was SHOULD) to retain
existing <msg-id>.

24. [00] (3.2.1) Removed provision (MAY) for outgoing gateway to change
Content-Transfer-Encoding.

25. [+1] (4.1) Application/news-transmission no longer handles batches.

26. {+1} was [-1]
(4.2,4.3) Removed requirement that
application/news-[groupinfo,checkgroups] MUST NOT be used except with
relevant control messages.

Alert! Duplicated issue number #27!

27A. [+1] (4.3) Application/news-[groupinfo,checkgroups] have a charset
parameter.

27B. [+1] was [-1] Russ[-1] Revised wording agreed.
(5) Nothing stated regarding Newsgroups header of Control
messages.

28. [00] Frank[-1]
(5) Subjects starting with 'cmsg' not accompanied by a Control
header MAY be rejected.

29. [+1] was [00]
(5.2) Newsgroup-names MUST be checked against disallowed names before
group control message is honoured.

30. [-1] (5.2) Nothing said about content of Approved header.
<[email-masked]> Tue, 2 Jan 2007 19:42:21 GMT Re: Approved header field content (was: Protocol changes in draft-allbery-usefor-usepro-00)

31. [+1] was [-1] with revised wording (not accepted, but still CHL +1)
(5.2.1) Newgroup message SHOULD include application/group-info.

32. [+1] was [-1] Russ[-1] Reworded
(5.2.1) Newgroup message containing other attachments in addition to
news-groupinfo to be multipart/related (was multipart/mixed).

33. [+1] Russ[+1](just)
(5.2.3) Checkgroups message still contains chkscope and chksernr
arguments.

34. [-1] was [00] with revised wording not accepted
(5.2.3) Checkgroups SHOULD contain sub-hierarchies excluded by
chkscope, for backwards compatibility.

35. [+1] (5.2.3) Chksernr MUST increase with successive checkgroups message.

36. [+1] was [00] (5.2.3) CHL[+1]
Body of checkgroups SHOULD be application/checkgroups, and
otherwise SHOULD treat as if it were.

37. [-1] was [00] Frank[-1]
(5.3) Body of cancel message MAY contain ...explanatory text.

38. [-1] revised wording not accepted
(5.3) No mention of who should be in From/Sender of cancel message.

39. [-1] Russ[-1] still under discussion
(5.5) The <relayer-name> in ihave and sendme control messages is now
optional.

40. [-1] (5.5) Format of batched news in response to sendme removed.

41. [+1] was [-1] (5.5) Russ[-1] new text written
The protocol for using the 'to.' newsgroup-name is omitted.

42. [-1] (5.6) Obsolete control messages SHOULD NOT be sent/honoured/

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24073
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Turning to issue tracking on the USEPRO document
Date: Mon, 8 Jan 2007 20:37:05 GMT
Lines: 80
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Here's a rundown of the current three open issues concerning USEPRO in the
>WG tracker:

>> #1051: USEPRO general: Document changes from RFC 1036
>> 
>>   Section 3.1 of USEPRO -03 should be updated to only list protocol
>>   related changes.

>This has now been moved to appendix A,

Yes, that is OK, apart from a few niggles I have noted to bring up.

There is also the matter of the transitional arrangements, but that can
also wait until we have resolved exactly which transitions are to survive.

>> #1083: USEPRO 7.5: Rules for Generating message-ID

>A question about where we should document rules for generating message IDs
>and the <cancel. $alz convention.  I think the context here concerned
>cancel messages, so the new section affected by this issue is 5.3.

>If we document an exception for $alz-convention cancels, we're overriding
>the uniqueness constraints defined in 3.6.4 of RFC 2822, which we inherit
>via USEFOR.  I'm not sure if we want to do this or not.  It makes me very
>uncomfortable; it's quite difficult to describe when this is okay and when
>it isn't.

>One possibility is that we don't document this at all and leave the
>constraints in RFC 2822 intact, which would mean that $alz-convention
>cancels would be an intentional protocol violation.

I agree. A useful hack best left to those who understand it.

>Other than the $alz-convention weirdness, I'm happy with what 3.6.4 of RFC
>2822 says about message ID construction and uniqueness and don't think we
>need to further elaborate.

Again OK. There is a draft on the group's website written in the very early
days which it might be useful to revive someday. But no need to say
anything now (I don't recall having discussed this in USEPRO-06 either).

>> #1093: USEPRO 2.3: Supported email addresses

>The conclusion here was:

>>   The USEPRO document will not say MUST or SHOULD about any of the
>>   addresses "abuse@identity", "usefor@identity" or "news@identity".

>Note, however, that I also dropped any NOTE about the addresses as well,
>which at least Charles has said he doesn't agree with, so perhaps this
>should become an issue about that.  If so, the relevant section is now
>3.2.

I would like to see some mention. RFC 2142 is a standards track RFC which
purports to affect Netnews (it should have been infomational, but wasn't).
So a mention is certainly in order (however disdainful), USEAGE is the
place for any serious discussion, though.

However, I think the first thing we need to do is to resolve whether to
accept the idea that <path-identities> in the form of FQDNs that are not
in the DNS are to be allowed, as Harald suggested. When we know that, then
we can review the wording of the rest of that section in a proper context.

Personally, I do not like the idea at all, and would rather put a SHOULD
NOT do it (which still leaves the door slightly ajar for those who have
some overriding need for it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24069
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Empty backlog
Date: Mon, 8 Jan 2007 20:08:13 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>That means that if there's something else that anyone expected me to
>respond to, you shouldn't wait in silence.  If it was a point of
>disagreement from one of the threads prior to the publication of -07, it's
>probably safe to read my silence as lack of agreement with any minor
>change to correct an issue.  It would therefore be appropriate at this
>time to send a message to the WG mailing list requesting a new issue be
>opened.

I have just reviewed my list of outstanding items, and have responded
further in a few threads which seem to have got stuck.

I also was also hoping for some response from you to:

Re: Injection-Date and reinjection
<[email-masked]> Wed, 3 Jan 2007 16:04:51 GMT

Re: Path header field
<[email-masked]> Mon, 1 Jan 2007 20:52:37 GMT

Re: Approved header field content (was: Protocol changes in draft-allbery-usefor-usepro-00) 
<[email-masked]> Tue, 2 Jan 2007 19:42:21 GMT

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24072
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Turning to issue tracking on the USEPRO document
Date: Mon, 8 Jan 2007 20:21:04 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Harald Alvestrand wrote:

>> usepro-06 should be quite sufficient.

>A pair of converging drafts could be also interesting 
>if Charles is willing to go to that trouble.

Probably not a good idea, unless things go completely off the rails :-( .

But there is a lot in USEPRO-06 that I shall be pressing to include in the
new USEPRO in due course (but we have quite enough on our agenda to keep
us busy at the moment, so they can wait awhile).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24074
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: USEPRO 5.5: ihave/sendme syntax
Date: Mon, 8 Jan 2007 20:58:07 GMT
Lines: 50
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>The primary reason why I had the relayer name be optional when the message
>IDs are in the control header is because that's what usepro-06 had.  (I
>later checked RFC 1036 and saw that it was optional there, and figured
>that's why we'd had it optional.)

Good Heavens! So it did! Evidently I just blindly copied RFC 1036, on the
grounds that it didn't really matter since I was deprecating it anyway.

>> In any case, the paragraph which follows explaining how to use these two
>> headers is written on the assumption that a <relayer-name> will be
>> provided.

>See the bottom of the second paragraph of the text:

>   If <relayer-name> is not given, it is determined from the origin of
>   the control message.

Ah!

>I think we have three options here:

> * Leave the text alone.  <relayer-name> is optional if the message IDs
>   are given in the control message, allowing the primary RFC 1036 syntax,
>   and mandatory if the message IDs are in the body.

> * Make <relayer-name> always optional.  This will make our syntax
>   identical with RFC 1036.

> * Make <relayer-name> always mandatory and remove the above sentence from
>   the text.  This will make our syntax identical with Son-of-1036.

>I have a mild preference for the third option but don't really care as
>long as we can agree on one choice and stick with it.

Agreed. My preference is for the third too. #! introduced apparently
complicated syntax for no apparent reason. #2 could result in an empty
header-body, which USEFOR does not allow.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24066
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: ISSUE: Injecting agents and From
Date: Mon, 8 Jan 2007 17:13:13 GMT
Lines: 70
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>>> inews as distributed with INN or the common news readers is a (part of
>>> a) posting agent, not an injecting agent.

>[...]

>>> You can see this by observing what actions it takes and what agent it
>>> talks to.  It sends messages to an injecting agent via POST and expects
>>> that agent to do the injection;

>> Not so. 'inews' existed long before the POST command was invented.

>inews *as distributed with INN or the common news readers* does indeed
>behave exactly as I describe.

Which is true, but does not answer the point. Are trn and nn not "common
newsreaders" any more?

>  As I mentioned explictly later, the version
>that comes with C News is much older and may well do something different.

>It draws the line between posting agent and injecting agent in the wrong
>place for the most common case of NNTP, and is therefore unnecessarily
>confusing.

I disagree. Historically (as in the days of BNews and Cnews), as I think I
have shown, 'inews' was the mechanism which caused injection to take
place, and filling in the From header was part of what it did. As I said,
the 'i' in 'inews' stands for 'inject' (just as the 'r' in 'rnews' stood
for 'relay').

When NNTP became the common mode of intereaction with newsreaders,
automatic filling in of the From could not then be done, but that did not
alter the situation wrt to existing usage - 'inews' did not suddenly
cease to become a means of injecting; rather in NNTP contexts it lost some
of its functionality.

>  It's a semantic distinction that I think is important in
>trying to understand which piece of software does what.  C News used
>without NNTP will be an unusual case.

USEPRO should not be concerned with which piece of software does what. The
fact remains that some injection mechanisms did (and still do) support
automatic deduction of From, even if those mechanisms are now relatively
uncommon, and USEPRO should recognise this.

The wording in USEPRO-06 did recognize it, but also indicated that it was
unusual, which seems to reflect the situation as it stands today. The
actualy wording used, in connection with headers omittable in a
proto-article, was "... (and even From if the particular injecting agent
can derive that information from other sources)". I see no reason why we
should not continue to say something of that nature.

ISSUE raised.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24070
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ISSUE: Control message propagation (was: Control message propagation)
Date: Mon, 8 Jan 2007 20:13:57 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Russ Allbery wrote:
> 
>>     To best ensure that it will be relayed to the same news servers
>>     as the original message, a cancel control message SHOULD have the
>>     same Newsgroups header field as the message it is cancelling.

>Proposed addition:

>| The groups in the Newsgroups header field MUST match at least one
>| of the affected newsgroups.

+1

Indeed, I would have expected MUST match all of them.

>For Rmgroup and Newgroup we have that anyway, for Checkgroups it's also
>sound, and for a Cancel it's required for transparency.  

>All other control messages apparently don't have a Newsgroups header 
>field.  If that's the case it has to be mentioned explicitly, USEFOR-11
>claims that this header field is "mandatory".

Yes, but checkgroups is the only one where it is not specified
(discounting the obsoleted ones), and I have already agreed that is for
USEAGE to discuss.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24071
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Mon, 8 Jan 2007 20:18:37 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Charles Lindsey wrote:

> [unknown control messages]
>> But maybe I am not as dubious as Frank :-) 

>I'd understand "MUST ignore" or "MUST reject".  But not simultaneously,
>these are conflicting strategies.  If we pick "ignore or reject", then
>there can't be a 2119 MUST, because there's no other choice - Russ' idea
>"try to guess what it means" is too odd for a "MUST NOT".

>We could say "SHOULD ignore" if "MUST ignore" is too strong.  The whole
>issue would be much clearer with a registry.  There are already tons of
>registries for mail, e.g. a "with" registry (SMTP, ESMTP, ESMTPA, etc.),
>it's okay if news has a "control" registry.

Yes, I agree the present wording is unsatisfactory.  

>And with an "RFC required" rule the work for IANA is minimal, they read
>all "IANA considerations", and note new registrations, like say "mvgroup
>is defined in [RFC 5555]", or "LMTPA is defined in [RFC 3848

But the only likely oddities (until we come to revive 'mvgroup') will be
private arrangements involving cooperating subnets (those seem to be the
ones Russ wants the rest of us to ignore/reject). And IANA does not want
to get involved in the private dealings of cooperating subnets.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Empty backlog
Organization: The Eyrie
Date: Sun, 07 Jan 2007 16:48:58 -0800
Sender: [email-masked]

At the time of the sending of this message, I have only one pending minor
change (to the wording of the Path header, noted in my message a few
minutes ago) for next weekend and no resolved issues resulting in changes.
I've also responded to all of the working group messages to which I felt I
had something useful to say.

That means that if there's something else that anyone expected me to
respond to, you shouldn't wait in silence.  If it was a point of
disagreement from one of the threads prior to the publication of -07, it's
probably safe to read my silence as lack of agreement with any minor
change to correct an issue.  It would therefore be appropriate at this
time to send a message to the WG mailing list requesting a new issue be
opened.

I will probably be fairly quiet on the mailing list until next weekend
(although what I find time to work on can be a bit unpredictable and I may
surprise myself).  It would probably be a good use of the next week to get
the issues opened that people want opened so that we have an idea of how
much we have remaining to deal with.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Turning to issue tracking on the USEPRO document
Organization: The Eyrie
Date: Sun, 07 Jan 2007 16:44:02 -0800
Sender: [email-masked]

Here's a rundown of the current three open issues concerning USEPRO in the
WG tracker:

> #1051: USEPRO general: Document changes from RFC 1036
> 
>   Section 3.1 of USEPRO -03 should be updated to only list protocol
>   related changes.

This has now been moved to appendix A, and I believe it only lists
protocol-related changes now.  My recommendation would be to close this
issue, and if anyone has any specific protocol-related changes that they
think are missing from that appendix (or that are present when they
shouldn't be), raise those issues on the working group, probably as "Minor
change" since that appendix isn't normative.

> #1083: USEPRO 7.5: Rules for Generating message-ID

A question about where we should document rules for generating message IDs
and the <cancel. $alz convention.  I think the context here concerned
cancel messages, so the new section affected by this issue is 5.3.

The $alz convention is really weird.  I'd said earlier that administrative
cancels were generally in compliance with the protocol without special
exceptions, but that's in the absence of this convention (which I wasn't
thinking about at the time).  If they use the $alz convention, that is a
special exception to all the message ID construction rules, since the
entire point of that convention is to ensure that multiple cancel messages
for the same message *will* collide, thus violating a basic principle of
Netnews in exchange for additional efficiency.  They violate not only our
uniqueness rules but the uniqueness rules inherited from RFC 2822.

If we document an exception for $alz-convention cancels, we're overriding
the uniqueness constraints defined in 3.6.4 of RFC 2822, which we inherit
via USEFOR.  I'm not sure if we want to do this or not.  It makes me very
uncomfortable; it's quite difficult to describe when this is okay and when
it isn't.

One possibility is that we don't document this at all and leave the
constraints in RFC 2822 intact, which would mean that $alz-convention
cancels would be an intentional protocol violation.  Given how much spam
cancelling has dropped off and given that it is, at its basis, an
intentional violation of the protocol done by people who know exactly what
they're doing to achieve a specific effect, I'm not sure that's a bad
idea.

Other than the $alz-convention weirdness, I'm happy with what 3.6.4 of RFC
2822 says about message ID construction and uniqueness and don't think we
need to further elaborate.

Regardless, this should probably stay open as an issue and be discussed
again now that we're working on USEPRO.  I don't see a really happy
solution.

> #1093: USEPRO 2.3: Supported email addresses

The conclusion here was:

>   The USEPRO document will not say MUST or SHOULD about any of the
>   addresses "abuse@identity", "usefor@identity" or "news@identity".

Note, however, that I also dropped any NOTE about the addresses as well,
which at least Charles has said he doesn't agree with, so perhaps this
should become an issue about that.  If so, the relevant section is now
3.2.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: ISSUE: USEPRO 3.2.1: delimiter for multiple Path identities (was: Path header field)
Organization: The Eyrie
Date: Sun, 07 Jan 2007 16:21:48 -0800
Sender: [email-masked]

Richard Clayton <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes

>>   5.  The agent MAY then prepend additional <path-identity>s for itself
>>       to the Path header field content, following each <path-identity>
>>       so added with either "!!" or "!".  This is permitted for agents
>>       that have multiple <path-identity>s (such as during a transition
>>       from one to another).  Each of these <path-identity>s MUST meet
>>       the requirements set out in Section 3.2.

> For !! to be of any use (which I continue to feel is entirely dubious,
> but that's water under the bridge) then surely the requirement in this
> last paragraph ought to be for !! (because the server must surely be
> claiming that self-to-self is a "validated" transfer)

Yes, the current wording was a cop-out and I had a feeling it was when
writing it.  It needs to be changed.

Currently, using <path-diagnostic> in general is RECOMMENDED but not
REQUIRED.  As mentioned in an earlier post about philosophy, I did this so
that, should the whole diagnostic thing turn out to be a bad idea for some
reason, or just not worth it, implementations would not be non-conforming
by declining to implement it.  That's what this "either !! or !" business
was trying to capture, poorly.

I think we have two options.  My preferred option would be to say:

  5.  The agent MAY then prepend additional <path-identity>s for itself
      to the Path header field content.  Each <path-identity> so added
      MUST be followed with either "!!" or "!".  Using "!!" is
      RECOMMENDED.  This is permitted for agents that have multiple
      <path-identity>s (such as during a transition from one to another).
      Each of these <path-identity>s MUST meet the requirements set out in
      Section 3.2.

This copies the MUST/SHOULD language of <path-diagnostic> in general,
although it's a bit awkward.  The other alternative is to just require the
<path-diagnostic> in this case:

  5.  The agent MAY then prepend additional <path-identity>s for itself
      to the Path header field content, following each <path-identity> so
      added with "!!".  This is permitted for agents that have multiple
      <path-identity>s (such as during a transition from one to another).
      Each of these <path-identity>s MUST meet the requirements set out in
      Section 3.2.

When configured to add multiple path identities, I believe INN would allow
specifying !! as the delimiter with no changes, but I'm not positive.
Other existing software may well require modification.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Minor change: Path wording (was: Path header field)
Organization: The Eyrie
Date: Sun, 07 Jan 2007 16:15:54 -0800
Sender: [email-masked]

I'm going to break my response up into two parts, one wording change and
one protocol issue.

Richard Clayton <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes

>>   3.  A relaying or serving agent SHOULD prepend a <path-diagnostic> to
>>       the Path header field content, where the <path-diagnostic> is
>>       chosen as follows:
>> 
>>       *  If the expected <path-identity> of the source of the article
>>          matches the leftmost <path-identity> of the Path header
>>          field's content, use "!" (<diag-match>).

> If you're not aware of the !! syntax (which is after all "new") then I
> think this would confuse, and people will be able to read it so as to
> miss out one of the !s

Agreed.

How about:

      *  If the expected <path-identity> of the source of the article
         matches the leftmost <path-identity> of the Path header
         field's content, use "!" (<diag-match>), resulting in two
         consecutive "!"s.

>>       *  If the expected <path-identity> of the source of the article
>>          does not match, use "!.MISMATCH." followed by the expected
>>          <path-identity> of the source or its IP address.
>> 
>>       *  If the relaying or serving agent is not willing or able to
>>          check the <path-identity>, use "!.SEEN." followed by the FQDN,
>>          IP address, or expected <path-identity> of the source.

> I'd then add

>         Note that previous versions of this standard did not require
>         <path-diagnostics> so that conformant systems would only add
>         single "!" characters between <path-identity> strings.

Taking into account the subsequent discussion, how about:

    Be aware that [RFC1036] did not include <path-diagnostic>.
    Implementations which predate this standard will add only single "!" 
    characters between <path-identity> strings.

If anyone disagrees with these wording changes, please speak up, either to
say that this should be an issue instead or to suggest further tweaks.
Unless this becomes an issue, I'll plan on making the wording changes next
Sunday.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: USEPRO 5.3: Newsgroups header of cancel
Date:  Sat, 06 Jan 2007 22:30:44 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Russ Allbery wrote:
 
>>> Does that mean that you're proposing something similar for
>>> checkgroups?
 
>> Yes.
 
> Just to be clear, any newsgroup that occurs in the checkgroups
> message would satisfy the requirement?

Yes.  As you wrote elsewhere that's typically admin group(s), but
we don't need that detail, let the posters pick what they consider
as the best affected group(s) wrt the checkgroups propagation.

If it's a very small TLH (say less than 15 groups) they might
also get away by listing them all, as in your SHOULD designed for
other types of control messages.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: ISSUE: USEPRO 5.3: Newsgroups header of cancel
Organization: The Eyrie
Date: Sat, 06 Jan 2007 12:48:05 -0800
Sender: [email-masked]

Frank Ellermann <[email-masked]> writes:
> Russ Allbery wrote:
 
>> Does that mean that you're proposing something similar for 
>> checkgroups?

> Yes.

Just to be clear, any newsgroup that occurs in the checkgroups message
would satisfy the requirement?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: ISSUE: USEPRO 5.3: Newsgroups header of cancel 
Date:  Sat, 06 Jan 2007 21:02:37 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 30
Sender: [email-masked]

Russ Allbery wrote:
 
> Fixing the header line for you to follow what the chairs asked for.

Thanks, I read 51 pending messages last in - first out, missing the
subject instructions (not that they're new, but I missed it anyway).

> I'm opposed, unsurprisingly.

Yes, IIRC Charles supported the "MUST at least match one" for Cancel.
My proposal to make it a general rule (= not only for Cancel) was a
new idea.
 
> Does that mean that you're proposing something similar for 
> checkgroups?

Yes.

>> other control messages apparently don't have a Newsgroups header
>> field.
 
> Of course they do.  There are just no special rules, so the standard 
> rules for a Newsgroups header applies.

Good.  There are no "affected newgroups" for a "version", "sendsys",
etc., so that won't conflict with the "SHOULD list all" or my proposed
"MUST match at least one" addition.

Frank

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: ISSUE: USEPRO 5.5: ihave/sendme syntax
Organization: The Eyrie
Date: Sat, 06 Jan 2007 11:50:33 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I've since reviewed RFC 1036 and Son-of-1036.  The former makes the
>> relayer name optional and requires that the message IDs be in the
>> control header field.  The latter makes the relayer name mandatory and
>> RECOMMENDS that the message IDs be put in the body.  I think the
>> following is the most reasonable compromise; it makes the relayer-name
>> mandatory if there are no message IDs (the form never allowed in RFC
>> 1036 but according to Son-of-1036 the most widely used in practice),
>> and if there are message IDs, permits the same syntax as RFC 1036.

> I don't see why the optionality or otherwise of the <relayer-name>
> should correlate in any way with whether the list of msg-ids is in the
> header or the body (except to ensure that these headers never have empty
> content, which does not sound like a particularly good reason).

I didn't think that RFC 1036 allowed message IDs in the body, which would
have created a link, but after reading it more closely, I see that it does
(in a confusing way).  So given that, I agree, there is no obvious link.

> So I would think it better to follow Son-of-1036 and make it obligatory
> regardless. I think we can assume that Henry had sufficient experience
> of the way Ihave and Sendme were used at that time that it is safe to
> follow his lead.

The primary reason why I had the relayer name be optional when the message
IDs are in the control header is because that's what usepro-06 had.  (I
later checked RFC 1036 and saw that it was optional there, and figured
that's why we'd had it optional.)

> In any case, the paragraph which follows explaining how to use these two
> headers is written on the assumption that a <relayer-name> will be
> provided.

See the bottom of the second paragraph of the text:

   If <relayer-name> is not given, it is determined from the origin of
   the control message.

I think we have three options here:

 * Leave the text alone.  <relayer-name> is optional if the message IDs
   are given in the control message, allowing the primary RFC 1036 syntax,
   and mandatory if the message IDs are in the body.

 * Make <relayer-name> always optional.  This will make our syntax
   identical with RFC 1036.

 * Make <relayer-name> always mandatory and remove the above sentence from
   the text.  This will make our syntax identical with Son-of-1036.

I have a mild preference for the third option but don't really care as
long as we can agree on one choice and stick with it.

For whatever it's worth, INN doesn't support message IDs in the control
header at all.  It requires they be in the body.  INN's implementation
also completely ignores the content of the control header, including any
relayer name, and always uses the origin of the control message.  I've
never gotten any complaints about how it works, which probably just
indicates that few people use these control messages since the current
behavior isn't compliant with any of the various specifications.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: ISSUE: USEPRO 5.3: Newsgroups header of cancel (was: Re: ISSUE: Control message propagation)
Organization: The Eyrie
Date: Sat, 06 Jan 2007 11:18:27 -0800
Sender: [email-masked]

Fixing the header line for you to follow what the chairs asked for.

Frank Ellermann <[email-masked]> writes:

> Russ Allbery wrote:
 
>>     To best ensure that it will be relayed to the same news servers
>>     as the original message, a cancel control message SHOULD have the
>>     same Newsgroups header field as the message it is cancelling.

> Proposed addition:

> | The groups in the Newsgroups header field MUST match at least one
> | of the affected newsgroups.

I'm opposed, unsurprisingly.

> For Rmgroup and Newgroup we have that anyway, for Checkgroups it's also
> sound,

Does that mean that you're proposing something similar for checkgroups?
If so, you should say so explicitly.

> and for a Cancel it's required for transparency.

> All other control messages apparently don't have a Newsgroups header
> field.

Of course they do.  There are just no special rules, so the standard rules
for a Newsgroups header applies.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Turning to issue tracking on the USEPRO document
Date:  Sat, 06 Jan 2007 15:07:14 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 12
Sender: [email-masked]

Harald Alvestrand wrote:

>> I suggest we refer to my present USEPRO as "CHASPRO"
>> whenever we have need to refer to it.
 
> usepro-06 should be quite sufficient.

A pair of converging drafts could be also interesting 
if Charles is willing to go to that trouble.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Injection-Date and reinjection
Date:  Sat, 06 Jan 2007 14:39:01 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 23
Sender: [email-masked]

Charles Lindsey wrote:

> this is a thing that is easily testable. I have just posted a series
> of articles "1 day stale", "2 days stale" up to "9 days stale" to
> misc.test, and I will see what the autoresponders tell me (and it
> would be useful for members of this list to look out for them also).

So far I don't see 4..9 days.  This group is horrible, thousands of
"NNTP monitor" messages.

The 3 days article is <news:[email-masked]>
Path: nnrp3.clara.net!monkeydust.news.clara.net!news.clara.net!
 wagner.news.clara.net!usenet-fr.net!nerim.net!oleane.net!oleane!
 keepthis.news.telefonica.de!telefonica.de!newsfeed.r-kom.de!
 newsfeed.freenet.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail

Please post the Message-IDs of your other tests if I should check
them directly.  The 3 days Path might be already unusual, the last
time I looked at it stuff from German servers reached clara.net via
Arcor, not via Oleane.

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Control message propagation
Date:  Sat, 06 Jan 2007 14:12:19 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 21
Sender: [email-masked]

Charles Lindsey wrote:

 [unknown control messages]
> But maybe I am not as dubious as Frank :-) 

I'd understand "MUST ignore" or "MUST reject".  But not simultaneously,
these are conflicting strategies.  If we pick "ignore or reject", then
there can't be a 2119 MUST, because there's no other choice - Russ' idea
"try to guess what it means" is too odd for a "MUST NOT".

We could say "SHOULD ignore" if "MUST ignore" is too strong.  The whole
issue would be much clearer with a registry.  There are already tons of
registries for mail, e.g. a "with" registry (SMTP, ESMTP, ESMTPA, etc.),
it's okay if news has a "control" registry.  

And with an "RFC required" rule the work for IANA is minimal, they read
all "IANA considerations", and note new registrations, like say "mvgroup
is defined in [RFC 5555]", or "LMTPA is defined in [RFC 3848]".

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  ISSUE: Control message propagation (was: Control message propagation)
Date:  Sat, 06 Jan 2007 13:50:17 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 20
Sender: [email-masked]

Russ Allbery wrote:
 
>     To best ensure that it will be relayed to the same news servers
>     as the original message, a cancel control message SHOULD have the
>     same Newsgroups header field as the message it is cancelling.

Proposed addition:

| The groups in the Newsgroups header field MUST match at least one
| of the affected newsgroups.

For Rmgroup and Newgroup we have that anyway, for Checkgroups it's also
sound, and for a Cancel it's required for transparency.  

All other control messages apparently don't have a Newsgroups header 
field.  If that's the case it has to be mentioned explicitly, USEFOR-11
claims that this header field is "mandatory".

Frank

To: [email-masked]
From: Frank Ellermann <[email-masked]>
Subject:  Re: Path header field 
Date:  Sat, 06 Jan 2007 13:32:17 +0100
Organization:  <URL:http://purl.net/xyzzy>
Lines: 27
Sender: [email-masked]

Richard Clayton wrote:

>         *  If the expected <path-identity> of the source of the article
>            matches the leftmost <path-identity> of the Path header
>            field's content, then prepend the <diag-match> string "!". So
>            in this, hopefully most common case, two !s will appear in
>            the path.

Makes sense.  I'd omit "hopefully", and I'd say "two adjacent !s" [...],
or '"!!" (two exclamation marks) will appear' [...]

>         Note that previous versions of this standard did not require
>         <path-diagnostics> so that conformant systems would only add
>         single "!" characters between <path-identity> strings.

Using "conformant" meaning "conformant with RFC 1036" is confusing, the
note would be clearer without this "conforman".  Actully I don't see 
why this note is needed at all, an ordinary "!" is explained in USEFOR.

> the requirement in this last paragraph ought to be for !! (because 
> the server must surely be claiming that self-to-self is a "validated"
> transfer)

ACK

Frank

From: Russ Allbery <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked], Harald Alvestrand <[email-masked]>, Lisa Dusseault <[email-masked]>
Subject: Re: Status of the USEFOR document
Organization: The Eyrie
Date: Fri, 05 Jan 2007 23:26:40 -0800
Sender: [email-masked]

Alexey Melnikov <[email-masked]> writes:

> Hi folks,

> After several discussions between Lisa, Harald and myself it became
> clear that making the reference to USEPRO normative would be the
> easiest/quickest way to remove remaining IESG DISCUSSes. This change
> will delay publication of the USEFOR document until the USEPRO document
> is approved by IESG.

> Any objections to doing that?

I have no objections.  I think it would probably be possible to rework
USEFOR so that it wouldn't have serious normative references, but I'm not
sure the result is that realistically useful.  I also think that
publishing USEPRO is an entirely reachable goal, and in our ideal world we
want them both anyway.

We can always go back and revisit this if we fail to achieve WG consensus
on USEPRO, or if it starts dragging out.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Fri,  5 Jan 2007 10:52:37 -0800 (PST)
Sender: [email-masked]

    Date: Friday, January 5, 2007 @ 10:52:36
  Author: eagle
Revision: 2225

Clarify charset wording for application/news-transmission around ASCII
compatibility of the newsgroup name.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-01-05 18:37:27 UTC (rev 2224)
+++ docs/usefor/usepro.xml	2007-01-05 18:52:36 UTC (rev 2225)
@@ -1396,8 +1396,8 @@
         not possible, UTF-8 <xref target="RFC3629" /> SHOULD be used.
         Regardless of the charset used, the constraints of the above
         grammar MUST be met and the &lt;newsgroup-name> MUST be
-        represented using the same octets as would be used with a charset
-        of US-ASCII.</t>
+        represented in that charset using the same octets as would be used
+        with a charset of US-ASCII.</t>
       </section>
 
       <section anchor="checkgroup" title="application/news-checkgroups">

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Minor change: USEPRO 4.2: charset wording (was: charset of application/news-{groupinfo,checkgroups})
Organization: The Eyrie
Date: Fri, 05 Jan 2007 10:51:53 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:

>>> It should be so, but I don't think it is deducible from the wording you
>>> have written. If you
>>>    s/MUST be represented using/MUST be represented in that charset using/
>>> it might work.

>> Well, that seems redundant to me, but if others feel like it adds clarity,
>> I don't mind adding it.

> Yes please, though I would really prefer a wording that explicitly spoke
> of having ASCII as a subset and/or of using charsets with escapable
> modes.

Adding "in that charset" at least seems uncontroversial (and I wanted to
test something else with the commit notification).  Changed in my copy.

> Either way, ASCII and UTF-8 would be the RECOMMENDED charsets.

Yup, that's already there.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Thu,  4 Jan 2007 21:03:18 -0800 (PST)
Sender: [email-masked]

    Date: Thursday, January 4, 2007 @ 21:03:18
  Author: eagle
Revision: 2220

Whitespace change to test commit notification to the mailing list.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2007-01-05 05:01:12 UTC (rev 2219)
+++ docs/usefor/usepro.xml	2007-01-05 05:03:18 UTC (rev 2220)
@@ -27,7 +27,7 @@
 ]>
 
 <rfc ipr="full3978" docName="draft-ietf-usefor-usepro-07" obsoletes="1036"
-  category="std">
+     category="std">
   <front>
     <title>Netnews Architecture and Protocols</title>

Date: Thu, 04 Jan 2007 17:05:09 -0500
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Harald Alvestrand <[email-masked]>
CC: [email-masked]
Subject: Re: Turning to issue tracking on the USEPRO document
Sender: [email-masked]

Harald Alvestrand wrote:
> 
> We believe that we have now reached the stage where it's useful to issue 
> the next revision of draft-allbery-usefor-usepro-00 as 
> draft-ietf-usefor-usepro-07, and start tracking issues in the issue 
> tracker again.

Note that there are 3 existing USEPRO tickets open in the tracker.  I 
assume that these issues are specific to -06 and earlier, but someone 
more familiar with them should double check.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-usefor-usepro-07.txt 
Date: Thu, 04 Jan 2007 15:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title		: Netnews Architecture and Protocols
	Author(s)	: R. Allbery, C. Lindsey
	Filename	: draft-ietf-usefor-usepro-07.txt
	Pages		: 49
	Date		: 2007-1-4
	
This document defines the architecture of Netnews systems and
   specifies the correct manipulation and interpretation of Netnews
   articles by software which originates, distributes, stores, and
   displays them.  It also specifies the requirements that must be met
   by any protocol used to transport and serve Netnews articles.

   Backward compatibility has been a major goal of this endeavour, but
   where this standard and earlier documents or practices conflict, this
   standard should be followed. In most such cases, current practice is
   already compatible with these changes.

   A companion Best Current Practice document [USEAGE], addressing
   requirements which are present for Social rather than Normative
   reasons is in preparation.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-07.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-usefor-usepro-07.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-usefor-usepro-07.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-usefor-usepro-07.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-07.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Xref: clerew local.usefor:24038
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: charset of application/news-{groupinfo,checkgroups}
Date: Thu, 4 Jan 2007 12:21:54 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:
>>>> Russ Allbery <[email-masked]> writes:

>>>>>         eightbit-utext      = utext / %d127-255

>>>> OK, <eightbit-utext> was clearly needed, but it still excludes those
>>>> ASCII characters not in <utext> notably CR, LF and HT,

>>> By HT I assume you mean HTAB?  That's is added back in by WSP in the
>>> newsgroup-description production, so only CR and LF are disallowed, which
>>> is intentional.

>> No, I meant that HT is not included in <utext> according to RFC 2822,
>> and hence is not in <eightbit-utext>. Whether we should care or not I am
>> not sure.

><utext> doesn't include whitespace, so <eightbit-utext> follows suit.  The
>only production that uses it reintroduces non-leading whitespace.

OK, that seems to cover it.

>>> Surely this is obviously invalid?  It would mean that the newsgroup
>>> name in the charset specified for the body of this type is different
>>> than the newsgroup name in ASCII, which is precisely what's forbidden.

>> It should be so, but I don't think it is deducible from the wording you
>> have written. If you
>>    s/MUST be represented using/MUST be represented in that charset using/
>> it might work.

>Well, that seems redundant to me, but if others feel like it adds clarity,
>I don't mind adding it.

Yes please, though I would really prefer a wording that explicitly spoke
of having ASCII as a subset and/or of using charsets with escapable modes.

Either way, ASCII and UTF-8 would be the RECOMMENDED charsets.

>> Suppose the two modes of the charset are ASCII and Japanese. ...

>In which case you can't use that character set, since the newsgroup name
>would be some random Japanese characters rather than a newsgroup that
>could be represented in ASCII.

OK, Ned Freed's explanations seem to confirm that all the current modal
charsets start out in ASCII, and would therefore be suitable.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 04 Jan 2007 10:15:52 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Moderator duties
Sender: [email-masked]

--On 3. januar 2007 19:25 +0000 Charles Lindsey <[email-masked]> 
wrote:

>
> In <[email-masked]> "Forrest J. Cavalier III"
> <[email-masked]> writes:
>
>> Charles Lindsey wrote:
>
>> I agree with this sentiment, but disagree that a reminder must appear in
>> the  document everywhere that it applies.  The reminders are out of
>> scope, in my view.  This is one of the significant improvements in
>> clarity in RUSSPRO.
>
> In this case it needs to be mentioned because leaving it out (see Russ's
> original version) you appear to be giving the impression that the
> Moderator is free to change anything whatsoever that he doesn't like, and
> we don't want to do that.

Don't speak for me.

Technical personal opinion: I don't want to claim that any change the 
moderator does is a *protocol* violation. It may be a violation of 
expectations, or of common sense, but I don't want to claim that it is a 
protocol violation.

> So we mention "recommended best practice" here (with reference to USEAGE)
> just to make sure we don't give any such wrong impression. The same may be
> required at other places. There may be further cases where such a mention
> is not so urgent, and there we can discuss whether it is helpful to put it
> in or not.
>
>> Are there any other RFCs that babysit the way that Charles suggests?  I'm
>> curious.  This isn't a rhetorical question.
>
> I think there are plenty of RFCs that give more than the bare facts of a
> protocol by drawing attention to reasons for features, possible
> implementation difficulties/strategies, gotchas, and the like.
>
> --
> Charles H. Lindsey ---------At Home, doing my own
> thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133
> Web: http://www.cs.man.ac.uk/~chl Email: [email-masked]      Snail:
> 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9      Fingerprint:
> 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
>
>

Date: Thu, 04 Jan 2007 10:10:05 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: Turning to issue tracking on the USEPRO document
Sender: [email-masked]

--On 3. januar 2007 19:46 +0000 Charles Lindsey <[email-masked]> 
wrote:

>
> In <[email-masked]> Harald Alvestrand
> <[email-masked]> writes:
>
>> We believe that we have now reached the stage where it's useful to issue
>> the next revision of draft-allbery-usefor-usepro-00 as
>> draft-ietf-usefor-usepro-07, and start tracking issues in the issue
>> tracker again.
>
> In that case I suggest we refer to my present USEPRO as "CHASPRO" whenever
> we have need to refer to it.

usepro-06 should be quite sufficient.

Cc: [email-masked]
Date: Wed, 03 Jan 2007 21:29:51 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: charset of application/news-{groupinfo,checkgroups}
To: Charles Lindsey <[email-masked]>
Sender: [email-masked]

> > Well, no, you don't, provided that without the escaping the newsgroup name
> > is still the same as for ASCII.  If it isn't, then you have a problem
> > because there's no way to do the escaping before the newsgroup name and
> > that charset is therefore banned.  Which is, I believe, correct, since
> > existing software would choke on the initial escaping character.

> Suppose the two modes of the charset are ASCII and Japanese. I am not
> familiar with such charsets, but I would not be surprised if the default
> was to show Japanese characters initially, until an escape character was
> encountered.

This is not how modal charsets work in practice. I don't have my copy of
ISO-2022 handy to check, but it may well allow the initial bindings to the
G0/G1/C0/C1 regions to be specified externally to be something other than
ASCII. (Heaven knows it allows all sorts of far more preposterous stuff, such
as sequences that do a mode switch only for the next character or creating
something that looks like, say, iso-8859-1 except the role of the 8th bit is
reversed.) Nevertheless, any such capability is not used in the various
iso-2022-* variants that have been defined, none of which allow anything
approaching the full set of capabilities of iso-2022.

In the specific case of Japanese, both iso-2022-jp and iso-2022-jp-2 always
start in ASCII. An escape sequence switches to some other character set. There
is also a requirement that things be switched back to ASCII at the end of each
text segment.

The Korean variant iso-2022-kr also starts in ASCII. It differs from the
Japanese variants in using a different set of capabilites. Specifically,
rather than rebinding the G0 set, iso-2022-kr binds the G1 set to Korean
and then uses SI/SO to switch between G0 and G1. (Note: This is all from
memory so some of the finer points may be wrong.)

The two Chinese charsets iso-2022-cn and iso-2022-cn-ext also start in 
ASCII. They then use an even more elaborate mix of iso-2022 capabilities than
Korean or Japanese.

I don't think iso-2022-tw (traditional as opposed to simplified Chinese) is
well defined, but the few places I've seen it crop up seem to indicate it
starts in ASCII like the others. (If anyone has a pointer to an actual
specification for it I'd like to have it.)

I suppose a fair question to ask is if there are other uses of iso-2022 that
would not be so restricted. AFAIK there are not: Things like X.400 generaltext
(or if you want a really obscure one, iso6937text) do allow the full repetoire
of iso-2022 trickery (although practically nobody ever implemented it all), but
there's no way in these protocols to specify the alternate set of initial
character set bindings that would be needed to start in something other than
ASCII.

Another question is if future charsets will be defined on top of iso-2022 that
behave differently. Of course nobody can predict the future, but I think
this is highly unlikely.

Of course there are lots of other CJK charsets, but these are the modal
ones. Charsets like EUC-TW or GB18030 all use 7bit characters to represent
ASCII and sequences of 8bit characters for everything else.

> In which case an attempt to display the checkgroups would
> render the newsgroup-name as gibberish (it would still work correctly as
> regards updating the active file, etc). Japanese users would have to tweak
> their displays to show it right (which is surely possible, because it will
> be a regular problem that they encounter, so it is not necessarily a show
> stopper but still needs drawing attention to).

I don't think this issue exists in practice. See above.

				Ned

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Moderator duties
Organization: The Eyrie
Date: Wed, 03 Jan 2007 21:59:27 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> In this case it needs to be mentioned because leaving it out (see Russ's
> original version) you appear to be giving the impression that the
> Moderator is free to change anything whatsoever that he doesn't like,
> and we don't want to do that.

Well, that's exactly what I want to do, because that's reality.  What I
don't want to give the impression that this is *recommended* or even
remotely best practice for the average moderated group.  But that doesn't
change what the moderator *can* do, which is something we also need to
make clear so that no one makes unwarranted assumptions.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: charset of application/news-{groupinfo,checkgroups}
Organization: The Eyrie
Date: Wed, 03 Jan 2007 21:56:49 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:
>>> Russ Allbery <[email-masked]> writes:

>>>>         eightbit-utext      = utext / %d127-255

>>> OK, <eightbit-utext> was clearly needed, but it still excludes those
>>> ASCII characters not in <utext> notably CR, LF and HT,

>> By HT I assume you mean HTAB?  That's is added back in by WSP in the
>> newsgroup-description production, so only CR and LF are disallowed, which
>> is intentional.

> No, I meant that HT is not included in <utext> according to RFC 2822,
> and hence is not in <eightbit-utext>. Whether we should care or not I am
> not sure.

<utext> doesn't include whitespace, so <eightbit-utext> follows suit.  The
only production that uses it reintroduces non-leading whitespace.

>> Surely this is obviously invalid?  It would mean that the newsgroup
>> name in the charset specified for the body of this type is different
>> than the newsgroup name in ASCII, which is precisely what's forbidden.

> It should be so, but I don't think it is deducible from the wording you
> have written. If you
>    s/MUST be represented using/MUST be represented in that charset using/
> it might work.

Well, that seems redundant to me, but if others feel like it adds clarity,
I don't mind adding it.

>> Well, no, you don't, provided that without the escaping the newsgroup
>> name is still the same as for ASCII.  If it isn't, then you have a
>> problem because there's no way to do the escaping before the newsgroup
>> name and that charset is therefore banned.  Which is, I believe,
>> correct, since existing software would choke on the initial escaping
>> character.

> Suppose the two modes of the charset are ASCII and Japanese. I am not
> familiar with such charsets, but I would not be surprised if the default
> was to show Japanese characters initially, until an escape character was
> encountered. In which case an attempt to display the checkgroups would
> render the newsgroup-name as gibberish (it would still work correctly as
> regards updating the active file, etc).

In which case you can't use that character set, since the newsgroup name
would be some random Japanese characters rather than a newsgroup that
could be represented in ASCII.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24031
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header field
Date: Wed, 3 Jan 2007 20:07:01 GMT
Lines: 61
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>what I'd add to that now is that any site with multiple machines doing
>peering, handling injections etc is very likely to have its own custom
>schemes for ensuring that databases are synchronised (or indeed there
>may just be a central store of data). These are unlikely to depend on
>scanning Path header fields, but of course they may.  However, when
>articles depart from these systems to the rest of the world they'd like
>to add a site-wide identity to show that they have been dealt with, so
>as to discourage them being offered again. Hence the addition of
>multiple path identities by such machines...

>hence we see

>        !newsfeed.gamma.ru!Gamma.RU!
>        !supernews.com!corp.supernews.com
>        !nx01.iad01.newshosting.com!newshosting.com
>        !news.tele.dk!news.tele.dk!small.news.tele.dk

>and doubtless many more (those were found in seconds in a current
>feed...)

Essentially, we need to allow sites to insert more Path entries than the
number of hosts the article actually passed through, just as we need to
allow them to insert fewer than that number of entries if the internal
structure of the site does not need to be exposed outwardly. Just so long
as the leftmost one is the one most likely to be recognized by their
peers, so that MISMATCH checking proceeds smoothly. What is good for
MISMATCH checking is not necessarily best for preventing articles being
sent to them more than once.

>>and can we please then incporate that into the examples somewhere. 

>I'm not sure it's to be encouraged for systems that are not so complex
>as major peering hubs.  Mind you, I sometimes wonder if there's going to
>be all that much left soon that isn't a major peering hub.

I think there are all sort of sites out there with special circumstances
that nobody else is aware of. So if we included an example, we would
probably say something like ...!news.demon.net!demon!incoming.demon.net!...
with and explanation like "although the site news.demon.net does not
actually include a host the with name 'demon', it included that
<path-nodot> because it was aware of a large number of (client) sites that
still knew it by that name". But change "demon" into some other foobarish
name, of course.

Interestingly, we have been talking a lot recently about home sites that
included their own personal newsserver. Demon is unusual in that it
possesses an unusually large number of such clients, since it initially set
all its clients up that way.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24026
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: charset of application/news-{groupinfo,checkgroups}
Date: Wed, 3 Jan 2007 18:08:37 GMT
Lines: 60
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>>>         eightbit-utext      = utext / %d127-255

>> OK, <eightbit-utext> was clearly needed, but it still excludes those
>> ASCII characters not in <utext> notably CR, LF and HT,

>By HT I assume you mean HTAB?  That's is added back in by WSP in the
>newsgroup-description production, so only CR and LF are disallowed, which
>is intentional.

No, I meant that HT is not included in <utext> according to RFC 2822, and
hence is not in <eightbit-utext>. Whether we should care or not I am not
sure.

>> 1. Suppose you specify a charset that works for the
>> <newsgroup-description> but renders the <newsgroup-name> as gibberish.

>Surely this is obviously invalid?  It would mean that the newsgroup name
>in the charset specified for the body of this type is different than the
>newsgroup name in ASCII, which is precisely what's forbidden.

It should be so, but I don't think it is deducible from the wording you
have written. If you
   s/MUST be represented using/MUST be represented in that charset using/
it might work.

>> 2. Suppose you specify a charset that can be escaped into or out of
>> ASCII.  Then you need to ensure that it starts out in ASCII mode, and
>> returns to ASCII mode before the moderation-flag, if any.

>Well, no, you don't, provided that without the escaping the newsgroup name
>is still the same as for ASCII.  If it isn't, then you have a problem
>because there's no way to do the escaping before the newsgroup name and
>that charset is therefore banned.  Which is, I believe, correct, since
>existing software would choke on the initial escaping character.

Suppose the two modes of the charset are ASCII and Japanese. I am not
familiar with such charsets, but I would not be surprised if the default
was to show Japanese characters initially, until an escape character was
encountered. In which case an attempt to display the checkgroups would
render the newsgroup-name as gibberish (it would still work correctly as
regards updating the active file, etc). Japanese users would have to tweak
their displays to show it right (which is surely possible, because it will
be a regular problem that they encounter, so it is not necessarily a show
stopper but still needs drawing attention to).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24029
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: More Path notes
Date: Wed, 3 Jan 2007 19:41:42 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:

>>>   Some existing implementations treat <path-identity> as case-
>>>   sensitive, some case-insensitive.  The <path-identity> therefore
>>>   SHOULD be all lowercase and implementations SHOULD compare identities
>>>   case-insensitively.

>> I would have chosen case-sensitively which should be OK if they obey the
>> first SHOULD, and lots of them do it anyway.

>Yeah, that was my original argument, but when I realized that it differs
>from common practice, I got nervous.  Any place we differ from common
>practice we should have very solid reasons for doing so.

I think you differ from common practice whichever you say. And if you say
one or the other then people may tend to assume they can rely on it, in
spite of the first SHOULD. So if you say "case-insensitive" they will tell
their peers that their site is called "FOO" and then put "foo" in their
Path. And if you say case-sensitive, then they will call their site "FOO"
even though a site "foo" already exists.

I think the second case is marginally more harmful, so it seems you have
talked me into agreeing with you :-) . Harald seems to agree.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24028
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Moderator duties
Date: Wed, 3 Jan 2007 19:25:16 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> "Forrest J. Cavalier III" <[email-masked]> writes:

>Charles Lindsey wrote:

>I agree with this sentiment, but disagree that a reminder must appear in the 
>document everywhere that it applies.  The reminders are out of scope, in
>my view.  This is one of the significant improvements in clarity in RUSSPRO.

In this case it needs to be mentioned because leaving it out (see Russ's
original version) you appear to be giving the impression that the
Moderator is free to change anything whatsoever that he doesn't like, and
we don't want to do that.

So we mention "recommended best practice" here (with reference to USEAGE)
just to make sure we don't give any such wrong impression. The same may be
required at other places. There may be further cases where such a mention
is not so urgent, and there we can discuss whether it is helpful to put it
in or not.

>Are there any other RFCs that babysit the way that Charles suggests?  I'm
>curious.  This isn't a rhetorical question.

I think there are plenty of RFCs that give more than the bare facts of a
protocol by drawing attention to reasons for features, possible
implementation difficulties/strategies, gotchas, and the like.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24027
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Wed, 3 Jan 2007 19:07:58 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>> Hmmmm! Please can somebody say what IS the correct Newsgroups header for a
>> checkgroups? I don't think any of our documents have mentioned this yet.

>There isn't a correct one from a protocol standpoint -- whatever achieves
>the propagation one wants.  Traditionally one uses the administrative
>announcement newsgroup for the hierarchy.

OK, sounds like a thing to be mentioned in USEAGE. Noted as such.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24030
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Turning to issue tracking on the USEPRO document
Date: Wed, 3 Jan 2007 19:46:40 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>We believe that we have now reached the stage where it's useful to issue 
>the next revision of draft-allbery-usefor-usepro-00 as 
>draft-ietf-usefor-usepro-07, and start tracking issues in the issue 
>tracker again.

In that case I suggest we refer to my present USEPRO as "CHASPRO" whenever
we have need to refer to it.

>Hoping to get this nailed down in less than one year....

We still have a long way to go, I fear.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Draft submitted, change policy
Organization: The Eyrie
Date: Wed, 03 Jan 2007 19:25:50 -0800
Sender: [email-masked]

I've submitted draft-ietf-usefor-usepro-07.txt for processing.

At this point, I intend to switch to a fairly strict system for processing
changes, based on the previous message from Harald on using issue
tracking.  Many of the remaining issues are controversial and I usually
have strong opinions on most of the controversies, and I want to keep a
clear line between that and the editing of the draft.  Here's my plan (and
please note in advance that I've not done this before and am likely to
make mistakes in following this plan; please feel free to call me on them
and if I change this plan, I'll let the WG know):

 * Any change which I judge to impact the protocol described by the
   document (as opposed to just the wording used to describe the
   protocol), that I believe is controversial, or that someone else
   believes is controversial I want to track as an issue.  That means I'm
   going to not apply substantive or controversial changes without an
   issue to track the resolution.  I intend to defer to the working group
   chairs on resolution of issues.  When I offer my opinion on a change,
   it will *always* be as an individual, not as editor, unless I
   explicitly say otherwise.

 * I will not make *any* change beyond housekeeping changes (updating the
   date, changing the I-D filename, that sort of thing) without mention on
   the mailing list.  I expect that mention will normally be in the form
   of someone suggesting better wording.  I'll try to follow up and say
   when such suggestions have been applied.

 * If I see a wording change that I believe is uncontroversial (grammar
   fixes or sentence order, that sort of thing, nothing that changes the
   substance of the document), I'll try to make note of it on the mailing
   list using a Subject header prefixed with "Minor Change" rather than
   "ISSUE".  Feel free to use this convention to note similar wording
   fixes that you feel can just be applied without further discussion.

I'm going to see if I can set up automated commit notification software so
that every change I make to the draft will be automatically sent to the
mailing list in diff form.  If this becomes annoying, let me know and I
can turn it off.  My feeling from previous working group discussions is
that this sort of transparency would be welcome, and I already have most
of a system in place for automating it so it's no extra effort for me.

My intention (which may not survive contact with reality) is to respond to
discussions as I have free time but to only apply changes resulting from
resolved issues (and probably also any accumulated minor changes) in batch
on Sundays.  This is a measure designed to help me control my time
committments to various volunteer projects.

It's not yet clear to me whether I'll have enough time to do a good job at
this; if not, with working group approval, I'll ask Ken to help or take
over.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control message propagation
Organization: The Eyrie
Date: Wed, 03 Jan 2007 19:01:42 -0800
Sender: [email-masked]

Richard Clayton <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes

>> the following under cancel control messages:
>> 
>>   A cancel control message SHOULD have the same Newsgroups header field
>>   as the message it is cancelling so that it will attempt to reach the
>>   same news servers as the original message.

> I don't think messages attempt anything :)

>    A cancel control message SHOULD have the same Newsgroups header field
>    as the message it is cancelling so as to best ensure that it will
>    be relayed to the same news servers as the original message.

> though if you dislike split infinitives then yet another phrase will be
> necessary :)

Definite improvement.  The "so as to" still felt awkward, so I also
inverted the sentence.  I now have:

    To best ensure that it will be relayed to the same news servers
    as the original message, a cancel control message SHOULD have the
    same Newsgroups header field as the message it is cancelling.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control message propagation
Organization: The Eyrie
Date: Wed, 03 Jan 2007 18:56:04 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> I now have:

>>   Exceptionally, control messages creating or removing newsgroups
>>   (newgroup or rmgroup control messages, for example) SHOULD be relayed
>>   if the affected group appears in its Newsgroups header field and the
>>   sending agent would have supplied and the receiving agent would have
>>   received the newsgroup affected by the control message had it existed,
>>   even if it currently does not.

> I presume that is intended to go after the current 2nd paragraph of 3.5.

> In which case, since that existing paragraph clearly indicates that it is
> a matter of the configuration of the sending and receiving relaying
> agents, you might as well continue with that terminology. So perhaps:

> "...if the affected group appears in its Newsgroups header field and the
> sending agent and receiving relaying agents are both configured to relay
> a newsgroup of that name (whether or not such a newsgroup yet exists)."

This seems like an uncontroversial improvement in wording (and it's
shorter!).  Changed in my copy, omitting the "yet" at the end since this
also applies to rmgroups and it may well be that the newsgroup was just
removed.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24022
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injection-Date and reinjection
Date: Wed, 3 Jan 2007 16:04:51 GMT
Lines: 301
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>First, one issue that's *not* directly part of this.  Even if we keep the
>current USEPRO method, we need to have a staleness check on the Date
>header field at injection.

Well there we have the first major disagreement. The whole point of
introducing Injection-Date was to remove the need for staleness checks
based on Date (except in cases where older software had not provided an
Injection-Date).

The Date header now indicates when the article was composed (following RFC
2822), so it can arise that the poster carries it around on his laptop for
several days before it actually gets injected. That could reasonably be a
week later (whereas Russ proposes to declare it stale after 72 hours,
which is far too short). Maybe even a fortnight if he writes the article
and then goes off on holiday for two weeks and omits to dialin to his
server before he goes. Clearly, leaving it for several months would be
stupid, but such timing issues are a matter of sensible practice, and not
a matter for the protocol.

>  We can't make a single leap to allowing stale
>Date header fields from not having any Injection-Date header field since
>most existing software is going to ignore the Injection-Date header field
>entirely and impose staleness checks on Date after injection.  Therefore,
>an article with a stale Date is going to propagate very poorly, and
>accepting such an article just to let it be dropped silently later by most
>of the Netnews network would be a poor interoperability choice.

OK, so let it propagate poorly. In fact, I don't think its propagation
will be all that bad. For sure, the fast transit backbone may decline it,
but it will still flood its way through the system via serving agents with
normal expiry times. I would have expected it to have reached most sites
within one day. And as Injection-Date becomes more widely implemented,
things will of course get better.

But this is a thing that is easily testable. I have just posted a series
of articles "1 day stale", "2 days stale" up to "9 days stale" to
misc.test, and I will see what the autoresponders tell me (and it would be
useful for members of this list to look out for them also). I expect to
see some long and unusual Paths for the staler ones, but I expect them
all to arrive eventually. We shall see.

>Now, let's look at the conditions under which this situation arises.

>....  The INN
>code to permit people to inject posts via IHAVE is something I committed
>only under protest, and I'm entirely content to have its behavior declared
>non-comformant by a standard.  It is a *very* specialized configuration
>for strange cases where the people involved know just what they're doing.

Not really. I regard it as a method of avoiding unnecessary wastage of
server resources in cases where articles are injected at more than one
server. If one of the servers has already received it, then you save the
trouble of transmitting the whole thing again only for it then to be
thrown away.

>That being said, there are disjoint Netnews networks that people want to
>connect, and multiple injection or proxying injection isn't always the
>best approach (for one, it requires a real-time connection with the remote
>injecting agent).  My definition of two disjoint Netnews networks is two
>Netnews networks between which there is no relaying agent to relaying
>agent connection.

Not so. You also have to ensure that there is at most only one *-agent to
injecting agent route.

>  Significant examples are:

> * The many people who do not have any relaying agreement with a large
>   Netnews network such as Usenet, only a reading agent relationship, but
>   still want the features of a full-fledged news server (longer article
>   retention, multiple users sharing the same spool, faster service to
>   reading agents, local newsgroups without requiring clients use multiple
>   servers, off-line operation).  This has become increasingly common over
>   the last five years for both personal and small organization use.

That is probably safe for personal use, but in the case of small
organizations, and increasingly as those organizations get larger, there
is the possibility that some well-meaning user in the organization will
create some surreptitious path/leak to the outer world, even if only for
some supposedly safe group that suddenly becomes unsafe because of an
unanticipated crosspost. It is a sad fact of life that such leaks DO
happen.

> * Ad hoc connections to a stand-alone news server that serves only a
>   particular hierarchy. ...

And again these can lead to unanticipated leaks.

>The goal is to handle these cases with a solution that has the
>following properties:

> * Netnews loop detection and prevention is preserved.

> * Articles are stamped with trustworthy injection information.  Since,
>   particularly in the first and third cases, the reason why no relaying
>   agreement is in place is precisely because the original injection
>   information can't be trusted, this requires replacing injection
>   information when the message is gated from one Netnews network to
>   another.

This is certainly true in the case of Injection-Info. The case in dispute
is Injection-Date which, where it already exists, can probably be trusted
to have an accurate date, or at worst a date too far into the past which
is a safe error so far as loop avoidance is concerned.

>My realization when reviewing the draft was that all or nearly all of the
>cases where this arises and is not simply a bug (such as treating relaying
>and injecting as interchangeable) can be meaningfully treated as disjoint
>Netnews networks.

Yes, I see now where you are coming from as treating local networks at the
edges of Usenet as "disjoint", but my concern is that such disjointedness
is too hard to police.

>  The case which Charles raises of a news server that
>acts as a relaying agent to one server and a posting agent to another
>server on the same network is an interesting possible exception; it's
>rather pathological, but I can see the argument that the end results, if
>done properly, are not distinguishable from multiple injection.

Yes, I accept "pathological" as a valid description of what I do, but my
view is that most cases of reinjection will turn out to be "pathological"
to some degree, and that it is impossible to foresee all the strange
circumstances that might be involved.

>  But let's
>put that aside for a moment and look at the alternatives before us in the
>more typical case.

There are essentially two possibilities:

1. Injection-Dates are ALWAYS rewritten by injecting agents (or
alternatively articles containing them are ALWAYS dropped).

2. Once an article has acquired an Injection-Date (whatever network put it
there), it ALWAYS retains that Injection-Date wherever it subsequently
goes.

#2 (which I advocate) is evidently safe against causing loops, but might
cause articles to be lost. So one MIGHT allow an exception for admins who
really REALLY knew what they were up to.

#1 is not safe against loops unless the admin who allows the rewriting
really REALLY knows what he is up to.

>Option 1 (my current draft)

>    Proto-articles may not contain Injection-Date.  Injecting agents must
>    reject any message that contains Injection-Date.  If an agent needs to
>    reinject a message, it must think of itself as a gateway, remove the
>    injection header fields (including Injection-Date), perform whatever
>    checks it needs to ensure that it's not creating a loop, and then
>    reinject the message, at which point it gets a new Injection-Date
>    header field with the current date.  This gateway is fully responsible
>    for not creating loops.

>Option 2 (current USEPRO draft)

>    For a given post, an injecting agent may either be doing injection or
>    reinjection based on whether injection headers fields are already
>    present.  It may decline to do reinjection.  If it doesn't, it may
>    rename an existing Injection-Info header to some other name or remove
>    it and must retain any existing Injection-Date header field.  It may
>    perform a staleness check against the Date header field if no
>    Injection-Date header field is present.  (The current USEPRO draft
>    doesn't say an injecting agent can perform staleness checks against an
>    existing Injection-Date header, but I presume that's simply an
>    oversight.)

Not an oversight. It was presumed that the immediately following serving
or relaying agent would do that check.

>    The injecting agent is responsible for not creating
>    loops, not the agent that offers the article for reinjection.

If the injecting agent retains the Injection-Date (maybe it should check
it is not into the future) then it has fulfilled its loop-avoiding
responsibility. If the offering agent has removed the Injection-Date, then
that is another matter.

>I've already talked about why I think the first option is cleaner and
>clearer from a protocol description and implementation standpoint.  Let me
>instead focus on failure modes.

>The primary risk of the first option is that it could create a slow loop
>in the presence of bidirectional gatewaying as follows:

Which is why USEPRO required an outgoing gateway to retain the
Injection-Date if possible, i.e if the other medium had some place to
record it. If the other medium cannot record it anyway, then the two
Options are equivalent, and both must rely on the circularity checking by
the gateways (which SHOULD be provided anyway).

>The second option does not have this same failure mode, and indeed it
>opens the door to fewer additional problems as long as we still have to
>enforce fresh Date header fields.  The failure mode with the second option
>comes when we want to drop the staleness check on a Date header field.
>Suppose that a stand-alone news server B wants to pull articles from a
>Netnews network A for local consumption.  B has a power failure and is
>off-line for an extended period of time (a week, say).  When it comes back
>up, it wants to catch up on all of the news from A that it missed while it
>was down.  Now, as long as we have to enforce Date staleness, it can only
>do this by ignoring a SHOULD at reinjection, since those articles will
>likely now have stale Dates, but it can be configured to do that.
>However, under option 2, catching up on gatewaying has now become
>impossible because it is REQUIRED to retain the Injection-Date header of
>the original message but that Injection-Date header is now stale and will
>therefore be rejected by any relaying or serving agent in B.

Well that is hardly a common scenario, and for sure B is going to have to
bypass some checks somewhere whichever Option we use. For sure, it no
longer cares about relaying stuff onwards (the rest of the net will have
seen those articles long since), so it only cares about its own serving
agent(s). Now for serving agents, staleness is determined by "the earliest
articles of which it keeps record" (in its history file). Since the expiry
time of serving agents is typically longer than a week, it may not even
need to bypass the staleness check at all, but if not then a temporary
change to that "earliest articles of which it keeps record" is all that is
needed. That might indeed let in a few articles it had actually seen
before the breakdown but which got sent again from A as part of the mixup,
but that is a small price to pay for a breakdown and will not lead to any
loops so long as the original Injection-Date is still retained, and
provided it does not promptly break down again for a further week. Indeed,
it is that magic "72 hours" which seems to be built into Option 1 that is
likely to be more of a problem than Injection-Date staleness.

>Now, my basic argument, apart from the simplicity of option 1, is that the
>failure scenario for option 2 is worse than the failure scenario for
>option 1.

Which doesn't seem to be born out by my analysis.

>Okay, phew.  Now back to Charles's case.

>Suppose we have a new server with a relaying agreement to another
>unreliable news server and a posting agreement with a much more reliable
>news server, and that news server wants to send outgoing articles via both
>paths.  Under option 2, the injecting agent that this server talks to must
>support reinjection (it's an optional feature in the current USEPRO
>draft), must enable it for that client, and then takes responsibility for
>doing the appropriate transformation, but Injection-Date is retained.
>Under option 1, this news server must either support multiple injection
>(far and away the best option) and inject at the remote server at the same
>time as the message is injected locally, or it must both relay the message
>and gateway it to the remote injecting agent.  The gatewaying is more
>complex, but can be done entirely locally and the article will then look
>to the remote server like a regular proto-article.  No special support on
>the remote server is therefore required; all of the work is born by the
>agent doing something unusual.  In either case, two slightly different
>copies of the message will exist with different trace headers, but this is
>acceptable; the same is true for multiple injection.

I agree that the only cases in which Option 2 could be problematical is
with a privately operated local server such as my own. In all other cases,
as I have tried to show, it works at least as well as, and often better
than, Option 1.

Now for a local server there are two problems:

1. It may operate offline, so there will be a delay between when it is
injected locally and when it is relayed (whether by proper relaying or by
posting), though it would be unusual for that delay to be long enough to
cause premature staleness. Hence the reason I mentioned the possibility of
allowing exceptions for admins who "really REALLY" knew what they were
about. In this case, "really REALLY" means being quite sure that there is
no possibility of unofficial leaks to the outer world (easy if he is the
sole user of the system, harder in a small organization where other users
are not so easily controlled). But in that case, a reasonable "cheat"
would be to delay adding the Injection-Date until the time came to make
the outgoing connection.

2. If he only has posting privileges at his outgoing feed (or for some of
his outgoing feeds) then he has to be sure that they will accept what is
technically a reinjection. Again, the obvious "cheat" is to omit the
Injection-Date, and for that he has to be just as "really REALLY" sure as
in the first case.

Yes, the burden of preventing nasty things happening rests with the agent
wanting to do the unusual things.

But note also that the alternative scenario you have suggested for
injecting only to the outgoing feed, or to the outgoing feed and the local
system simultaneously, only works if you have an "always on" connection to
the outside.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 3 Jan 2007 13:20:10 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Path header field
Sender: [email-masked]

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>If Richard
>Clayton is listening, could he please repeat his reasons 

from June 2005, how time flies :(

>why the
><path-nodot> "demom" was needed (and is still used) by demon's servers,

What I said in http://www.imc.org/ietf-usefor/mail-archive/msg01381.html
was:

<quote>

It would also be unwise to put any existing servers in a position that
they found themselves feeling that to conform to modern standards they
should change their identity...

Demon Internet (the first UK dialup ISP in the early 1990s) has used
!demon! since before it was called Demon Internet or even before it sent
news over NNTP. When an error was made with a new peering machine and
this tag was omitted a number of customers (with simple single-user
"leaf" systems who only swapped articles with Demon) started fetching
articles and (trying to) re-inject them all. Because servers elsewhere
in the cluster, generally, already had a copy of the article this went
on unnoticed for weeks until occasionally the customer presented the
article earlier than the peering machine.... and when the article was
spam it tripped detectors and we found the problem.

What I'm trying to illustrate is that path identities are not only
important to the few thousand machines that peer news on a high volume
basis, but these identities are nailed into hundred of thousands of end
user systems. We outlaw (or deprecate) existing path identities, whether
dating from UUCP or more recent conventions, at our peril.

</quote>

what I'd add to that now is that any site with multiple machines doing
peering, handling injections etc is very likely to have its own custom
schemes for ensuring that databases are synchronised (or indeed there
may just be a central store of data). These are unlikely to depend on
scanning Path header fields, but of course they may.  However, when
articles depart from these systems to the rest of the world they'd like
to add a site-wide identity to show that they have been dealt with, so
as to discourage them being offered again. Hence the addition of
multiple path identities by such machines...

hence we see

        !newsfeed.gamma.ru!Gamma.RU!
        !supernews.com!corp.supernews.com
        !nx01.iad01.newshosting.com!newshosting.com
        !news.tele.dk!news.tele.dk!small.news.tele.dk

and doubtless many more (those were found in seconds in a current
feed...)

>and can we please then incporate that into the examples somewhere. 

I'm not sure it's to be encouraged for systems that are not so complex
as major peering hubs.  Mind you, I sometimes wonder if there's going to
be all that much left soon that isn't a major peering hub.

>The
>whole <path-nodot> business was heavily influenced by his example, but I
>forget the precise reasoning now.

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

Date: Wed, 3 Jan 2007 11:51:52 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Control message propagation (was: draft-allbery-usefor-usepro-00 errata)
Sender: [email-masked]

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes

>the following under cancel control messages:
>
>   A cancel control message SHOULD have the same Newsgroups header field
>   as the message it is cancelling so that it will attempt to reach the
>   same news servers as the original message.

I don't think messages attempt anything :)

   A cancel control message SHOULD have the same Newsgroups header field
   as the message it is cancelling so as to best ensure that it will
   be relayed to the same news servers as the original message.

though if you dislike split infinitives then yet another phrase will be
necessary :)

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

Date: Wed, 3 Jan 2007 12:42:00 +0000
To: [email-masked]
From: Richard Clayton <[email-masked]>
Subject: Re: Path header field (was: draft-allbery-usefor-usepro-00 errata)
Sender: [email-masked]

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes

>3.2.1.  Constructing the Path Header Field
>
>   If a relaying or serving agent receives an article from an injecting
>   or serving agent that is part of the same news server, it MAY leave
>   the Path header field of the article unchanged.  Otherwise, every
>   injecting, relaying, or serving agent that accepts an article MUST
>   update the Path header field as follows.  Note that the Path header
>   field content is constructed from right to left by prepending
>   elements.
>
>   1.  The agent MUST prepend "!" to the Path header field content.
>
>   2.  An injecting agent SHOULD prepend the <path-diagnostic>
>       "!.POSTED", optionally followed by "." and the FQDN or IP address
>       of the source, to the Path header field content.
>
>   3.  A relaying or serving agent SHOULD prepend a <path-diagnostic> to
>       the Path header field content, where the <path-diagnostic> is
>       chosen as follows:
>
>       *  If the expected <path-identity> of the source of the article
>          matches the leftmost <path-identity> of the Path header
>          field's content, use "!" (<diag-match>).

If you're not aware of the !! syntax (which is after all "new") then I
think this would confuse, and people will be able to read it so as to
miss out one of the !s

could it say

        #1   All agents MUST always first prepend "!" ..

        #2   An injecting agent SHOULD then prepend ...

        #3   A relaying or serving agent SHOULD then prepend ...

        *  If the expected <path-identity> of the source of the article
           matches the leftmost <path-identity> of the Path header
           field's content, then prepend the <diag-match> string "!". So 
           in this, hopefully most common case, two !s will appear in
           the path.

>       *  If the expected <path-identity> of the source of the article
>          does not match, use "!.MISMATCH." followed by the expected
>          <path-identity> of the source or its IP address.
>
>       *  If the relaying or serving agent is not willing or able to
>          check the <path-identity>, use "!.SEEN." followed by the FQDN,
>          IP address, or expected <path-identity> of the source.

I'd then add

        Note that previous versions of this standard did not require
        <path-diagnostics> so that conformant systems would only add
        single "!" characters between <path-identity> strings.

>   4.  The agent MUST then prepend its own <path-identity> to the Path
>       header field content.
>
>   5.  The agent MAY then prepend additional <path-identity>s for itself
>       to the Path header field content, following each <path-identity>
>       so added with either "!!" or "!".  This is permitted for agents
>       that have multiple <path-identity>s (such as during a transition
>       from one to another).  Each of these <path-identity>s MUST meet
>       the requirements set out in Section 3.2.

For !! to be of any use (which I continue to feel is entirely dubious,
but that's water under the bridge) then surely the requirement in this
last paragraph ought to be for !! (because the server must surely be
claiming that self-to-self is a "validated" transfer)

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

Date: Wed, 03 Jan 2007 12:12:23 +0100
From: Harald Alvestrand <[email-masked]>
To: [email-masked]
Subject: Turning to issue tracking on the USEPRO document
Sender: [email-masked]

We believe that we have now reached the stage where it's useful to issue 
the next revision of draft-allbery-usefor-usepro-00 as 
draft-ietf-usefor-usepro-07, and start tracking issues in the issue 
tracker again.

As soon as the draft is out, please switch to the logic of raising 
issues by writing a message to the mailing list saying "ISSUE: USEPRO 
<section>: <subject>".

The chairs will enter an issue in the tracker if one or more of the 
following things is true:
- the chairs agree it is an issue that needs discussion
- one or more persons apart from the person raising the issue agrees 
that it is an issue, and says so on the mailing list ("+1" can be used, 
but be clear on what you're responding to - +1 on a refutation is not 
the same as +1 on the original issue!)

The chairs will not enter an issue in the tracker if:
- they don't think it is an issue
- the editor responds with "yes, that's an error, fixed in my copy" or 
equivalent before the issue can be tracked, and nobody else disasgrees 
with that statement.

Hoping to get this nailed down in less than one year....

Date: Wed, 03 Jan 2007 09:24:18 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: Xref and relaying agents
Sender: [email-masked]

Charles Lindsey wrote:

>>No, because changes made by serving agents would only be seen by posting
>>agents.  Serving agents do not sent messages to any other news server.
>>Only relaying agents do that.
>>    
>>
>
>Ah! I see! We carefully said that the order of the steps in the various
>"Duties" sections could be varied so long as the final effect was the
>same, but we omitted to say the same for the order in which serving and
>relaying were done.
>
>You could cover it in the definitions section 1.4 by saying:
>
>'A "relaying agent" accepts articles from injecting agents, serving
>agents, or other relaying agents and distributes them ...".'
>
>possibly with corresponding tweaks in the varius "Duties" sections.
>
No.

Date: Wed, 03 Jan 2007 09:20:51 +0100
From: Harald Alvestrand <[email-masked]>
To: Charles Lindsey <[email-masked]>
Cc: [email-masked]
Subject: Re: Son of 1036
Sender: [email-masked]

Charles Lindsey wrote:

>I have had some email correspondence with Henry Spencer, and it seems he
>is at last putting Son-of-1036 into a form suitable for an
>informational/historic RFC, which we can therefore refer to in our drafts.
>
>  
>
If you could relay the conversation to the chairs, that would be greatly 
appreciated.
Henry hasn't answered any recent messages to any mail I've sent to any 
address I've had for him.

                     Harald

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Scope of application/* media types
Organization: The Eyrie
Date: Tue, 02 Jan 2007 23:43:27 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:
>> Charles Lindsey <[email-masked]> writes:

>>> In this case, the side efect is that a line gets added/changed in the
>>> Newsgroups file.

>> I don't agree with this interpretation, and I don't see anything in the
>> text of the application/news-groupinfo registration that reflects this.

> I doubt if you would be sending an application/group-info in an
> article/message unless you thought that the recipient might want to use
> it for exactly that purpose.

In my role as control message issuer for the Big Eight hierarchies, I
would exchange such mail messages routinely.  I wouldn't be processing
such parts to change a newsgroups file; I'd use it as input into the
database used to send subsequent control messages.

>> It sounds to me like you're confusing the newgroup control message with
>> the application/news-groupinfo MIME media type.  They're not the same
>> thing.

> But are clearly related.

They're related only insofar as we need application/news-groupinfo to
clearly express the complete action of a newgroup message, but the former
has no inherent connection to the latter.  It's a one-way link.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: application/news-groupinfo in newgroup
Organization: The Eyrie
Date: Tue, 02 Jan 2007 23:51:11 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> In that case we need to look into checkgroups. In a hierarchy that does
> not customarily bother with <newsgroup-description>s, I would expect a
> checkgroups message to be of the form:

> example.unmoderated.group<HTAB>
> example.moderated.group<HTAB> (Moderated)

Yeah, I was wondering about that.  Currently, we don't allow that content
in a checkgroups message, but I've seen this in the wild (without the
trailing HTAB, in fact) and I believe that some servers honor checkgroups
of this form.  I'd have to check to be sure, though.

I don't think it's unreasonable to say that if you want to issue a
checkgroups, you have to provide a description, even if it's something
lame like "?".  But it's probably worth discussing whether we want to
permit the empty description explicitly.

>>> So you need something like:

>>>     The application/group-info MUST be included if the group is
>>>     moderated, in which case it MUST include a <moderation-flag>. For
>>>     other groups, it MAY be omitted if there is no
>>>     <newsgroup-description> to be provided.

>> Why do we need something like that?  There's a perfectly usable moderation
>> flag in the Control header field.

> OK, point taken there. The question is whether it would be normal for
> such an application/group-info to be present for at least all moderated
> groups (answer probably 'yes'), in which case should that be reinforced
> with at least a 'SHOULD' (answer uncertain, but the 'SHOULD' you already
> have covers it anyway)?

I don't see why a moderated group should be different from an unmoderated
one when it comes to newgroup control messages.  If there's no
description, there's no description; for newgroup, unlike checkgroups,
nothing relies on the description for any protocol function beyond
updating the optional newsgroups file.  (Up to and including reading agent
recognition of moderated newsgroups; that's done via the LIST ACTIVE
flag.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Xref and relaying agents
Organization: The Eyrie
Date: Tue, 02 Jan 2007 23:40:31 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:

> Ah! I see! We carefully said that the order of the steps in the various
> "Duties" sections could be varied so long as the final effect was the
> same, but we omitted to say the same for the order in which serving and
> relaying were done.

> You could cover it in the definitions section 1.4 by saying:

> 'A "relaying agent" accepts articles from injecting agents, serving
> agents, or other relaying agents and distributes them ...".'

> possibly with corresponding tweaks in the varius "Duties" sections.

> My problem was with your wording under Duties of a Relaying Agent where
> you said:

> 9. It MAY delete any Xref header field present and MAY add a new Xref
> header field with any valid content. ...

> That will be confusing to a reader who has not spotted the subtlety of
> the particular order of happenings that we have described, and he will
> ask himself "of what possible use is an Xref header added by a
> _relaying_ agent?". And of course the answer is "none" because it was
> not really the relaying agent that you had in mind, but just an artefact
> of the way things were described.

I think that approach is more confusing than mine and would prefer to keep
the approach that I used in my draft.  The agents are a conceptual model,
and conceptual models are more useful the simpler we can keep them.  We're
always going to have to do some mapping of conceptual agents to real
software and the boundaries are always going to be messy; the best we can
manage is an approximation.  But the purpose of the conceptual model is to
provide a clear view of the overall flow so that one has the theory to
then apply to a particular implementation, and the simpler we can keep the
model while still expressing how Netnews works, the easier that will be.

> I think the simplest thing is to admit that relaying agents may obtain
> their input from serving agents, as I have suggested above, and then
> that Step 9 for relaying agents can be reduced to a simple "MAY delete
> XREFs".

Except they don't obtain their input from serving agents when you look at
the architecture of news servers that implement those roles using separate
programs.  This is confusing primarily because such servers are unusual
(and necessary for only very high reader volume) and most servers instead
implement a simpler combined relaying and serving agent.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Injecting agents and From
Organization: The Eyrie
Date: Tue, 02 Jan 2007 23:34:22 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> inews as distributed with INN or the common news readers is a (part of
>> a) posting agent, not an injecting agent.

[...]

>> You can see this by observing what actions it takes and what agent it
>> talks to.  It sends messages to an injecting agent via POST and expects
>> that agent to do the injection;

> Not so. 'inews' existed long before the POST command was invented.

inews *as distributed with INN or the common news readers* does indeed
behave exactly as I describe.  As I mentioned explictly later, the version
that comes with C News is much older and may well do something different.

> For sure, fashions have changed since then. But I am sure there are
> still people using updated versions of those early newsreaders (such as
> nn and trn) which still call inews, taking advantage of the automatic
> fillng in of From, and those will all work happily with Bnews, Cnews,
> INN, and for all I know any other newsservers which provide the 'inews'
> interface.

Sure, they can continue to delegate some of their posting agent duties to
inews if they have an inews on hand that does them.  If they are used with
C News, that inews will apparently also do injection; if they are used
with INN, that inews will hand the post off to a separate injecting
agent.  The newsreader doesn't have to care.  Neither, so far as I can
tell, does the language in our draft.

[...]

> I see no reason why a similar wording should not continue to be used.

It draws the line between posting agent and injecting agent in the wrong
place for the most common case of NNTP, and is therefore unnecessarily
confusing.  It's a semantic distinction that I think is important in
trying to understand which piece of software does what.  C News used
without NNTP will be an unusual case.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control message propagation
Organization: The Eyrie
Date: Tue, 02 Jan 2007 23:27:42 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

>> That's apparently a derivation from usepro-06 in your text, you have
>> "MAY send or accept whatever" with a "MUST ignore or reject unknown".

> For those who are lost, those two statements appear in a single sentence
> in section 5, and do seem confusing when put together. Also, I don't
> understand what "reject" means (as dintinct from "ignore" or "not
> honour") in this context.

The same as it means in all other contexts.  Rejecting the message means
refusing to accept it and process it.

>> IMO that can't fly if relaying agents reject unknown stuff.  We could
>> create a IANA registry of control <verb>s, reserving x-whatever for
>> experimental / unofficial <verb>s, and anything else to be defined in
>> an RFC.  With a general "MUST ignore unknown", limiting "MAY reject
>> unknown" to injecting agents.  Or better removing the "reject" option.

> I think what the sentence is really trying to say is:

> "Agents MAY make arrangements to accept other control message types than
> those specified below, but otherwise MUST ignore all unrecognized
> types."

No, that wasn't what I was trying to say.  The above is functionally
equivalent to what I have now, but I think explicitly mentioning the
always-present option to reject a message is clearer.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24001
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Scope of application/* media types (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 19:28:55 GMT
Lines: 71
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 26. [-1] (4.2,4.3) Removed requirement that
>>>> application/news-[groupinfo,checkgroups] MUST NOT be used except with
>>>> relevant control messages.

>> Application types are intended to cause applications to be invoked, with
>> possible side effects.

>Could you provide a reference for this?  I don't see this statement in RFC
>2046.  The closest that it has is:

>   The "application" media type is to be used for discrete data which do
>   not fit in any of the other categories, and particularly for data to
>   be processed by some type of application program.  This is
>   information which must be processed by an application before it is
>   viewable or usable by a user.

OK. Perhaps I was carried away by the propensity of Microsoftware to
execute anything that is remotely executable even if the user just blinks
:-( .

And it is certainly the case that application types are mostly intended to
be processed by application programs that may have side effects beyond
fancy forms of display. And RFC 2046 goes to considerable lengths to warn
agains possible side effects of obeying an application/postscript.

>> In this case, the side efect is that a line gets added/changed in the
>> Newsgroups file.

>I don't agree with this interpretation, and I don't see anything in the
>text of the application/news-groupinfo registration that reflects this.

I doubt if you would be sending an application/group-info in an
article/message unless you thought that the recipient might want to use it
for exactly that purpose.

>It sounds to me like you're confusing the newgroup control message with
>the application/news-groupinfo MIME media type.  They're not the same
>thing.

But are clearly related.

>> OK, in that case what you need to say is:

>>     This media type is intended to be used in conjunction with the
>>     newgroup control message as described in section 5.2.1. It MUST NOT be
>>     automatically invoked in any other context without explicit human
>>     intervention.

>> and similarly for checkgroups.

>I disagree with adding anything like this to our document.

OK. In which case the existing sentences at the start of 4.2 and 4.3 will
suffice.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23999
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Xref and relaying agents (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 18:07:19 GMT
Lines: 65
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 20. [00] (3.5) Relaying agent MAY add a new Xref header.

>>> Because the same piece of software also implements a serving agent,
>>> which has to add the Xref header, and the same code path is used for
>>> all articles received by the server whether they are for further
>>> relaying or local serving.

>> Well if that is the case it was nominally the serving function that
>> added the Xref (section 3.6).

>No, because changes made by serving agents would only be seen by posting
>agents.  Serving agents do not sent messages to any other news server.
>Only relaying agents do that.

Ah! I see! We carefully said that the order of the steps in the various
"Duties" sections could be varied so long as the final effect was the
same, but we omitted to say the same for the order in which serving and
relaying were done.

You could cover it in the definitions section 1.4 by saying:

'A "relaying agent" accepts articles from injecting agents, serving
agents, or other relaying agents and distributes them ...".'

possibly with corresponding tweaks in the varius "Duties" sections.

My problem was with your wording under Duties of a Relaying Agent where
you said:

9. It MAY delete any Xref header field present and MAY add a new Xref
header field with any valid content. ...

That will be confusing to a reader who has not spotted the subtlety of the
particular order of happenings that we have described, and he will ask
himself "of what possible use is an Xref header added by a _relaying_
agent?". And of course the answer is "none" because it was not really the
relaying agent that you had in mind, but just an artefact of the way
things were described.

I think the simplest thing is to admit that relaying agents may obtain
their input from serving agents, as I have suggested above, and then that
Step 9 for relaying agents can be reduced to a simple "MAY delete XREFs".

>> Now whether such an agent first stores the article and then relays what
>> it has stored, or whether it relays first and then stores after is none
>> of our business.

And so our document should reflect that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24003
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: application/news-groupinfo in newgroup (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 20:06:40 GMT
Lines: 75
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 31. [-1] (5.2.1) Newgroup message SHOULD include application/group-info.

>Its absence may mean that no newsgroups file entry is created at all.  The
>newsgroups file is an optional feature of NNTP and need not be
>implemented, nor need the newsgroups file be accurate or complete.  See
>section 7.6.6 of RFC 3977:

>   The list MAY omit newsgroups for which the information is unavailable
>   and MAY include groups not available on the server.  The client MUST
>   NOT assume that the list is complete or that it matches the list
>   returned by LIST ACTIVE.

Hmmm! With hindsight I might have said that, where a Newsgroups file
existed, the Active file SHOULD be a subset of it. Oh well!

>Newsgroups are not required to have descriptions unless you want to send a
>checkgroups for them.  Some Netnews networks (such as the servers on which
>microsoft.* is hosted) simply don't use them.

>> Likewise for checkgroups.

In that case we need to look into checkgroups. In a hierarchy that does
not customarily bother with <newsgroup-description>s, I would expect a
checkgroups message to be of the form:

example.unmoderated.group<HTAB>
example.moderated.group<HTAB> (Moderated)

And many NNTP servers would probably keep a Newsgroups file containing
exactly that. Or at least the (Moderated) ones.

In which case I would expect a newgroup message for a moderated group to
contain an application/group-info with

example.moderated.group<HTAB> (Moderated)

so that such NNTP servers would get their Newsgroups files set up
accordingly.

>> So you need something like:

>>     The application/group-info MUST be included if the group is moderated,
>>     in which case it MUST include a <moderation-flag>. For other groups, it
>>     MAY be omitted if there is no <newsgroup-description> to be provided.

>Why do we need something like that?  There's a perfectly usable moderation
>flag in the Control header field.

OK, point taken there. The question is whether it would be normal for such
an application/group-info to be present for at least all moderated groups
(answer probably 'yes'), in which case should that be reinforced with at
least a 'SHOULD' (answer uncertain, but the 'SHOULD' you already have
covers it anyway)?

So I seem to have accepted your wording, but maybe there is still a need to
indicate that checkgroups are still appropriate even in hierarchies where
there are no <newsgroup-description>s.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24002
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Approved header field content (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 19:42:21 GMT
Lines: 65
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 30. [-1] (5.2) Nothing said about content of Approved header.

>>>> Surely, it SHOULD identify the person/identity/role of the issuer, ...

>>> Intentional change.

>>> The content of the Approved header serves no protocol purpose, and
>>> USEFOR already adequately covers the definition of its content.
>>> Control message authorization is done on the basis of the Sender or
>>> From header (preferrably in combination with a digital signature).

>The description in USEFOR is:

>   The Approved header field indicates the mailing addresses (and
>   possibly the full names) of the persons or entities approving the
>   article for posting.  Its principal uses are in moderated articles
>   and in group control messages; see [I-D.ietf-usefor-usepro].

Which certainly implies that an Approved header needs to contain an email
address identifying the person responsible for
posting/authorizing/whatever (which may well be the same as what is in the
From/Sender, or it may be the 'role' which the From/Sender person claims
to be fulfilling).

But, either way, USEPRO needs to ensure that the Approved header contains
what USEFOR says it is supposed to contain. You have covered this properly
in section 3.8 in the case of moderators. All I am asking is that you
cover it with similar wording in the case of group control messages.

>The Netnews protocol currently does not deal with authorization at all (an
>obvious flaw noted in Security Considerations).  Any authorization
>information you want to use has to be derived from the underlying
>transport protocol or from unstandardized extensions such as digital
>signatures.

No, that is 'authentication'. But I have written about 'authorization' in
another thread.

>If you're referring to group control mesages, said identity is checked
>against the *From or Sender* header field, not the Approved header, at
>least in INN.  INN ignores the contents of the Approved header.  I don't
>know if C News uses the contents of the Approved header field for control
>message permissions, but my impression from having maintained control.ctl
>for some years is that most everyone uses From/Sender.

Yes, I have looked at CNews and I was wrong. It looks at the From (not
even Sender AFAICS) and compares that with what it is configured to
honour.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24004
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Tue, 2 Jan 2007 20:51:41 GMT
Lines: 53
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Russ Allbery wrote:

>> The standard makes it clear that anyone who wishes can invent a new
>> control message and start sending them.

>That's apparently a derivation from usepro-06 in your text, you have
>"MAY send or accept whatever" with a "MUST ignore or reject unknown".

For those who are lost, those two statements appear in a single sentence
in section 5, and do seem confusing when put together. Also, I don't
understand what "reject" means (as dintinct from "ignore" or "not honour")
in this context.

>IMO that can't fly if relaying agents reject unknown stuff.  We could
>create a IANA registry of control <verb>s, reserving x-whatever for
>experimental / unofficial <verb>s, and anything else to be defined in
>an RFC.  With a general "MUST ignore unknown", limiting "MAY reject
>unknown" to injecting agents.  Or better removing the "reject" option.

I think what the sentence is really trying to say is:

"Agents MAY make arrangements to accept other control message types than
those specified below, but otherwise MUST ignore all unrecognized types."

>The IANA registry stuff is fairly simple, Harald has written a kind of
>"howto" (2434bis), and "defined in an RFC" is straight forward.

No, please keep IANA out of this. IANA has quite enough to do already :-( .

>#######################################################################

>  [A cancel SHOULD match the newsgroups of the original article]
>>> Is that good enough for a MUST instead of a SHOULD ?  Or an explicit
>>> OPTION to ignore cancels violating the SHOULD ?

>> Two reasons not to make it a MUST:

I basically agree with Frank here, but I have explained my reasons in
another thread.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24007
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Son of 1036
Date: Tue, 2 Jan 2007 21:40:25 GMT
Lines: 14
Sender: [email-masked]

I have had some email correspondence with Henry Spencer, and it seems he
is at last putting Son-of-1036 into a form suitable for an
informational/historic RFC, which we can therefore refer to in our drafts.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24005
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Tue, 2 Jan 2007 21:00:16 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Frank Ellermann <[email-masked]> writes:

>> Spam cancels intentionally violate official rules, following their own
>> standards.

>Spam cancels and other sorts of administrative cancels are compliant with
>my protocol spec and I want to keep it that way.  They're an authorization
>problem, not a protocol problem.

Discussion of cancels and who may issue them (1st, 2nd, and 3rd parties
and all that) was moved into USEAGE way back (it was not a clear-cut
decision and I could have been persuaded to move them back, but only one
person ever suggested doing so). So let us leave it that way.

>I think the current text is fine.

But that does not mean I agree with the current text (again, I addressed
this in another thread).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24006
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Tue, 2 Jan 2007 21:38:09 GMT
Lines: 73
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Frank Ellermann <[email-masked]> writes:

>> It was your idea to add the OPTION of new control verbs, with the odd
>> effect of talking about newgroup + rmgroup "for example", as if there
>> are others.

>No, it was not my idea.  It's how Netnews actually works and has for
>longer than I've been involved in it.  From the first version of INN, you
>could drop in a handler for a brand new control message by doing nothing
>more than putting a shell script in a directory, and people did so to
>various ends.  Adding a new group control message would also require
>changing one line of code in innd that classifies control messages as
>"newgroup-like" for propagation purposes, but that's trivial.  That new
>control verbs weren't allowed was a pure artificial creation of previous
>documents (assuming one even read them that way) which I have attempted to
>bring in line with reality.

The fact that it is easy to drop a new control message type into existing
servers is no argument for making it a readily available feature of the
protocol. It only makes it suitable for use in "cooperating subnets" (a
term which Henry invented for S-o-1036, and which I see you have removed
from your draft). It does not make it suitable for use throughout the
whole of Usenet.

If you want to introduce a new control message type for Usenet as a whole,
then it is essential that everybody knows in advance exactly how it is
supposed to operate, and it needs very careful preparation, since you only
get one chance to get it right. So it really needs an RFC, whether
experimental or standards-track (and I don't think even experimental is
sufficient for Usenet as a whole since, once the experiment is out, there
is no way to put it back in the bag). By all means try out your experiment
on a cooperating subnet, but generally speaking control messages are so
few and far between that it is difficult to obtain evidence of whether the
experiment works, except by artificially creating and removing test groups
in all sorts of contrary ways (which is essentially what I did when I
tested mvgroup in CNews, and an cooperating subnet of 1 sufficed for
that).

So I am essentially dubious about the "MAY accept other control message
types" unless it is coupled with mention on some sort of cooperating
subnet, or with mention of an extension RFC. But maybe I am not as dubious
as Frank :-) .

>>> Since relaying agents may reject messages for any reasons they chose,

>> This is not the case, compare usepro-06 chapter 6.3 points 1..8.  They
>> must / should / may reject articles for the reasons specified in 1..8.

>I didn't realize that anyone in the working group didn't consider this
>principle implicit and assumed.  I'm glad, in this case, that I made it
>explicit in my draft.  Section 3.1 of my draft says:

I agree with Russ here. USEPRO always made it clear that individual sites
could always refuse to accept any article "just because it is Friday" if
they were so minded. The one thing they were NOT allowed to do was to
alter it if they did not like it (beyond a few defined circumstances).

Not that this makes much difference for relayers because, if one site
refuses to relay an article, the flooding algorithm will still find a way
to get it to pretty well every other site.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24000
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Serving agents and duplicates (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 19:08:23 GMT
Lines: 49
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:

>I now have:

(Presumably in section 3.6)

>   3.  It MUST reject any article that has already been successfully
>       sent to it, based on the Message-ID header field of the article.
>       To satisfy this requirement, a relaying agent normally keeps a
>       database of message identifiers it has already accepted.

>   4.  It SHOULD reject any article that matches an already-received and
>       honored cancel message or Supersedes header field following the
>       same rules as a relaying agent (Section 3.5).

OK. That is better. Though in fact it might wait until it sees the actual
article before deciding whether to honor the cancel.

>which is basically the same text as for relaying agents.

>> OK, but people will think it odd that you mention the obscure case of
>> cancels which arise first here, but make to mention of the common case.
>> Perhaps an extra step to say that it SHOULD then process any control
>> messages (including cancels) in acccordance with section 5.

>This would need to be added to relaying agents as well.

ITYM serving agents. Relaying agents should always just pass the cancel
message on and let serving agents downstream decide for themselves whether
to honour it.

But for serving agents, in addition to the wording above, you need at the
least a NOTE or a remark in parentheses to the effect that

"Cancel messages and Supersedes header fields arriving later then the
affected article are to be processed as described in section 5.3."

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23998
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Injecting agents and From (was: Re: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 17:45:34 GMT
Lines: 103
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 3. [-1] (3.3.1,3.4,3.9.2) From not omittable in proto-article

>>>> There is existing usage where the injecting agent fills in From header
>>>> (not possible in NNTP, of course)

>>> Intentional change.

>>> What injecting agent supports this?

>> INN apparently (see below).

>> Newsreaders such as rn, trn, nn, and others of that generation had (and
>> still have) the capability to interact directly with the newspool
>> (either on the same host, or more likely NFS mounted from some server)
>> rather than going via NNTP (which did not exist when they were first
>> written). They injected articles by calling a program 'inews' which, in
>> the absence of an explicit From:, assumed the poster was the user who
>> had called 'inews'.

>inews as distributed with INN or the common news readers is a (part of a)
>posting agent, not an injecting agent.

Ah! So it boils down to a matter off semantics. Surely, when the first
newsreaders were written (RN and earlier, before NNTP came on the scene),
their final act when posting an article was to hand it off to 'inews',
which came with the software of the server (which would be BNews in those
days). What does the 'i' in 'inews' stand for, if not for 'inject'. For
sure, people using UNIX never configured individual pieces of software
with their own name and id, because they naturally assumed that the system
would pick it up from /etc/passwd and the fact that they were logged in as
their own id.

>  You can see this by observing what
>actions it takes and what agent it talks to.  It sends messages to an
>injecting agent via POST and expects that agent to do the injection;

Not so. 'inews' existed long before the POST command was invented. AFAICT
from the O'Reilly Book, which was the definitive tutorial on administering
Usenet from 1986 onwards, the principal interfaces to BNews were 'inews'
and 'rnews'. Inews had an amazing collection of parameters (you could
even create groups locally with it or issue control messages).

>Our agent distinctions are largely based on how INN and similar news
>servers work.  C News, being a much earlier implementation, may not have
>the same distinctions, or they may be much less clear.  Some
>implementations written prior to our standard will combine different
>agents into one program, so it's possible that in C News inews combines
>functions of a posting agent and an injecting agent.

In CNews, 'inews' IS the injecting agent, which performs all the relevant
checks, sends it to the moderator if needed, and otherwise puts it into
the input queue alongside stuff received via 'rnews', whence it eventually
gets stored locally and relayed onwards. If stuff arrives via the NNTP
POST command, then it merely generates a call on 'inews'. It is certainly
not regarded as a posting agent (a crude posting agent 'readnews' is
provided - not really suited to serious newsreading) and there is a script
'postnews' which prompts you for Newsgroup and Subject, lets you use your
favourite editor to construct the article (adding other headers, even
From:, as you wish), and finally calls 'inews'. So 'inews' is not regarded
in any sense as a posting agent, although it would likely be called
directly by scripts which generated articles automatically (such as
modbots).

For sure, fashions have changed since then. But I am sure there are still
people using updated versions of those early newsreaders (such as nn and
trn) which still call inews, taking advantage of the automatic fillng in
of From, and those will all work happily with Bnews, Cnews, INN, and for
all I know any other newsservers which provide the 'inews' interface.

So there is no reason not to recognise that, even if the usage is not so
common as it once was. The wording I put into USEPRO (when discussing
headers that could be omitted from proto-articles) was:

"...(and even From if the particular injecting agent can derive that
information from other sources.)"

which seems to me an accurate description of the situation I have
described. I see no reason why a similar wording should not continue to be
used.

>INN's inews used to redundantly perform a few (although not all by any
>means) of the functions of an injecting agent, such as mailing posts to
>the moderators of moderated newsgroups.

Sure, 'inews' used to have all sorts of Bells and Whistles, and even CNews
provides a cutdown version 'injnews' which it recommends for routine use.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 03 Jan 2007 02:36:08 +0100
From: Harald Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>, [email-masked]
Subject: Path case sensitivity (Re: More Path notes)
Sender: [email-masked]

I thought we'd been here before, but the context differed:
USEFOR 3.1.3:

   Some software will try to match the <id-right> of a <msg-id> in a
   case-insensitive fashion; some will match it in a case-sensitive
   fashion.  Implementations MUST NOT generate a Message-ID where the
   only difference from another Message-ID is the case of characters in
   the <id-right> part.

USEFOR 3.1.4:

  A <component> SHOULD NOT consist solely of digits and SHOULD NOT
  contain uppercase letters.  Such <component>s MAY be used only to
  refer to existing groups that do not conform to this naming scheme,
  but MUST NOT be used otherwise.

     NOTE: All-digit <component>s conflict with one widely used storage
     scheme for articles.  Mixed-case groups cause confusion between
     systems with case-sensitive matching and systems with case-
     insensitive matching of <newsgroup-name>s.

So it seems that we have established a tradition where two different kinds 
of matching are out there in the wild that we note the fact, and specify 
something that works in all cases.

In this case, I think case-insensitive works in all cases - and since this 
is a procedures document, it's appropriate to specify the matching rule to 
use when executing the procedure, while such a statement wouldn't be 
appropriate in USEFOR.

But - matching the language in USEFOR 3.1.4 - I think "SHOULD" in both 
cases is appropriate. I don't see a big argument for declaring 
case-sensitive matchers unconditionally non-conformant. (But I could easily 
live with MUST too)

                    Harald

--On 2. januar 2007 10:46 -0800 Russ Allbery <[email-masked]> wrote:

>
> Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>
>>> The more I research this, though, the more I think that while the above
>>> is true, it's not ideal.  Poking around a bit, it looks like a *lot* of
>>> existing implementations are case-insensitive, and I'm not sure I want
>>> to declare that incorrect.  Accordingly, I now have the following text,
>>> which I think is an improvement.  Please, everyone, let me know if you
>>> disagree:
>
>>>   Some existing implementations treat <path-identity> as case-
>>>   sensitive, some case-insensitive.  The <path-identity> therefore
>>>   SHOULD be all lowercase and implementations SHOULD compare identities
>>>   case-insensitively.
>
>> First SHOULD is OK, but I am no sure why you then choose
>> case-insensitivity, since that is more work in an intensively used piece
>> of code.
>
> Because existing software looks mostly case-insensitive, indicating that
> the performance issue has not been an issue in practice and raising the
> possibility that something is relying on that behavior given how
> widespread it is.  So this seems safer to me, plus given that it's not
> been a problem in practice, your argument that hostnames are inherently
> case-insensitive seems fairly well-founded.
>
>> I would have chosen case-sensitively which should be OK if they obey the
>> first SHOULD, and lots of them do it anyway.
>
> Yeah, that was my original argument, but when I realized that it differs
> from common practice, I got nervous.  Any place we differ from common
> practice we should have very solid reasons for doing so.
>
> --
> Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>
>
>

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: More Path notes
Organization: The Eyrie
Date: Tue, 02 Jan 2007 10:46:26 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>> The more I research this, though, the more I think that while the above
>> is true, it's not ideal.  Poking around a bit, it looks like a *lot* of
>> existing implementations are case-insensitive, and I'm not sure I want
>> to declare that incorrect.  Accordingly, I now have the following text,
>> which I think is an improvement.  Please, everyone, let me know if you
>> disagree:

>>   Some existing implementations treat <path-identity> as case-
>>   sensitive, some case-insensitive.  The <path-identity> therefore
>>   SHOULD be all lowercase and implementations SHOULD compare identities
>>   case-insensitively.

> First SHOULD is OK, but I am no sure why you then choose
> case-insensitivity, since that is more work in an intensively used piece
> of code.

Because existing software looks mostly case-insensitive, indicating that
the performance issue has not been an issue in practice and raising the
possibility that something is relying on that behavior given how
widespread it is.  So this seems safer to me, plus given that it's not
been a problem in practice, your argument that hostnames are inherently
case-insensitive seems fairly well-founded.

> I would have chosen case-sensitively which should be OK if they obey the
> first SHOULD, and lots of them do it anyway.

Yeah, that was my original argument, but when I realized that it differs
from common practice, I got nervous.  Any place we differ from common
practice we should have very solid reasons for doing so.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:23996
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: More Path notes (was: Protocol changes in draft-allbery-usefor-usepro-00)
Date: Tue, 2 Jan 2007 16:29:54 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Charles Lindsey <[email-masked]> writes:
>> Russ Allbery <[email-masked]> writes:
>>> Charles Lindsey <[email-masked]> writes:

>>>> 1. [-1] (2.2) <path-identity> - Possibility to use a FQDN not
>>>> resolvable in the DNS.

>>>> (This was a suggestion by Harald, & 1 of 2 alternatives in USEPRO)

OK, I responded to this in an earlier thread.

>>>> 2. [00] (3.2) <path-identity>s are case-sensitive.

>> OK, I will buy that.

And now you aren't selling it :-( .

>The more I research this, though, the more I think that while the above is
>true, it's not ideal.  Poking around a bit, it looks like a *lot* of
>existing implementations are case-insensitive, and I'm not sure I want to
>declare that incorrect.  Accordingly, I now have the following text, which
>I think is an improvement.  Please, everyone, let me know if you disagree:

>   Some existing implementations treat <path-identity> as case-
>   sensitive, some case-insensitive.  The <path-identity> therefore
>   SHOULD be all lowercase and implementations SHOULD compare identities
>   case-insensitively.

First SHOULD is OK, but I am no sure why you then choose
case-insensitivity, since that is more work in an intensively used piece
of code. I would have chosen case-sensitively which should be OK if they
obey the first SHOULD, and lots of them do it anyway. Or let them do it
anyway they wish, provided they understand the risk (which is no worse
then being sent some articles you have already seen).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 02 Jan 2007 08:23:40 -0500
From: "Forrest J. Cavalier III" <[email-masked]>
To: [email-masked]
Subject: Re: Moderator duties
Sender: [email-masked]

Charles Lindsey wrote:

> In <[email-masked]> Russ Allbery <[email-masked]> writes:
> 
> 
>>I went over the text in Duties of a Moderator about what a moderator is
>>allowed to do and came up with the following rewording.  Do folks feel
>>that this is an improvement?  Does this address some of the concerns with
>>the previous language?
> 
> 
>>  There are no protocol restrictions on what criteria are used for
>>  accepting or rejecting messages or on what modifications a moderator
>>  may make to a message (both header fields and body) before injecting
>>  it.  Recommended best practice, however, is to make the minimal
>>  required changes.  Moderators need to be aware that modifications
>>  made to articles may invalidate signatures created by the poster or
>>  previous moderators.  See [USEAGE] for further best-practice
>>  recommendations.
> 
> 
> Yes, that will work.
> 
> But it does establish the point that I was tying to make, namely that the
> standard does need to admit that there is more to running Usenet than bare
> adherence to a standard.

I agree with this sentiment, but disagree that a reminder must appear in the 
document everywhere that it applies.  The reminders are out of scope, in
my view.  This is one of the significant improvements in clarity in RUSSPRO.

Are there any other RFCs that babysit the way that Charles suggests?  I'm
curious.  This isn't a rhetorical question.

To: [email-masked]
Xref: clerew local.usefor:23993
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Moderator duties
Date: Tue, 2 Jan 2007 10:39:08 GMT
Lines: 59
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I went over the text in Duties of a Moderator about what a moderator is
>allowed to do and came up with the following rewording.  Do folks feel
>that this is an improvement?  Does this address some of the concerns with
>the previous language?

>   There are no protocol restrictions on what criteria are used for
>   accepting or rejecting messages or on what modifications a moderator
>   may make to a message (both header fields and body) before injecting
>   it.  Recommended best practice, however, is to make the minimal
>   required changes.  Moderators need to be aware that modifications
>   made to articles may invalidate signatures created by the poster or
>   previous moderators.  See [USEAGE] for further best-practice
>   recommendations.

Yes, that will work.

But it does establish the point that I was tying to make, namely that the
standard does need to admit that there is more to running Usenet than bare
adherence to a standard.

Whether this is expressed by reference to some "hierarchy administrators"
who are appointed by some undefined external mechanism, or by reference to
some "moderation policy (aka 'charter') for the group", again established
by undefined external mechanism, or by reference to "some established
practice/custom/consensus of the network", or to "recommended best
practice" (with pointer to USEAGE) is just a matter of taste - some such
phraseology is needed.

"Recommended best practice" is about as weak a phrase as you could get,
but it works for this case, and possibly for some other cases that we will
encounter.

What is not so clear is the matter of control messages, where section 5.1
speaks of "authorization" with the expectation that agents will magically
know who is "authorized" but no indication where the "authority" comes
from. The related matter of "authentication" is also vague, but at least
we know that cryptographic methods exist to solve this and will hopefully
be covered in some later document (though I will be pressing later on for
something a little stronger in the present text, but that can wait for
now).

But for "authority", we still need some hint that sources for such
authority do exist (varying by hierarchy) and that they are a matter of
consensus, and may not be universally accepted. I don't see at the moment
how "recommended best practice" could be applied to cover that situation,
but we may be able to find some other phaseology for it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: Control message propagation
Organization: The Eyrie
Date: Mon, 01 Jan 2007 22:16:34 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Frank Ellermann <[email-masked]> writes:

>> Same here, a "checkgroups" listing _all_ newsgroups in the Newsgroups
>> header field would be odd.  Get rid of "for example", the qualifier is
>> confusing without "mvgroup".

> Hmmmm! Please can somebody say what IS the correct Newsgroups header for a
> checkgroups? I don't think any of our documents have mentioned this yet.

There isn't a correct one from a protocol standpoint -- whatever achieves
the propagation one wants.  Traditionally one uses the administrative
announcement newsgroup for the hierarchy.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:23991
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: to.* newsgroups (was: draft-allbery-usefor-usepro-00 errata)
Date: Mon, 1 Jan 2007 17:51:02 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I'm convinced.  I now have:

>   These control messages are normally sent as point-to-point articles
>   between two sites and not then sent on to other sites.  Newsgroups
>   beginning with "to." are reserved for such point-to-point
>   communications and are formed by prepending "to." to a <relayer-name>
>   to form a <newsgroup-name>.  Articles with such a group in their
>   Newsgroups header fields SHOULD NOT be sent to any news server other
>   than the one identified by <relayer-name>.

Fine!

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23992
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Path header field
Date: Mon, 1 Jan 2007 20:52:37 GMT
Lines: 159
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>As promised, here is the new section.  I also made the corresponding
>updates to the other sections (removing the description and replacing it
>with a reference to this section) and moved the Path header field example.

>3.2.1.  Constructing the Path Header Field

>   If a relaying or serving agent receives an article from an injecting
>   or serving agent that is part of the same news server, it MAY leave
>   the Path header field of the article unchanged.  Otherwise, every
>   injecting, relaying, or serving agent that accepts an article MUST
>   update the Path header field as follows.  Note that the Path header
>   field content is constructed from right to left by prepending
>   elements.

I think that is more restrictive than we intended. The idea was that if
there was a "farm" of servers at one site, splitting the various duties
(incoming articles, outgoing articles, local storage of articles) amongst
themselves in some manner, it was not necessary to include every server
the article actually passed through (i.e. we do not need to see, from the
outside, the detailed internal structure of that site).

>   1.  The agent MUST prepend "!" to the Path header field content.

>   2.  An injecting agent SHOULD prepend the <path-diagnostic>
>       "!.POSTED", optionally followed by "." and the FQDN or IP address
>       of the source, to the Path header field content.

I don't think that we ever agreed to permit a <diag-identity> after
"POSTED", though we did discuss it. I have no great objection if others
want it (semantically, it is rather like "SEEN"). However, if allowed, it
needs to indicate that the FQDN or IP address is indeed a <diag-identity>,
in which case the question arises of why you have not permitted a
<path-nodot> (which the syntax would allow).

>   3.  A relaying or serving agent SHOULD prepend a <path-diagnostic> to
>       the Path header field content, where the <path-diagnostic> is
>       chosen as follows:

>       *  If the expected <path-identity> of the source of the article
>          matches the leftmost <path-identity> of the Path header
>          field's content, use "!" (<diag-match>).

At this point, you need a definition of "expected". I see that you mention
"trusted" sources and agents in 3.4 and 3.5 (but not 3.6). In USEPRO, I
had a generic definition in section 7, but using "verified" rather than
"trusted". I think you either need such a generic definition in this
section, or you need to define "expected" by reference to those earlier
mentions.  And I would have thought that "verified" was a better term than
"expected", but that is less of an issue than ensuring the term in
question is consistently defined somehow.

>       *  If the expected <path-identity> of the source of the article
>          does not match, use "!.MISMATCH." followed by the expected
>          <path-identity> of the source or its IP address.

Again, you are actually constructing a <diag-match> here, but this time
you DO allow a <path-nodot>.

>       *  If the relaying or serving agent is not willing or able to
>          check the <path-identity>, use "!.SEEN." followed by the FQDN,
>          IP address, or expected <path-identity> of the source.

But now the <path-nodot> is gone again :-( .

>   4.  The agent MUST then prepend its own <path-identity> to the Path
>       header field content.

>   5.  The agent MAY then prepend additional <path-identity>s for itself
>       to the Path header field content, following each <path-identity>
>       so added with either "!!" or "!".  This is permitted for agents
>       that have multiple <path-identity>s (such as during a transition
>       from one to another).  Each of these <path-identity>s MUST meet
>       the requirements set out in Section 3.2.

You need to be careful here. Certainly an agent with several
<path-identity>s can prepend a selection of them, but it MUST ensure that
the last (leftmost) prepended one is its "official identity" by which it
will be known to its peers; otherwise, it risks getting MISMATCHed.

>   Any agent which modifies the Path header field MAY fold it by
>   inserting FWS immediately after any <path-identity> or <diag-other>
>   it added (see section 3.1.5 of [USEFOR] for allowable locations for
>   FWS).

OK, the general structure of what you have written is fine. The only extra
item I can think of at the moment is that you need to say that you have
only defined three <diag-keyword>s (there is a pointer to USEPRO regarding
this in USEFOR), and that other <diag-keyword>s MUST/SHOULD NOT be used,
except those beginning with "X" (for sites that claim a need to say
something that cannot be expressed with the three we have provided, or for
other such experimental reasons).

The section on Path Examples presumably follows here here. If Richard
Clayton is listening, could he please repeat his reasons why the
<path-nodot> "demom" was needed (and is still used) by demon's servers,
and can we please then incporate that into the examples somewhere. The
whole <path-nodot> business was heavily influenced by his example, but I
forget the precise reasoning now.

>>  * (Still under discussion.)  We may wish to reintroduce the possibility
>>    of a path-identity that is not a resolvable name in DNS.

>This was already in my draft.  I have:

>   The <path-identity> used by an agent may be chosen via one of the
>   following methods (in decreasing order of preference):

>   1.  The fully-qualified domain name (FQDN) of the system on which the
>       agent is running.

If this means A and AAAA records then please, for the removal of all
doubt, make that explicit.

>   2.  A fully-qualified domain name (FQDN) within a domain affiliated
>       with the administrators of the agent and guaranteed to be unique
>       by the administrators of that domain.  For example, the
>       uniqueness of server.example.org could be guaranteed by the
>       administrator of example.org even if there is no DNS record for
>       server.example.org itself.

Again, if this is meant to include MX records and CNAMES, please make it
explicit, because the wording does not immediately suggest that, and it
will hopefully be the common case.

However, I am still opposed to allowing domain-names for which no DNS
exists, and if that gets taken out then there is little point in keeping
this paragraph, and the MX and CNAMEs may as well go back in #1.

>   3.  Some other (arbitrary) name .....

>The most common current practice is to use the FQDN of the news server,
>not a mail server, as a path identity.  If one wants to use a mail server,
>2. still allows for that; it doesn't require that the name *not* exist in
>DNS.

>In my original draft, I intentionally removed any protocol-required
>connection between a <path-identity> and a contact address.

There hasn't been any such protocol-required connection for a long time.
We decided to let RFC 2142 (muddled though it is) speak for itself.
However, that it no reason not to draw attention to it as USEPRO did
(slightly differently in the two alternatives, of which I prefer the
freestanding second version which came from Harald); observe that it
carefully refrains from saying that you MUST/SHOULD follow it. I would be
equally happy yo see it in a NOTE, but it needs to be mentioned somewhere.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23989
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Mon, 1 Jan 2007 17:35:07 GMT
Lines: 19
Sender: [email-masked]

In <[email-masked]> Frank Ellermann <[email-masked]> writes:

>Same here, a "checkgroups" listing _all_ newsgroups in the Newsgroups
>header field would be odd.  Get rid of "for example", the qualifier is
>confusing without "mvgroup".

Hmmmm! Please can somebody say what IS the correct Newsgroups header for a
checkgroups? I don't think any of our documents have mentioned this yet.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23988
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Control message propagation
Date: Mon, 1 Jan 2007 17:32:25 GMT
Lines: 79
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I now have:

>   Exceptionally, control messages creating or removing newsgroups
>   (newgroup or rmgroup control messages, for example) SHOULD be relayed
>   if the affected group appears in its Newsgroups header field and the
>   sending agent would have supplied and the receiving agent would have
>   received the newsgroup affected by the control message had it existed,
>   even if it currently does not.

I presume that is intended to go after the current 2nd paragraph of 3.5.

In which case, since that existing paragraph clearly indicates that it is
a matter of the configuration of the sending and receiving relaying
agents, you might as well continue with that terminology. So perhaps:

"...if the affected group appears in its Newsgroups header field and the
sending agent and receiving relaying agents are both configured to relay a
newsgroup of that name (whether or not such a newsgroup yet exists)."

>for the propagation rule, the following under group control messages:

>   Group control messages affecting specific groups (newgroup and
>   rmgroup control messages, for example) SHOULD include the <newsgroup-
>   name> for the group or groups affected in their Newsgroups header
>   field.  Other newsgroups MAY be included in the Newsgroups header
>   field so that the control message will reach more news servers, but
>   due to the special relaying rules for group control messages (see
>   Section 3.5) this is normally unnecessary and may be excessive.

>the following under cancel control messages:

>   A cancel control message SHOULD have the same Newsgroups header field
>   as the message it is cancelling so that it will attempt to reach the
>   same news servers as the original message.

I think I am persuaded by Frank's argument to s/SHOULD/MUST/ in both those
cases (even though USEPRO has "SHOULD"). As to the two exceptions you
mentioned, a man who cancels an article in a test group might actually
prefer to receive confirmation that is had reached the usual
auto-responders, and spam cancellers are in some sense already breaking
the rules, so what the heck if they break some more? But even there, the
spam cancel needs to propagate everywhere the article propagated, and I do
not buy the argument about hierarchies that have special groups for
cancels. Surely NNTP servers are going to put them all in control.cancel
anyway, so that special group would appear to be empty anyway?

>and the following in Security Considerations:

>   Cancel control messages are not required to have the same Newsgroups
>   header field as the messages they are cancelling and, since they are
>   sometimes processed before the original message is received, it may
>   not be possible to check that they do.  This allows a malicious
>   poster to inject a cancel control message for an article in a
>   moderated newsgroup without adding an Approved header field to the
>   control message, and to hide malicious cancel control messages from
>   some reading agents by using a different Newsgroups header field so
>   that the cancel control message is not accepted by all news servers
>   that accepted the original message.

And I do not at all follow the scenarios you envisage there. If the cancel
arrives before its matching article at some server which needs to act on
it, who cares about checking its newsgroups header anymore? And a malicious
canceller who wanted to hide the cancel from certain sites would do better
to preload the Path header.

But if you s/SHOULD/MUST/, then none of this arises anyway.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23990
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ihave/sendme syntax (was: draft-allbery-usefor-usepro-00 errata)
Date: Mon, 1 Jan 2007 17:48:20 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Russ Allbery <[email-masked]> writes:

>I've since reviewed RFC 1036 and Son-of-1036.  The former makes the
>relayer name optional and requires that the message IDs be in the control
>header field.  The latter makes the relayer name mandatory and RECOMMENDS
>that the message IDs be put in the body.  I think the following is the
>most reasonable compromise; it makes the relayer-name mandatory if there
>are no message IDs (the form never allowed in RFC 1036 but according to
>Son-of-1036 the most widely used in practice), and if there are message
>IDs, permits the same syntax as RFC 1036.

I don't see why the optionality or otherwise of the <relayer-name> should
correlate in any way with whether the list of msg-ids is in the header or
the body (except to ensure that these headers never have empty content,
which does not sound like a particularly good reason).

So I would think it better to follow Son-of-1036 and make it obligatory
regardless. I think we can assume that Henry had sufficient experience of
the way Ihave and Sendme were used at that time that it is safe to follow
his lead.

In any case, the paragraph which follows explaining how to use these two
headers is written on the assumption that a <relayer-name> will be
provided.

If any pair of existing relayers is currently using this protocol without
<relayer-name>s, then good luck to them, and 'lang may their lum reek'. 

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Russ Allbery <[email-masked]>
To: [email-masked]
Subject: Re: charset of application/news-{groupinfo,checkgroups}
Organization: The Eyrie
Date: Mon, 01 Jan 2007 10:20:39 -0800
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Russ Allbery <[email-masked]> writes:

>>         newsgroup-description
>>                             = eightbit-utext *( *WSP eightbit-utext )
>>         moderation-flag     = %x28.4D.6F.64.65.72.61.74.65.64.29
>>                                  ; case sensitive "(Moderated)"
>>         eightbit-utext      = utext / %d127-255

> OK, <eightbit-utext> was clearly needed, but it still excludes those
> ASCII characters not in <utext> notably CR, LF and HT,

By HT I assume you mean HTAB?  That's is added back in by WSP in the
newsgroup-description production, so only CR and LF are disallowed, which
is intentional.

> so you could not use EBCDIC (where I am sure those octets correspond to
> useful characters) nor any charset which could be escaped into or out of
> ASCII and in which those octets had some other meaning when escaped.

Right.

> Yes, that almost gets you there, and maybe near enough. But it does result
> in some oddities:

> 1. Suppose you specify a charset that works for the
> <newsgroup-description> but renders the <newsgroup-name> as gibberish.

Surely this is obviously invalid?  It would mean that the newsgroup name
in the charset specified for the body of this type is different than the
newsgroup name in ASCII, which is precisely what's forbidden.

> 2. Suppose you specify a charset that can be escaped into or out of
> ASCII.  Then you need to ensure that it starts out in ASCII mode, and
> returns to ASCII mode before the moderation-flag, if any.

Well, no, you don't, provided that without the escaping the newsgroup name
is still the same as for ASCII.  If it isn't, then you have a problem
because there's no way to do the escaping before the newsgroup name and
that charset is therefore banned.  Which is, I believe, correct, since
existing software would choke on the initial escaping character.

> 3. Totally stupid charsets such as EBCDIC are excluded.

> 4. The ISO 8859-* series will work for now, but not if/when
> <newsgroup-name>s in UTF-8 appear.

Right.

> All those snags are implicit in the wording you have written, but are
> not immediately evident to a reader not already familiar with these
> issues. So it would be kinder to use a wording which made them more
> evident, even at the expense of a little extra text. That would include
> explicit mention of ASCII as a subset and of the alternative possibility
> of escaped charsets, including the necessity to ensure they were somehow
> in ASCII mode at the proper places.

As above, I think escaped charsets don't work as easily as you're
implying.  I'm not sure that I'm convinced for the rest, but if you have a
language suggestion in mind, I'd certainly consider it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:23986
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: charset of application/news-{groupinfo,checkgroups}
Date: Mon, 1 Jan 2007 16:20:43 GMT
Lines: 72
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Here's what I currently have:

>   The content of the application/news-groupinfo body part is defined
>   as:

...........

>         newsgroup-description
>                             = eightbit-utext *( *WSP eightbit-utext )
>         moderation-flag     = %x28.4D.6F.64.65.72.61.74.65.64.29
>                                  ; case sensitive "(Moderated)"
>         eightbit-utext      = utext / %d127-255

OK, <eightbit-utext> was clearly needed, but it still excludes those ASCII
characters not in <utext> notably CR, LF and HT, so you could not use
EBCDIC (where I am sure those octets correspond to useful characters) nor
any charset which could be escaped into or out of ASCII and in which those
octets had some other meaning when escaped.

>   This unusual format is backward-compatible with the scanning of the
>   body of newgroup control messages for descriptions done by Netnews
>   implementations that predate this specification.  Although optional,
>   the <newsgroups-tag> SHOULD be included for backward compatibility.

>   The <newsgroup-description> MUST NOT contain any occurrence of the
>   string "(Moderated)" within it.  Moderated newsgroups MUST be marked
>   by appending the case sensitive text " (Moderated)" at the end.

>   While a charset parameter is defined for this media type, most
>   existing software does not understand MIME header fields or correctly
>   handle descriptions in a variety of charsets.  Using a charset of US-
>   ASCII where possible is therefore RECOMMENDED; if not possible, UTF-8
>   [RFC3629] SHOULD be used.  Regardless of the charset used, the
>   constraints of the above grammar MUST be met and the <newsgroup-name>
>   MUST be represented using the same octets as would be used with a
>   charset of US-ASCII.

Yes, that almost gets you there, and maybe near enough. But it does result
in some oddities:

1. Suppose you specify a charset that works for the
<newsgroup-description> but renders the <newsgroup-name> as gibberish.

2. Suppose you specify a charset that can be escaped into or out of ASCII.
Then you need to ensure that it starts out in ASCII mode, and returns to
ASCII mode before the moderation-flag, if any.

3. Totally stupid charsets such as EBCDIC are excluded.

4. The ISO 8859-* series will work for now, but not if/when
<newsgroup-name>s in UTF-8 appear.

All those snags are implicit in the wording you have written, but are not
immediately evident to a reader not already familiar with these issues. So
it would be kinder to use a wording which made them more evident, even at
the expense of a little extra text. That would include explicit mention of
ASCII as a subset and of the alternative possibility of escaped charsets,
including the necessity to ensure they were somehow in ASCII mode at the
proper places.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:23987
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Decision: Let's get draft-allbery-usefor-usepro-00 right and publish it as draft-ietf-usefor-usepro-07
Date: Mon, 1 Jan 2007 16:34:53 GMT
Lines: 58
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>Frank Ellermann <[email-masked]> writes:
>> Russ Allbery wrote:
> 
>>> The situation only gets interesting when the poster themselves has
>>> provided the message ID, which is an entirely different case.

>> It's the interesting case - I don't care about any provisional M-ID used
>> for the communication between the server where the article was
>> submitted, and the moderator(s).  If the first (and only the first)
>> moderator somehow identifies a provisional M-ID he's free to beautify
>> it.  But modifying an original M-ID is utter dubious, and it could cause
>> complete confusion if a later (not the first) moderator starts to modify
>> an already approved M-ID.

>How?  Could you provide some specific examples of what interoperability
>problems you believe could result?  As previously mentioned, cancels are
>not an insolvable issue; cancels of a message in a moderated group have to
>be approved by the moderator anyway, at which point the correct message ID
>can be used.

Yes, but for that to work a moderator would have to keep a list of
original message-ids vs new message-ids, and I can't see many moderators
being willing to go to that trouble.

And then there the posters who try to cancel the message themselves by
Approving it themselves. Naughty, but it's going to happen, as Frank has
said.

Then there are the people who keep records of their own message-ids so
that they can immediately identify followups to their own articles. If
that does not work, then they will perceive it as an interoperability
matter (I don't think you can restrict "interoperability" only to features
specified by the protocol and ignore interoperability with widely
implemented practices - indeed we have already extended it to situations
of that nature).

And then there are people who mail and post their article at the same
time, perhaps to a mailing list, and perhaps in a manner which causes it
to appear in Usenet under its original ID (any maybe that IS a case to be
cuahgt and stopped, but maybe not).

Essentially, it is far far safer to make it clear that miderators 'must'
use the original ID _unless_ they have some very good reason for changing
it, and that is exactly the sort of situation that "SHOULD" was invented
for.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5